Repository navigation
Trading unattended hosting and recovery recipe: Dagu start-all evidence chain, fail-closed NYSE-session guard, restart drill, restic journal recovery (operation gated on T2/T3) - #736
Conversation
… schedule, journal snapshot and restic procedure (unit T PR-4) - Dagu 2.16.6 (pin unchanged) runs as `dagu start-all` with `coordinator.enabled: false` set in config.yaml, sourced from startall.go:32-45 and loader.go:1311-1319. - The unit keeps Restart=on-failure. The research DAG's four variables are delivered through the DAG's own env block, because the unit's empty-environment launcher discards inherited variables. - The named schedule is CRON_TZ=America/New_York, weekdays at 16:30. A precondition step runs the locked exchange-calendars 4.13.2 session check and skips non-session days; early closes keep the 16:30 run. - journal_recovery.py copies each SQLite journal into staging through the Online Backup API and freezes the inventory (sha256, bytes). A restic backup treats every nonzero exit, including exit 3, as failure. `check --read-data-subset` rotates n/7 within seven days. Restore is compared against the frozen inventory, then integrity_check and required row counts. A failing snapshot control and a failing restore control are kept. - Same-host drills (process restart, local backup-check-restore) are scripts for the operator to run on the operating host, which is user decision 2. Off-host recovery needs user decision 3. - The 2.16.6 -> 2.18.2 scheduler comparison is a source read only; the pin is held provisionally. - Offline proof: restic 0.19.1 on scratch directories, with two SQLite journals in WAL mode written to during the copy. Backup, a 1/1 check, restore and compare all pass, and both controls exit 1. A local_integration receipt and the decision record are added. Built by GPT-6.1 Sol through the packaged SDK worker (round t-pr4-r1); committed by the coordinator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the final hot-file commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (hot-file protocol: every hot-file edit in the last commit) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n bound, WAL-safe copy, restic grouping, operator docs, input freshness, behavioral tests p1: - session_day.py brackets each query from December 1 of the previous year to January 31 of the next. A weekday New Year's Day and an observed January 2 now skip as non-session days instead of failing with DateOutOfBounds. Real-calendar tests against exchange-calendars 4.13.2 cover 2026-01-01, 2027-01-01, 2034-01-02, 2026-11-26, 2026-11-27 and 2026-12-24. A date the calendar really does not cover is still a hard failure. - config.yaml.example restores the seven env_passthrough entries research-runtime uses. Dagu's allowlist behaviour is documented. p2: - The check rotation (subsets 1/7 to 7/7) must complete within seven days in UTC, measured from the invocation's start, with tests at 24 h + 60 s and across the 2026-11-01 fall-back. - Connection.backup uses pages=-1 inside a source read transaction, with --backup-timeout and --max-journal-bytes as documented parameters. A failed copy publishes no inventory. - restic backup uses --group-by host,tags with a stable host id and a journal-recovery tag. The matching forget grouping is documented, and nothing is deleted. - The README restores the operator content other documents cite, amended for start-all: the pinned checksum download, the 0600 config, the network-exposure warning, the auth observation, the 2.17.2 boundary, the systemctl steps, loginctl enable-linger and the WSL lifetime caveat. - Portable templates use plain env. Receipts name the rtk fallback the builder ran through. - The sanitizer resolves the scratch root, scrubs both path forms and refuses to publish while any absolute path remains. - SDK_ENV cites adoption/sdk/requirements-linux-x86_64-py313.lock:241. The reference to the unlanded PR-1 path is removed. - LEAN_EVENTS freshness: the research operator's local simulation job owns publishing it, atomically, after the actual close and by 16:25 America/New_York. Missing, stale or future-dated input is reported visibly and skips the run. - Behavioral tests cover the real calendar and native Dagu chains, the drill guards, the sanitizer, lock contention and stale locks, jitter and DST, and a content-altering restore control. Upstream-first rule: each glue piece keeps a pin-cited justification. The rotation lock is replaced by a nonblocking, process-owned flock (fcntl, CPython 3.12.3). Open: - The cron v3.0.1 Next() probe needs a Go toolchain, which is not available here. - Operating-host acceptance waits on user decision 2; off-host recovery waits on user decision 3. Built by GPT-6.1 Sol through the packaged SDK worker (rounds t-pr4-r2 and r2b); committed by the coordinator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…jects (pre-push and blind-export fixes) - upstream-glue-review-r2.json records each glue piece's outcome under `upstream_outcome` instead of `decision`. The values are glue-review outcomes, not arm labels. The blind-checkout test classifies every value under a label-like key, and "kept" and "replaced" were unclassified, so the pre-push hook refused the previous push. - Recorded restic JSON output is now stored as parsed objects or a sanitized projection (message_type, counts, snapshot id prefix, exit) instead of raw JSON-text lines. The receipts' checksums and byte counts were updated. The blind export's prose_exposure check calls Path.is_file() on long strings and raised ENAMETOOLONG on those lines. That fragility in tools/sota-convergence/export_isolation_check.py:527 is a separate foundation follow-up. The coordinator ran test_blind_checkout and test_us_equities_hosting outside the builder's sandbox: OK (2 skipped). Built by GPT-6.1 Sol through the packaged SDK worker (rounds t-pr4-r2c and r2d); committed by the coordinator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the final hot-file commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (hot-file protocol: every hot-file edit in the last commit) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ut, upstream systemd kill, WAL-only online backup, separate control groups Fixes from the Claude Opus read of r2: - **session_day.** Missing, stale or future session input now prints its token and exits 1. That fails the native Dagu run and blocks dependents, instead of producing a succeeded run. Holidays and before-close checks keep their skip behaviour. - **Restart drill fault delivery.** It uses `systemctl --user kill --kill-whom=main --signal=SIGKILL` (systemd/systemd@v255 man/systemctl.xml:541-549, 2377-2388) in place of a custom os.kill, keeping the identity guards. It also requires more than 150 s before the due slot, rechecks just before fault delivery, and reports a late restart separately. - **journal_recovery.** The online backup checks the native PRAGMA journal_mode and refuses non-WAL sources unless --allow-non-wal is given, citing sqlite/sqlite@version-3.53.1 src/backup.c:346-354, 383-389. Control snapshots use their own restic host and tag groups, equity-research-recovery-control and journal-recovery-control. - **Receipt.** The r2 receipt names the published main base 4c89741, and identifies the tested working tree by integration_source_sha256. - **Test helper.** The native Dagu test helper no longer raises KeyError when TMPDIR is unset. - **Docs.** Manual run, stop, history and replay are documented with native CLI citations. - **Evidence.** Fresh scratch evidence covers all six deployed DAG steps, native history filtering, the unavailable-input failures and the restic controls. Restic output is stored as parsed JSON. Each behavioural fix has a regression that failed (exit 1) against the r2 code and passes now. Still open, as recorded: - user decision 2: the operating host's restart, dispatch, reboot, missed-run and independent-alert acceptance; - user decision 3: the independent recovery destination, key recovery and a second-host restore; - cron DST Next is unexecuted, because no Go toolchain is available; - no Python 3.11 interpreter. Built by GPT-6.1 Sol through the packaged SDK worker (rounds t-pr4-r3 and r3b); committed by the coordinator. The coordinator ran tests.test_us_equities_hosting and tests.test_blind_checkout outside the sandbox, plus validate, evidence_manifest --check and git diff --check, all exit 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the final hot-file commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (hot-file protocol: every hot-file edit in the last commit) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-input test on the r3 contract, replay window, single-unit drill, portable helper - upstream-glue-review-r2.json and the r2 receipt's artifact entry are restored to their round-2 bytes. The round-3 additions move to upstream-glue-review-r3.json, which is pinned in the r3 receipt and the registry. - The native skip test uses before_close, the remaining exit-0 skip token, as a defence-in-depth precondition check. It no longer encodes the round-2 stale-input contract. - The README replay recipe states the valid window: the same New York date, after that session's close, with input newer than the close. It also requires `dagu status --run-id` to show every research node succeeded before a replay counts as recovery. - drill_process_restart.py refuses --unit values that match more than one unit. A regression covers glob patterns. - The native Dagu test helper invokes the SDK's own interpreter name instead of a hard-coded `python`. Built by GPT-6.1 Sol through the packaged SDK worker (round t-pr4-r4); committed by the coordinator. User decisions 2 and 3 (operating host and independent recovery) remain open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the final hot-file commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (hot-file protocol: every hot-file edit in the last commit) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d109af7a36
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…(CodeQL sensitive-name finding) CodeQL flagged the drill's printed cleanup boolean as a clear-text password: it was a field of a variable named for a password. The value printed was never secret; it said only whether the throwaway restic key file was deleted. The local variable is now unlock_file, the scratch file throwaway-unlock-file, and the report and printed field throwaway_unlock_file_deleted. Unchanged: secrets.token_urlsafe(48), mode 0600, delivery through restic --password-file, and deletion in finally. The README describes the unlock file and the renamed field. The six historical evidence files keep their recorded field names. Checks: tests.test_us_equities_hosting 53 OK. The offline restic 0.19.1 drill passed for two concurrently written SQLite journals (backup, subset 1/1 check, restore, comparison), both failing controls exited 1, and the run printed throwaway_unlock_file_deleted=true. Built by GPT-6.1 Sol through the packaged SDK worker (round t-pr4-r5); committed by the coordinator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the final hot-file commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (hot-file protocol: every hot-file edit in the last commit) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…egistry plus the owned rows) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the final hot-file commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (hot-file protocol: every hot-file edit in the last commit) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rse, journal durability, rotation deadline, 16:25 cutoff, status-byte hashing, round-3 receipt registration) Each Codex review thread on #736 was judged on the merits at e97a603. All six were valid; each fix has a regression that failed before it and passes now. - drill_process_restart.py, restart permission: substring matching accepted `run_dags: false` anywhere, including comments and inactive sections. The config is now parsed with upstream PyYAML (yaml/pyyaml 6.0.3, pinned in the README), and the drill requires the active permission value and Dagu's documented legacy override to be false. The drill fails closed with an install pointer when PyYAML is absent, and the regression skips there. - journal_recovery.py write_new: the final mode 0400 is applied before the file fsync, then the parent directory is fsynced and closed, so the external oracle survives power loss before recovery is accepted. - journal_recovery.py rotation: a completed rotation that ends with next == 1 now carries the previous cycle's deadline. An overdue rollover requires a successful full read before reset. - session_day.py: the session date's 16:25 America/New_York publication cutoff is enforced, and late input exits 1. The guard and replay documentation are updated. - drill_process_restart.py, status digest: the status bytes are captured once, then parsed and hashed from the same buffer, and polling continues when the entry disappears before the read. - manifests/evidence.json: the round-3 acceptance receipt is added to receipts[] with its original metadata. Historical proofs and receipts are unchanged. tests.test_us_equities_hosting: 59 OK with the locked calendar SDK and PyYAML; with plain python3, 59 run OK with 15 skipped. validate.py and evidence_manifest --check exit 0. Built by GPT-6.1 Sol through the packaged SDK worker (round t-pr4-r6); committed by the coordinator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the final hot-file commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (hot-file protocol: every hot-file edit in the last commit) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…YAML overlay instead of an SDK install, and replay limits stated This fixes the three P2s from the Claude Opus delta read of r6. All three came from r6's fixes. - Rotation: the default maximum rotation age is now 8 elapsed days, while keeping 7 subsets and a daily run. The fall-back week (16:30 New York, 7d 1:00) and +60 s/day jitter no longer trip the bound. Cycles that are truly overdue (8d + 1s, completed cycles included) are still refused. The new regressions failed against r6 and pass now. - PyYAML: the hash-locked SDK_ENV is left unchanged. The drill runs through an isolated `uv run --no-project --python "$SDK_ENV/bin/python" --with PyYAML==6.0.3` overlay, following the repository precedent in tools/runtime-worker-evidence/checks.json. The missing-parser pointer and the provenance are updated. - Replay: the README states plainly that a missing, stale or late input failure cannot be replayed for that session. A replay after a failure in a later step still works, with unchanged, on-time input on the same New York date. Checks: tests.test_us_equities_hosting 59 OK through the pinned PyYAML overlay, and 59 OK with 11 skipped under plain python3. validate.py and evidence_manifest --check exit 0. Built by GPT-6.1 Sol through the packaged SDK worker (round t-pr4-r7); committed by the coordinator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the final hot-file commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (hot-file protocol: every hot-file edit in the last commit) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Claude session native-agent-stack-5f: landing at head Observed main Required checks at this head: 7 pass . Unresolved review threads: 0. macOS (advisory). The user decided on 2026-10-05 that macOS checks are advisory. Ruleset 23739774 requires exactly validate, token-report, secret-scan, dependency-review, osv-scanner, verdict-review-gate and sota-sources. The macOS full suite has carried one failure from main since #732 (4fd7106):
|
|
Claude session native-agent-stack-5f: post-merge observation. Landed as |
…the new-WSL handbook (registry last) CI validate at ef48bca failed one test: tests.test_new_wsl_handbook reported "stale generated output: docs/new-wsl-handbook.md". The earlier merge changed handbook inputs, the merged catalog among them, without regenerating the handbook. This merge brings in main 2d849ba (#713, #726, #736 and the other afternoon landings). Only the registry conflicted; it is main's registry with this branch's rows re-registered. The handbook was regenerated with `python3 scripts/build_new_wsl_handbook.py --write`, and --check passes. Its receipt records the new outputs, the current generator and profile digests, and an eighth regenerations entry with the previous outputs. Checks: validate.py exit 0; validate_convergence --all-recorded exit 0; evidence_manifest --check passed (10,240 files); tests.test_architecture_pin_source, test_new_wsl_handbook, test_ecosystem_manifest and test_stack_lifecycle OK. uv.lock and pyproject.toml of the 2604 runtime are unchanged, so the 2604 qualification still carries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scope
What this PR changes: the trading lane's unattended-hosting and recovery recipe for the evidence DAG. It covers:
systemctl --user kill;Dagu stays at the held 2.16.6 (the trading-lane hold, docs/decisions/2026-09-25-workstation-sota-refresh.md:459). No service is installed, started or signalled by this PR.
Base commit:
4c897418fe35a030a1188ae447eaf31c893f8effLane:
lane:tradingOwned paths touched: blueprints/us-equities/hosting/** (recipe, DAG, guards, drill, recovery, evidence), tests/test_us_equities_hosting.py, docs/decisions/2026-10-05-trading-unattended-hosting-recovery.md, the evidence receipts r1-r3, manifests/evidence.json (last commit, per the hot-file protocol).
Open gates (user decisions; operation waits on them):
This PR establishes the recipe and scratch-root evidence only.
SOTA sources
dagu status --run-id, and the scheduler tick planner; cited in the decision record at pinned source lines.kill --kill-whom=main --signal) and :320, :1826-1833 (unit patterns).--stdin-from-command, host and tag grouping.Evidence-class table
Local commands run
Review
Built by GPT-6.1 Sol through the packaged SDK worker (rounds t-pr4-r1 to r4); committed by the coordinator.
Decision record
docs/decisions/2026-10-05-trading-unattended-hosting-recovery.md
Checklist
Recorded residuals (bounded review, 2026-10-05)
Claude Opus delta read r4 at
d109af7a3607returned ACCEPT with no p1. Under the bounded review rule these p2 items land as recorded residuals, each owed as a follow-up:docs/decisions/2026-10-05-trading-unattended-hosting-recovery.md:132-136 (vs blueprints/us-equities/hosting/evidence/upstream-glue-review-r3: Under 'Round 2 native reuse and repairs', the repointed link says the r3 review 'records pinned installed help, release notes and source' and then lists the restic 0.19.1 --stdin-from-command, SQLite 3.53.1 .backup/.dump/sqlite3_rsync, CPython backup/subprocess/flock, Dagu preconditions/retries/handlers and ecal checks.Recorded residuals (bounded review, 2026-10-05; Claude Opus 5.5 cross-family read (session 5f) of round t-pr4-r5 at 8892fbb at 8892fbb)
Claude Opus 5.5 cross-family read (session 5f) of round t-pr4-r5 at 8892fbb at
8892fbb4ab88returned ACCEPT with no p1. Under the bounded review rule these p2 items land as recorded residuals, each owed as a follow-up:blueprints/us-equities/hosting/README.md:383-384, compared with the evidence it cites at :397-399, :408 and :420-424: Non-blocking documentation gap introduced by r5.blueprints/us-equities/hosting/drill_local_recovery.py:73-82 and :195-202: Pre-existing, unchanged by r5, non-blocking residual for the 'deletion on every path' check.🤖 Generated with Claude Code