test(recovery): pin RC v2 saga semantics across v0.12 + v0.13 (Wave 4 P3-4) - #171
Merged
Ghenghis merged 2 commits intoMay 10, 2026
Conversation
… 7/7 gates green) Wave 1 of the v0.13 production promotion swarm cleared all 7 hard gates. This PR flips the resolver default from v0.12 (hermes-agent-fresh) to v0.13 (hermes-agent-v013-canary). v0.12 remains the opt-in fallback via HERMES_AGENT_CHECKOUT=G:/Github/hermes-agent-fresh. Hard gates (all PASS, evidence below) 1. MiniMax accepted=true — P1-1 (ev_84c2ba93c25d4017) 2. DeepSeek accepted=true — P1-1 (ev_6fa1536b411822db) 3. Canary runtime smoke passes — P1-2 (8/8 imports, 38-subcommand CLI, 10 MCP tools, redaction default-ON per upstream PR #21193) 4. v0.12 rollback path proven — P1-2 (resolver mid-process flip 4/4 reads correct; production HEAD bit-identical pre/post) 5. No secret leak — P1-1 + P1-4 (only env-var NAMES + host labels in evidence; auth_scheme is static "Authorization: Bearer <redacted>"; OWASP A09 review clean) 6. No v0.12 regression — P1-2 (production checkout bit-identical; git status --short empty post-run) 7. BLK-013 fixed by PR #159 — P1-4 (bounded CLI runner endpoint shipped with --network=none + --read-only + --cap-drop=ALL + sha256-only stderr; 6/6 new tests + 52/52 adjacent regression PASS) Bonus discovery (P1-3) - Upstream PR #21561 already merged the Windows TUI signal.SIGPIPE/ SIGHUP guard on 2026-05-08 (uses hasattr() pattern + SIGBREAK fallback). Canary at v2026.5.7 (498bfc7, 2026-05-07) predates this fix, so the TUI/PTY surface still requires advancing the canary checkout to >= e93bfc6c to gain native Windows TUI support. Core CLI + MCP + redaction + provider chat all work TODAY without that advance, which is why this promotion is safe. Patch - DEFAULT_AGENT_CHECKOUT now points to v0.13 (canary path). - New V012_FALLBACK_CHECKOUT constant exposes the v0.12 path for operators + tests that need explicit fallback semantics. - CANARY_AGENT_CHECKOUT retained as alias for back-compat. Tests added/updated (11/11 PASS) - test_default_is_v013_post_promotion: env unset → v0.13 - test_v012_fallback_via_env: env=hermes-agent-fresh → v0.12 - test_resolver_constants_are_correct: DEFAULT == CANARY == v0.13; V012_FALLBACK == hermes-agent-fresh - test_a5_per_call_env_flip: post-promotion semantic (v0.13 ↔ v0.12) - test_a4_agent_updates_repo_path_per_call: same semantic via agent_updates._repo_path() - test_v013_default_when_env_unset: route helper + resolver agree - (4 source-level pin tests retained from PR #155) Verification - py_compile: OK - Focused tests: 11/11 pass - Pre-push hook: passed Rollback - Operators set HERMES_AGENT_CHECKOUT=G:/Github/hermes-agent-fresh in the FastAPI process env to revert to v0.12 mid-process (no restart; per-call resolver from PR #155). - Or revert this PR (1 commit) to restore v0.12 as the file-level default. - Production v0.12 checkout at G:/Github/hermes-agent-fresh remains byte-identical (HEAD 73bf3ab1, v2026.4.30, status clean) — never modified by canary work. Companion PRs - PR #155 (8544bbc): per-call env-switch resolver (Wave A4 + A5) - PR #157 (b8277db): canary smoke results 7/1/0 - PR #158 (5a32fe0): production v0.13 + multi-version action plan - PR #159 (open): BLK-013 bounded task — provides hard gate 7 Hermes-Agent-Only Wave 1 swarm provenance - P1-1 Live provider probes: PASS (ev_84c2ba93c25d4017 + ev_6fa1536b411822db) - P1-2 Canary runtime + production-untouched: PASS (5/5 sub-checks) - P1-3 Upstream Windows guard: ALREADY MERGED upstream (PR #21561) - P1-4 BLK-013: PR #159 OPEN (6/6 + 52/52 regression PASS) - P1-5 Promotion: this PR References - Upstream v0.13.0 release: https://github.com/NousResearch/hermes-agent/releases/tag/v2026.5.7 - Upstream redaction default-ON: NousResearch/hermes-agent#21193 - Upstream Windows guard: NousResearch/hermes-agent#21561 - 12-Factor App rule III (config in env): https://12factor.net/config Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… P3-4) Mission: confirm Recovery Controller v2 freeze/thaw + saga compensation behave identically against the v0.12 fallback checkout (G:/Github/hermes-agent-fresh) and the v0.13 production default (G:/Github/hermes-agent-v013-canary). Per the action plan §P3-4 user requirement: "freeze/snapshot/MCP-lock semantics unchanged". This is a TEST-ONLY PR. recovery_controller.py source is NOT modified. New tests (8 logical, 9 collected — test 5 parameterized v012/v013): 1. test_freeze_run_creates_record_under_v013_default — env unset, freeze produces RecoveryRun with locked_files + pre_snapshot_ids + freeze_event_utc. 2. test_freeze_run_creates_record_under_v012_fallback — env=v0.12, identical record shape (full payload-key contract). 3. test_thaw_run_releases_locks_v013 — release exactly once, locked_files cleared. 4. test_thaw_run_releases_locks_v012 — parity with v0.13. 5. test_compensate_freeze_failure_under_both_versions — Postgres-style rollback: snapshot fails on file 3 of 3; release_mcp_files called once for the full set, mark_recovery_outcome retry_failed written, run transitions to RETRY_FAILED. Same on v0.12 + v0.13 (parametrized). 6. test_freeze_run_proof_event_records_active_version — auto-skipped until P2-6 (proof event version tagging) lands; detection scans RecoveryRun.__dataclass_fields__ for version_label / version_tag / active_checkout / hermes_agent_version. 7. test_recovery_run_dataclass_field_set_unchanged — pin: field set is identical across env-driven importlib.reload of recovery_controller. 8. test_freeze_thaw_round_trip_identical_across_versions — saga step ORDER (record_step_failure → lock_mcp_files → snapshot_file ×N → release_mcp_files) is byte-identical across versions. Sources cited (2-source minimum per brief): * Saga pattern (Garcia-Molina + Salem 1987) — Temporal docs: https://temporal.io/blog/saga-pattern-made-easy * PostgreSQL transaction-rollback semantics: https://www.postgresql.org/docs/current/tutorial-transactions.html Constraints honored: * recovery_controller.py source NOT modified. * No filesystem requirement on actual v0.12/v0.13 directories — all v1 collaborators stubbed (lock_mcp_files, release_mcp_files, snapshot_file, record_step_failure, mark_recovery_outcome, append_mcp_evidence). * RC v2 commits 3-5 (autonomous mode, propose/review/apply, UI panel) out of scope. Verification: * New file alone: pytest 8 passed, 1 skipped (test 6 P2-6 fence). * Combined (this file + test_v012_fallback_regression_pin.py + test_v013_default_regression_pin.py + test_agent_checkout_resolver.py + test_recovery_controller_freeze.py): 41 passed, 1 skipped, 1 deselected (BLK-013 route test pre-existing failure unrelated to RC v2 — relies on PR #159 not in this branch's ancestry). Hermes evidence chain: PASS Task ID: P3-4-RC-CROSS-VERSION-2026-05-09 hermes_run_gate: pytest 8/8 + combined 41+/41+ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Ghenghis
changed the base branch from
claude/v013-promotion-flip
to
feat/hermes3d-7-complete-gui-repo-wiring
May 10, 2026 01:40
Ghenghis
merged commit May 10, 2026
8575f43
into
feat/hermes3d-7-complete-gui-repo-wiring
1 check passed
Ghenghis
added a commit
that referenced
this pull request
May 10, 2026
Exercises the Hermes Agent Recovery Loop end-to-end against the v0.13 production-default stack. Confirms PR #149 (RC v2 commits 1+2: scaffold + freeze_run + thaw_run + saga compensation), PR #154 (list_active_runs route), and PR #171 (cross-version saga semantics pin) hold under a full failure -> classify -> freeze -> snapshot -> fix -> review -> apply -> thaw -> resume sequence. Drill is a pytest.mark.integration test pair: 1. test_recovery_loop_drill_end_to_end (happy path): - Synthetic runner_fail failure on cli_runner.exec - Real start_recovery, freeze_run, thaw_run (saga step 2 + compensation) - MOCKED MiniMax/DeepSeek/apply (per feedback_no_paid_services rule; real wiring lands with RC v2 commits 3-5) - Asserts canonical 8-phase ordering matches DRILL_PHASES - Asserts no _SECRET_PATTERNS regex matches in any captured payload (OpenAI/Anthropic, AWS, GitHub PAT, Slack, PEM) - Asserts saga ordering invariants (lock-before-snapshot, review-before-apply, apply-before-thaw) 2. test_recovery_loop_drill_hard_escalate_short_circuits_freeze: - provider_auth (hard-escalate class) refuses freeze - No locks acquired; no fix_proposal/review/apply events fire Drill rule honored: source NOT modified (recovery_controller.py untouched). 8-phase event sequence captured + redacted in the handoff doc. Sources cited: - Garcia-Molina & Salem (1987) "Sagas," ACM SIGMOD Record 16(3):249-259 - User MEMORY feedback_recovery_loop (STRICT 2026-05-09): canonical failure -> classify -> freeze -> snapshot -> MiniMax fix -> DeepSeek review -> apply -> re-run -> resume sequence Test results: - 04_testing/pytest/integration/test_recovery_loop_drill.py: 2/2 PASS - All recovery suites (freeze + ledger-locking + runs-route + drill): 21/21 PASS Hermes evidence chain: PASS Task ID: w5-5-recovery-loop-e2e-drill hermes_run_gate: pytest 2/2 + 21/21 cross-suite Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 10, 2026
Ghenghis
added a commit
that referenced
this pull request
May 10, 2026
Exercises the Hermes Agent Recovery Loop end-to-end against the v0.13 production-default stack. Confirms PR #149 (RC v2 commits 1+2: scaffold + freeze_run + thaw_run + saga compensation), PR #154 (list_active_runs route), and PR #171 (cross-version saga semantics pin) hold under a full failure -> classify -> freeze -> snapshot -> fix -> review -> apply -> thaw -> resume sequence. Drill is a pytest.mark.integration test pair: 1. test_recovery_loop_drill_end_to_end (happy path): - Synthetic runner_fail failure on cli_runner.exec - Real start_recovery, freeze_run, thaw_run (saga step 2 + compensation) - MOCKED MiniMax/DeepSeek/apply (per feedback_no_paid_services rule; real wiring lands with RC v2 commits 3-5) - Asserts canonical 8-phase ordering matches DRILL_PHASES - Asserts no _SECRET_PATTERNS regex matches in any captured payload (OpenAI/Anthropic, AWS, GitHub PAT, Slack, PEM) - Asserts saga ordering invariants (lock-before-snapshot, review-before-apply, apply-before-thaw) 2. test_recovery_loop_drill_hard_escalate_short_circuits_freeze: - provider_auth (hard-escalate class) refuses freeze - No locks acquired; no fix_proposal/review/apply events fire Drill rule honored: source NOT modified (recovery_controller.py untouched). 8-phase event sequence captured + redacted in the handoff doc. Sources cited: - Garcia-Molina & Salem (1987) "Sagas," ACM SIGMOD Record 16(3):249-259 - User MEMORY feedback_recovery_loop (STRICT 2026-05-09): canonical failure -> classify -> freeze -> snapshot -> MiniMax fix -> DeepSeek review -> apply -> re-run -> resume sequence Test results: - 04_testing/pytest/integration/test_recovery_loop_drill.py: 2/2 PASS - All recovery suites (freeze + ledger-locking + runs-route + drill): 21/21 PASS Hermes evidence chain: PASS Task ID: w5-5-recovery-loop-e2e-drill hermes_run_gate: pytest 2/2 + 21/21 cross-suite Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wave 4 P3-4: cross-version regression suite for Recovery Controller v2 (RC v2)
freeze/thaw + saga compensation. Confirms the saga step ordering and MCP-lock
semantics behave identically under v0.12 fallback (
G:/Github/hermes-agent-fresh)and v0.13 production default (
G:/Github/hermes-agent-v013-canary).Per the action plan §P3-4 user requirement: "freeze/snapshot/MCP-lock semantics unchanged."
This is a TEST-ONLY PR —
recovery_controller.pysource is not modified.What's covered (8 logical tests, 9 collected — test 5 parametrized v012/v013)
test_freeze_run_creates_record_under_v013_default— env unset, freezeproduces a
RecoveryRunwithlocked_files+pre_snapshot_ids+freeze_event_utc. Resolver sanity asserts the v0.13 path.test_freeze_run_creates_record_under_v012_fallback— env=v0.12, identicalrecord shape; full payload-key contract pin.
test_thaw_run_releases_locks_v013— release exactly once, locked_filescleared.
test_thaw_run_releases_locks_v012— parity with v0.13.test_compensate_freeze_failure_under_both_versions— Postgres-stylerollback: snapshot fails on file 3 of 3 → 2 acquired locks released, partial
snapshot path NOT applied, mark_recovery_outcome retry_failed written, run
transitions to RETRY_FAILED. Same on v0.12 + v0.13 (parametrized).
test_freeze_run_proof_event_records_active_version— conditionallyskipped until P2-6 (proof event version tagging) lands. Detection scans
RecoveryRun.__dataclass_fields__forversion_label/version_tag/active_checkout/hermes_agent_version. Will auto-flip to a realassertion once the version field is added.
test_recovery_run_dataclass_field_set_unchanged— pin: field set isidentical across env-driven
importlib.reloadofrecovery_controller.test_freeze_thaw_round_trip_identical_across_versions— saga step ORDER(
record_step_failure→lock_mcp_files→snapshot_file×N →release_mcp_files) is byte-identical across versions.Sources cited (2-source minimum per brief)
https://temporal.io/blog/saga-pattern-made-easy
https://www.postgresql.org/docs/current/tutorial-transactions.html
Constraints honored
recovery_controller.pysource NOT modifiedcollaborators stubbed (
lock_mcp_files,release_mcp_files,snapshot_file,record_step_failure,mark_recovery_outcome,append_mcp_evidence)1bd8e5bdb2d013a30c1d1327,owner
claude-lead-p3-4-rc-smoke, task_idP3-4-RC-CROSS-VERSION-2026-05-09)Hermes evidence chain: PASS
P3-4-RC-CROSS-VERSION-2026-05-09claude-lead-p3-4-rc-smokeTest plan
pytest 04_testing/pytest/unit/test_recovery_controller_cross_version.py -v→ 8 passed, 1 skipped (test 6, P2-6 fence)
test_recovery_controller_cross_version.py + test_v012_fallback_regression_pin.py + test_v013_default_regression_pin.py + test_agent_checkout_resolver.py + test_recovery_controller_freeze.py→ 41 passed, 1 skipped, 1 deselected
(BLK-013 route test pre-existing failure unrelated to RC v2 — relies on
PR feat(BLK-013): real bounded CLI runner task on hardened docker sandbox #159 not in this branch's ancestry; will re-pass once PR test(v0.12-pin): regression-pin 8 fallback surfaces (Wave 2 P2-3) #163 / test(v0.13): regression-pin v0.13 production default (Wave 2 P2-4) #165
stack rebases onto a base that includes feat(BLK-013): real bounded CLI runner task on hardened docker sandbox #159)
🤖 Generated with Claude Code