test(hermes-agent): post-promotion smoke runner (Wave 3 P1-7)

[Ghenghis]

Summary

Wave 3 P1-7. Re-runs all 8 smokes from PR #157 against the now-promoted v0.13 default. PR #160 squash 3158a4e flipped services/agent_checkout.py:DEFAULT_AGENT_CHECKOUT to Path("G:/Github/hermes-agent-v013-canary"); this PR confirms zero regression and upgrades Smoke 7 (BLK-013 bounded task) from N/A to PASS using the endpoint shipped in PR #159 with monkeypatched subprocess.run.

Result: 8 PASS / 0 FAIL / 0 N/A. Net change vs PR #157: Smoke 7 N/A → PASS. Every smoke that PR #157 reported PASS is still PASS.

Pre-flight pins (both PASS)

Check	Required	Actual
services/agent_checkout.py:DEFAULT_AGENT_CHECKOUT	Path("G:/Github/hermes-agent-v013-canary")	matches
git -C G:/Github/hermes-agent-fresh rev-parse HEAD	73bf3ab1b22314ed9dfecbb59242c03742fe72af (v2026.4.30)	matches; git status --short empty

Smoke results (vs PR #157 baseline)

#	Smoke	PR #157	P1-7	Delta
1	Hermes Agent imports (8 packages)	PASS	PASS	unchanged
2	MCP tools — 10 @mcp.tool() at lines 471/528/561/618/670/699/733/769/823/839	PASS	PASS (exact line match)	unchanged
3	MiniMax build_probe_request — Authorization: Bearer <redacted>	PASS	PASS (key value never leaks)	unchanged
4	DeepSeek graceful RuntimeError (PR #145/#148 fix)	PASS	PASS (env-var NAME absent from message)	unchanged
5	OpenCode preflight	PASS	PASS (detected=True, version_status=pass)	unchanged
6	OpenHands preflight	PASS	PASS-with-finding-F1 (binary detected; --version probe occasionally exceeds 8s budget — non-blocking)	unchanged criterion
7	BLK-013 bounded task	N/A (endpoint not shipped)	PASS via TestClient + mocked subprocess.run; hardened-docker argv pinned; no secret leak; stderr only as sha256	upgraded N/A → PASS
8	Rollback to v0.12 (env-flip drill)	PASS	PASS (5/5 mid-process flips correct via PR #155 per-call resolver)	unchanged

What this PR adds

• 04_testing/pytest/unit/test_v013_post_promotion_smoke.py — 11 test cases (2 pre-flight pins + 8 smokes + 1 aggregate). Runs in ~20 s, no live HTTP, no docker spawn.
• 03_implementation/docs/handoffs/HERMES_AGENT_V013_POST_PROMOTION_SMOKE_2026-05-09.md — full smoke results, findings, source links.

Pytest output: 11 passed in 19.67s. Pre-push hook also ran 39 unit tests and printed pre-push: branch 'claude/v013-post-promotion-smoke' OK.

Smoke 7 detail — what was actually exercised

The bounded-task endpoint is exercised through the production FastAPI router via TestClient. code_history.subprocess.run is monkey-patched to capture argv. Assertions cover:

• Endpoint returns 200 with status="ok", accepted=True, network_mode="none", timeout_s=30.
• Hardened-docker argv: --network=none, --read-only, --memory=512m, --cpus=1, --pids-limit=128, --cap-drop=ALL, --security-opt=no-new-privileges, --tmpfs /tmp:noexec,nosuid,....
• Bounded prompt reaches the inner CLI (-t <prompt> carries "JSON array of names" and "max 50 items").
• No secret host paths mounted (g:/private, /.aws, ${home} absent from joined argv).
• No provider env vars passed to docker (OPENAI_API_KEY, ANTHROPIC_API_KEY, MINIMAX_API_KEY, DEEPSEEK_API_KEY, HUGGINGFACE_TOKEN absent; -e absent entirely).
• Stderr never surfaces in the response body — only stderr_sha256 (64 hex chars).
• append_mcp_evidence called with kind="code_cli_runner_bounded_task".

Findings (non-blocking)

F1 — OpenHands --version probe occasionally exceeds 8 s timeout on slow Windows hosts; manual invocation completes in ~8.6 s and prints OpenHands CLI 1.16.0. Detection (detected=True, the PR #157 PASS criterion) is unchanged. Recommended follow-up (separate PR): raise _cli_runner_status version-probe timeout from 8 s to 15 s, OR memoize the probe within a process. Out of scope for this PR.

Constraints honored

• Read-only on canary code (no edits to G:/Github/hermes-agent-v013-canary/ or G:/Github/hermes-agent-fresh/).
• No secret values printed or stored.
• Live HTTP probes intentionally deferred (PR docs(hermes-agent): v0.13 canary smoke results — 7 PASS / 1 N/A / 0 FAIL #157 already banked accepted=true for both providers; integration path verified by mocking subprocess + inspecting argv).
• MCP locks held + chained evidence (ev_4003d541281ee381) recorded for task P1-7-V013-POST-PROMOTION-SMOKE. Locks released cleanly post-commit.

Online research

Primary — Argo Rollouts blue-green smoke gate (argoproj.github.io/rollouts): re-run smokes against the new active version after promotion, before declaring stable. Cross-comparison — PR #157 baseline 7 PASS / 1 N/A / 0 FAIL was the explicit motivator for PR #160; this PR closes the loop. Supporting — Kubernetes post-flight pattern (testkube.io/glossary/post-flight-testing) and 12-Factor App rule III (12factor.net/config).

Hermes evidence chain: PASS

• Task ID: P1-7-V013-POST-PROMOTION-SMOKE
• hermes_run_gate: locks acquired + chained evidence + locks released
• Pre-flight evidence: ev_4003d541281ee381 (entry_hash 9869a3a7269d8b6a8fe30202737dcd801207d9163914c0d1a1410aaef09b0ac0, prev_hash 67f0dbfbd79058f5a464b44cde74d7c5751eceacc723e6a4625d3991f9505da0)

Test plan

• Pre-flight: DEFAULT_AGENT_CHECKOUT is v0.13 canary path
• Pre-flight: production v0.12 HEAD unchanged at 73bf3ab1
• Smoke 1: 8/8 imports
• Smoke 2: 10/10 MCP tools at exact baseline lines
• Smoke 3: MiniMax config-OK, no key leak
• Smoke 4: DeepSeek RuntimeError, no name leak
• Smoke 5: OpenCode detected=True
• Smoke 6: OpenHands detected=True (with F1 finding)
• Smoke 7: BLK-013 bounded task via TestClient — PASS
• Smoke 8: Rollback to v0.12 via env flip — 5/5 transitions correct
• Aggregate pytest: 11 passed in 19.67s
• No regression vs PR docs(hermes-agent): v0.13 canary smoke results — 7 PASS / 1 N/A / 0 FAIL #157 baseline

Handoff signal

Post-promotion stable. v0.13 default is runtime-verified against the same eight-smoke contract that motivated the promotion. v0.12 ripcord remains operative mid-process. Wave 3 P1-7 closes clean.

🤖 Generated with Claude Code

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 94ded8cf-8517-4795-9f71-6fe46ce20bbc

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/v013-post-promotion-smoke

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}