ci(P3-3): multi-version regression gate (Wave 2) by Ghenghis · Pull Request #172 · Ghenghis/Hermes3D

Ghenghis · 2026-05-10T01:07:30Z

Summary

Adds .github/workflows/hermes-agent-versions.yml — a multi-OS GHA workflow that runs the per-call resolver suite (PR feat(hermes-agent): canary env-switch resolver (Wave A4 + A5 combined) #155) plus BOTH regression-pin suites (PR test(v0.12-pin): regression-pin 8 fallback surfaces (Wave 2 P2-3) #163 v0.12 fallback + PR test(v0.13): regression-pin v0.13 production default (Wave 2 P2-4) #165 v0.13 default) on ubuntu-24.04 + windows-latest with Python 3.11 whenever any of the 10 gated contract paths change.
Adds 03_implementation/docs/handoffs/HERMES_AGENT_MULTI_VERSION_CI_2026-05-09.md — handoff doc covering rationale, what is and is NOT tested, when to add OS / Python rows, debug runbook, sources cited.

Why

Wave 1 (PR #160 squash 3158a4e) promoted Hermes Agent v0.13 to production default; v0.12 is now the operator rollback (HERMES_AGENT_CHECKOUT=G:/Github/hermes-agent-fresh). A future commit could silently break either side. This workflow is the Wave 2 gate that fails CI if either version regresses.

windows-latest is the production-host parity row (operator desktops run Windows; Tenacity Release ships a Windows TUI guard). ubuntu-24.04 is the server-host parity row.

Design notes

fail-fast: false so one OS does not mask the other (GHA matrix docs).
No pip install -e . — pin tests are env-mocked + path-mocked; verified by reading both pin files (no .exists() on the v0.12/v0.13 paths, only Path equality).
HERMES_AGENT_CHECKOUT is left UNSET in the env block — resolver defaults to v0.13, and the v0.12 suite uses monkeypatch.setenv internally.
shell: bash on the test steps so backslash line continuation works on both runners (Git Bash on Windows).
Path filters cover the 6 source files + 3 test files + this workflow itself (10 paths) so unrelated PRs do not burn matrix time.

What this does NOT test

Live HTTP probes against MiniMax / DeepSeek (manual operator drill).
Real subprocess execution of git/npm/pytest gates (the runners cannot host those repos; surface 5 stubs subprocess.run).
Filesystem existence of G:/Github/hermes-agent-fresh or G:/Github/hermes-agent-v013-canary (only Path equality).

Sources cited

GitHub Actions matrix strategy: https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs
pytest skip / xfail multi-OS: https://docs.pytest.org/en/stable/how-to/skipping.html
12-Factor App config rule III: https://12factor.net/config

Test plan

yamllint on hermes-agent-versions.yml: 2 cosmetic warnings only (document-start + truthy on:), 0 errors. Matches existing ci.yml warning profile.
GHA-shape validator (custom python check): PASS — matrix [ubuntu-24.04, windows-latest], python ['3.11'], fail-fast: false, permissions.contents: read, concurrency.cancel-in-progress: true, 10 path filters, 5 steps; env contains PYTHONPATH but explicitly NOT HERMES_AGENT_CHECKOUT.
Local pytest re-run on the 3 gated suites: 27 passed in 2.40s on Windows + Python 3.14 (CI runs on Python 3.11).
First-run feedback after merge: trigger via Actions → hermes-agent-versions → Run workflow → branch claude/v013-promotion-flip. Expected URL: https://github.com/Ghenghis/Hermes3D/actions/workflows/hermes-agent-versions.yml.

Persistence rule (windows-latest first-run)

If the first GHA run red-bars on Windows-latest (normal failure mode for first-of-kind multi-OS workflows), debug per section 5 of the handoff doc and push a follow-up NEW commit (never --amend). The workflow re-runs automatically because the workflow file is in its own gated paths list.

Hermes evidence chain: PASS
Task ID: P3-3-MULTI-VERSION-CI-2026-05-09
hermes_run_gate: yamllint clean + local pytest re-run on same suites

Co-Authored-By: Claude Opus 4.7 (1M context)

… P2-4) Adds 8 focused unit tests that lock down the post-PR #160 (squash 3158a4e) production v0.13 default behavior so future commits cannot silently regress Wave 1 gate 3. Surfaces pinned (one test each): 1. agent_updates._repo_path() -> v0.13 path when env unset. 2. module_runtime.BUILTIN_RUNTIME_PROBES['hermes_agent']['path'] resolves to v0.13 when env unset at module-import (importlib.reload). 3. code_history.SOURCE_REPOS[id='nous_hermes_agent'].local_path resolves to v0.13 when env unset at module-import (importlib.reload). 4. db.load_modules.SOURCE_OVERRIDES['hermes_agent']['local_path'] resolves to v0.13 when env unset at module-import (importlib.reload). 5. agent_updates._run_update_checks runs against the v0.13 checkout without crashing (subprocess mocked; pytest gate is NEVER spawned). 6. MiniMax + DeepSeek probe requests build correctly with v0.13 active (no live HTTP; bearer scheme + Accept header asserted). 7. CLI runner detection works (OpenCode 1.4.3-hermes3d + OpenHands CLI 1.16.0 per PR #157 baseline) under v0.13 default. 8. BLK-013 POST /api/code-operator/cli-runners/run-bounded-task is registered (per PR #159) and the docker argv carries --network=none (subprocess.run mocked; no real container spawn). Sister coverage to PR P2-3 (test_v012_fallback_regression_pin.py): same 8 surfaces, opposite env state. Neither file duplicates the existing test_agent_checkout_resolver.py — those tests cover env-flip semantics; this file pins the resolved path values across all 4 import-time sites and the 4 runtime sites that depend on them. Refs: - PR #160 squash 3158a4e (Wave 1 promotion, 7/7 gates green) - PR #159 0f42dda (BLK-013 hardened bounded-task runner) - PR #157 b8277db (canary smoke baseline; CLI runner versions) - pip-tools regression-pin pattern (pin a default in lock file + re-assert on every compile run); 12-Factor App config rule III. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Pin v0.12 (G:/Github/hermes-agent-fresh, v2026.4.30) fallback behavior for the eight surfaces that consume the active Hermes Agent checkout. After PR #160 promoted v0.13 to default, v0.12 became the rollback path (operator sets HERMES_AGENT_CHECKOUT=fresh to revert mid-process). If a future v0.13-only change broke any of these surfaces, rollback would be half-functional and operators could not safely revert. Surfaces pinned (one test each): 1. agent_updates._repo_path() — per-call resolver 2. module_runtime.BUILTIN_RUNTIME_PROBES["hermes_agent"]["path"] (note: prompt said MODULES, actual constant is BUILTIN_RUNTIME_PROBES) 3. code_history.SOURCE_REPOS[id="nous_hermes_agent"].local_path 4. db/load_modules.SOURCE_OVERRIDES["hermes_agent"]["local_path"] 5. _run_update_checks does not crash under v0.12 (canary-venv-free) 6. Provider config (MiniMax + DeepSeek) — config layer only, no HTTP 7. CLI runner detection (OpenCode + OpenHands) — env-only, no PATH dep 8. agent_config.last_run + proof_events writes round-trip cleanly, no v0.13 cross-version contamination at write time Surfaces 2-4 are module-level constants captured at import; the test mutates env then importlib.reload(module) so the literals re-resolve. Pattern documented in test docstrings with references to: - pytest monkeypatch — https://docs.pytest.org/en/stable/how-to/monkeypatch.html - importlib.reload — https://docs.python.org/3/library/importlib.html - nox per-version isolation — https://nox.thea.codes/en/stable/tutorial.html 8/8 pass; 11 existing resolver tests still pass (19/19 combined). No production source modified — read-only on src/, test-only addition.

Add .github/workflows/hermes-agent-versions.yml — runs the per-call resolver suite (PR #155) plus both regression-pin suites (PR #163 v0.12 fallback + PR #165 v0.13 default) on ubuntu-24.04 + windows-latest with Python 3.11 whenever any contract surface changes. Why - Wave 1 (PR #160 squash 3158a4e) promoted Hermes Agent v0.13 to the production default; v0.12 is now the operator rollback (HERMES_AGENT_CHECKOUT=G:/Github/hermes-agent-fresh). - A future commit could silently break either side. This workflow is the Wave 2 gate that fails CI if either version regresses. - Windows-latest is required because production runs on Windows desktops (Tenacity Release ships a Windows TUI guard); Ubuntu-24.04 is the server-host parity row. Design notes - fail-fast: false so one OS does not mask the other. - No `pip install -e .` — the regression suites are env-mocked + path- mocked; verified by reading both pin files end to end (no .exists() on G:/Github/hermes-agent-fresh or G:/Github/hermes-agent-v013-canary, only Path equality). - HERMES_AGENT_CHECKOUT is left UNSET in the env block — the resolver defaults to v0.13, and the v0.12 suite uses monkeypatch.setenv internally (per pytest's recommended pattern). - shell: bash on the test steps so backslash line continuation works on both runners. Local re-run gate: 27 passed in 2.40s (Windows + Python 3.14, will run on Python 3.11 in CI). yamllint: 2 cosmetic warnings only, identical profile to existing ci.yml. Hermes evidence chain: PASS Task ID: P3-3-MULTI-VERSION-CI-2026-05-09 hermes_run_gate: yamllint clean + local pytest re-run on same suites Sources cited (read for this PR): - GitHub Actions matrix strategy: https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs - pytest skip / xfail multi-OS guidance: https://docs.pytest.org/en/stable/how-to/skipping.html - 12-Factor App config rule III: https://12factor.net/config Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

gemini-code-assist · 2026-05-10T01:07:34Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

coderabbitai · 2026-05-10T01:07:37Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8ef3ba03-0668-4b47-8126-045cd2f02719

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch claude/p3-3-multi-version-gha-proof

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Ghenghis and others added 3 commits May 9, 2026 17:46

Ghenghis changed the base branch from claude/v013-promotion-flip to feat/hermes3d-7-complete-gui-repo-wiring May 10, 2026 01:40

Ghenghis merged commit 12dc53e into feat/hermes3d-7-complete-gui-repo-wiring May 10, 2026
1 check passed

Ghenghis deleted the claude/p3-3-multi-version-gha-proof branch May 10, 2026 01:41

Ghenghis mentioned this pull request May 10, 2026

docs(audit): fix 20 v0.13 promotion doc-audit findings (W5-8 follow-up) #176

Merged

5 tasks

Navigation Menu

ci(P3-3): multi-version regression gate (Wave 2) - #172

ci(P3-3): multi-version regression gate (Wave 2)#172
Ghenghis merged 3 commits into
feat/hermes3d-7-complete-gui-repo-wiringfrom
claude/p3-3-multi-version-gha-proof

Ghenghis commented May 10, 2026

Uh oh!

gemini-code-assist Bot commented May 10, 2026

Uh oh!

coderabbitai Bot commented May 10, 2026

Review skipped

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Ghenghis commented May 10, 2026

Summary

Why

Design notes

What this does NOT test

Sources cited

Test plan

Persistence rule (windows-latest first-run)

Uh oh!

gemini-code-assist Bot commented May 10, 2026

Uh oh!

coderabbitai Bot commented May 10, 2026

Review skipped

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant