fix(ci): repair red main — busy-mode test + missing checkout in skills-index workflows - #83974
Merged
Merged
Conversation
…s-index workflows Three separate reds on main. Two are fixed here; the third needs no code. 1. tests/gateway/test_multiplex_busy_input_mode.py (blocks every merge) Fails "Python tests / Run tests slice 5/12" and therefore "All required checks pass". Semantic merge conflict between two PRs merged ~1h apart: a31be48 fix(gateway): respect routed profile busy modes (added the test) c8f235a feat(gateway): allow selective multiplex profile serving (added the gate) c8f235a taught _profile_name_for_source to reject a route whose target profile is not in the served set (profiles_to_serve). Each PR was green on its own base; neither ran against the other's merge result. The test asserts a route to profile "research" resolves to that profile's busy mode, but never patches profiles_to_serve — so it reads the runner's REAL on-disk profiles. "research" is not among them, the route is rejected before the busy-mode snapshot is consulted, and the assertion gets the gateway default: WARNING gateway.run: Rejecting profile route 'research-chat': target profile 'research' is not served AssertionError: assert 'interrupt' == 'steer' Patch profiles_to_serve for the assertion — the same seam every sibling test in tests/gateway/test_profile_resolution.py already patches (test_route_inside_allowlist_resolves, test_route_outside_allowlist_rejects). This also removes an ambient-state dependency: the test previously passed or failed based on which profiles happened to exist on the machine running it. Verified passing under an empty HERMES_HOME. Test-only. The serving gate from c8f235a is correct and left intact. 2. Skills-index workflows: local action used without actions/checkout check-freshness has failed on all 12 of its last 12 scheduled runs: ##[error]Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '.../.github/actions/get-app-token'. Did you forget to run actions/checkout before running your local action? ./.github/actions/get-app-token is a LOCAL composite action and cannot resolve without the repo on disk. skills-index-freshness.yml had no checkout step at all. The step is gated on `status != 'ok'`, so the watchdog broke exactly when it was supposed to file its issue — the live index is currently 521.4h stale (limit 26h) and nobody was told. An audit of all workflows for this bug class found one more instance: skills-index.yml's `trigger-deploy` job, which re-triggers the docs deploy so a refreshed index reaches the live site. Its sibling `build-index` job checks out; this one did not. That is plausibly why the index went stale in the first place. Both are fixed; the audit now reports zero remaining jobs that use a local action without a prior checkout. Pinned to the same actions/checkout SHA used by the other 35 call sites. 3. "Publish inline E2E evidence" — no fix needed Failed once at 13:33Z on a transient TLS error reaching api.github.com ("certificate is not valid for any names") while installing a gh extension. The last 25 runs of that workflow are 25/25 success. Infra blip, not a code defect.
OutThisLife
approved these changes
Aug 11, 2026
kshitijk4poor
enabled auto-merge (rebase)
August 11, 2026 15:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Main is red. Three distinct failures; two are fixed here, the third needs no code.
1.
test_multiplex_busy_input_mode.py— blocks every mergetest_profile_route_and_nonmultiplexed_resolution_preserve_boundariesfails onmain, taking down
Python tests / Run tests slice 5/12and thereforeAll required checks pass. Every PR is currently blocked behind this.Cause
A semantic merge conflict between two PRs (both by @tmchow) that merged
57 minutes apart:
a31be4809829746d→c8f235a1#83400 taught
_profile_name_for_sourceto reject a route whose target profileis not in the served set (
profiles_to_serve). #83550 added a test that routesto profile
researchwithout ever declaring it served. Each is correct alone;together the gate rejects the route the test depends on.
Why CI didn't catch it: #83400 was never re-tested after #83550 landed.
0eb96d7aat 2026-08-10 19:23:59Z —Run tests slice 5/12✅,All required checks pass✅.The test literally did not exist on the commit CI approved:
So this is not a flake and not a bad review — it's a stale-base merge. The
first execution of gate + test together happened on main, in the merge result
that no CI run ever covered.
The mechanism, concretely: the test never patches
profiles_to_serve, so itreads the runner's real on-disk profiles.
researchisn't among them, theroute is rejected before the busy-mode snapshot is ever consulted, and the
assertion receives the gateway default instead:
Fix: patch
profiles_to_servearound the assertion — the same seam everysibling test in
tests/gateway/test_profile_resolution.pyalready patches(
test_route_inside_allowlist_resolves,test_route_outside_allowlist_rejects).This also removes an ambient-state dependency. The test previously passed or
failed depending on which profiles happened to exist on the machine running it;
it now passes under an empty
HERMES_HOME.Test-only — no production behavior changes. The serving gate from #83400 is
correct and is left fully intact.
2. Skills-index workflows: local action used without
actions/checkoutcheck-freshnesshas failed all 12 of its last 12 scheduled runs:./.github/actions/get-app-tokenis a local composite action and can't resolvewithout the repo on disk.
skills-index-freshness.ymlhad no checkout step atall.
The real-world impact is worse than one red check: that step is gated on
status != 'ok', so the watchdog only broke at the exact moment it wassupposed to work. The live skills index is right now 521.4h stale against
a 26h limit, and the alarm designed to tell us never fired.
Auditing every workflow for this bug class turned up one more instance —
skills-index.yml'strigger-deployjob, which re-triggers the docs deploy soa refreshed index actually reaches the live site. Its sibling
build-indexjobchecks out; this one didn't. That is plausibly why the index went stale in the
first place: the rebuild ran, but the deploy was never kicked.
Both are fixed. The audit now reports zero remaining jobs that invoke a local
action without a prior checkout. Pinned to the same
actions/checkoutSHA usedby the other 35 call sites in the repo.
3.
Publish inline E2E evidence— no fix neededFailed once at 13:33Z on a transient TLS error reaching
api.github.com(
certificate is not valid for any names) while installing aghextension.The last 25 runs of that workflow are 25/25 success. Infra blip, not a defect —
flagging it so it isn't mistaken for a real regression.
Verification
tests/gateway/test_multiplex_busy_input_mode.py— 15/15 pass after the fix.HERMES_HOMEto confirm the ambient-state dependency is gone.tests/gateway/test_multiplex_busy_input_mode.py+test_profile_resolution.py+tests/hermes_cli/test_profiles.py— 76 passed, 2 skipped.ruffclean.Note:
tests/gateway/test_api_server.py::TestHealthDetailedEndpoint::test_health_detailed_returns_okfails in my local environment (
assert 'degraded' == 'ok'), but it failsidentically on unmodified main and its CI slice is green — pre-existing and
environment-specific, unrelated to this change.