Skip to content

fix(ci): repair red main — busy-mode test + missing checkout in skills-index workflows - #83974

Merged
kshitijk4poor merged 1 commit into
NousResearch:mainfrom
kshitijk4poor:fix/ci-main-red
Aug 11, 2026
Merged

kshitijk4poor merged 1 commit into
NousResearch:mainfrom
kshitijk4poor:fix/ci-main-red

Conversation

@kshitijk4poor

@kshitijk4poor kshitijk4poor commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Main is red. Three distinct failures; two are fixed here, the third needs no code.

1. test_multiplex_busy_input_mode.py — blocks every merge

test_profile_route_and_nonmultiplexed_resolution_preserve_boundaries fails on
main, taking down Python tests / Run tests slice 5/12 and therefore
All required checks pass. Every PR is currently blocked behind this.

Cause

A semantic merge conflict between two PRs (both by @tmchow) that merged
57 minutes apart:

PR merge commit merged (UTC) role
#83550 — fix(gateway): respect routed profile busy modes a31be480 2026-08-11 04:51:47 added the test
#83400 — feat(gateway): allow selective multiplex profile serving 9829746dc8f235a1 2026-08-11 05:48:24 added the gate

#83400 taught _profile_name_for_source to reject a route whose target profile
is not in the served set (profiles_to_serve). #83550 added a test that routes
to profile research without ever declaring it served. Each is correct alone;
together the gate rejects the route the test depends on.

Why CI didn't catch it: #83400 was never re-tested after #83550 landed.

The test literally did not exist on the commit CI approved:

$ git merge-base --is-ancestor a31be480 0eb96d7a   # test in tested head?
NO — tested head predates the test

$ git show 0eb96d7a:tests/gateway/test_multiplex_busy_input_mode.py \
    | grep -c test_profile_route_and_nonmultiplexed
0

So this is not a flake and not a bad review — it's a stale-base merge. The
first execution of gate + test together happened on main, in the merge result
that no CI run ever covered.

The mechanism, concretely: the test never patches profiles_to_serve, so it
reads the runner's real on-disk profiles. research isn't among them, the
route is rejected before the busy-mode snapshot is ever consulted, and the
assertion receives the gateway default instead:

WARNING gateway.run: Rejecting profile route 'research-chat':
                     target profile 'research' is not served
AssertionError: assert 'interrupt' == 'steer'

Fix: patch profiles_to_serve around the assertion — the same seam every
sibling test in tests/gateway/test_profile_resolution.py already patches
(test_route_inside_allowlist_resolves, test_route_outside_allowlist_rejects).

This also removes an ambient-state dependency. The test previously passed or
failed depending on which profiles happened to exist on the machine running it;
it now passes under an empty HERMES_HOME.

Test-only — no production behavior changes. The serving gate from #83400 is
correct and is left fully intact.

2. Skills-index workflows: local action used without actions/checkout

check-freshness has failed all 12 of its last 12 scheduled runs:

##[error]Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under
'.../.github/actions/get-app-token'. Did you forget to run
actions/checkout before running your local action?

./.github/actions/get-app-token is a local composite action and can't resolve
without the repo on disk. skills-index-freshness.yml had no checkout step at
all.

The real-world impact is worse than one red check: that step is gated on
status != 'ok', so the watchdog only broke at the exact moment it was
supposed to work.
The live skills index is right now 521.4h stale against
a 26h limit, and the alarm designed to tell us never fired.

Auditing every workflow for this bug class turned up one more instance —
skills-index.yml's trigger-deploy job, which re-triggers the docs deploy so
a refreshed index actually reaches the live site. Its sibling build-index job
checks out; this one didn't. That is plausibly why the index went stale in the
first place
: the rebuild ran, but the deploy was never kicked.

Both are fixed. The audit now reports zero remaining jobs that invoke a local
action without a prior checkout. Pinned to the same actions/checkout SHA used
by the other 35 call sites in the repo.

3. Publish inline E2E evidence — no fix needed

Failed once at 13:33Z on a transient TLS error reaching api.github.com
(certificate is not valid for any names) while installing a gh extension.
The last 25 runs of that workflow are 25/25 success. Infra blip, not a defect —
flagging it so it isn't mistaken for a real regression.

Verification

  • Reproduced the test failure deterministically on main HEAD before fixing.
  • tests/gateway/test_multiplex_busy_input_mode.py — 15/15 pass after the fix.
  • Re-ran under an empty HERMES_HOME to confirm the ambient-state dependency is gone.
  • tests/gateway/test_multiplex_busy_input_mode.py + test_profile_resolution.py + tests/hermes_cli/test_profiles.py — 76 passed, 2 skipped.
  • Both workflow files parse as valid YAML with correct step structure; workflow diffs are pure additions (no steps altered or lost).
  • ruff clean.

Note: tests/gateway/test_api_server.py::TestHealthDetailedEndpoint::test_health_detailed_returns_ok
fails in my local environment (assert 'degraded' == 'ok'), but it fails
identically on unmodified main and its CI slice is green — pre-existing and
environment-specific, unrelated to this change.

…s-index workflows

Three separate reds on main. Two are fixed here; the third needs no code.

1. tests/gateway/test_multiplex_busy_input_mode.py (blocks every merge)

Fails "Python tests / Run tests slice 5/12" and therefore "All required
checks pass". Semantic merge conflict between two PRs merged ~1h apart:

  a31be48 fix(gateway): respect routed profile busy modes             (added the test)
  c8f235a feat(gateway): allow selective multiplex profile serving    (added the gate)

c8f235a taught _profile_name_for_source to reject a route whose target
profile is not in the served set (profiles_to_serve). Each PR was green on
its own base; neither ran against the other's merge result.

The test asserts a route to profile "research" resolves to that profile's
busy mode, but never patches profiles_to_serve — so it reads the runner's
REAL on-disk profiles. "research" is not among them, the route is rejected
before the busy-mode snapshot is consulted, and the assertion gets the
gateway default:

  WARNING gateway.run: Rejecting profile route 'research-chat':
                       target profile 'research' is not served
  AssertionError: assert 'interrupt' == 'steer'

Patch profiles_to_serve for the assertion — the same seam every sibling
test in tests/gateway/test_profile_resolution.py already patches
(test_route_inside_allowlist_resolves, test_route_outside_allowlist_rejects).

This also removes an ambient-state dependency: the test previously passed
or failed based on which profiles happened to exist on the machine running
it. Verified passing under an empty HERMES_HOME.

Test-only. The serving gate from c8f235a is correct and left intact.

2. Skills-index workflows: local action used without actions/checkout

check-freshness has failed on all 12 of its last 12 scheduled runs:

  ##[error]Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under
  '.../.github/actions/get-app-token'. Did you forget to run
  actions/checkout before running your local action?

./.github/actions/get-app-token is a LOCAL composite action and cannot
resolve without the repo on disk. skills-index-freshness.yml had no
checkout step at all. The step is gated on `status != 'ok'`, so the
watchdog broke exactly when it was supposed to file its issue — the live
index is currently 521.4h stale (limit 26h) and nobody was told.

An audit of all workflows for this bug class found one more instance:
skills-index.yml's `trigger-deploy` job, which re-triggers the docs deploy
so a refreshed index reaches the live site. Its sibling `build-index` job
checks out; this one did not. That is plausibly why the index went stale
in the first place. Both are fixed; the audit now reports zero remaining
jobs that use a local action without a prior checkout.

Pinned to the same actions/checkout SHA used by the other 35 call sites.

3. "Publish inline E2E evidence" — no fix needed

Failed once at 13:33Z on a transient TLS error reaching api.github.com
("certificate is not valid for any names") while installing a gh
extension. The last 25 runs of that workflow are 25/25 success. Infra
blip, not a code defect.
@kshitijk4poor
kshitijk4poor requested a review from a team August 11, 2026 15:41
@kshitijk4poor kshitijk4poor added the ci-reviewed applied to manually approve dangerous changes label Aug 11, 2026
@kshitijk4poor
kshitijk4poor enabled auto-merge (rebase) August 11, 2026 15:55
@kshitijk4poor
kshitijk4poor merged commit f51aa6a into NousResearch:main Aug 11, 2026
105 of 107 checks passed
@alt-glitch alt-glitch added type/test Test coverage or test infrastructure comp/gateway Gateway runner, session dispatch, delivery P1 High — major feature broken, no workaround sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation labels Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-reviewed applied to manually approve dangerous changes comp/gateway Gateway runner, session dispatch, delivery P1 High — major feature broken, no workaround sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation type/test Test coverage or test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants