Repository navigation
harness: a belt-spawned turn waits in line for its serving seat instead of exiting - #13066
Merged
Merged
Conversation
…ad of exiting
A Pending seat bind (every admissible serving group busy) ended the worker
with exit 1 before any provider call: the dashboard read "Agent failed",
the belt recorded a yield, and a continuation turn was spent on a turn that
never reached a model (srv2, 2026-10-02: group-b saturated by traffic outside
the seat ledger, group-a down). harness_seat already names the remedy --
retry in a moment -- and the worker now performs it in place.
- harness_cli harness_bind_seat_waiting: worker, reviewer, auditor and
supervisor re-ask every 60s, bounded at 1440 waits; each wait is announced
as turn.placement_waiting on the attempt's own event stream. An unwritable
announcement or a failed sleep refuses at once; the cap exits pending as
before. The wait never places a turn beside unexplained occupants.
- roadmap_provider_events: HarnessPlacementWaiting -> ProviderAwaitingSeat
{ detail } (non-terminal); the jq projector keeps wait, wait_cap, reason.
- Every ProviderExecutionState consumer gains the arm: active agent segment
with the reason, no commit, handoff awaiting worker, not terminal.
- Witnesses: the harness's own wait line through the real reader; the
projector keeps the reason; a placed turn leaves the wait; a waiting worker
is active and holds its lineage; a wait ending in a nonzero exit is failed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…card over the closed coproduct) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A belt-spawned harness turn (worker, reviewer, auditor, supervisor) that finds no admissible serving seat now waits in line instead of exiting.
Why
harness_seatalready classifies "every admissible group busy" asHarnessSeatPending, and says the remedy is to ask again in a moment. Butharness_cliturned it intoExitFailure { code: 1 }before any provider call. Three things followed:On srv2 (2026-10-02/03), this used up all 3 turns of the
shell-dag-live-deploy-restart-tailscalelineage. Group-b was saturated by traffic outside the seat ledger, and group-a was down. None of those turns ran a model round, and the supervisor's verdict said so itself.What
gunbc.harness.harness_cliharness_bind_seat_waiting. On Pending, the harness announcesturn.placement_waiting(wait n of cap, plus the reason) on the attempt's own event stream, sleeps 60s, and re-binds. It's bounded at 1440 waits; at the cap it exits pending as before, loudly.gunbc.harness.harness_turnharness_event_placement_waiting. The event's producer.gunbc.roadmap_provider_events:HarnessPlacementWaiting, which maps toProviderAwaitingSeat { detail }, a non-terminal state.wait,wait_capandreason. Without that, an unknown type projects to{type}alone and loses the reason.ProviderExecutionStateconsumer gains the arm. The agent segment shows as active — waiting for a serving seat — …reason; there's no commit, the handoff awaits the worker, and the attempt isn't terminal, so the lineage holds and no second attempt is spawned.Evidence
roadmap_provider_events_witness_test: the harness's own wait line goes through the real reader, the reason and counter reach the detail, the label isawaiting_seat, a placed turn leaves the wait behind, and the projector keeps the reason.roadmap_workflow_progress_witness_test: a waiting worker is an active agent with its reason, and a wait ending in a nonzero exit is failed.roadmap_belt_actuate_witness_test: a waiting worker is not terminal until it exits.srv2/placement-wait, the same change on the old deployed revisioncc5bbdcc): provider_events 25 pass, workflow_progress 29 pass, belt_actuate 140 pass and 1 fail.the_two_production_lineages_escalate_and_convene_naming_their_derived_supervisor, also fails atcc5bbdccwithout this change (baseline run).e1cc8709).9e12b415dd3899e4emittedturn.placement_waitingand waited about 100 times. The issue page showed Agent · active — waiting for a serving seat — wait 9 of 1440: group-a … unavailable; group-b: the engine reports 7 occupant(s) while the seat ledger admitted 0.01feb0b65dee1377) were seated through the ledger once group-b drained.Known follow-ups (not in this PR)
stale_session) until a belt tick's teardown pass runs, so with the timer off they have to be cleared by hand.🤖 Generated with Claude Code