Skip to content

harness: a belt-spawned turn waits in line for its serving seat instead of exiting - #13066

Merged
gunbai-bot[bot] merged 2 commits into
mainfrom
roadmap/e2e-fabric-run
Oct 3, 2026
Merged

gunbai-bot[bot] merged 2 commits into
mainfrom
roadmap/e2e-fabric-run

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

A belt-spawned harness turn (worker, reviewer, auditor, supervisor) that finds no admissible serving seat now waits in line instead of exiting.

Why

harness_seat already classifies "every admissible group busy" as HarnessSeatPending, and says the remedy is to ask again in a moment. But harness_cli turned it into ExitFailure { code: 1 } before any provider call. Three things followed:

  • The dashboard read Agent failed.
  • The belt recorded a yield without a captured submission.
  • A continuation turn was spent on a turn that never reached a model.

On srv2 (2026-10-02/03), this used up all 3 turns of the shell-dag-live-deploy-restart-tailscale lineage. Group-b was saturated by traffic outside the seat ledger, and group-a was down. None of those turns ran a model round, and the supervisor's verdict said so itself.

What

  • gunbc.harness.harness_cli harness_bind_seat_waiting. On Pending, the harness announces turn.placement_waiting (wait n of cap, plus the reason) on the attempt's own event stream, sleeps 60s, and re-binds. It's bounded at 1440 waits; at the cap it exits pending as before, loudly.
    • Refused, an unwritable announcement, or a failed sleep each end the wait at once.
    • The wait only defers asking. It never places a turn beside occupants the ledger can't explain.
  • gunbc.harness.harness_turn harness_event_placement_waiting. The event's producer.
  • gunbc.roadmap_provider_events:
    • New event kind HarnessPlacementWaiting, which maps to ProviderAwaitingSeat { detail }, a non-terminal state.
    • The jq projector keeps wait, wait_cap and reason. Without that, an unknown type projects to {type} alone and loses the reason.
  • Every ProviderExecutionState consumer gains the arm. The agent segment shows as active — waiting for a serving seat — …reason; there's no commit, the handoff awaits the worker, and the attempt isn't terminal, so the lineage holds and no second attempt is spawned.

Evidence

  • Witnesses:
    • roadmap_provider_events_witness_test: the harness's own wait line goes through the real reader, the reason and counter reach the detail, the label is awaiting_seat, a placed turn leaves the wait behind, and the projector keeps the reason.
    • roadmap_workflow_progress_witness_test: a waiting worker is an active agent with its reason, and a wait ending in a nonzero exit is failed.
    • roadmap_belt_actuate_witness_test: a waiting worker is not terminal until it exits.
  • Run on srv2 against the deployed binary (deploy branch srv2/placement-wait, the same change on the old deployed revision cc5bbdcc): provider_events 25 pass, workflow_progress 29 pass, belt_actuate 140 pass and 1 fail.
    • The one failure, the_two_production_lineages_escalate_and_convene_naming_their_derived_supervisor, also fails at cc5bbdcc without this change (baseline run).
  • Production route: deployed to srv2 (revision e1cc8709).
    • Attempt 9e12b415dd3899e4 emitted turn.placement_waiting and waited about 100 times. The issue page showed Agent · active — waiting for a serving seat — wait 9 of 1440: group-a … unavailable; group-b: the engine reports 7 occupant(s) while the seat ledger admitted 0.
    • Later, the supervisor turn and the renewed worker turn (01feb0b65dee1377) were seated through the ledger once group-b drained.
  • Not yet run on this exact head: CI is the first run of the witnesses at main + this commit.

Known follow-ups (not in this PR)

  • The page header still reads "Agent working" while waiting; the segment line underneath is correct.
  • A launch with a missing worktree is admitted and shown as "Workspace · complete". Launch should check that the workspace exists.
  • Exited worker and supervisor units block re-dispatch (stale_session) until a belt tick's teardown pass runs, so with the timer off they have to be cleared by hand.

🤖 Generated with Claude Code

…ad of exiting

A Pending seat bind (every admissible serving group busy) ended the worker
with exit 1 before any provider call: the dashboard read "Agent failed",
the belt recorded a yield, and a continuation turn was spent on a turn that
never reached a model (srv2, 2026-10-02: group-b saturated by traffic outside
the seat ledger, group-a down). harness_seat already names the remedy --
retry in a moment -- and the worker now performs it in place.

- harness_cli harness_bind_seat_waiting: worker, reviewer, auditor and
  supervisor re-ask every 60s, bounded at 1440 waits; each wait is announced
  as turn.placement_waiting on the attempt's own event stream. An unwritable
  announcement or a failed sleep refuses at once; the cap exits pending as
  before. The wait never places a turn beside unexplained occupants.
- roadmap_provider_events: HarnessPlacementWaiting -> ProviderAwaitingSeat
  { detail } (non-terminal); the jq projector keeps wait, wait_cap, reason.
- Every ProviderExecutionState consumer gains the arm: active agent segment
  with the reason, no commit, handoff awaiting worker, not terminal.
- Witnesses: the harness's own wait line through the real reader; the
  projector keeps the reason; a placed turn leaves the wait; a waiting worker
  is active and holds its lineage; a wait ending in a nonzero exit is failed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…card over the closed coproduct)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 3, 2026
Merged via the queue into main with commit 2d8bfeb Oct 3, 2026
4 checks passed
@gunbai-bot
gunbai-bot Bot deleted the roadmap/e2e-fabric-run branch October 3, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants