Skip to content

The harness binds to a serving replica; one Spark down no longer refuses every dispatch - #10617

Merged
briansrls merged 1 commit into
mainfrom
harness-serving-roster
Sep 5, 2026
Merged

briansrls merged 1 commit into
mainfrom
harness-serving-roster

Conversation

@briansrls

Copy link
Copy Markdown
Contributor

Why

First dispatch on srv1 after #10611 got past admission (data-launch="admitted") and refused at provider selection: harness backend did not answer http://192.168.1.232:30000/v1/models. Measured from srv1 at that moment: .232 → no answer, .225 → 200 advertising deepseek-v4-flash:iq3s-split. harness_observe_backend folded the roster and answered with the first non-serving replica, so one Spark down refused every dispatch while the other served. Worse, even had it passed, the attempt's replica was picked by seed from the declared list, so a seed landing on .232 would fail on its first turn.

What

  • gunbc.harness.harness_backend: HarnessBackendRoster = Serving { model, serving, declined } | NoneServing { model, declined }, each replica observed once; declined carries the typed per-replica standing (countable, never absorbed). harness_serving_backend_url(shape, seed) binds a URL from the serving set or refuses naming every replica's cause. The all-or-nothing fold is deleted.
  • gunbc.harness.harness_wire: harness_replica_pick(seed, replicas) — the pure pick over an arbitrary roster; harness_backend_url_for_address.
  • gunbc.harness.harness_cli: every entry point (probe, worker, reviewer, auditor) binds the backend at process start and exits with the roster refusal before the first POST if nothing serves; the four copies of the exit match collapse into harness_turn_exit.
  • gunbc.roadmap_dispatch_actuator selection and gunbc.roadmap_belt_actuate provider preflight admit when any replica serves; refusal detail lists each declined replica.
  • Witness: the_replica_pick_binds_within_the_roster_it_is_handed — a one-element roster is chosen whatever the seed says (RED against the declared-list pick), two elements still spread by seed.

Verified

  • gunbc compile --target dag: 0 blocking on roadmap_serve, roadmap_belt_tick_cli, harness_cli.
  • claim_batch --hermetic over harness_chat_shape_test: 11/11.
  • Wet receipt from srv1 with .232 down: see the comment below (harness probe report run from this branch).

🤖 Generated with Claude Code

https://claude.ai/code/session_019iXPtAqzPaNBZbaaM5rxrY

…ses every dispatch

harness_observe_backend folded the replica roster into one standing and answered with the first
non-serving replica's, so with .232 unreachable and .225 advertising the model, dispatch on srv1
refused at provider selection (measured 2026-09-05, first dispatch after #10611). The roster's point
is that attempts spread across it, so the reading was all-or-nothing over a set.

harness_backend now observes each replica once and keeps both halves: HarnessBackendRoster carries the
serving addresses and the declined replicas with their typed standing, so what is down is counted and
never absorbed. Selection and the belt preflight admit when any replica serves and refuse naming every
replica's cause when none does. Each harness entry point binds its backend URL at process start from
the serving set (harness_serving_backend_url; the seed spreads attempts within it) and exits with the
roster's refusal before the first POST when nothing serves. harness_wire gains the pure pick over an
arbitrary roster, witnessed against the deleted behaviour: a one-element roster is chosen whatever
the seed says.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019iXPtAqzPaNBZbaaM5rxrY
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-05T22:13:48.054616Z 0065ed5 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0065ed5b09

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +147 to +148
let folded = fold(harness_engine_replica_addresses, init: HarnessRosterFold { serving: [], declined: [] }, f: (acc, a) => {
let standing = harness_observe_replica(address: a)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bound each replica probe before walking the roster

When one declared replica accepts a connection but never answers /v1/models while another replica is healthy, every roster observation stalls at this fold: harness_observe_replica calls http.Client.Get, whose transport is only curl -fsS with no connection or overall timeout (dag/extdeps/http/client.dag:52-58). Because the probe, worker, reviewer, and auditor entry points now all perform this full-roster observation before selecting a URL, they never reach the healthy replica or the turn's request deadline, so one wedged Spark can still block all dispatches instead of becoming a declined member; use a bounded GET for each replica.

Useful? React with 👍 / 👎.

@briansrls

Copy link
Copy Markdown
Contributor Author

Wet receipt from srv1, branch head 0065ed5b09, run 2026-09-05 ~21:40Z, no deploy (built in a scratch worktree, gunbc run --function harness_probe_cli):

replica 192.168.1.232: 000
replica 192.168.1.225: 200
probe_exit=0
✓ http.Client.PostJsonFromFile done in 5 minutes

With .232 unreachable the probe bound to .225 and completed a real multi-round turn (events file: turn.started, round.usage steps 1–5+ with tool calls, exit 0). On main the same roster refuses at selection before any POST (harness backend did not answer http://192.168.1.232:30000/v1/models, measured on srv1 earlier tonight after #10611 landed).

@briansrls
briansrls merged commit d79e5a6 into main Sep 5, 2026
4 checks passed
@briansrls
briansrls deleted the harness-serving-roster branch September 5, 2026 23:47
gunbai-bot Bot pushed a commit that referenced this pull request Sep 11, 2026
…ures; the private gap as a typed stall; the within-tree half re-homed

Answers review 63772 / 63786 (a transcribed measurement with no producer) and the
manager's 2026-09-11 rulings, by landing the producer rather than softening the
claim.

THE INSTRUMENT. v1_compiler.bin.joint_claim_join computes, per change, the
export-surface entries removed (declared | variants | reexported, base minus
head) and the import claims added and retired, from only the diff-touched files
on both sides through namespace_wave_admission base_records and diff_sides, and
joins every admitted pair. A claim is joined only while LIVE when the entry
leaves: not retired by the removing change itself, and in retrospective mode not
retired by any change that landed between the pair. raw_candidates is reported
beside findings so the width the exclusion removes stays a number. Seven
fixture-boundary controls, real parser, real join: one positive control per
instance shape (variant deleted, name renamed, re-export dropped), one RED per
exclusion, one for the pair predicate, one for a claim already at the base.

WHAT IT ANSWERS, cited by invocation in the plan and not transcribed:
`joint_claim_join 6a54695^..f078c59` (first-parent main since
2026-08-28, default 3-day window) reports subjects=373 raw_candidates=236
findings=4 -- exactly the three 2026-09-10 instances (instance 1 is two names).
The two candidates the first-exclusion-only run reported as findings were both a
third commit retiring the claim between the pair (#10617 for the harness one; the
DensityMarketedMax one likewise); two artefacts with one cause was a defect in the
retrospective's pair predicate, built in as the second exclusion with its own
control rather than described.

TWO CORRECTIONS THE INSTRUMENT MADE TO THE SCOPE. Instance 2 (#10923 x #10925)
was a MOVE, not a dropped re-export: the base declared
mutation_status_is_commit_ambiguous in secret_provision_actuator and #10923
re-homed it; the instrument reports `(declared)` and the plan's table now says so.
The dropped re-export stays covered by the surface and keeps its control.

THE RULINGS. Checkpoint 2 stops at the pull-requests:read permission row on a
required job (the operator's trade); the dashboard reader is costed in section
4.1 and preferred on every axis but authority. The private gap is a typed
GuaranteeStall row (joint_claim_join_private_corpus_unindexed_stall) with the
overlay's extra-source-root trigger as its grounding, not prose. The within-tree
field-set case is NOT a checkpoint here: its home is
witness_that_fails_to_compile_is_absent_rather_than_red, and this change appends
a receipt there widening the trigger from claim-root files to the ruled capability
-- every construction site of a type whose field set changed, regardless of gate
closure -- per section 4b(3), rather than minting a second authority.

Hand Rust is enumerated in gunbc.joint_claim_join_seed_growth (33 items, no impl
block, every one citable) and rostered in seed_growth_admission.

Verified remotely: clippy -D warnings clean on the bin, 7/7 tests, the
retrospective above, and v1_src_dag_parse over the corpus with every new .dag
present: 5461 files parse-clean, no findings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QzQcQRu3WKxJuWV2zaiHSW
briansrls pushed a commit that referenced this pull request Sep 11, 2026
…t 1: joint_claim_join, the instrument behind its figures (#11044)

* Scope the jointly-incompatible-open-PR mitigation: a claim-channel x surface join over the declaration index, advisory-first

Three 2026-09-10 reds on main were pairs of individually-green, textually
disjoint PRs whose union does not compile. This records the scope of a
mitigation and builds nothing, answering the four questions the brief asked
before any shape:

THE CUT. All three public instances are one finding kind, ImportMemberAbsent,
and only one was a deletion: #10865 was a rename, and #10923 dropped a
RE-EXPORT with no declaration deleted anywhere. The cut is therefore the
surface predicate the declarations rider already applies, import_surface_has
(declared | variants | reexported) as a base-minus-head DELTA, not the kind of
edit. The general case is every claim channel the index carries -- imports,
citations, rostered rows (#10769, #10718 are the same shape through the other
two) -- and the residuals the index cannot see are named: field rename,
arity/signature, and pure freshness (private #49 x #50).

THE COST. namespace_wave_admission already reconstructs a base index beside
the head index from only the diff-touched files, per run, in the witnesses
lane; the join is A(I) & R(D) minus the claims D itself retires, a set
intersection, not a pairwise compile. Measured retrospectively over 374
first-parent commits since 2026-08-28: the raw join is a 207-hit candidate
list (the superset trap); with the one exclusion it is 5 hits, 4 of them the
three brief instances, the fifth an ordering artefact an open-PR-set join
excludes by construction. Recall 3/3 on the window's ImportMemberAbsent reds.

THE PLACEMENT. No new job (roster closed): a rider on the parse phase, like
rostered_row_join, advisory, reporting and annotating without pushing a phase
failure; the one emitted-workflow change is a pull-requests: read permission
row. The freshness caveat of a push-time verdict is stated as the reason this
sits beneath the ceiling.

THE REFUSAL. One finding per removed entry x claiming site: the (module,
name) and which surface it left, the sibling PR and head, the claiming module,
in_declaration and SourceLocation; NotEvaluated when the open-PR set is
unobservable, never an empty hit set.

gunbc-private: no parse phase, the overlay is outside DAG_PARSE_SWEEP_ROOTS,
so the join is public-only today; 113/114 private modules import from public,
a cross-repo exposure no public PR is ever compiled against, named and left
to warm-badger-62's lane.

Ceiling stays with gunbc.plans.ci_merge_freshness and the landed, deferred
receipt_is_admissible; this record retires with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QzQcQRu3WKxJuWV2zaiHSW

* chore: regenerate drifted generated artifacts (ci auto-heal)

Ledger-Repair-Judged: docs/design-rung-drops.md

* Checkpoint 1: joint_claim_join, the instrument behind the scope's figures; the private gap as a typed stall; the within-tree half re-homed

Answers review 63772 / 63786 (a transcribed measurement with no producer) and the
manager's 2026-09-11 rulings, by landing the producer rather than softening the
claim.

THE INSTRUMENT. v1_compiler.bin.joint_claim_join computes, per change, the
export-surface entries removed (declared | variants | reexported, base minus
head) and the import claims added and retired, from only the diff-touched files
on both sides through namespace_wave_admission base_records and diff_sides, and
joins every admitted pair. A claim is joined only while LIVE when the entry
leaves: not retired by the removing change itself, and in retrospective mode not
retired by any change that landed between the pair. raw_candidates is reported
beside findings so the width the exclusion removes stays a number. Seven
fixture-boundary controls, real parser, real join: one positive control per
instance shape (variant deleted, name renamed, re-export dropped), one RED per
exclusion, one for the pair predicate, one for a claim already at the base.

WHAT IT ANSWERS, cited by invocation in the plan and not transcribed:
`joint_claim_join 6a54695^..f078c59` (first-parent main since
2026-08-28, default 3-day window) reports subjects=373 raw_candidates=236
findings=4 -- exactly the three 2026-09-10 instances (instance 1 is two names).
The two candidates the first-exclusion-only run reported as findings were both a
third commit retiring the claim between the pair (#10617 for the harness one; the
DensityMarketedMax one likewise); two artefacts with one cause was a defect in the
retrospective's pair predicate, built in as the second exclusion with its own
control rather than described.

TWO CORRECTIONS THE INSTRUMENT MADE TO THE SCOPE. Instance 2 (#10923 x #10925)
was a MOVE, not a dropped re-export: the base declared
mutation_status_is_commit_ambiguous in secret_provision_actuator and #10923
re-homed it; the instrument reports `(declared)` and the plan's table now says so.
The dropped re-export stays covered by the surface and keeps its control.

THE RULINGS. Checkpoint 2 stops at the pull-requests:read permission row on a
required job (the operator's trade); the dashboard reader is costed in section
4.1 and preferred on every axis but authority. The private gap is a typed
GuaranteeStall row (joint_claim_join_private_corpus_unindexed_stall) with the
overlay's extra-source-root trigger as its grounding, not prose. The within-tree
field-set case is NOT a checkpoint here: its home is
witness_that_fails_to_compile_is_absent_rather_than_red, and this change appends
a receipt there widening the trigger from claim-root files to the ruled capability
-- every construction site of a type whose field set changed, regardless of gate
closure -- per section 4b(3), rather than minting a second authority.

Hand Rust is enumerated in gunbc.joint_claim_join_seed_growth (33 items, no impl
block, every one citable) and rostered in seed_growth_admission.

Verified remotely: clippy -D warnings clean on the bin, 7/7 tests, the
retrospective above, and v1_src_dag_parse over the corpus with every new .dag
present: 5461 files parse-clean, no findings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QzQcQRu3WKxJuWV2zaiHSW

* State the population compile's boundary on the row: compilability of the claim-root population, not evaluation of it

A file that compiles and whose test fns are declared but never reached from the
entry passes that wall untouched (cool-badger-34's four fns, 2026-09-11). That
hole is discriminating_arm_built_but_never_enrolled's, named as the neighbour so
the word POPULATION is not later cited for a scope the trigger never claimed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QzQcQRu3WKxJuWV2zaiHSW

---------

Co-authored-by: gunbc-ci-auto-heal <gunbc-ci-auto-heal@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant