Repository navigation
The harness binds to a serving replica; one Spark down no longer refuses every dispatch - #10617
Conversation
…ses every dispatch harness_observe_backend folded the replica roster into one standing and answered with the first non-serving replica's, so with .232 unreachable and .225 advertising the model, dispatch on srv1 refused at provider selection (measured 2026-09-05, first dispatch after #10611). The roster's point is that attempts spread across it, so the reading was all-or-nothing over a set. harness_backend now observes each replica once and keeps both halves: HarnessBackendRoster carries the serving addresses and the declined replicas with their typed standing, so what is down is counted and never absorbed. Selection and the belt preflight admit when any replica serves and refuse naming every replica's cause when none does. Each harness entry point binds its backend URL at process start from the serving set (harness_serving_backend_url; the seed spreads attempts within it) and exits with the roster's refusal before the first POST when nothing serves. harness_wire gains the pure pick over an arbitrary roster, witnessed against the deleted behaviour: a one-element roster is chosen whatever the seed says. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019iXPtAqzPaNBZbaaM5rxrY
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0065ed5b09
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let folded = fold(harness_engine_replica_addresses, init: HarnessRosterFold { serving: [], declined: [] }, f: (acc, a) => { | ||
| let standing = harness_observe_replica(address: a) |
There was a problem hiding this comment.
Bound each replica probe before walking the roster
When one declared replica accepts a connection but never answers /v1/models while another replica is healthy, every roster observation stalls at this fold: harness_observe_replica calls http.Client.Get, whose transport is only curl -fsS with no connection or overall timeout (dag/extdeps/http/client.dag:52-58). Because the probe, worker, reviewer, and auditor entry points now all perform this full-roster observation before selecting a URL, they never reach the healthy replica or the turn's request deadline, so one wedged Spark can still block all dispatches instead of becoming a declined member; use a bounded GET for each replica.
Useful? React with 👍 / 👎.
|
Wet receipt from srv1, branch head With .232 unreachable the probe bound to .225 and completed a real multi-round turn (events file: |
…ures; the private gap as a typed stall; the within-tree half re-homed Answers review 63772 / 63786 (a transcribed measurement with no producer) and the manager's 2026-09-11 rulings, by landing the producer rather than softening the claim. THE INSTRUMENT. v1_compiler.bin.joint_claim_join computes, per change, the export-surface entries removed (declared | variants | reexported, base minus head) and the import claims added and retired, from only the diff-touched files on both sides through namespace_wave_admission base_records and diff_sides, and joins every admitted pair. A claim is joined only while LIVE when the entry leaves: not retired by the removing change itself, and in retrospective mode not retired by any change that landed between the pair. raw_candidates is reported beside findings so the width the exclusion removes stays a number. Seven fixture-boundary controls, real parser, real join: one positive control per instance shape (variant deleted, name renamed, re-export dropped), one RED per exclusion, one for the pair predicate, one for a claim already at the base. WHAT IT ANSWERS, cited by invocation in the plan and not transcribed: `joint_claim_join 6a54695^..f078c59` (first-parent main since 2026-08-28, default 3-day window) reports subjects=373 raw_candidates=236 findings=4 -- exactly the three 2026-09-10 instances (instance 1 is two names). The two candidates the first-exclusion-only run reported as findings were both a third commit retiring the claim between the pair (#10617 for the harness one; the DensityMarketedMax one likewise); two artefacts with one cause was a defect in the retrospective's pair predicate, built in as the second exclusion with its own control rather than described. TWO CORRECTIONS THE INSTRUMENT MADE TO THE SCOPE. Instance 2 (#10923 x #10925) was a MOVE, not a dropped re-export: the base declared mutation_status_is_commit_ambiguous in secret_provision_actuator and #10923 re-homed it; the instrument reports `(declared)` and the plan's table now says so. The dropped re-export stays covered by the surface and keeps its control. THE RULINGS. Checkpoint 2 stops at the pull-requests:read permission row on a required job (the operator's trade); the dashboard reader is costed in section 4.1 and preferred on every axis but authority. The private gap is a typed GuaranteeStall row (joint_claim_join_private_corpus_unindexed_stall) with the overlay's extra-source-root trigger as its grounding, not prose. The within-tree field-set case is NOT a checkpoint here: its home is witness_that_fails_to_compile_is_absent_rather_than_red, and this change appends a receipt there widening the trigger from claim-root files to the ruled capability -- every construction site of a type whose field set changed, regardless of gate closure -- per section 4b(3), rather than minting a second authority. Hand Rust is enumerated in gunbc.joint_claim_join_seed_growth (33 items, no impl block, every one citable) and rostered in seed_growth_admission. Verified remotely: clippy -D warnings clean on the bin, 7/7 tests, the retrospective above, and v1_src_dag_parse over the corpus with every new .dag present: 5461 files parse-clean, no findings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzQcQRu3WKxJuWV2zaiHSW
…t 1: joint_claim_join, the instrument behind its figures (#11044) * Scope the jointly-incompatible-open-PR mitigation: a claim-channel x surface join over the declaration index, advisory-first Three 2026-09-10 reds on main were pairs of individually-green, textually disjoint PRs whose union does not compile. This records the scope of a mitigation and builds nothing, answering the four questions the brief asked before any shape: THE CUT. All three public instances are one finding kind, ImportMemberAbsent, and only one was a deletion: #10865 was a rename, and #10923 dropped a RE-EXPORT with no declaration deleted anywhere. The cut is therefore the surface predicate the declarations rider already applies, import_surface_has (declared | variants | reexported) as a base-minus-head DELTA, not the kind of edit. The general case is every claim channel the index carries -- imports, citations, rostered rows (#10769, #10718 are the same shape through the other two) -- and the residuals the index cannot see are named: field rename, arity/signature, and pure freshness (private #49 x #50). THE COST. namespace_wave_admission already reconstructs a base index beside the head index from only the diff-touched files, per run, in the witnesses lane; the join is A(I) & R(D) minus the claims D itself retires, a set intersection, not a pairwise compile. Measured retrospectively over 374 first-parent commits since 2026-08-28: the raw join is a 207-hit candidate list (the superset trap); with the one exclusion it is 5 hits, 4 of them the three brief instances, the fifth an ordering artefact an open-PR-set join excludes by construction. Recall 3/3 on the window's ImportMemberAbsent reds. THE PLACEMENT. No new job (roster closed): a rider on the parse phase, like rostered_row_join, advisory, reporting and annotating without pushing a phase failure; the one emitted-workflow change is a pull-requests: read permission row. The freshness caveat of a push-time verdict is stated as the reason this sits beneath the ceiling. THE REFUSAL. One finding per removed entry x claiming site: the (module, name) and which surface it left, the sibling PR and head, the claiming module, in_declaration and SourceLocation; NotEvaluated when the open-PR set is unobservable, never an empty hit set. gunbc-private: no parse phase, the overlay is outside DAG_PARSE_SWEEP_ROOTS, so the join is public-only today; 113/114 private modules import from public, a cross-repo exposure no public PR is ever compiled against, named and left to warm-badger-62's lane. Ceiling stays with gunbc.plans.ci_merge_freshness and the landed, deferred receipt_is_admissible; this record retires with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzQcQRu3WKxJuWV2zaiHSW * chore: regenerate drifted generated artifacts (ci auto-heal) Ledger-Repair-Judged: docs/design-rung-drops.md * Checkpoint 1: joint_claim_join, the instrument behind the scope's figures; the private gap as a typed stall; the within-tree half re-homed Answers review 63772 / 63786 (a transcribed measurement with no producer) and the manager's 2026-09-11 rulings, by landing the producer rather than softening the claim. THE INSTRUMENT. v1_compiler.bin.joint_claim_join computes, per change, the export-surface entries removed (declared | variants | reexported, base minus head) and the import claims added and retired, from only the diff-touched files on both sides through namespace_wave_admission base_records and diff_sides, and joins every admitted pair. A claim is joined only while LIVE when the entry leaves: not retired by the removing change itself, and in retrospective mode not retired by any change that landed between the pair. raw_candidates is reported beside findings so the width the exclusion removes stays a number. Seven fixture-boundary controls, real parser, real join: one positive control per instance shape (variant deleted, name renamed, re-export dropped), one RED per exclusion, one for the pair predicate, one for a claim already at the base. WHAT IT ANSWERS, cited by invocation in the plan and not transcribed: `joint_claim_join 6a54695^..f078c59` (first-parent main since 2026-08-28, default 3-day window) reports subjects=373 raw_candidates=236 findings=4 -- exactly the three 2026-09-10 instances (instance 1 is two names). The two candidates the first-exclusion-only run reported as findings were both a third commit retiring the claim between the pair (#10617 for the harness one; the DensityMarketedMax one likewise); two artefacts with one cause was a defect in the retrospective's pair predicate, built in as the second exclusion with its own control rather than described. TWO CORRECTIONS THE INSTRUMENT MADE TO THE SCOPE. Instance 2 (#10923 x #10925) was a MOVE, not a dropped re-export: the base declared mutation_status_is_commit_ambiguous in secret_provision_actuator and #10923 re-homed it; the instrument reports `(declared)` and the plan's table now says so. The dropped re-export stays covered by the surface and keeps its control. THE RULINGS. Checkpoint 2 stops at the pull-requests:read permission row on a required job (the operator's trade); the dashboard reader is costed in section 4.1 and preferred on every axis but authority. The private gap is a typed GuaranteeStall row (joint_claim_join_private_corpus_unindexed_stall) with the overlay's extra-source-root trigger as its grounding, not prose. The within-tree field-set case is NOT a checkpoint here: its home is witness_that_fails_to_compile_is_absent_rather_than_red, and this change appends a receipt there widening the trigger from claim-root files to the ruled capability -- every construction site of a type whose field set changed, regardless of gate closure -- per section 4b(3), rather than minting a second authority. Hand Rust is enumerated in gunbc.joint_claim_join_seed_growth (33 items, no impl block, every one citable) and rostered in seed_growth_admission. Verified remotely: clippy -D warnings clean on the bin, 7/7 tests, the retrospective above, and v1_src_dag_parse over the corpus with every new .dag present: 5461 files parse-clean, no findings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzQcQRu3WKxJuWV2zaiHSW * State the population compile's boundary on the row: compilability of the claim-root population, not evaluation of it A file that compiles and whose test fns are declared but never reached from the entry passes that wall untouched (cool-badger-34's four fns, 2026-09-11). That hole is discriminating_arm_built_but_never_enrolled's, named as the neighbour so the word POPULATION is not later cited for a scope the trigger never claimed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzQcQRu3WKxJuWV2zaiHSW --------- Co-authored-by: gunbc-ci-auto-heal <gunbc-ci-auto-heal@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Why
First dispatch on srv1 after #10611 got past admission (
data-launch="admitted") and refused at provider selection:harness backend did not answer http://192.168.1.232:30000/v1/models. Measured from srv1 at that moment: .232 → no answer, .225 → 200 advertisingdeepseek-v4-flash:iq3s-split.harness_observe_backendfolded the roster and answered with the first non-serving replica, so one Spark down refused every dispatch while the other served. Worse, even had it passed, the attempt's replica was picked by seed from the declared list, so a seed landing on .232 would fail on its first turn.What
gunbc.harness.harness_backend:HarnessBackendRoster=Serving { model, serving, declined }|NoneServing { model, declined }, each replica observed once;declinedcarries the typed per-replica standing (countable, never absorbed).harness_serving_backend_url(shape, seed)binds a URL from the serving set or refuses naming every replica's cause. The all-or-nothing fold is deleted.gunbc.harness.harness_wire:harness_replica_pick(seed, replicas)— the pure pick over an arbitrary roster;harness_backend_url_for_address.gunbc.harness.harness_cli: every entry point (probe, worker, reviewer, auditor) binds the backend at process start and exits with the roster refusal before the first POST if nothing serves; the four copies of the exit match collapse intoharness_turn_exit.gunbc.roadmap_dispatch_actuatorselection andgunbc.roadmap_belt_actuateprovider preflight admit when any replica serves; refusal detail lists each declined replica.the_replica_pick_binds_within_the_roster_it_is_handed— a one-element roster is chosen whatever the seed says (RED against the declared-list pick), two elements still spread by seed.Verified
gunbc compile --target dag: 0 blocking onroadmap_serve,roadmap_belt_tick_cli,harness_cli.claim_batch --hermeticoverharness_chat_shape_test: 11/11.🤖 Generated with Claude Code
https://claude.ai/code/session_019iXPtAqzPaNBZbaaM5rxrY