Skip to content

Refuse the heal job's binary/source revision skew; stop reporting every missing function as "no main" - #7401

Merged
briansrls merged 1 commit into
mainfrom
heal-binary-source-skew
Jul 29, 2026
Merged

briansrls merged 1 commit into
mainfrom
heal-binary-source-skew

Conversation

@briansrls

Copy link
Copy Markdown
Contributor

Two defects from one incident. The first made three innocent PRs red; the second is why it took hours to find.

1. The heal job pairs two different revisions

heal_generated_artifacts runs a binary and a source tree from different commits:

  • build compiles the default pull_request checkout — the merge ref
  • heal checks out github.head_ref — the branch head — because it pushes the regeneration back
  • heal then runs that merge-ref binary against the branch-head tree

Nothing in the model pairs them. A generated artifact's identity is (authority source × compiler), and this job silently combined two revisions of that product. ci_heal_branch_checkout_note reasons carefully about push semantics, permissions, loop termination and fork PRs — and calls build's output "the release gunbc binary", treating a projection of one revision as a context-free tool.

Latent until a seed change adds a Rust→.dag call. #7348 did: its Rust calls gunbc.observation_seed_render.scoped_run_begin_write_measured. Every open PR whose branch head predated it went red at once — #7392, #7394, #7395, identical failure, none of them at fault — while main stayed green because ci_regen_heal_if skips this job on push. Invisible where it landed, blamed on three PRs that didn't cause it.

Reproduced by execution, one binary both times:

tree result
pre-#7348 branch head error: scoped run observation projection failed: no main function found — byte-identical to CI
same tree, advance merged in ExitSuccess, zero drift

Why the comparand is github.sha, not origin/main

The first cut compared against origin/main. That was wrong, per operator ruling: origin/main names several different things — this runner's, a developer's, the actual source of truth — so a guard keyed on it asserts a relationship to whichever main this box last fetched, and can race an advance mid-run.

There is no ambiguity to resolve, because the question was never about main. It is "was this binary compiled from a revision this tree can satisfy?" — and github.sha is the merge commit build checked out, known exactly, in the same workflow run. Unresolvable provenance refuses rather than assuming agreement.

Why it's narrow

A blanket "tree must equal the built-from revision" would red nearly every PR to catch a rare pairing — a wall priced in other people's time. Only seed (src/v1) commits can introduce a Rust→.dag expectation, so a .dag-only advance cannot skew a compiled binary and doesn't red the PR.

Verified in both directions:

tree seed commits missing verdict
the incident's branch point 2 — one of them is 445bac31c (#7348) refuse, naming the responsible commit
current main 0 pass
same branch, post-merge 0 pass — the live heal on #7395 is the receipt

Out of scope on purpose: preventing a stale branch from merging. That needs a source-of-truth model the repo doesn't have yet (SCM lane, after v1 deletion), and this guard doesn't pretend to supply one — it protects the one pairing this job has agency over. Landed as a declared Scaffold: validation where construction (heal building from the tree it heals) costs ~10min of build against heal's current ~2min, with that named as the dissolution trigger.

2. The diagnostic named the wrong function

ctx.lookup_fn(entry_fn).ok_or(InterpError::NoMainFunction)?

Both run_in_context and run_in_context_with_args threw away the name they had just failed to look up, so every missing named function surfaced as no main function found. main_wet was never missing — the missing thing was a projection function — and the message sent the investigation at the wrong target for hours.

NoSuchFunction { name } already existed. Both sites now use it. Those were NoMainFunction's only two constructors, so the variant is deleted rather than left unconstructible; its one consumer (run_claim_failure_receipt) had to match both variants precisely because of this bug and now matches the single honest arm.

Proven by a discriminating run — same failure, under a simulated skew:

before: scoped run observation projection failed: no main function found
after:  scoped run observation projection failed: no such function: scoped_run_begin_write_measured

Validation

  • whole-tree gunbc compile --target dag — 0 blocking errors
  • generated_artifact_drift_test 8/8, incl. witness_committed_is_fixed_point + RED control — ci.yml is regenerated from the model, not hand-edited
  • observation_seed_scoped_run_witness_test 10/10
  • cargo test --test scoped_run_observation 7/7
  • cargo fmt --all --check clean; emitted guard bash -n clean

Noted, not fixed here

Both git hooks misreport a missing tool as a content failure — the same defect class as #2. .githooks/pre-push and .githooks/pre-commit run cargo fmt --all --check, and when cargo isn't on PATH they print "rustfmt drift" / "fmt drift in staged Rust". There is no drift; the check never ran. Both are registered generated artifacts, so the fix belongs at their .dag authority — worth its own change.

🤖 Generated with Claude Code

…ry missing function as "no main"

The generated-artifact heal ran a binary and a source tree from DIFFERENT
revisions. `build` compiles the default pull_request checkout — the merge ref —
while `heal` checks out `github.head_ref`, the branch head, because it pushes the
regeneration back. Nothing paired them. A generated artifact's identity is
(authority source x compiler), and the job silently combined two revisions of it.

Latent until a seed change adds a Rust->.dag call. #7348 landed the scoped-run
observation seed, whose Rust calls
gunbc.observation_seed_render.scoped_run_begin_write_measured; every open PR whose
branch head predated it went red at once (#7392, #7394, #7395 — identical failure,
none of them at fault), while main stayed green because ci_regen_heal_if skips
this job on push. The regression was invisible where it landed and blamed three
PRs that did not cause it.

Reproduced by execution: ONE binary built from the merge ref, run against a
pre-#7348 tree reproduces the exact CI failure; run against the same tree with the
advance merged in it exits 0 with zero drift.

The guard compares the tree against `github.sha` — the revision the build job
actually compiled — not against `origin/main`. Per operator ruling 2026-07-29,
`origin/main` names several different things (this runner's, a developer's, the
real source of truth) and can race an advance mid-run; the binary's own provenance
has no such ambiguity and is known exactly within the same workflow run. An
unresolvable provenance refuses rather than assuming agreement.

It is deliberately narrow: only SEED (src/v1) commits can introduce a Rust->.dag
expectation, so a .dag-only advance cannot skew a compiled binary and does not red
the PR. At the incident's branch point the set is two commits, one of which IS
445bac3 (#7348) — the guard names the responsible commit in its own refusal; at
current main and post-merge it is empty and silent. Preventing a stale branch from
MERGING is deliberately out of scope: that needs a source-of-truth model the repo
does not have yet (SCM lane, after v1 deletion). Landed as a declared Scaffold —
validation where construction (heal building from the tree it heals) was too
expensive today — with that named dissolution trigger.

Second, independent defect, and the reason this cost hours to find: both
run_in_context and run_in_context_with_args threw away the name they had just
failed to look up and reported InterpError::NoMainFunction, so EVERY missing named
function surfaced as "no main function found". main_wet was never missing.
NoSuchFunction { name } already existed; both sites now use it, and since those
were NoMainFunction's only two constructors the variant is deleted rather than
left unconstructible. Its one consumer, run_claim_failure_receipt, had to match
both variants precisely because of this bug and now matches the single honest arm.
Proven by a discriminating run: the same failure now reads
"no such function: scoped_run_begin_write_measured".

Verified: whole-tree compile 0 blocking errors; generated_artifact_drift_test 8/8
including witness_committed_is_fixed_point (ci.yml regenerated from the model, not
hand-edited) and its RED control; observation_seed_scoped_run_witness_test 10/10;
cargo test --test scoped_run_observation 7/7; cargo fmt --all --check clean;
emitted guard bash -n clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 29, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant