Repository navigation
Own PR #9275 to green (private CARGO_HOME/RUSTUP_HOME per job): re-run its cancelled build job, close the fleet-desired.yml shared-home gap in-PR, and classify every generated self-hosted Rust job PrivateMutableHome or ImmutableToolchain with no unclassified job admitted - #9342
Closed
gunbai-bot[bot] wants to merge 11 commits into
Closed
gunbai-bot[bot] wants to merge 11 commits into
gunbai-bot[bot] wants to merge 11 commits into
Conversation
… slots stop rewriting each other's binaries
LINE STOP. The required evidence path was corrupting itself. rustfmt refused with
ETXTBSY on srv4-11 and srv3-11 seven minutes apart, killing the build lane on two
unrelated PRs, and srv1-15 resolved rustc while cargo had vanished (exit 127). The
compiler's own refusal named the cause exactly: the tool "was resolved at admission
from PATH ... and ran there: --version was executed successfully before this phase
began. So it has been removed, replaced or made unusable while this run was executing;
this is not a missing or broken installation."
CAUSE: each host runs many runner slots under one /home/ghrunner, and every witness job
began by installing rustup into it. One slot's install rewrites a toolchain another slot
is executing.
THE CONSTRUCTION ALREADY EXISTED. fleet-converge has wiped and set HOME, CARGO_HOME and
RUSTUP_HOME under RUNNER_TEMP since someone hit this before -- its pin step is literally
named "isolated RUSTUP_HOME has no default toolchain". witnesses.yml had no such step and
called setup-rust-toolchain into the shared default. So this is applying a proven in-repo
pattern, not designing a fix.
WHY A THIRD MODULE. ci_isolate_toolchain_step carried THREE concepts: home isolation,
build tuning (MAKEFLAGS, CARGO_BUILD_JOBS) and cache policy (sccache). Importing it
wholesale would have imposed a CARGO_BUILD_JOBS derived from the SELECTED runner target's
RAM-speed budget on a job with a different subject -- a build-policy change smuggled
inside an outage fix. So the intent is decomposed into ci_toolchain_home_isolation_steps
and ci_fleet_build_tuning_steps, and gunbc.toolchain_workflow_steps holds the two steps
both workflows genuinely share. The witness workflow does not import the fleet prelude
(it would borrow a cross-job artifact architecture it does not have) and does not
duplicate the shell strings (that is the §3 nicknaming violation).
THE COMPOSED FLEET PIPELINE IS BYTE-IDENTICAL BY CONSTRUCTION -- same seven steps, same
order, same builder -- and that was the discriminating control on the refactor. Verified
by execution: regenerating produced ZERO toolchain, isolate, rustup, CARGO_HOME,
MAKEFLAGS or sccache line changes in fleet-converge.yml.
THE CAPABILITY CLOSURE CAUGHT A REAL FORK, which is the part I did not predict. Adding
the pin step made the_live_witness_floor_job_closes_its_capabilities FAIL: the pin
consumes RustupCapability and the witness toolchain step declared only
[Cargo, Rustfmt], while fleet declares [Cargo, Rustc, Rustfmt, Rustup] for the IDENTICAL
setup_rust_action. One action, two capability stories. The under-declaration is corrected
rather than the consumer weakened -- the action really does install rustup and rustc --
and it went unnoticed only because nothing consumed Rustup here until now.
THE DIAGNOSTIC NOW FAILS CLOSED. It printed ls -l "${CARGO_HOME:-$HOME/.cargo}/bin",
whose :- fallback made a MISSING isolation step look like a legitimate shared-home mode.
It now refuses with ToolchainHomesNotIsolated when either home is unset, so the condition
this PR fixes cannot silently return.
YAML REGENERATED FROM THE EMITTER, not hand-edited: I rebuilt the toolchain binary (3m)
rather than hand-apply, because this change adds steps to two jobs and hand-application
at that structure is not the deterministic substitution the budget-tree.md case was.
SEPARATELY FOUND, DELIBERATELY NOT INCLUDED: .github/workflows/fleet-converge.yml is
STALE ON MAIN against its own emitter -- an unrelated secret-hygiene change (auth headers
moved from -H arguments into files) is authored but never regenerated. My heal run picked
it up; I reverted it so this PR does not absorb somebody else's change under a toolchain
title. It needs its own cut.
Witnesses: workflow_capability_closure_witness_test 4/4 PASS including the live witness
floor job, the live fleet-converge build job (the control), and the two ordering REDs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…violations inside a declaration body The v2-emission phase refused with 7 hard diagnostics, all mine, all one cause: source annotation sits inside a declaration body. Only module-item grain is modeled; move it above the declaration it describes. I put a seven-line // block INSIDE the prepared_bound_steps() list literal, beside the binding it explains. DESIGN §4c admits only standalone leading blocks attached to MODULE-SCOPE declarations, and each // line counts as its own annotation -- hence seven diagnostics from one comment. Re-homed above fn prepared_bound_steps, reworded to name the binding it now sits above rather than pointing at "below". WHY MY LOCAL RUN WAS GREEN AND CI WAS NOT, because that is the reusable part. The capability witnesses passed 4/4 locally through claim_batch, which evaluates and is LENIENT about in-body annotations. The refusal lives in strict preparation / compile, which is what CI runs. So a green witness suite is not evidence a .dag file is admissible -- the two ask different questions, and the cheap strict check is a compile of the entry. Verified this time by structure rather than by rerunning the lenient path: zero indented // lines inside any declaration body across all four files this PR touches. WHAT THE FAILED RUN ALSO PROVED, and it is the thing the PR is for. The isolation WORKED on a real runner. srv1-14 reported HOME, CARGO_HOME and RUSTUP_HOME all resolved under the per-job RUNNER_TEMP, the new fail-closed diagnostic passed rather than refusing, and the job then built for eighteen minutes through regen with no ETXTBSY and no missing cargo -- the exact failure that killed three lanes tonight did not recur. The rustup line "$HOME differs from euid-obtained home directory" is a non-fatal notice; fleet-converge has produced it since it adopted the same construction. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
# Conflicts: # .github/workflows/witnesses.yml # dag/gunbc/witness_floor_workflow.dag
…des of the three conflicts
Merge of origin/main (11 commits) conflicted in witness_floor_workflow.dag and its
emitted witnesses.yml. All three hunks were BOTH-SIDES-WANTED, not either/or:
1. imports -- my capability widening [Cargo, Rustc, Rustfmt, Rustup]
AND main's new repo_self_build_all_bins_command.
2. prepared_bound_steps -- main parameterized it as (build_script: String) for
#9270's two-lane bootstrap split; my capability-fork note is
kept beside main's two-lanes note, both at module-item grain.
3. the step list -- my pin-step binding AND main's
witness_floor_build_step(build_script: build_script).
Taking either side alone would have silently dropped the other's change, which is why
each hunk is resolved by composition rather than by choosing.
THE .yml WAS NOT HAND-RESOLVED. It is a committed generated artifact, so a hand-merged
one would be a guess wearing the authority of a generated file. Instead: reset it to
main, rebuild the toolchain binary against the MERGED tree (2m53s), and regenerate. Both
jobs carry Isolate toolchain homes and Pin rustup default, in the required order, and
main's build-lane change survives.
fleet-converge.yml again regenerated with the same unrelated secret-hygiene drift it
carries on main (8 insertions, 4 deletions, ZERO toolchain/isolate/rustup/MAKEFLAGS/
sccache lines -- measured, not assumed). Reverted again; still needs its own cut.
Capability closure re-run on the MERGED tree, RC=0, 4/4 PASS including the
fleet-converge control and the two ordering REDs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…d mark the extracted shell carrier
Both findings from review 56091, verified against the tree before acting.
FINDING 2 IS CORRECT AND IS FIXED. The refusal was hand-built shell control flow inside
a concat chain in gunbc.witness_floor_workflow -- the medium-as-string form. The accepted
construction in this repository is a named command row composed through the pipeline, and
ci_pin_rustup_cargo_bin_missing_command is nearly identical in shape:
if [ -z "$CARGO_BIN" ]; then echo "::error::..."; exit 1; fi
So the refusal is now ci_toolchain_homes_isolated_or_refuse_command, a data row in
v2.workflow.ci_workflow_run_emit beside its four siblings, and the probe composes it
rather than spelling shell. Emitted identically into both jobs.
FINDING 1 IS ACTIONED, WITH ITS PREMISE CORRECTED. The marker is added -- a Scaffold
disposition plus an unbound dissolution trigger, matching ci_native_cache_root_script's
construction, emitted as a leading comment in the script. But "the new shell carrier" is
not what ci_toolchain_home_isolation_script is: it authors NO new shell. It emits four
PRE-EXISTING command rows unchanged (ci_isolate_rm_rustup_cargo_command and the three
echo rows) that ci_isolate_toolchain_script already emitted. What this PR added was a
COMPOSITION, not a medium.
The marker is still right, because the carrier genuinely is orch-emitted foreign-executor
shell pending the same shell-to-intent migration. WHAT THE MARKER DOES NOT CLOSE, named
in its own trigger rather than left for a reader to trip over: ci_isolate_toolchain_script
and ci_pin_rustup_default_script are the same class and were unmarked before this PR too.
Marking only the extracted subset leaves an inconsistency this change neither creates nor
closes, and the trigger says all three dissolve together.
ONE SELF-INFLICTED DEFECT ON THE WAY, caught by the compiler rather than by review: moving
the string dropped its brace escapes, and ${CARGO_HOME:-} is interpolation syntax in .dag.
The module index refused with the exact remedy ("for a literal brace write \{"), which is
what a fail-closed diagnostic is for.
Regenerated from the emitter against a rebuilt binary. fleet-converge.yml again showed
only the unrelated secret-hygiene drift it carries on main -- ZERO toolchain/isolate/
rustup/MAKEFLAGS/sccache lines, measured -- and was reverted again.
Capability closure re-run: RC=0, 4/4 PASS including the fleet-converge control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e standing from the emitted job fleet-desired.yml installed a Rust toolchain into /home/ghrunner on srv1, where many runner slots share one home -- the same defect witnesses.yml carried, with the remedy sitting one import away in gunbc.toolchain_workflow_steps the whole time. The job now composes toolchain_home_isolation_step ahead of the install and toolchain_pin_rustup_default_step behind it, in witness-floor's own order. gunbc.toolchain_home_standing carries the obligation rather than the remedy: it folds Workflow.jobs and classifies each job PrivateMutableHome or ImmutableToolchain off its emitted steps -- against the authority's own command rows and the cited setup_rust_action, never a text pattern -- and refuses on a shared-home install, an isolation with no install, or an undecidable expression runner. The predicate is conjoined into all three emitters, so a job added without a standing emits no yaml at all. action_ref_eq moves from gunbc.ci_heal_credential to extdeps.github.actions: it is a fact about ActionRef, and a second consumer would have made a credential-healing module the authority on action identity.
…rs without its merge-attribute row Not this branch's change. #9284 added the stage0 host file and the artifact is behind its authority by exactly the one derived row that file entails, so every branch's regen gate reproduces it. Carried here because the required run regenerates before it compares.
…er, so it owes its own obligation review 56168 is right on the doctrine. docs/plans/scaffold-admission-doctrine.md admits a SHARED dissolution obligation only where the artifacts have the same terminal construction, fire on the same observation, AND CANNOT SURVIVE ONE ANOTHER. ci_pin_rustup_default_script fails the first and third: it selects a TOOLCHAIN through rustup, which is not the per-job filesystem-and-environment effect ci_toolchain_home_isolation_script waits on, so when that effect lands the isolation shell dissolves and the pin shell is still standing. The previous trigger swept it in as a sibling that "dissolves together". That is precisely the single broad trigger laundering an expanding collection of temporary work the doctrine names, so the claim is WITHDRAWN rather than reworded, and the pin script now carries its own Scaffold disposition and its own unbound_dissolution naming the toolchain-selection effect it actually waits on. ZERO EXECUTABLE SHELL MOVED, and that is checked rather than asserted: filtering the emitted workflow diff to lines that are not dissolve-on comments returns EMPTY across all three artifacts. The change is four added comment lines and one corrected comment line. ONE CLAIM IN THIS PR IS NARROWED BY IT. The branch said fleet-converge.yml does not move a byte; it now moves by exactly one line, the pin step's own marker. The step's NAME and its executable body are still unchanged, which is what the byte-identity claim existed to protect -- the cost-floor exempt roster keys on step name against the live workflow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rier on its own declaration review 56168, both findings. The Bool predicate is deleted rather than given a disposition receipt. std.keyed_roster's own carrier already rules against it in one sentence -- consumers inspect the canonical typed verdict directly, with no parallel *_holds predicate that collapses the coproduct -- and the collapse was not free here: it discarded the job_id and the ToolchainHomeRefusal at the exact seam the emitter needed them, which is why all three emitters named one static string. admit_workflow_toolchain_homes returns WorkflowToolchainHomeAdmission, the refusing arm reproduces itself through the fold, and each cause now renders its own remedy -- the three the ToolchainHomeRefusal note promised and the emission was not keeping. WitnessFloorGenerationRefused and FleetDesiredGenerationRefused gained a reason field to carry it; a toolchain-home refusal and a capability-closure refusal previously reached the operator as the same sentence. The witness keeps a Bool, because a witness returns Bool by construction -- the collapse belongs at the assertion, where the discarded fields have no remaining consumer. ci_pin_rustup_default_script carries its own Scaffold and DissolutionCondition. Being named in the isolation carrier's trigger text bound nothing removable, and this PR triples the step's reach. ci_isolate_toolchain_script stays unmarked and the declaration says so: this PR does not touch it, and marking it would put a comment line into an emitted script no finding is about. Emitted yaml changes by marker comment only; no semantic delta.
…oolchain-home-isolation # Conflicts: # .github/workflows/fleet-converge.yml # .github/workflows/fleet-desired.yml # .github/workflows/witnesses.yml # src/v2/workflow/ci_workflow_run_emit.dag
Contributor
Author
|
Closing: this is a duplicate of #9275, which merged at 15:07:43Z as
Measured per file, branch
So there is nothing to gain and something to lose: zero new content, plus a three-way merge against a feature that landed after this branch stopped moving. Not flipping to ready — auto-review and CI on a stale duplicate would spend a ~55 minute floor cycle to confirm what is already on — sent from eager-crane-791 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-opened by session-dashboard for session
eager-crane-791.Pushing to
toolchain-home-isolationadvances this PR.Worker attestation
Before flipping this PR to ready for review, confirm each item:
npm test,cargo test) and the result.Closes #Ndirective.Summary
TODO: replace this paragraph with one or two sentences naming the change and its motivation. Reviewers read this first.
Test plan