Repository navigation
Complete private Rust-home isolation across generated workflows - #9301
Closed
gunbai-bot[bot] wants to merge 7 commits into
Closed
gunbai-bot[bot] wants to merge 7 commits into
gunbai-bot[bot] wants to merge 7 commits into
Conversation
… slots stop rewriting each other's binaries
LINE STOP. The required evidence path was corrupting itself. rustfmt refused with
ETXTBSY on srv4-11 and srv3-11 seven minutes apart, killing the build lane on two
unrelated PRs, and srv1-15 resolved rustc while cargo had vanished (exit 127). The
compiler's own refusal named the cause exactly: the tool "was resolved at admission
from PATH ... and ran there: --version was executed successfully before this phase
began. So it has been removed, replaced or made unusable while this run was executing;
this is not a missing or broken installation."
CAUSE: each host runs many runner slots under one /home/ghrunner, and every witness job
began by installing rustup into it. One slot's install rewrites a toolchain another slot
is executing.
THE CONSTRUCTION ALREADY EXISTED. fleet-converge has wiped and set HOME, CARGO_HOME and
RUSTUP_HOME under RUNNER_TEMP since someone hit this before -- its pin step is literally
named "isolated RUSTUP_HOME has no default toolchain". witnesses.yml had no such step and
called setup-rust-toolchain into the shared default. So this is applying a proven in-repo
pattern, not designing a fix.
WHY A THIRD MODULE. ci_isolate_toolchain_step carried THREE concepts: home isolation,
build tuning (MAKEFLAGS, CARGO_BUILD_JOBS) and cache policy (sccache). Importing it
wholesale would have imposed a CARGO_BUILD_JOBS derived from the SELECTED runner target's
RAM-speed budget on a job with a different subject -- a build-policy change smuggled
inside an outage fix. So the intent is decomposed into ci_toolchain_home_isolation_steps
and ci_fleet_build_tuning_steps, and gunbc.toolchain_workflow_steps holds the two steps
both workflows genuinely share. The witness workflow does not import the fleet prelude
(it would borrow a cross-job artifact architecture it does not have) and does not
duplicate the shell strings (that is the §3 nicknaming violation).
THE COMPOSED FLEET PIPELINE IS BYTE-IDENTICAL BY CONSTRUCTION -- same seven steps, same
order, same builder -- and that was the discriminating control on the refactor. Verified
by execution: regenerating produced ZERO toolchain, isolate, rustup, CARGO_HOME,
MAKEFLAGS or sccache line changes in fleet-converge.yml.
THE CAPABILITY CLOSURE CAUGHT A REAL FORK, which is the part I did not predict. Adding
the pin step made the_live_witness_floor_job_closes_its_capabilities FAIL: the pin
consumes RustupCapability and the witness toolchain step declared only
[Cargo, Rustfmt], while fleet declares [Cargo, Rustc, Rustfmt, Rustup] for the IDENTICAL
setup_rust_action. One action, two capability stories. The under-declaration is corrected
rather than the consumer weakened -- the action really does install rustup and rustc --
and it went unnoticed only because nothing consumed Rustup here until now.
THE DIAGNOSTIC NOW FAILS CLOSED. It printed ls -l "${CARGO_HOME:-$HOME/.cargo}/bin",
whose :- fallback made a MISSING isolation step look like a legitimate shared-home mode.
It now refuses with ToolchainHomesNotIsolated when either home is unset, so the condition
this PR fixes cannot silently return.
YAML REGENERATED FROM THE EMITTER, not hand-edited: I rebuilt the toolchain binary (3m)
rather than hand-apply, because this change adds steps to two jobs and hand-application
at that structure is not the deterministic substitution the budget-tree.md case was.
SEPARATELY FOUND, DELIBERATELY NOT INCLUDED: .github/workflows/fleet-converge.yml is
STALE ON MAIN against its own emitter -- an unrelated secret-hygiene change (auth headers
moved from -H arguments into files) is authored but never regenerated. My heal run picked
it up; I reverted it so this PR does not absorb somebody else's change under a toolchain
title. It needs its own cut.
Witnesses: workflow_capability_closure_witness_test 4/4 PASS including the live witness
floor job, the live fleet-converge build job (the control), and the two ordering REDs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…violations inside a declaration body The v2-emission phase refused with 7 hard diagnostics, all mine, all one cause: source annotation sits inside a declaration body. Only module-item grain is modeled; move it above the declaration it describes. I put a seven-line // block INSIDE the prepared_bound_steps() list literal, beside the binding it explains. DESIGN §4c admits only standalone leading blocks attached to MODULE-SCOPE declarations, and each // line counts as its own annotation -- hence seven diagnostics from one comment. Re-homed above fn prepared_bound_steps, reworded to name the binding it now sits above rather than pointing at "below". WHY MY LOCAL RUN WAS GREEN AND CI WAS NOT, because that is the reusable part. The capability witnesses passed 4/4 locally through claim_batch, which evaluates and is LENIENT about in-body annotations. The refusal lives in strict preparation / compile, which is what CI runs. So a green witness suite is not evidence a .dag file is admissible -- the two ask different questions, and the cheap strict check is a compile of the entry. Verified this time by structure rather than by rerunning the lenient path: zero indented // lines inside any declaration body across all four files this PR touches. WHAT THE FAILED RUN ALSO PROVED, and it is the thing the PR is for. The isolation WORKED on a real runner. srv1-14 reported HOME, CARGO_HOME and RUSTUP_HOME all resolved under the per-job RUNNER_TEMP, the new fail-closed diagnostic passed rather than refusing, and the job then built for eighteen minutes through regen with no ETXTBSY and no missing cargo -- the exact failure that killed three lanes tonight did not recur. The rustup line "$HOME differs from euid-obtained home directory" is a non-fatal notice; fleet-converge has produced it since it adopted the same construction. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
# Conflicts: # .github/workflows/witnesses.yml # dag/gunbc/witness_floor_workflow.dag
…des of the three conflicts
Merge of origin/main (11 commits) conflicted in witness_floor_workflow.dag and its
emitted witnesses.yml. All three hunks were BOTH-SIDES-WANTED, not either/or:
1. imports -- my capability widening [Cargo, Rustc, Rustfmt, Rustup]
AND main's new repo_self_build_all_bins_command.
2. prepared_bound_steps -- main parameterized it as (build_script: String) for
#9270's two-lane bootstrap split; my capability-fork note is
kept beside main's two-lanes note, both at module-item grain.
3. the step list -- my pin-step binding AND main's
witness_floor_build_step(build_script: build_script).
Taking either side alone would have silently dropped the other's change, which is why
each hunk is resolved by composition rather than by choosing.
THE .yml WAS NOT HAND-RESOLVED. It is a committed generated artifact, so a hand-merged
one would be a guess wearing the authority of a generated file. Instead: reset it to
main, rebuild the toolchain binary against the MERGED tree (2m53s), and regenerate. Both
jobs carry Isolate toolchain homes and Pin rustup default, in the required order, and
main's build-lane change survives.
fleet-converge.yml again regenerated with the same unrelated secret-hygiene drift it
carries on main (8 insertions, 4 deletions, ZERO toolchain/isolate/rustup/MAKEFLAGS/
sccache lines -- measured, not assumed). Reverted again; still needs its own cut.
Capability closure re-run on the MERGED tree, RC=0, 4/4 PASS including the
fleet-converge control and the two ordering REDs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…d mark the extracted shell carrier
Both findings from review 56091, verified against the tree before acting.
FINDING 2 IS CORRECT AND IS FIXED. The refusal was hand-built shell control flow inside
a concat chain in gunbc.witness_floor_workflow -- the medium-as-string form. The accepted
construction in this repository is a named command row composed through the pipeline, and
ci_pin_rustup_cargo_bin_missing_command is nearly identical in shape:
if [ -z "$CARGO_BIN" ]; then echo "::error::..."; exit 1; fi
So the refusal is now ci_toolchain_homes_isolated_or_refuse_command, a data row in
v2.workflow.ci_workflow_run_emit beside its four siblings, and the probe composes it
rather than spelling shell. Emitted identically into both jobs.
FINDING 1 IS ACTIONED, WITH ITS PREMISE CORRECTED. The marker is added -- a Scaffold
disposition plus an unbound dissolution trigger, matching ci_native_cache_root_script's
construction, emitted as a leading comment in the script. But "the new shell carrier" is
not what ci_toolchain_home_isolation_script is: it authors NO new shell. It emits four
PRE-EXISTING command rows unchanged (ci_isolate_rm_rustup_cargo_command and the three
echo rows) that ci_isolate_toolchain_script already emitted. What this PR added was a
COMPOSITION, not a medium.
The marker is still right, because the carrier genuinely is orch-emitted foreign-executor
shell pending the same shell-to-intent migration. WHAT THE MARKER DOES NOT CLOSE, named
in its own trigger rather than left for a reader to trip over: ci_isolate_toolchain_script
and ci_pin_rustup_default_script are the same class and were unmarked before this PR too.
Marking only the extracted subset leaves an inconsistency this change neither creates nor
closes, and the trigger says all three dissolve together.
ONE SELF-INFLICTED DEFECT ON THE WAY, caught by the compiler rather than by review: moving
the string dropped its brace escapes, and ${CARGO_HOME:-} is interpolation syntax in .dag.
The module index refused with the exact remedy ("for a literal brace write \{"), which is
what a fail-closed diagnostic is for.
Regenerated from the emitter against a rebuilt binary. fleet-converge.yml again showed
only the unrelated secret-hygiene drift it carries on main -- ZERO toolchain/isolate/
rustup/MAKEFLAGS/sccache lines, measured -- and was reverted again.
Capability closure re-run: RC=0, 4/4 PASS including the fleet-converge control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 26, 2026
gunbai-bot
Bot
changed the title
Contributor
Author
|
Closing as overtaken by the now-authoritative #9275 lane. Fleet-desired coverage and derived classification are already present there; the one unique hostile legacy-writer discriminator has been extracted against #9275's current head as commit 06c6f02 and handed to its exclusive owner. This thread remains as the implementation/test receipt. — sent from jolly-koi-76 |
Contributor
Author
|
Forwarding correction: commit 06c6f02 was withdrawn after review showed its readers and hostile writer used disjoint paths, so it could never go red. The corrected two-arm discriminator is now PR #9341: shared-path readers must fail under the antagonist, isolated readers must pass, and deleting the isolation binding makes the private arm alias the legacy fixture and refuse. — sent from jolly-koi-76 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the private-toolchain-home construction from #9275 across every generated self-hosted workflow, including
fleet-desired. Job classification is derived from each modeled workflow's authoritativeJob.steps: every job isPrivateMutableHomeorImmutableToolchain, and generation refuses by job id if anysetup-rust-toolchainwriter is not preceded by privateHOME,CARGO_HOME, andRUSTUP_HOME.The emitted workflows also run a hostile discriminator: two readers execute private rustup dispatchers while a third actor repeatedly unlinks and reinstalls a dispatcher in a legacy shared-home fixture. The private readers must finish unaffected. This distinguishes directory isolation from #9290's superseded cooperation protocol, which depended on every writer honoring PATH/snapshot coordination.
Why
setup-rust-toolchainguards installation withcommand -v rustup. Reused self-hosted runners have rustup on disk, but non-login shells do not inherit the profile PATH edit, so the guard falls through and jobs repeatedly rewrite one shared dispatcher. The observed failures include both ETXTBSY (write through a busy executable) and ENOENT (unlink/replace after--versionsucceeded). Private mutable homes cover both because they remove sharing rather than matching one write pattern.The homes live under
RUNNER_TEMP, which GitHub scopes and clears per job. A fresh runner therefore installs successfully into empty private homes, and reruns receive a fresh mutable home without a parallel run-id/attempt/job naming scheme.The #9290 PATH-prepend and before/after inode comparison are absent; #9290 is closed as superseded.
Test plan
cargo build --release -p v1-compiler --bin gunbc— PASSdag/test/claim/toolchain_workflow_isolation_witness_test.dag— PASS; full native emission was stopped after an unrelated long-running emit phaseclaim_batch --hermetic --source-root dag --source-root src/v2 --entry dag/test/claim/toolchain_workflow_isolation_witness_test.dag --functions live_witness_jobs_derive_as_isolated_from_the_workflow_authority,live_fleet_desired_jobs_derive_as_isolated_from_the_workflow_authority,live_fleet_converge_jobs_derive_as_isolated_from_the_workflow_authority,generated_workflow_yaml_is_checked_against_the_derived_classification,hostile_legacy_writer_before_isolation_is_discriminated,the_same_writer_after_isolation_is_admitted— 6/6 PASSgunbc run --source-root dag --source-root src/v2 --entry dag/tools/generated_artifact_gate.dag --function main_wet— PASS; committed workflow projections regeneratedgit diff --check— PASScargo fmt --all --check— PASS