Skip to content

Phase 3b: emit_host rust/python smokes -> typed argv (ratchet 20->19) - #6478

Merged
briansrls merged 42 commits into
mainfrom
session/witty-ibex-317
Jul 11, 2026
Merged

briansrls merged 42 commits into
mainfrom
session/witty-ibex-317

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Jul 11, 2026 •

Copy link
Copy Markdown
Contributor

Fourth increment of the ShellProgram-sidecar dissolution (after #6457 workdir-by-construction, #6461 compositional serializer keystone, #6467 emit(Retry, Bash) slice 0).

What

run_rust_smoke / run_python_smoke in dag/tools/emit_host_transport.dag are runtime-present sites (gunbc invokes every step), so per the shell-emission-model rule they become typed argv service operations — no bash serialized or executed:

  • shell.Mktemp.Dir / shell.Mkdir.Parents / shell.Remove.RecursiveForce (existing ops)
  • shell.Write.File (new): POSIX tee with the transport's first-class stdin: binding (the cron.Replace precedent) — file content never rides through a heredoc
  • cargo.Build.BuildManifest (new): env -C {workdir} CARGO_TARGET_DIR={target_dir} cargo build --manifest-path {manifest_path} — workdir-by-construction, mid-token env interpolation
  • extdeps.python (new module): python.Interpreter.RunFile, cited to docs.python.org cmdline
  • gunbc.WitnessBin.Run for the fixture binary (reused — the fixture IS a witness bin; a fresh invoke op would be a §3 nickname)

Stronger assertion

The old wc -c | tr -d ' ' byte-count check strengthens to content equality: the expected string is built in-substrate — repeat_string(from_code_point(0), byte_size_count(stdout_byte_count)) — from the same count authority the fixtures use, and the 5-NUL fixture bytes survive the transport's trailing-whitespace trim. Discriminating RED control (dag/test/claim/emit_host_typed_smoke_test.dag): a count-identical 5-space fixture must compare unequal — exactly the class the count check was blind to. Plus golden shape pins (non-empty, length 5) so the golden cannot silently degenerate to "".

Cleanup improvement: rm -rf now runs unconditionally; the old set -e script leaked the tempdir on mid-script failure.

Ratchet

dag/tools/emit_host_transport.dag drops its extdeps.languages.bash.program import → bash_program_importer_count_baseline 20 → 19.

Proven by execution

  • run_rust_smoke → true, run_python_smoke → true (typed path, content equality)
  • composed emit_host_gate main → ExitSuccess (all 5 smoke rows incl. unchanged go/ts/node arms)
  • all 3 new witnesses green; importer-count ratchet green at 19 with both planted-RED controls green

🤖 Generated with Claude Code

briansrls and others added 30 commits July 10, 2026 20:20
…wall on the whole-tree emit path

Fold-family productions extended so the fold no longer refuses word
Concat/CmdSubst or stmt WithRedir (all four Redir variants): new
concat_parts/word-compound/with_redir production families, lex tokens,
kind-tag emit transforms, and recursive bundle arms. Byte-identity vs
serialize_bash proven by execution: five depth-2 oracle tests plus the
depth-4 assign_root_stmt manual probe (ROOT=$('git' 'rev-parse'
'--show-toplevel' 2>/dev/null || 'pwd') byte-exact). The delegated
fail-closed RED control repoints from WithRedir (now native) to Heredoc
(still delegated) so the boundary guard stays discriminating.

Measured cost wall, declared on-carrier (bash_program_emit_cost_wall_note):
whole-tree backward row-selection is ~alternatives^depth (1s flat stmt,
64s for the single depth-4 stmt, DNF >8min for the full witness_bin
program) because formal_production_unique_lhs_exact_match deep-validates
every candidate per level and the descent re-validates each level again.
Real-program oracles therefore stay MANUAL probes, not test fns (a
discovery-run test would hang the local floor); the probe note on the
test carrier names the dissolution triggers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t runtime)

Phase-3 shape for the shared witness invocation, per the shell-emission-model
rule that runtime-present sites are argv invocations wearing bash syntax.
run_witness_bin_program and run_gunbc_claims no longer build a ShellProgram,
serialize it, and shell out - they realize the same intent as typed service
operations:

- git.Core.Toplevel (new op beside CurrentBranch/HeadCommit) resolves the
  repo root the bash path derived with a cd/rev-parse dance
- shell.Test.IsExecutable (new, POSIX test -x) replaces the test-not-
  executable guard and the post-build artifact verification
- cargo.Build.BuildInheritEnv (new) is the build-if-absent arm; it inherits
  process env exactly as the old sh -c cargo build did (cargo.Build.Build's
  env Map default is both a behavior change and unrecordable by the claim-
  run fixture writer, which fail-closes on Map values)
- gunbc.WitnessBin.Run (new) invokes the witness binary by absolute path
  with a spliced args list

Proven by execution: six op probes incl. argv[0] interpolation, list splice,
and negative RED controls (typed_witness_invocation_test); pattern A
end-to-end (v1_dag_parse witness, 0.43s wall vs bash round-trip); the
run_gunbc_claims rider (extdeps_external_authority uri witnesses); and the
flip-level RED control (unbuildable bin -> false, refused not fabricated).
The ShellProgram builders stay as the canonical fixtures for the bash
row-emit byte-identity oracles; importer ratchet unchanged at 27.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rs 27 -> 20

Second typed-argv tranche, riding the host_prelude flip:

- dag_compile_clean_transport: the four perturb receipts and the gate are
  one shared typed composition (Mktemp.Dir, Filesystem.Write for the
  perturb module - the heredoc dissolves, WitnessBin.Run on gunbc,
  Remove.RecursiveForce), parameterized by expected compile outcome. The
  ShellProgram builders, their serialize-shape witnesses, the marker rows,
  and both RealizationDispatch scaffold rows dissolve: with typed args the
  invocation facts are the construction, so the spelling-drift class those
  witnesses validated is unwritable (DESIGN 5) - the witness test file and
  the duplicate_computation compile-clean subject go with them (noted on
  the carrier; the lens keeps its emit_determinism subjects).
- dag_compile_clean_seam: the RawLine pile (cp -r, sed, $VAR-captured
  compiles, grep -c, echo SEAM_RESULT) becomes typed ops end to end - the
  sed is Filesystem.Read + replace + Write, the three compiles return
  typed results, and the SEAM_RESULT string protocol plus its parser
  dissolve; the receipt is a typed record and the synthetic RED controls
  construct it directly.
- dag_compile_clean_shard / shard_totality: typed staging compile and a
  Find.Files op (the '| sort' was cosmetic - the totality algebra is
  set-membership both ways).
- source_root_ingest_transport, compiler_closure_ingest_transport,
  self_host_realized_comparison_gate: typed compositions over the shared
  helpers (run_witness_bin_program / run_gunbc_claims).
- New extdeps ops: shell.Mktemp.Dir/DirWithTemplate, shell.Mkdir.Parents,
  shell.Remove.RecursiveForce, shell.Copy.File/Recursive, shell.Find.Files,
  shell.Test.IsNonEmpty (review fix: restores the test -s zero-byte guard
  the bash emit_verifications carried).
- Importer ratchet re-frozen 27 -> 20; seven stale exception-roster rows
  removed; ratchet + planted RED controls + clean-tree + roster-soundness
  all green.

Verified by execution: seam algebra + seam live probe, totality live,
shard exemplar compile, cross-tree green + optional-skew red receipts,
ingest gate (incl. a live cold-build refusal: remote-routed cargo produced
no local artifact and the non-empty/executable verification failed closed),
closure ingest gate, pattern A, full co-rooted corpus compile x2 clean.

Deferred with in-tree notes: floor_diff_observe (affected-set CI's diff
observer - avoid colliding with that lane), emit_determinism_transport
(last duplicate_computation ShellProgram subject), emit_host_transport
(go/node arms are genuine toolchain-bootstrap windows; rust arm needs an
env-pinning build op), bmc/ubuntu/srv3 provisioning tails.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… regression)

The dissolved bash prelude carried cd "$ROOT" - the construction that made
relative --source-root values and cargo build unambiguous. The typed flip
dropped it: WitnessBin.Run, BuildInheritEnv, and Find.Files ran in inherited
process cwd. Fix: each op now takes a required workdir input spelled as GNU
env -C (coreutils env(1)), so a call that does not say where it runs is a
type error, not a latent host-state dependency. All call sites thread
git.Core.Toplevel(). New discriminating probes: pwd under workdir=/tmp and
workdir=toplevel; ingest gate re-run green from a repo subdirectory (the
exact failure class flagged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
briansrls and others added 9 commits July 11, 2026 00:17
…, Bash); on_exhausted wired

Precondition first (plan-doc FLAG): Retry.on_exhausted was a declared-but-unread
field - the emitter matched it as _ and the row baked exhaustion as fixed
tokens. Wired end to end: seg_h split into four fixed segments with three
shared-binding occurrences of an exhausted spelling, the emitter emits the
on_exhausted Pipeline and threads it through realize_retry (arity 8), and a
new teeth test reds the bytes when on_exhausted changes.

Cutover: gunbc.ci_spec.ci_cargo_eagain_retry_core no longer concatenates bash -
it builds the Retry intent (body run, two LogMatches escalation levels,
exhausted pipeline: the policy facts ci_spec owns) and renders through
orch_emit_step over the bash grammar rows. First dag-tree module importing
v2.* (gunbc ci loads both roots; workflow->compiler is the allowed layering
direction). The String seam's refusal arm emits a loud poison marker that reds
both the drift gate and the yaml parse gate - refuse, never widen; noted on
the carrier. The hand-authored blob moved verbatim to the test module as an
emitter-independent golden; the intent fixture module now delegates to the
production intent so tests exercise the value production emits.

Receipts: 5 retry emit tests green (byte-identity vs golden, production seam
vs golden, env/exhausted teeth); ci_yaml_parse_witness (the CI drift+parse
gate) ExitSuccess - committed ci.yml byte-identical through the new path;
full corpus compile 0 diagnostics. Attribution: the local 'gunbc ci' top-level
failure (dash rejecting set -o pipefail) reproduces on clean origin/main -
pre-existing, not this change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
run_rust_smoke/run_python_smoke are runtime-present sites: replace the
ShellProgram builders (mktemp/mkdir/heredoc/EnvPrefixed/wc -c pipeline)
with typed service operations - shell.Mktemp/Mkdir/Write/Remove,
cargo.Build.BuildManifest (env -C workdir + CARGO_TARGET_DIR),
python.Interpreter.RunFile (new cited extdeps.python),
gunbc.WitnessBin.Run for the fixture binary.

shell.Write.File is the one net-new capability: POSIX tee with the
transport's first-class stdin binding (cron.Replace precedent), so file
content never rides through a heredoc.

The wc -c byte-count assertion strengthens to CONTENT equality: the
expected string is built in-substrate (repeat_string of
from_code_point(0)) from the same stdout_byte_count authority, and the
5-NUL fixture bytes survive the transport's trailing-whitespace trim.
Discriminating RED control: a count-identical 5-space fixture must
compare unequal - the exact class the old count check was blind to.
Cleanup now runs unconditionally (set -e leaked the tempdir on
mid-script failure).

Proven by execution: both smokes true, composed emit_host_gate
ExitSuccess (all 5 rows), importer-count ratchet green at 19 with both
planted-RED controls.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gunbai-bot gunbai-bot Bot changed the title ShellProgram -> DAG (Wave 1-3, on 1) Phase 3b: emit_host rust/python smokes -> typed argv (ratchet 20->19) Jul 11, 2026
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review July 11, 2026 03:50
@gunbai-bot

gunbai-bot Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Addressed cursor's REQUEST_CHANGES in 61f6932: verified valid — Filesystem.Write (extdeps.filesystem.filesystem_io) is the established write authority and the phase-3a sibling migration (dag_compile_clean_transport.dag) already routes heredoc bodies through it. The tee-based shell.Write.File is dropped (shell.dag reverts to its pre-PR shape) and both fixture writes plus the RED-control write now use Filesystem.Write. Re-proven by execution post-fix: both smokes true, count-identical RED control true, composed emit_host_gate ExitSuccess. — sent from witty-ibex-317

@gunbai-bot

gunbai-bot Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

CI-failing investigation: the red on db7fd7a is the 60-minute floor timeout, not a defect in this diff. The run reached the timeout at batch-2 with no algebra.dag panic and no witness failure — this branch's base predates the #6459/#6484 cli_run regression (fixed in main by #6493), so it never hit that panic; it hits only the systemic floor-cost wall that's timing out every broad-corpus run today (the #6469 perf incident; step-timeout treadmill exhausted per the operator's merge-past ruling).

This diff is the sanctioned merge-past case:

  • Local receipts, green by execution: run_rust_smoke → true, run_python_smoke → true (typed path, content equality), composed emit_host_gate main → ExitSuccess (all 5 rows), importer-count ratchet green at 19 with both planted-RED controls, the count-identical 5-space RED control → true.
  • No main regression: the diff touches only extdeps/python, extdeps/rust/cargo_build, emit_host_transport, emit_host_typed_smoke_test, and the realization_vocabulary_containment baseline — none of them cli_run.rs, ci_spec, or any floor/timeout authority the recent main merges changed. ci_spec is untouched, so the floor-timeout value cannot regress. mergeable=CLEAN.
  • Reviews: 4 dashboard approvals (claude ×2, cursor ×2), zero findings, no REQUEST_CHANGES.

Not churning the head to chase a green floor run (that resets the 4 approvals to gamble on a wall that's timed out all day). Recommending merge-past on these receipts per the standing policy. — sent from witty-ibex-317

@briansrls
briansrls merged commit 9ae9931 into main Jul 11, 2026
2 of 3 checks passed
@briansrls
briansrls deleted the session/witty-ibex-317 branch July 11, 2026 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant