Skip to content

SHELL-DAG: migrate six retained_foreign call sites to typed operations - #8621

Merged
briansrls merged 2 commits into
mainfrom
session/quick-stag-61
Aug 20, 2026
Merged

briansrls merged 2 commits into
mainfrom
session/quick-stag-61

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Dissolves six confirmed retained_foreign (shell-string-concat) call sites onto typed shell/extdeps operations (dashboard node adhoc-a84c2852-5b9), per §3/§5: construction over string-building, reusing existing single authorities rather than forking.

Sites migrated

  1. dag/gunbc/package_delivery.dag observe_protocol_schema_identity — codex app-server generate-json-schema -o invocation → new codex_app_server.Cli.GenerateJsonSchema (added to extdeps.llm.codex_app_server, which already cites this executable/protocol — not a new product surface).
  2. Same function — find ... -type f -name '*.json' | sort listing → new shell.Find.FilesByNameSorted (extdeps/shell.dag), following the existing FilesAndSymlinksWithMode positional-parameter precedent so no path/glob can escape a quote.
  3. finish_bisect_generate_protocol_schema_dir — duplicate of Add SVG viz, test helpers, and makegen scaffold #1, same new op.
  4. relocate_build_root_to_unique_publishing — rmdir '<staged>' && mv -T '<build_root>' '<staged>' → shell.Remove.EmptyDirectory + shell.Move.NoReplaceDirectory (two new operations on the existing shell.Remove/shell.Move services).
  5. rename_publishing_noreplace — mv -T → shell.Move.NoReplaceDirectory.
  6. dag/tools/interpreter_dispatch_bijection_real_roster_transport.dag real_roster_cargo_run — multi-line bash (set -euo pipefail; cd; export; two cargo test --ignored --exact invocations) → new cargo.Build.TestWithCwdEnv (extdeps/rust/cargo_build.dag), composing env -C <workdir> <VAR=val...> <cargo> test <args> as one argv vector, called twice sequentially from real_roster_cargo_test_run with explicit caller-side short-circuit (cargo_outcome_holds) preserving fail-fast semantics without shell chaining.

No sites were left as retained_foreign — all six are fully migrated.

Architectural calls made (documented per task brief)

  • RunArgv vs narrow op: used a narrow, domain-specific codex_app_server.Cli.GenerateJsonSchema instead of shell.Exec.RunArgv + ProcessArgvExpansion/CliSurface, which has zero production consumers today and is disproportionate machinery for a one-off fixed 4-token argv — matches the narrow-fixed-argv-per-tool pattern used everywhere else.
  • rmdir vs rm -rf: shell.Remove.EmptyDirectory uses rmdir (refuses when non-empty), preserving the original script's safety property rather than silently loosening it.
  • Site 4 control flow: computed a plain Bool in each branch rather than unifying two different services' anonymous {success: Bool} output records across an if/else, to avoid a speculative cross-type merge.
  • Site 6 cwd+env: reused the env -C {workdir} argv-splice precedent already in cargo_build.dag (BuildInheritEnv, BuildManifest), adding a List<String> env-var-words parameter following the extra_args precedent — no new grammar/transport machinery needed.

Follow-up: state-space conflation in site 4 (review on this PR, 69f9364)

relocate_build_root_to_unique_publishing originally collapsed two distinct typed outcomes — shell.Remove.EmptyDirectory failing (placeholder not empty: something else wrote into a name this call believed was exclusively its own) vs. shell.Move.NoReplaceDirectory failing (ordinary move refusal against a confirmed-empty, confirmed-ours placeholder) — into one Bool at the binding. Under that Bool, the not-empty case ran shell.Remove.RecursiveForce on the placeholder, i.e. deleted a directory whose contents this call could not account for. Nobody flagged that as a defect during initial review; it fell out of the decomposition once the two outcomes were split into UniquePublishingRelocateOutcome { Ready | PlaceholderNotEmpty | Refused }. The PlaceholderNotEmpty arm now skips the cleanup; RecursiveForce only runs on the move-refusal arm, where emptiness is established.

Caveat on the compile-clean claim below: compile_clean_diagnostic_is_advisory admits WhereRefinementUnenforced onto the non-blocking allowlist for any non-literal refined value, and where refinements are unchecked at runtime for casts whose kernel is String (which is NonEmptyStr's case) — so "0 blocking errors" is not evidence that any non-literal as NonEmptyStr/as FilePath cast in this diff actually holds. The two new as NonEmptyStr casts added here are on string literals, not non-literal expressions, so they fall outside that gap; no claim in this PR rests on an unchecked refinement.

Incidental fixes

  • Two DESIGN §4c violations: explanatory // comments were nested inside a service { } body (module-item grain only is admitted) — relocated into top-level data ..._note declarations in shell.dag and cargo_build.dag, matching the files' existing convention.
  • An interpolation hazard: the new cwd/env note's prose contained a literal {workdir}, which the compiler parsed as an unbound interpolation reference (undefined variable 'workdir') — reworded to plain prose.
  • A missing NonEmptyStr import in cargo_build.dag (advisory-level, fixed for correctness).

Verification

  • Entry-scoped gunbc compile on both touched leaf files is 0 blocking errors on both:
    • dag/gunbc/package_delivery.dag: 395 advisory diagnostics, all the file's pre-existing where-refinement unenforced pattern on non-literal FilePath/NonEmptyStr casts, unrelated to this change.
    • dag/tools/interpreter_dispatch_bijection_real_roster_transport.dag: 14 advisory diagnostics, same pre-existing pattern plus two on my own new as NonEmptyStr casts, consistent with the file's existing style.
  • gunbc.retained_shell_script's retained_foreign carries no fixed caller-count assertion, and none of these six sites appear in host_language_transport_script's wall_residue_live_test.dag roster — no lens update required.
  • dag/test/claim/cargo_execution_placement_witness_test.dag (hermetic controls covering the placement-decision logic site 6 also uses) passes — ran w_local_runner_admits_its_observed_path via gunbc run --claim-run.
  • The interpreter_dispatch_bijection_real_roster witness (real git clone + cold cargo build, ~692s wall) was not executed in this session — its own test file documents it as "a cadence job, not a session job" requiring a runner with ~11 GiB and a local cargo. gunbc run --dry-run on the unmodified real_roster_cargo_decision confirms it hits the expected hermetic-mock wall at shell.Which.Check (pre-existing behavior, unrelated to this change).
  • Full 3-root compile-clean gate not run locally (documented OOM on remote dispatch, per project memory) — relying on CI.

Test plan

  • CI compile-clean gate green
  • Reviewer spot-checks the six diffs against the six cited sites

Brian Searls and others added 2 commits August 20, 2026 02:25
Dissolves six confirmed retained_foreign (shell-string-concat) call sites
in favor of typed shell/extdeps operations, per §3/§5: construction over
string-building, single authority reused where it exists.

Sites migrated (all in dag/gunbc/package_delivery.dag unless noted):
1. observe_protocol_schema_identity — codex app-server generate-json-schema
   invocation -> new codex_app_server.Cli.GenerateJsonSchema (added to the
   existing extdeps.llm.codex_app_server module, which already cites this
   executable/protocol).
2. observe_protocol_schema_identity — `find ... | sort` listing -> new
   shell.Find.FilesByNameSorted (extdeps/shell.dag), following the existing
   FilesAndSymlinksWithMode positional-parameter precedent so no path/glob
   can escape a quote.
3. finish_bisect_generate_protocol_schema_dir — duplicate of #1, same new op.
4. relocate_build_root_to_unique_publishing — `rmdir && mv -T` ->
   shell.Remove.EmptyDirectory + shell.Move.NoReplaceDirectory (both new
   operations on the existing shell.Remove/shell.Move services).
5. rename_publishing_noreplace — `mv -T` -> shell.Move.NoReplaceDirectory.
6. dag/tools/interpreter_dispatch_bijection_real_roster_transport.dag,
   real_roster_cargo_run — multi-line bash (set -euo pipefail; cd; export;
   two `cargo test --ignored --exact` invocations) -> new
   cargo.Build.TestWithCwdEnv (extdeps/rust/cargo_build.dag), composing
   `env -C <workdir> <VAR=val...> <cargo> test <args>` as one argv vector,
   called twice sequentially from real_roster_cargo_test_run with explicit
   caller-side short-circuit (cargo_outcome_holds) preserving the fail-fast
   semantics without shell chaining.

Architectural calls made along the way (documented per task brief):
- Used a narrow, domain-specific op (codex_app_server.Cli.GenerateJsonSchema)
  instead of shell.Exec.RunArgv + ProcessArgvExpansion/CliSurface, which has
  zero production consumers today and is disproportionate machinery for a
  one-off fixed 4-token argv.
- shell.Remove.EmptyDirectory uses `rmdir` (refuses if non-empty) rather than
  `rm -rf`, preserving the original script's safety property.
- relocate_build_root_to_unique_publishing computes a plain Bool in each
  branch rather than unifying two different services' anonymous output
  records across an if/else, to avoid a speculative cross-type merge.

Also fixes two DESIGN §4c violations surfaced while landing the new
operations: explanatory `//` comments were nested inside a `service { }`
body (module-item grain only) — relocated into top-level `data ..._note`
declarations in shell.dag and cargo_build.dag, matching the file's existing
convention. And fixes an accidental interpolation hazard: the new cwd/env
note's prose contained a literal `{workdir}`, which the compiler parsed as
an unbound interpolation reference — reworded to plain prose.

Verification: entry-scoped `gunbc compile` on both touched leaf files
(package_delivery.dag, interpreter_dispatch_bijection_real_roster_transport.dag)
is 0 blocking errors on both (395 / 14 pre-existing-pattern advisories,
where-refinement-unenforced on non-literal FilePath/NonEmptyStr casts,
unrelated to this change). gunbc.retained_shell_script's retained_foreign
carries no fixed caller-count assertion, and none of these six sites appear
in the host_language_transport_script wall_residue_live_test.dag roster, so
no lens update is required. The interpreter_dispatch_bijection_real_roster
witness (real git clone + cold cargo build, ~692s wall) was not executed in
this session — its own test file documents it as "a cadence job, not a
session job" requiring a runner with ~11 GiB and a local cargo; the
hermetic cargo_execution_placement_witness_test.dag controls (which cover
the placement-decision logic this site also uses) pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WaiMx97tLCspkMg9rDkfjS
…ation

review on #8621 (eager-crane-282): the mktemp placeholder being non-empty
(an exclusivity violation -- something else wrote into a name this call
believed was exclusively its own) and an ordinary move refusal (destination
exists, cross-device, permissions) against a confirmed-empty placeholder
were collapsed into one Bool at the binding, even though the new typed
operations (shell.Remove.EmptyDirectory, shell.Move.NoReplaceDirectory)
had just created the distinction.

Introduces UniquePublishingRelocateOutcome (Ready | PlaceholderNotEmpty |
Refused) so the caller sees the two failure states separately and reports
a distinct cause string for each. The not-empty arm no longer runs the
recursive-force cleanup, since rm -rf on a placeholder this call does not
own would destroy whatever the other writer put there; cleanup stays on
the move-refusal arm where the placeholder is confirmed empty and ours.

rename_publishing_noreplace is left untouched (pre-existing, out of scope
per the review).
@briansrls
briansrls merged commit d106ec3 into main Aug 20, 2026
1 check passed
@briansrls
briansrls deleted the session/quick-stag-61 branch August 20, 2026 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant