Skip to content

shell→dag: exhaustive per-instance census (§4) — the tracked punch-list - #7065

Merged
briansrls merged 7 commits into
mainfrom
census-shell-to-dag
Jul 22, 2026
Merged

briansrls merged 7 commits into
mainfrom
census-shell-to-dag

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

What

Adds §4 — the exhaustive per-instance census to docs/plans/shell-to-dag-residual-census-and-arc-completion.md: the complete, current (@ origin/main) list of every shell-string-construction site, so every instance is tracked to closure. Docs-only (+115 lines).

Why now

A recent wave (#7004, #7006, and the closed srv* cluster) counted relocations as migrations — it moved ShellCommand{script: concat(...)} out of the intent file into a new *_script.dag file and realized a typed variant by calling that concat and stuffing the raw string back into ShellOnHost{script}. Net raw-shell-string construction: unchanged; new §3 coproduct-nickname debt added. This census exists so no site is counted done until the concat is gone, not homed elsewhere.

What's in it

  • 4.0 Completeness method — the P1–P8 grep patterns that partition every construction/carrier form (proves exhaustive, not sampled).
  • 4.A–4.H — every instance with file:line, classified by ACTION:
    • A relocation regression (fake-migrated *_script.dag) → dissolve to the already-modeled extdeps op; delete the concat AND the nickname variant
    • B direct ShellCommand{script} still in intent
    • C runtime-present shell.Exec.Run with a string/_script body
    • D ssh.Session.Exec command-string vs typed ExecArgv
    • E foreign-executor/bootstrap (legit emit, bounded roster)
    • F bottom transport + the porous TransportScript brand (the Phase-3 construction wall)
    • G bash-AST emit vocab (emit-internal, already confined Phase 4: lock shell→intent via generalized realization_vocabulary_containment #6854)
    • H oracle/test retainers
  • Scope in flight — what sleek-crab-621 (in-mission A1/A2) and keen-deer C5 Wave C5: typed-argv exec at realization edge (deploy probes, #5828 slice) #6946 (A3 access) own, and what's deferred (srv* A5, multi-op A4).

The key finding surfaced

The striking fact: almost everything hand-concatenated is already modeled in extdeps/ (systemctl.ShowProperty etc., access.PosixEffectivePrincipal, curl, sccache, git argv). The fix for most rows is "call the op." Only the hostname sites need a new extdeps.os.hostname op modeled first.

Each row is discharged by deletion of the concat (green-by-execution + an injection-RED), never relocation. §4 folds into the host_language_transport_script lens going live — the construction wall that makes new sites unwritable.

🤖 Generated with Claude Code

briansrls and others added 3 commits July 22, 2026 14:22
§1–§3 recorded direction at #6507; this adds §4, the complete current
per-instance list grounded at origin/main so every shell-string-construction
site is tracked to closure — motivated by the relocation regression (#7004,
#7006, closed srv* cluster) that counted concat-relocations as migrations.

Completeness method (P1–P8 grep patterns) partitions every construction/carrier
form into action-classes:
  A. relocation regression (fake-migrated *_script.dag) → dissolve to the
     already-modeled extdeps op; delete the concat AND the nickname variant
  B. direct ShellCommand{script} still in intent
  C. runtime-present shell.Exec.Run with a string/_script body
  D. ssh command-string vs typed ExecArgv
  E. foreign-executor/bootstrap (legit emit, bounded roster)
  F. bottom transport + the porous TransportScript brand (Phase-3 wall)
  G. bash-AST emit vocab (emit-internal, confined)
  H. oracle/test retainers

Each row is discharged by DELETION of the concat (green-by-execution +
injection-RED), never relocation. §4 dissolves into the
host_language_transport_script lens going live (the construction wall).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… 41399)

review 41399 (cursor) correctly caught that §4 was mis-grounded on a divergent
worktree (still pre-#7006, so it showed direct ShellCommand{script} sites that
no longer exist on main) and that line-numbered rows drift (DESIGN §6). Fixes:

- Re-anchor every row on file + symbol NAME, not line numbers (drift-proof).
- §4.B: there are 0 live direct ShellCommand construction sites on origin/main —
  all became nickname variants (now correctly in §4.A). §4.B is residue only
  (host_effect_plan placeholder + host_effect.dag type def). Removed the wrong
  fleet_show/host_identity ShellCommand rows.
- §4.A: correct construction sites are the nickname variants
  (SystemdUnitMemory*Read in fleet_show, HostIdentityShortHostnameRead, etc.)
  plus the INLINE builders P5 had missed (fleet_runner_unit_property_read_script,
  fleet_runner_width_count_read_script, host_converge_slice1 *_script fns).
- §4.0 P5: broadened to inline `fn … -> String` builders outside *_script.dag.
- §4.D: corrected/completed the ssh_session_exec command-string inventory
  (test -x, command -v ×2, id -u/-g, ssh_session_exec_script).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in 71843de — all four findings in review 41399 were valid. Root cause: §4 was grep-grounded on a divergent local worktree that still held the pre-#7006 ShellCommand{script} sites, so the rows drifted from origin/main.

  • §4.B misclassification — correct: there are 0 live direct ShellCommand{script} construction sites on main; all became nickname variants. §4.B is now residue-only (the host_effect_plan {script: ""} placeholder + the host_effect.dag type def); the fleet_show/host_identity rows moved to §4.A as their real nickname-variant construction sites (SystemdUnitMemoryPropertiesRead/SystemdUnitMemoryMaxRead, HostIdentityShortHostnameRead).
  • P5 under-scope — broadened to inline fn … -> String builders outside the *_script.dag glob, and §4.A now inventories them by name (fleet_runner_unit_property_read_script, fleet_runner_width_count_read_script, host_converge_slice1's …_memory_max_read/set/enumerate_units_script).
  • §4.D line drift — corrected and completed the ssh_session_exec command-string inventory (test -x, command -v ×2, id -u/-g, ssh_session_exec_script).

Deeper fix per your DESIGN §6 point: every row is now keyed on file + symbol name, not line numbers — the anchor is stable across line moves, so the ledger can't drift the way it just did. The section is explicitly an interim tracker that dissolves into the enforcement lens (§4.F), which is the drift-proof authority. — sent from calm-ferret-849

briansrls and others added 3 commits July 22, 2026 14:46
…eipts

Answers "for every instance that wants to call a bash script, what is the path?"
with filenames + receipts, grounded @ 78f43c3.

Headline: the whole arc needs only ~4 NEW typed ops —
  - extdeps.os.hostname (Read/Set)           [new file]
  - systemd.Systemctl.ListUnits              [add to existing systemctl.dag]
  - extdeps.os.id (uid/gid/user)             [new file]
  - ssh.Session.ExecArgv                     [add; in flight C5 #6946]
Everything else CALLS AN OP THAT ALREADY EXISTS (receipts in §5.B): Clock.Now
for `date`, shell.Which.Check for `command -v`, shell.Find.IsExecutable for
`test -x`, git.Core.Show for `git show`, systemd.Systemctl.ShowProperty/
SetProperty for systemctl, Filesystem.Write for file writes, WitnessBin.Run/
cargo.Build for the witness transports.

Four paths per instance: 5.B call-existing-op · 5.A add-one-op-then-call ·
5.C emit-for-foreign-executor (bounded roster) · 5.D deferred (srv*/C5/nbd).
5.E names the enabler that must come first — brand TransportScript + typed-argv
realization edge so the string sink is unreachable and 5.B can't be faked by
argv_join (the sleek-crab #7064 failure mode).

Receipts: op inventory (present) + new-ops (absent) both verified @ 78f43c3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
review 41467 (cursor) correctly caught that §4.B's "0 direct ShellCommand sites"
was false: live_deploy/readiness.dag constructs ShellCommand{script:...} in the
MULTILINE form (`ShellCommand {` then `script:` next line), which my single-line
P1 pattern could not match. Fixes:

- §4.0: P1 is now multiline (`ShellCommand\s*\{\s*script:`); added an explicit
  note that the search must be slurped/multiline, not line-at-a-time — that gap
  is what hid these sites. Disambiguated match-arms (host_effect_realize/
  fleet_converge_cli/ci_deploy_access_observe `ShellCommand{script:_} =>`) as
  destructuring, not construction.
- §4.B: NOT zero — 2 live direct sites in readiness.dag
  (live_deploy_healthz_probe_script_for_port, live_deploy_unit_diagnosis_command),
  both runtime-present via host_effect_apply_gated on srv1 LocalShell, with their
  intent.dag builders and dissolve-to paths. Flagged the `| tail` + defensive
  `exit 0` in unit_diagnosis as a §5 absorbing fallback (intent.dag's own comment
  admits it masks systemctl's nonzero).
- §5.A: add systemd.Systemctl.Status (models exit-3-for-dead-unit, retires the
  tail/exit-0 fallback). §5.B: healthz curl → http.Client.Get (already exists,
  no new op).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in 8cc98cb — review 41467 was correct, and it's a real completeness hole. Root cause: my P1 pattern (ShellCommand *{ *script:) was single-line, so it could not match live_deploy/readiness.dag's multiline construction (ShellCommand { then script: on the next line). A census claiming grep-completeness with a line-at-a-time pattern isn't actually complete — that's the honest failure.

  • §4.B is no longer 'zero' — it now lists the two live direct sites (live_deploy_healthz_probe_script_for_port, live_deploy_unit_diagnosis_command), both runtime-present via host_effect_apply_gated on srv1 LocalShell, with their intent.dag builders and dissolve-to paths.
  • P1 is now multiline (ShellCommand\s*\{\s*script:) with an explicit note that the search must be slurped/multiline, not line-at-a-time. I re-ran it multiline-aware across dag/gunbc and confirmed readiness.dag is the only construction site the single-line pass missed — the other ShellCommand{script:_} => hits in host_effect_realize/fleet_converge_cli/ci_deploy_access_observe are match-arm destructuring, now noted as such.
  • §5 updated: healthz curl → http.Client.Get (already exists, no new op); unit diagnosis → a new systemd.Systemctl.Status op added to the §5.A finite list. I also flagged the | tail + defensive exit 0 in live_deploy_unit_diagnosis_command as a §5 absorbing fallback — intent.dag's own comment admits it masks systemctl's nonzero, which the typed op models instead.

The deeper takeaway matches your DESIGN §5 point: a prose census is only as complete as its search method, which is exactly why §4 is framed as an interim tracker that dissolves into the enforcement lens (§4.F) — the lens reads the Node tree structurally and can't be fooled by a line-vs-multiline pattern. — sent from calm-ferret-849

@gunbai-bot

gunbai-bot Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Both findings in review 41476 valid, fixed in 0a01303:

Also in this push: a wind-down PR ledger (operator request) replacing 'Scope in flight' — the state of every arc PR (landed / open / closed-srv* / not-started), flagging that #7004 and #7006 merged as 'progress' but are relocations, not true dissolutions. — sent from calm-ferret-849

@briansrls
briansrls merged commit 1ff64bf into main Jul 22, 2026
1 check passed
@briansrls
briansrls deleted the census-shell-to-dag branch July 22, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant