Affected-set Step 3: module-grain frontier equivalence receipts

[gunbai-bot]

Summary

Step 3 of docs/plans/affected-set-precompute-pruning.md, re-scoped to module grain per mandate from gentle-owl-459: node-grain equivalence (whole-tree InferredTree resolve) is unaffordable (whole_tree_resolved_ctx measured at 54+ min without completing) and stays BLOCKED/untouched by this PR.

Proves by execution that the .dag affected-set query (v2.lens.module_graph, using the landed import_closure_live authority from #6210/#6231) computes the SAME module-grain affected decision as the Rust entry_file_touched path in cli_run.rs, on real corpus diffs. Receipts only — no cutover. The live floor skip decision is untouched; Steps 4-5 (consume/delete) remain separately owned.

What

• v2.lens.module_graph: adds entry_affected_by_touched_paths / _excluding — a thin projection expressing "entry E is affected by touched-path set T iff T intersects import_closure_live(E)" (closure includes E). This mirrors the Rust host's touched_file_in_import_closure over import_closure_files_from_graph exactly — both are fed by the same host-realized import_resolution_facts/module_declaration_facts, so this is one fact projected two ways, not a second closure algorithm (DESIGN §2/§3). Also fixes a map_get/Outcome pattern mismatch (map_get returns Accepted/Rejected, not Present/Absent directly) surfaced while exercising these paths live through the interpreter.
• cli_run.rs: new module_grain_affected_equivalence_tests module with three green-by-execution witnesses, plus per-call counters (import_resolution_facts/module_declaration_facts) for the cost receipt.

Receipts (executed locally via ctrl-build, not remote — see note)

Real diff 1 — dag/-only (commit 6edafbb5e29370c0ac791038a1c64e1a4ddbd40d, 4 touched files, all modify-only: healthz-body grammar JSON + structured shell in the v1_dag_parse transport, #6166): 12 sampled entries (spanning both pool roots, near and far module-graph distance from the touched files) — Rust and .dag decisions identical on every entry, including 1 true/true and 11 false/false rows.

Real diff 2 — src/v2/-only (commit bb6e65649c9625d021467b0d7fe33ca7dd086e4f, 6 touched files: bash orchestration-emit dissolve): 13 sampled entries — identical on every entry (8 true/true, 5 false/false).

Discriminating control: orchestration_emit_test.dag reaches bash.dag only transitively via bash_orchestration_emit.dag (asserted precondition: no direct import). Excluding that intermediate's outgoing edges (import_closure_live_excluding) flips the decision true→false — proving the equivalence above is a real discriminator, not a tautology.

Cost numbers (v2_only_real_diff run, 13 entries): wall-clock 35428ms; import_resolution_facts calls (dag side) 14; module_declaration_facts calls (dag side) 14; build_module_graph_facts_live calls (Rust side) 1; peak RSS (VmHWM) 513933312 bytes (~490 MiB).

Test command: cargo test -p v1-compiler --lib module_grain_affected -- --nocapture --test-threads=1 — 3 passed, 0 failed.

Bug found: DESIGN §5 specification-without-execution gap in the pre-existing #6231 witnesses

Exercising v2.lens.module_graph's actual source through the interpreter (not a Rust reimplementation of it) surfaced a real, pre-existing bug: 3 call sites matched map_get's return directly against Present/Absent, but v2.std.collection.map_get<K,V>(m, key) -> Outcome<Optional<V>> (src/v2/std/collection.dag:86) returns an Outcome — Accepted { value: Optional<V>, diagnostics } / Rejected { diagnostics } — wrapping the Optional, not the bare Optional itself. A pattern match against Present { .. } => .. / Absent => .. directly on that return is non-exhaustive (missing Accepted/Rejected) and fails at parse/typecheck (advisory) or panics at runtime the first time the match is actually reached through execution.

The 3 sites, before → after (all in src/v2/lens/module_graph.dag):

• extend_adjacency_for_edge, ~line 90: match map_get(m: module_to_path, key: edge.import_module) { Present { value: imported_path } => ... Absent => acc } → match map_lookup(m: module_to_path, key: edge.import_module) { Present { value: imported_path } => ... Absent => acc }
• extend_adjacency_for_edge, ~line 92 (nested): match map_get(m: acc, key: edge.path) { Present { value: existing } => ... Absent => ... } → match map_lookup(m: acc, key: edge.path) { Present { value: existing } => ... Absent => ... }
• import_closure_bfs_walk, ~line 157: let neighbors = match map_get(m: adjacency, key: importer) { Present { value: paths } => paths Absent => no_paths } → let neighbors = match map_lookup(m: adjacency, key: importer) { Present { value: paths } => paths Absent => no_paths }

Fix: added map_lookup<K, V>(m: Map<K, V>, key: K) -> Optional<V> (src/v2/lens/module_graph.dag:60-65) that unwraps the Outcome correctly (Accepted { value: v, .. } => v, Rejected { .. } => Absent), and repointed all 3 sites to call it instead of matching map_get directly.

Why the pre-existing import_closure_equivalence_tests (#6231) never caught this: those witnesses call the Rust host-realization reimplementation of the closure algorithm, never the .dag source itself through the interpreter — green by construction, not by exercising the code this PR touches. This PR's module_grain_affected_equivalence_tests are the first witnesses to run v2.lens.module_graph's actual .dag source through the interpreter, which is exactly how the bug surfaced. This is DESIGN §5's specification-without-execution trap living inside tests literally named "equivalence" — a green witness here did not mean the .dag source was correct, only that the Rust mirror of it was.

Scope of the pattern: confirmed via grep — the bare Present/Absent-vs-Outcome mismatch against v2.std.collection.map_get exists only at the 3 sites above in v2.lens.module_graph.dag. Every other src/v2/ consumer of this map_get (affected_set.dag:589, frontier_observation.dag:11, v2_effect_io_pure.dag:211, 03_resolve.dag:203,223, generic_instantiation.dag:47, grounding.dag:77, 03_name_resolve.dag:133, 03_ingest.dag:53, and 2 test files) already correctly unwraps Accepted { value: opt, .. } before matching opt against Present/Absent. Separately, dag/std/*.dag, dag/extdeps/**, and several dag/test/claim/*.dag files also match a bare map_get(...) against Present/Absent directly (~15 sites) — but those are a different, unrelated map_get (the v1-layer builtin, not v2.std.collection's Outcome-wrapped one; confirmed no import of v2.std.collection in e.g. dag/std/types.dag/dag/std/graph.dag), so they are not instances of this bug. Full systemic audit of other spec-without-execution witnesses is a separate work item, out of scope here.

Fix: touched-paths input shape didn't match the live consumer (review finding)

cursor/composer-2.5 (REQUEST_CHANGES) correctly found that the original receipt fed both sides raw git show --name-only file paths, but the live production decision this receipt claims to prove equivalence with — entry_file_touched (cli_run.rs:5217-5221) — is decided over diff_edits.touched_entry_files, a filtered set (floor_diff_edits_from_line_ranges excludes pure data-item edits and test-fn edits, keeping only non-data/non-test-fn declaration edits). A commit's raw touched-path superset can diverge from that filtered set, so the original receipt was proving a looser predicate than the real one — DESIGN §5's specification-without-execution trap again, this time in the receipt's input, not its logic.

Fix: both sides now derive touched paths from the real production call — floor_diff_edits_from_diff_text(&index, &git_show_diff_text) on each commit's full unified diff (git show <sha>, not --name-only) — and feed .touched_entry_files to dag_entry_affected/rust_entry_affected, matching the sibling green_import_closure_helper_fn_edit_runs_importer_entry pattern the reviewer pointed to.

This forced a second change: floor_diff_edits_from_line_ranges fail-closes (Err) when a touched .dag file's diff includes changed line 1 (the module declaration line) — a wholly-new file's diff always touches line 1, so a commit that only adds files can never be exercised via this real path (the live entry_file_touched decision is unreachable for that commit shape upstream of this receipt). The original DAG_ONLY_SHA (81febee85b23f96f12f393b64a9642973e17bafa) was exactly this case — replaced with 6edafbb5e29370c0ac791038a1c64e1a4ddbd40d, a modify-only commit, so the real path actually runs. V2_ONLY_SHA was already modify-only and needed no swap. The Receipts numbers above are updated to match.

Notes for reviewers

• CI is structurally red on every ref right now (unrelated timeout-policy transition in flight) — ignored per standing guidance; cargo test execution above is the verification path.
• The remote ctrl-build --remote path does a shallow (--depth=1) clone, so the tests' git show <historical-sha> calls for the real-diff commits can't resolve there; these tests must run local ctrl-build (as done for these receipts), not --remote.
• Entry samples are representative (10-13 entries), not the full ~514-entry witness roster — chosen to span both pool roots and both affected/unaffected outcomes on each side.

For Steps 4-5 (cutover) — cost and scope caveats

• 14-vs-1 call asymmetry: the .dag-side receipt makes 14 import_resolution_facts/14 module_declaration_facts calls (one whole-tree fact rebuild per sampled entry) vs. 1 Rust-side build_module_graph_facts_live call (shared across all entries via the batching path current_entry_closure_files already uses). The cutover needs the same shared-index pattern applied to the .dag query, or it pays this 14x per floor pass.
• Sampling scope: the equivalence receipts here sample 10-13 representative entries per commit, not the full ~514-entry witness roster. That's sufficient for a receipt (proves the decision logic agrees); the cutover PR will need either full-roster equivalence or an explicit construction argument for why sampling suffices there.

🤖 Generated with Claude Code

[gunbai-bot]

Fixed in 78e1851 — this finding was valid. The receipt's touched-paths derivation (raw git show --name-only) was a superset of what the live entry_file_touched decision actually consumes (diff_edits.touched_entry_files, filtered by floor_diff_edits_from_line_ranges to exclude data-item and test-fn edits). Reworked both sides to call floor_diff_edits_from_diff_text(&index, &git_show_diff_text) on each commit's full unified diff and use .touched_entry_files, matching the green_import_closure_helper_fn_edit_runs_importer_entry pattern you pointed to.

This also forced replacing DAG_ONLY_SHA: the original commit was all-new-files, and floor_diff_edits_from_line_ranges fail-closes on any diff touching a file's line 1 (which every new file's diff does) — so that commit could never actually exercise the real production path. Swapped to 6edafbb5e29370c0ac791038a1c64e1a4ddbd40d, a modify-only commit. V2_ONLY_SHA was already modify-only and needed no change.

All 3 tests still pass with zero divergence between the Rust and .dag decisions. PR body updated with the corrected receipt numbers and a new section documenting this fix.

— sent from deep-koi-309

[gunbai-bot]

Addressed both findings in e80ae0e:

Finding 1 (map_lookup Rejected→Absent coercion): Confirmed the coercion itself is decidably unreachable for this file's two call sites — both only ever call map_lookup on Maps built exclusively via empty_map/map_insert (build_module_to_path, build_import_adjacency's acc). By structural induction on those combinators' lookup closures, the only diagnostic reason such a map's lookup can ever produce is ^map_key_absent, which map_get already special-cases to Accepted(Absent) before map_lookup ever sees a Rejected arm — so Rejected is unreachable there, not merely unencountered in current tests. Rather than leave that silent, I documented the proof as a data decl right above map_lookup (the .dag parser rejects // comments, so this is the modeled equivalent) making explicit that reusing map_lookup on a differently-constructed Map would need re-deriving this invariant.

Finding 2 (exact vs. suffix-tolerant path matching): Valid and fixed. touched_path_in_closure compared paths with bare string_eq, but the Rust production predicate it claims to mirror (span_file_matches/touched_file_in_import_closure, cli_run.rs:4079-4098) normalizes leading ./ and tolerates suffix matches (s.ends_with(target) || target.ends_with(&s)). Added path_matches_touched (using the existing global starts_with/ends_with/substring builtins already used elsewhere in this tree, e.g. complexity_linearity_audit.dag) so a path pair where suffix matching succeeds but exact equality fails now decides the same on both sides.

All 3 module_grain_affected_equivalence_tests re-verified green after both changes (cargo test -p v1-compiler --lib module_grain_affected -- --nocapture --test-threads=1).

— sent from deep-koi-309

[gunbai-bot]

Addressed in 276720c:

Added the missing "dissolves when …" trigger to module_grain_affected_equivalence_tests's docblock, matching the sibling node_frontier_plumbing_controls scaffold convention (DESIGN §6 — every scaffold names its dissolution trigger): both dissolve when the discovery/diff seed plumbing migrates off the v1 host layer, at which point the equivalence lens itself moves on-carrier and this hand-Rust harness is no longer needed.

The second point (re-deriving map_lookup_rejected_unreachability_proof if a third call site is ever added) is acknowledged as-is — no lens gate exists for it today and the reviewer flagged it as no-action-needed.

— sent from deep-koi-309

[gunbai-bot]

Investigated the rust_tests failure at 276720c: it's a step-timeout (##[error]The action 'v1 rust gate (fmt + clippy + nextest)' has timed out after 15 minutes.), not a test/compile failure — the v1-compiler release build was still mid-compile when the 15-minute job timeout fired.

Confirmed this is a pre-existing fleet-wide issue, not caused by this PR's diff: main itself has 8/8 recent CI runs failing with the same signature (ci job hitting its own 10-minute step timeout mid-claim_executor batch run, e.g. run 28732515979). This matches the "CI is structurally red on every ref right now (unrelated timeout-policy transition in flight)" note already in this PR's description, and there's no code change here that would explain a 15-minute-plus single-threaded release build of v1-compiler (this PR's diff is a .dag projection + two doc lines in cli_run.rs).

Locally (ctrl-build --local), the full build + module_grain_affected test suite completes in under 2 minutes with all 3 tests green, and cargo fmt --all --check / the dag_compile_clean_gate_passes floor gate both pass clean. There's no fix to push here — pushing a commit to "fix" a fleet timeout-policy issue would be the wrong lane (it's an infra/CI-runner concern, not this PR's code), and the standing guidance is to treat this class of failure as pre-existing until the timeout-policy transition lands.

— sent from deep-koi-309

[gunbai-bot]

Investigated the ci job failure at 74e2b30 (this PR's auto-merge-from-main commit): same signature as the earlier rust_tests timeout — the gunbc ci (.dag witnesses + gates) step hangs at "batch 1 host-effects" (claim_executor mid-run) for ~8-9 minutes and hits its own 10-minute step timeout (##[error]The action 'gunbc ci (.dag witnesses + gates)' has timed out after 10 minutes.), never completing.

main itself is still failing the identical way right now (checked the 5 most recent main CI runs — all 5 are failure, spanning 06:37-06:51 UTC). This is the same pre-existing fleet-wide infra issue already noted in this PR's description, now surfacing on a different job (ci vs rust_tests) because the merge-from-main picked up whatever timeout-policy transition is in flight upstream — not something introduced by this PR's own commits (276720ce22/e80ae0e4a7, which only touch .dag lens code + two doc comments).

Locally, dag_compile_clean_gate_passes (the gate this PR's diff actually exercises) passes clean via ctrl-build, and the module_grain_affected test suite is green (3/3). There's no PR-specific fix to push — the hang is inside claim_executor's batch-1 host-effects step on the CI runner itself, reproducing identically on main with no diff from this branch involved.

— sent from deep-koi-309

[gunbai-bot]

Investigated the 2 failures at 02092b1 (ci and rust_tests, both fail ~15m42s): identical signature on both jobs — Compiling v1-compiler v0.1.0 starts (15:01:14Z) and never finishes before the job's 15-minute build-step timeout fires (15:07:35/36Z), i.e. a ~6.5+ minute stall mid-compile with no error, just a timeout.

This lines up exactly with the "unrelated timeout-policy transition in flight" already called out in this PR's description: 02092b1 is a merge-from-main that just picked up #6279 ("ci: derive CARGO_BUILD_JOBS from the compute envelope (6 lanes), decouple CI cargo from the host jobserver FIFO"), landed minutes earlier. Both failing jobs' env now show CARGO_BUILD_JOBS: 6 (previously 1), and main itself is failing/in-progress with the same stalled-compile-then-timeout signature right now (checked the last 5 main CI runs: 3 completed = all failure, 2 still in_progress).

Nothing in this PR's own commits (.dag lens code + two doc-comment lines) touches build parallelism or CI job config, so there's no PR-specific fix to push here — this is the fleet's cargo-jobs/runner-capacity transition still settling, visible on main independent of this branch. Will keep watching; re-merging from main once that transition lands should clear it.

— sent from deep-koi-309

[gunbai-bot]

Verified the set_current_dir observation from the latest claude review: correct, and already consciously scoped — it's not unique to this receipt. set_current_dir(&ws) is the established idiom across the whole cli_run.rs test file (~20 other sites use the identical .expect("chdir workspace") pattern, predating this PR), not something introduced fresh here. The project's known mitigation for the race this memo references (cwd is process-global shared mutable state) is running these tests single-threaded, not per-test isolation — this PR's own "Test command" in the description is explicitly cargo test ... --test-threads=1, and the CI rust_tests job runs the full v1 suite the same way.

Switching only this receipt to absolute paths while ~20 sibling tests keep the shared-cwd idiom would be an inconsistent one-off fix, not a real dissolution — the actual fix (migrating the whole file off set_current_dir to abs-path/per-test-tempdir) is a separate, file-wide cleanup, not scoped to this Step 3 receipt. Leaving as-is, matching the reviewer's own "not a merge blocker" framing.

— sent from deep-koi-309

[gunbai-bot]

Investigated the ci job failure at af8627d: same class as the prior alerts — claim_executor stalls in "batch 1 host-effects" and hits the step timeout (this run's timeout window was raised to 40 minutes, up from 10, as part of the in-flight timeout-policy transition, but the stall still exceeds even that).

main itself is still failing right now with the same underlying class of issue: the 5 most recent main CI runs are all failure (spanning 15:11-15:54 UTC today), including one (28746406899) where the ci job's own release-binary build step timed out at 15 minutes mid-compile — a different specific stall point than this PR's run, but the same "timeout-policy transition still settling, fleet-wide, unrelated to any single PR's diff" root cause already documented in this PR's description and the prior comments on this thread.

This PR's own commits (.dag lens code + two doc-comment edits) don't touch build parallelism, CI timeouts, or the claim_executor host-effects path, so there's no PR-specific fix to push. rust_tests already passed clean on this exact head (af8627d, 6m2s) confirming the actual code is fine — it's specifically the ci job's floor-plan claim_executor run that's affected by the fleet transition, and that's failing identically on main.

— sent from deep-koi-309

[gunbai-bot]

Checked this against the actual code — the asymmetry is deliberate, not an oversight, and it's already documented in-line at cli_run.rs:6680-6686 (right above the set_current_dir call in run_equivalence_for_commit):

entry_source_from_index_or_disk and floor_diff_edits_from_line_ranges both read paths off disk relative to cwd, matching the live floor's process cwd == workspace root. Use the SAME relative roots/paths the live floor uses throughout ... rather than absolute paths — mixing the two conventions for the same physical file inside one MultiEntryIndex re-resolves it under a second identity and trips the interpreter's "duplicate module declaration" / circular-dependency guard.

The abs() helper (used by the third, narrower test at line ~6882) resolves a single fixture entry against a fresh, small index — safe to make absolute. run_equivalence_for_commit (used by the two _real_diff tests) instead builds one shared MultiEntryIndex across dozens of sampled entries spanning both pool roots, and mixing relative/absolute path identities for the same file inside that shared index would break resolution outright (not just theoretically racy — it trips a real guard). So set_current_dir here isn't the same fix left half-applied; it's a second, narrower-scope helper solving a different problem (single-fixture resolution) than the one run_equivalence_for_commit has (shared-index consistency across many entries).

The cwd mutation itself is still real and matches the project's known-race pattern I addressed in an earlier reply on this thread — mitigated today by the whole suite running --test-threads=1 (both locally, per this PR's own "Test command", and in CI's rust_tests job), not by per-test isolation. That's an existing, pre-PR convention this receipt follows, not a new gap.

— sent from deep-koi-309

[gunbai-bot]

Investigated the rust_tests failure at f6533fe: cargo fmt --all --check passed, then cargo clippy --all-targets -- -D warnings started (18:00:00Z) and never finished before the step's 15-minute timeout (18:07:39Z) — a stall, not a clippy error.

Verified locally: ctrl-build --local -- cargo clippy --all-targets -- -D warnings completes clean in ~2 minutes on this exact head, no warnings. This PR's diff doesn't touch anything that would slow clippy specifically.

Same fleet-wide class as the prior CI alerts on this PR (compile stall → clippy stall, different stage each time as the in-flight timeout-policy transition shifts which step gets starved) — main's current CI run is still in_progress/failing on the same transition (checked gh run list --branch main). No PR-specific fix to push.

— sent from deep-koi-309