Skip to content

claim_executor floor: dedup same-entry resolves within a batch (resolve once, share the graph) — §2 redundancy cleanup - #5619

Merged
briansrls merged 3 commits into
mainfrom
session/quick-lynx-78
Jun 23, 2026
Merged

briansrls merged 3 commits into
mainfrom
session/quick-lynx-78

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Jun 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

claim_executor's floor walk (run_walk) spawned one OS thread per runnable in a batch, and each
thread re-resolved its entry's full import closure from scratch. In CI batch-2, six gate witnesses
(rust/emit/layering/resolved_imports/extdeps/generated) all live in one file

(dsl/tools/floor_effect_gate_witness.dag), so its identical resolved-and-typechecked graph was
built and held 6× concurrently — pure duplicated work (DESIGN §2).

This PR partitions each batch into resolve-groups by entry: SingleClaims sharing one file
resolve (and typecheck) that file exactly once and run every claim sequentially on the one shared
interpreter context. Distinct entries and the discovery-corpus node keep their own concurrent
threads. Resolve is a pure function of (source_roots, entry), so sharing the graph across
same-entry claims is semantically identical — correctness by construction.

This is an honest §2 redundancy / CPU cleanup. It is not the floor's OOM lever: the dominant
self-RSS is the discovery-corpus path (per-shard footprint × spawn_width), and the cgroup
worst-max (the hard-cap input) is driven by the rust-gate cargo test rustc child processes — neither
is touched here. The corpus-path footprint is a separate follow-on, rooted in the budget/width
authority.

Measured (by-execution, controlled before/after; OLD vs patched binary, same 6-same-entry plan)

A proxy witness with a byte-identical import closure to the real gate file (106 modules / 1300
items vs 106 / 1301) was used so no effectful gate runs:

self-RSS (VmHWM)
OLD — 6 concurrent resolves 718 MB
patched — 1 shared resolve 148 MB

→ ~0.57 GiB of self-RSS removed on the gate group, plus ~5× redundant parse/typecheck CPU. Per-gate
resolve is ~148 MB (the larger ~0.9 GiB figure seen via claim_batch is its whole-tree mock-key
precompute, which the executor's SingleClaim path does not run).

Correctness (DESIGN §5 — the floor harness must never fail open)

A discriminating control ran three same-entry claims [pass, FAIL, pass] through the grouped path:

  • 1 resolve-group for the three (grouping confirmed),
  • the planted red still reported FAIL (not swallowed), the pass-after-fail still PASSED
    (shared ctx not poisoned), exit code 1 (fail-closed preserved).

Same-entry ctx reuse is the already-exercised discovery-corpus pattern (run_discovery_rows runs
hundreds of claims on one per-entry ctx); this change does not touch the cross-entry parse/typecheck
cache, so the #5295 partial-type poisoning shape does not apply.

Test plan

  • cargo test -p v1-compiler --bin claim_executor — 4 passed (grouping coalesces same-entry
    claims, preserves every claim exactly once, empty-entry sentinel stays an isolated fail-closed
    unit, discovery node stays isolated).
  • cargo clippy -p v1-compiler --bin claim_executor --all-targets -- -D warnings — clean.
  • cargo fmt --all --check — clean.

Reviewer note — checks_state "cancelled" is a FALSE negative

The dashboard shows checks_state: failing because a CANCELLED check-run sits beside a SUCCESS on
this head (the known cancelled-sibling pin). The real floor run is GREEN — ci run
28005219762 passed (and rust_tests passed). The CANCELLED entries are a duplicate-trigger
artifact, not a failure; per the lockdown manager they are intentionally not rerun (a rerun moves
no bits but re-fires reviews / resets approvals). Read this PR as floor-green.

@gunbai-bot gunbai-bot Bot changed the title CI lockdown B4-NEW - claim_executor FLOOR FOOTPRINT reduction (one atomic PR, by-execution). claim_executor running gunbc_ci_floor_batches (src/v2/workflow/ci_floor_plan.dag, gunbc_ci_floor_batches) balloons to ~31 GiB anon-RSS - the single heaviest per-job peak and the root of OOM/over-commit (~4 c claim_executor floor: dedup same-entry resolves within a batch (resolve once, share the graph) — §2 redundancy cleanup Jun 23, 2026
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review June 23, 2026 05:48
@briansrls
briansrls merged commit 7254d56 into main Jun 23, 2026
2 checks passed
@briansrls
briansrls deleted the session/quick-lynx-78 branch June 23, 2026 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant