Skip to content

Witness cost class: import-closure pollution — zero confirmed exemplars - #8495

Merged
briansrls merged 1 commit into
mainfrom
session/gentle-badger-188
Aug 19, 2026
Merged

briansrls merged 1 commit into
mainfrom
session/gentle-badger-188

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Work item: node://adhoc-0e6f8ea5-158 ("Witness cost class: import-closure pollution — root-cause 2-3 exemplars, land a reusable closure-narrowing repair, restore the modules with an exact planned/problem/collateral delta"), child of royal-hawk-392.

Honest result: zero confirmed exemplars. Three candidates were investigated; all three were falsified as import-closure pollution, each remeasured against the required floor's own [floor-witness-slow] log line (not a standalone harness run, which was found to overstate cost ~2-3x due to a once-per-claim vs once-per-MultiEntryIndex amortization difference).

The three candidates

  1. extdeps_scope_placement_gate_loudness_witness (PR Take the extdeps-scope loudness probe out from behind a 718-module import #8432's 718→62 module import-closure narrowing). The narrowing is real and DESIGN.md §3-legitimate, and standalone resolve time dropped (30.7s → 1.0s). But the floor's own last post-repair line (red_seed_runner_failure_detail_projects_located_receipt, run 32177951514, head 59653f569e, a confirmed descendant of the repair commit ff56305b731) shows 75926ms, BUDGET-REFUSED against the 1552ms ceiling, +1.00GB RSS growth (the worst single claim in that run) — statistically unchanged from its pre-repair floor history (42983-97260ms). The module is now excluded from the floor entirely by #8457's test.claim.long. rename, so no current floor line exists at all. Right repair, wrong quantity measured.

  2. Class B rust_test_fixtures rows 1-2 (declared_import_closure_lists_rust_import, ambient_pool_provider_preserves_listed_import). A paired same-host, same-dispatch comparison found the narrow probe-backed entry and the heavy transport-backed entry statistically indistinguishable (~70s/~80s standalone, both). The floor's own line (24674ms/31603ms) confirms the same non-divergence at its own lower, amortized magnitude.

  3. dissolution_census unbound-literal witnesses. Root-caused to a different mechanism: compile_dag_rust_emit_check's uncached path rebuilds a full corpus-rooted module index per call, memoized only under a real --required-floor run — a first-toucher shared-fill signature (42129ms vs 825ms, same module, no import-closure difference). This is class-2 per-claim memoization, not closure-narrowing scope, and out of this lane.

The common object

All three point at the same underlying cost: a fixed, corpus-wide load/index term paid once per MultiEntryIndex rather than once per entry's import closure. This corroborates — does not re-derive — DESIGN.md's floor-shared-compute-memoization open thread on build_both_closure_edge_index, whose closure-size-independence this lane's measurements are consistent with on three independent subjects.

What's retained (not cost repairs)

  • src/v2/workflow/class_b_import_closure_probe.dag (new): rows 1-2's shape+call, relocated out of class_b_import_closure_transport.dag per the same DESIGN.md §3 relocation-not-refork discipline already used for src/v2/compiler/inferred_tree.dag (also included here, previously split from 04_infer.dag). Both splits genuinely narrow dependency footprints and are legitimate modeling corrections — neither is claimed to buy measured cost.
  • src/v1/stage0/src/cli_run.rs: adds CLASS_B_PROBE_REL to CLASS_B_GATE_INPUT_ENTRIES — a real collateral correctness fix so the gate's input closure covers the relocated module.
  • Three .dag note strings updated with the corrected, floor-sourced magnitudes and explicit UNCONFIRMED/FALSIFIED framing.

The reusable deliverable

The paired same-process, same-host narrow-vs-heavy comparison methodology, baselined against the floor's own CI job log rather than a standalone run:

gh api repos/:owner/:repo/actions/runs/<run_id>/jobs --jq '.jobs[0].id'
gh api --allow-escape-sequences repos/:owner/:repo/actions/jobs/<job_id>/logs

This is the two-sided acceptance bar for any future import-closure-pollution candidate in this population: cost must collapse when the closure narrows AND not collapse under repetition in one process — measured against the floor's own line, never a standalone run.

Test plan

  • cargo fmt --all --check (pre-commit/pre-push hooks passed)
  • Remote release build of v1-compiler compiles cleanly with these changes applied (verified via background dispatch; the one failure was a CLI-flag usage error in my own verification invocation, not a compile failure — the Finished release profile line preceded it)
  • Full floor pass — not run from this session per the resource-sharing constraint below; the module changes are note-only or dependency-closure-only, no test-body semantics changed

Note: a full required-floor run was attempted from this session and OOM-killed (session containers share sessions.slice's aggregate memory cap; a single claim on this population alone grows RSS to 10+GB on the CI runner). Per-identity floor lines were instead pulled from existing CI job logs via the gh api route above, which is the now-established correct method for this class of measurement.

🤖 Generated with Claude Code

Investigated three candidates for the witness-cost-class lane
(dashboard node://adhoc-0e6f8ea5-158): the extdeps_scope_placement_gate
loudness witness (PR #8432's 718→62 module import-closure narrowing),
the Class B rust_test_fixtures rows 1-2, and the dissolution_census
empty-literal witness. All three were falsified as import-closure
pollution, each against the required floor's own [floor-witness-slow]
line rather than a standalone harness run:

- extdeps_scope_placement_gate_loudness_witness: the closure narrowing
  is real (718/10.1MB -> 62/0.49MB modules) and standalone resolve time
  dropped, but the floor's own last post-repair line for
  red_seed_runner_failure_detail_projects_located_receipt is 75926ms
  (BUDGET-REFUSED against 1552ms, +1.00GB RSS, worst claim in the run)
  — unchanged in order of magnitude from its pre-repair floor history.
  Right repair, wrong quantity measured.

- Class B rows 1-2 (declared_import_closure_lists_rust_import,
  ambient_pool_provider_preserves_listed_import): a paired same-host,
  same-dispatch comparison found the narrow probe-backed entry and the
  heavy transport-backed entry statistically indistinguishable
  (~70s/~80s both). The floor's own line (24674ms/31603ms) confirms
  the same non-divergence at its own (lower, amortized) magnitude.

- dissolution_census unbound-literal witnesses: root-caused to a
  different mechanism entirely — compile_dag_rust_emit_check's
  uncached path rebuilds a full corpus-rooted module index per call,
  memoized only under a real --required-floor run, producing a
  first-toucher shared-fill signature (42129ms vs 825ms) with no
  import-closure difference between the two rows. Class-2 per-claim
  memoization, not closure-narrowing scope.

All three point at the same underlying object: a fixed corpus-wide
load/index cost paid once per MultiEntryIndex rather than once per
entry's import closure — corroborating, not re-deriving,
DESIGN.md's floor-shared-compute-memoization thread on
build_both_closure_edge_index.

The class_b_import_closure_probe split (mirroring the already-landed
inferred_tree split) is retained as a DESIGN.md §3 relocation-not-
refork dependency correctness improvement, not a cost repair. The
CLASS_B_GATE_INPUT_ENTRIES roster fix in cli_run.rs is a genuine
collateral correctness fix for that split.

Deliverable: the paired same-process, same-host narrow-vs-heavy
comparison methodology, baselined against the floor's own CI job log
(gh api .../jobs/<id>/logs --allow-escape-sequences) rather than a
standalone run — the reusable diagnostic for any future import-
closure-pollution candidate in this population.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@briansrls
briansrls merged commit f585233 into main Aug 19, 2026
1 check passed
@briansrls
briansrls deleted the session/gentle-badger-188 branch August 19, 2026 03:01
gunbai-bot Bot pushed a commit that referenced this pull request Aug 19, 2026
… witnesses

Turns PR #8495's hedged "most plausibly the corpus-wide load/index
cost" into typed verdicts, grounded in a real --required-floor pass
(#8455's shared_fill ledger post-dates #8457's quarantine of these
modules by name, so no floor run ever had both simultaneously; a
throwaway, never-merged probe PR (#8563, run 32291153434 / job
96192007757) temporarily renamed the three identities for one CI
pass to observe them, then was closed without merging).

- declared_import_closure_lists_rust_import and
  ambient_pool_provider_preserves_listed_import: every
  module_graph_facts/module_path_index/reference_edges fill either
  row touches is keyed on this witness's own narrow
  class_b_declared_import_pool_roots, paid_by exactly this module,
  consumer_modules=1 -> SharedFillExclusiveToOneModule. The run's
  TOTAL line (fills=17, shared_fill_ms=0) shows zero
  SharedFillSharedAcrossModules exemplars anywhere in that floor
  pass. Attributed fill covers ~17.4-18.5s of each row's cost; the
  remainder is real per-call cost outside the ledger's surface.

- red_seed_runner_failure_detail_projects_located_receipt: appears
  in zero of the run's 17 shared_fill rows, neither as payer nor
  consumer. The standard-pool fills its own entry resolution would
  touch are all paid_by=<outside-fold> (pre-warmed during floor
  preparation before any claim runs) -- refuting the shared-fill
  mechanism as this identity's cost source entirely. Its ~54-76s
  cost and RSS growth (worst single claim in both observed runs)
  sit outside the ledger's instrumented surface.

Both notes now report a denominator-honest verdict rather than a
plausibility, per DESIGN.md's empty-observation-narrow rule.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ARER4f3NVk6ZwMvHNCBaUb
briansrls pushed a commit that referenced this pull request Aug 19, 2026
… witnesses (#8566)

Turns PR #8495's hedged "most plausibly the corpus-wide load/index
cost" into typed verdicts, grounded in a real --required-floor pass
(#8455's shared_fill ledger post-dates #8457's quarantine of these
modules by name, so no floor run ever had both simultaneously; a
throwaway, never-merged probe PR (#8563, run 32291153434 / job
96192007757) temporarily renamed the three identities for one CI
pass to observe them, then was closed without merging).

- declared_import_closure_lists_rust_import and
  ambient_pool_provider_preserves_listed_import: every
  module_graph_facts/module_path_index/reference_edges fill either
  row touches is keyed on this witness's own narrow
  class_b_declared_import_pool_roots, paid_by exactly this module,
  consumer_modules=1 -> SharedFillExclusiveToOneModule. The run's
  TOTAL line (fills=17, shared_fill_ms=0) shows zero
  SharedFillSharedAcrossModules exemplars anywhere in that floor
  pass. Attributed fill covers ~17.4-18.5s of each row's cost; the
  remainder is real per-call cost outside the ledger's surface.

- red_seed_runner_failure_detail_projects_located_receipt: appears
  in zero of the run's 17 shared_fill rows, neither as payer nor
  consumer. The standard-pool fills its own entry resolution would
  touch are all paid_by=<outside-fold> (pre-warmed during floor
  preparation before any claim runs) -- refuting the shared-fill
  mechanism as this identity's cost source entirely. Its ~54-76s
  cost and RSS growth (worst single claim in both observed runs)
  sit outside the ledger's instrumented surface.

Both notes now report a denominator-honest verdict rather than a
plausibility, per DESIGN.md's empty-observation-narrow rule.


Claude-Session: https://claude.ai/code/session_01ARER4f3NVk6ZwMvHNCBaUb

Co-authored-by: Brian Searls <briansearls1@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant