Skip to content

Measure what the slow floor witnesses actually cost, before any are pared - #8464

Merged
briansrls merged 6 commits into
mainfrom
session/valiant-eagle-570
Aug 19, 2026
Merged

briansrls merged 6 commits into
mainfrom
session/valiant-eagle-570

Conversation

@briansrls

@briansrls briansrls commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

gunbc#8455 established that the floor's per-row wall time is not always the row's cost, and named the repair it did not perform — attribute shared cost to the entry rather than its first row. This is that repair, plus the mechanism root-caused to source.

The mechanism, named

The floor builds a fresh evaluation frame per claim, so the sharing #8455 measured is not an eval memo — a per-claim memo cannot survive between claims. It is the seed's process-global corpus-scan caches, each filled at most once per process and read free thereafter.

The #8455 specimen resolves exactly. g2_data_reference_under_selection's specimen_projection reaches v2.lens.live_read_classification live_read_selection_manifest_live → dependency_resolution_facts_live, whose seed realization cli_run.rs reference_resolution_facts performs "one O(corpus) parse pass over the pool, cached" behind REFERENCE_EDGE_CACHE. The first claim pays the pass (12197ms); the second reads the map (31ms). Same call, 393x, 35ms apart.

Why the existing number cannot answer the paring question

Nothing is computed twice — the caches are correct, and the sharing is the saving. What is wrong is the attribution, in the direction that decides things, because the ceiling refuses per row:

  • a refusal names the first toucher, whose own cost may be milliseconds;
  • splitting such a row moves the charge to whichever fragment runs first (witness_decomposition_does_not_reduce_entry_cost_note, observed rather than predicted);
  • removing such a row does not remove its cost either — the next claim to touch the fill pays the same seconds.

What lands

cli_run/shared_fill.rs records, per fill: the cache, the key it is declared on, what the fill cost, which claim paid it, and every later claim and module that read it. Eleven caches instrumented — reference_edges, module_graph_facts, module_path_index, and the eight whole-corpus census reports.

gunbc.witness_row_cost owns the model: SharedFillPayer, and a three-state SharedFillDisposition because there are three remedies — outside-fold (preparation's cost, which no witness can be pared to recover), exclusive (one module owns it and loses it with itself), shared (survives any single removal). Collapsing the first into either of the others is how a removal comes to be justified by a saving it cannot deliver.

gunbc.observation_ci_render owns the line; the seed mirrors it and both sides assert one literal, so a format drift reds one of the pair rather than forking silently.

The quantity a removal decision needs, joined against the [floor-witness-slow] rows the floor already prints:

what removing module M saves = M's rows' wall time
                             - fills M paid that are `shared`
                             (+ nothing at all for `outside-fold` fills)

What this is not

It refuses nothing, skips nothing and changes no verdict; it prints a table (DESIGN §5's sanctioned stopped-line audit — it reports, it does not green). It makes no witness faster and takes no position on which rows should be pared. unattributed_hits states the shared figure as a lower bound: reads served by a fill this ledger did not observe.

Receipt — run 32192150969, 9425 claims, 19 fills

The run is main's inherited red (failed=0 stale_quarantine=5 budget_refused=102), not this diff's; the fold completed and the ledger reported.

One row's entire cost is a fill 29 other modules read. module_path_index key=dag+src/v2, fill_ms=51508, paid by dissolution_census_witness_test.unbound_dissolution_empty_literal_refuses and read by 139 claims across 29 modules. That payer is one of five rows in the corpus with an exact cost — 51620ms — so 99.8% of its measured cost is a computation 29 other modules consume. It is the run's only shared fill. Removing that witness recovers ~112ms and hands 51.5s to whichever of the other 29 goes first.

And the opposite, one row away on the same list. The 86741ms red_seed_runner_failure_detail_projects_located_receipt paid a 29017ms module_graph_facts fill and a 24965ms reference_edges fill, both exclusive — nobody else reads either. Its cost is genuinely its own and it is a real paring target. Nothing in the per-row wall time distinguishes the two cases.

~108s is outside-fold — paid before any claim ran. Quarantining every witness on the floor would not move it.

The prediction, confirmed on an experiment nobody had to build (measured by tidy-lark-471): across #8457's quarantine, in one module and one run, single_refinement_carrier_emits_no_unsupported_cast went 746ms → 45941ms while its sibling stayed flat at 763ms. The fill its neighbour used to pay re-landed on it; the sibling rides it free in both eras. Quarantining a first toucher relocates the bill.

Second receipt — run 32196069889: the same fill changing hands

The first receipt reported consumer counts; this one names them, and caught the same fill change payers.

run 32192150969   module_path_index  dag+src/v2   fill_ms=51508
                  paid_by  dissolution_census…unbound_dissolution_empty_literal_refuses
                  read by  139 claims / 29 modules        shared

run 32196069889   module_path_index  dag+src/v2   fill_ms=45075
                  paid_by  emitter_nested_refinement_cast…single_refinement_carrier
                  read by   94 claims / 20 modules        shared

Same cache, same key, paid_by changing hands after #8457 quarantined the first payer. The second payer is main's only failing witness. On this run it measured 45784ms, of which 45075ms is this fill — ~98.5%, and that share is pinned to the run: the same row measured 45784, 45941, 46714, 50532 and 52982 across five runs the same day, so the fill fraction is robust and the denominator is not. Nearly all of the remaining red is one build that 20 other modules read for free.

The consumer set also shrank across the quarantine — 139 claims / 29 modules → 94 / 20. A consumer is a free rider: it reads a fill someone else built, so its marginal cost for that fill is ~zero. Quarantining a rider removes coverage and recovers nothing on this axis; quarantining the payer is what produced the red. (The first receipt had counts only, so those nine modules cannot be named retroactively — which is exactly why the line now names them.) The riders are named, so the next hop is a prediction rather than a warning: evaluation order is alphabetical, so quarantining this payer hands ~45s to emitter_optional_payload_cast_witness_test. The fix is #8470, which pays this index in preparation; I reviewed it rather than writing a second one.

A correction to this instrument, from its own first receipt. The reported TOTAL fill_ms=302362 was an inclusive sum: these caches compose, so a module-graph fill contains the path-index and reference-edge fills it triggers — visible in the receipt's own rows, 17890 = 12054 + 5141 + ~695. Fills now carry self and inclusive separately, the total sums self (~180s), and a planted two-level test pins the arithmetic. The payer/consumer/disposition columns were never affected. Two bounds on reading them, both from measurements handed over today and recorded in the design note: run-to-run variance is ~18% for a near-ceiling row on byte-identical input (sharp-raven-273), and 84 of main's 97 budget-refused rows are pinned at their interrupt point and carry no cost to decompose (tidy-lark-471, correcting an earlier reading).

Coordinated with tidy-lark-471, sharp-raven-273, and #8455/#8457.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BXMNpLmt4mGzThb7pfHVJS

…ited

gunbc#8455 established that the floor's per-row wall time is not always the
row's cost, and named the repair it did not perform. This is that repair, plus
the mechanism root-caused to source.

THE MECHANISM. The floor builds a fresh evaluation frame per claim, so the
sharing #8455 measured is not an eval memo — one cannot survive between claims.
It is the seed's process-global corpus-scan caches, each filled once per process
and read free thereafter. The specimen resolves exactly: g2's specimen_projection
reaches v2.lens.live_read_classification live_read_selection_manifest_live ->
dependency_resolution_facts_live, whose seed realization cli_run.rs
reference_resolution_facts performs an O(corpus) parse pass behind
REFERENCE_EDGE_CACHE. The first claim pays it (12197ms); the second reads the map
(31ms).

WHY IT MATTERS. Nothing is computed twice — the caches are correct and the
sharing is the saving. The attribution is wrong, in the direction that decides
things, because the ceiling refuses per row: a refusal names the first toucher,
whose own cost may be milliseconds. Splitting such a row moves the charge to
whichever fragment runs first; REMOVING it does not remove the cost either,
because the next claim to touch the fill pays the same seconds.

WHAT LANDS. cli_run/shared_fill.rs records per fill: the cache, its declared key,
what the fill cost, which claim paid, and every later claim and module that read
it. gunbc.witness_row_cost owns the model — SharedFillPayer, and a three-state
SharedFillDisposition because three remedies: preparation cost no witness can be
pared to recover, a fill one module owns, and a fill that outlives any single
removal. gunbc.observation_ci_render owns the line; the seed mirrors it and both
sides assert one literal, so a format drift reds one of the pair.

Eleven caches instrumented. Reporting only: nothing refuses, skips, or greens.
unattributed_hits counts reads served by a fill this ledger did not observe, so
the shared figure states itself as a lower bound rather than as the whole of the
sharing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BXMNpLmt4mGzThb7pfHVJS
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review August 18, 2026 21:57
Brian Searls and others added 3 commits August 18, 2026 22:19
…nloaded the whole render module

dag does not admit \' in a string literal. One character at line 803 col 40
made gunbc.observation_ci_render unparseable, and every function in it then
read as not-found across three witness files -- 60+ cascading diagnostics from
one escape. Reworded rather than escaped, so nothing depends on which escapes
the grammar admits.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BXMNpLmt4mGzThb7pfHVJS
The first floor receipt (run 32192150969, 19 fills) exposed a defect in this
instrument. These caches compose — building the module graph builds the
reference edges and the path index on its way — so an outer fill's wall
contains its inner fills'. Summing them counted every inner scan once per
enclosing scan.

Measured, in the receipt's own numbers: a 17890ms module_graph_facts fill
contained a 12054ms module_path_index fill and a 5141ms reference_edges fill,
leaving ~695ms of its own. The reported TOTAL fill_ms=302362 was inclusive; over
self time it is ~180s. The per-fill payer, consumer and disposition columns were
never affected — only the durations nested.

Fills now carry both. fill_ms is SELF and the total sums it; inclusive_ms rides
beside it because it is what names the caller that CAUSED the inner scans, which
a self-only figure would hide. a_nested_fill_is_not_counted_twice_in_the_total
pins the arithmetic on a planted two-level fill, and would have caught this.

WHAT THE RECEIPT FOUND, now in the design note. One row's entire measured cost is
a fill 29 other modules read: unbound_dissolution_empty_literal_refuses is one of
five rows in the corpus with an exact cost (51620ms) and 51508ms of it is a
module_path_index fill that 139 claims across 29 modules consume — 99.8%, the
run's only `shared` fill. Removing that witness recovers ~112ms and hands 51.5s
to whichever of the other 29 goes first.

And the opposite, one row away on the same list: the 86741ms
red_seed_runner_failure_detail_projects_located_receipt paid a 29017ms module
graph fill and a 24965ms reference-edge fill, both EXCLUSIVE — nobody else reads
either. Its cost is its own and it is a real paring target. Nothing in the
per-row wall time distinguishes the two cases, which is the whole reason this
ledger exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BXMNpLmt4mGzThb7pfHVJS
TWO CORRECTIONS, both from readers of the first receipt.

NAME THEM, DO NOT COUNT THEM. The row line reported consumer_modules=29 and
nothing else, so it could not answer the question the relocation raised: did the
module that INHERITED a bill read that exact fill before? That is an identity
join, and DESIGN section 5 rules a count is not one -- the same rule the floor's
own completeness checks are held to. The line now carries the sorted module
list, so a relocation is checkable at identity grain rather than by matching
magnitudes across two runs on two machines.

BOUND THE VERDICT. `exclusive` means no other module in the EXECUTED population
read that fill on THIS run. It does not establish that none ever would: remove
the payer and a would-be consumer may pay it next run, which is the first-touch
trap one level up and is not visible from inside a single run. Two things bound
the doubt, and both are now on the carrier rather than left implied. A fill is
keyed on its declared inputs, so an exclusive fill on a key no other consumer
requests is exclusive by construction rather than by luck. And the residue is
decidable by execution -- run the fold with the payer absent and read the ledger
again -- so until that run exists a recovery estimated from an exclusive row is
an upper bound, stated as one.

The design note also records why the ledger measures on the floor path at all: a
witness whose subject is the corpus is not made cheap by shrinking the corpus.
The 71s row is 70-85ms in a 67-module closure and 71060ms against the floor's
2376-module subject with nothing about the witness changed, so isolation is
structurally unable to price that class.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BXMNpLmt4mGzThb7pfHVJS
Brian Searls and others added 2 commits August 18, 2026 23:42
…e red

Run 32196069889 on this branch, 9001 claims, 8 fills, failed=1 on exactly the
row every branch inherits. The named-consumer column earns itself immediately.

  run 32192150969  module_path_index dag+src/v2  51508ms
                   paid_by dissolution_census...unbound_dissolution_empty_literal_refuses
                   139 claims / 29 modules      shared

  run 32196069889  module_path_index dag+src/v2  45075ms
                   paid_by emitter_nested_refinement_cast...single_refinement_carrier
                    94 claims / 20 modules       shared

Same cache, same key, the paid_by field changing hands after #8457 quarantined
the first payer. The second payer is main's only failing witness at exactly
45784ms, of which 45075ms is this fill -- 98.5% of the entire remaining red is
one build that 20 other modules read for free.

The riders are named, so the next hop is a prediction rather than a warning:
evaluation order is alphabetical, so quarantining this payer hands ~45s to
emitter_optional_payload_cast_witness_test. That would be the fourth hop of a
chain that has already run three times. The fix is gunbc#8470, which pays the
index in preparation; its self-verifying post-condition is this row reappearing
as paid_by=<outside-fold>.

Still the only shared fill in the run. The nesting correction is visible in the
same receipt: module_graph_facts on dag reports fill_ms=2990 against
inclusive_ms=55048, so 52s of it was the scans it triggered -- the term the
first receipt's total counted twice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BXMNpLmt4mGzThb7pfHVJS
…modules cost

TWO CORRECTIONS FROM A READER OF THE SECOND RECEIPT.

The 98.5 percent was stated as if the denominator were stable. It is not: the
same row measured 45784, 45941, 46714, 50532 and 52982 across five runs on one
day. The fill FRACTION is the robust part and the total is not, so the share is
now pinned to its run and the claim that survives every reading is stated
instead -- nearly all of the remaining red is one build that 20 modules read.

And the two receipts price the quarantine itself, which I had not read out of my
own data. The consumer set went from 139 claims across 29 modules to 94 across
20. A consumer is a FREE RIDER: it reads a fill someone else already built, so
its marginal cost for that fill is approximately zero. Quarantining a rider
removes coverage and recovers nothing on this axis, while quarantining the PAYER
is what handed the bill onward and produced main's sole red.

The limit on that reading is stated with it rather than left for someone to
find: the first receipt carried consumer COUNTS only, so those nine modules
cannot be named retroactively. The delta is consistent with riders having been
quarantined; it does not identify which. That is the same count-versus-identity
gap the named-consumer column closed, showing up one level out -- from here the
question is a set difference rather than an inference.

So the 20-module list reads two ways at once: who inherits the bill if the
current payer is quarantined, and who is first in line to return once the fill
belongs to preparation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BXMNpLmt4mGzThb7pfHVJS
@briansrls
briansrls merged commit 18ff838 into main Aug 19, 2026
1 check passed
@briansrls
briansrls deleted the session/valiant-eagle-570 branch August 19, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant