Skip to content

Native-realize the dependency-fact composition: census 180,600ms failing to 68,224ms passing - #8162

Merged
briansrls merged 5 commits into
mainfrom
session/gentle-dove-143-native-dep-facts
Aug 12, 2026
Merged

briansrls merged 5 commits into
mainfrom
session/gentle-dove-143-native-dep-facts

Conversation

@briansrls

@briansrls briansrls commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

The partition-closure census exceeded its long-lane ceiling (180,600 ms against 180,000 ms). This native-realizes the composition that was the cost.

Result

before after
dependency_resolution_facts_live 112,261 ms 216 ms
w_live_corpus_partition_closure_holds 180,600 ms — failing 68,224 ms — passing
Long-lane ceiling 180,000 ms unchanged

62% under the ceiling. No budget raised, no cadence moved, no witness relocated. The census landed within 0.5% of what the localization predicted (68,555 vs 68,224).

The cause, measured rather than reasoned

Per term, one process, live corpus:

import_resolution_facts_live                      146 ms
reference_derived_import_resolution_facts_live      5 ms
union + dedup of the two (interpreted)        104,943 ms
dependency_resolution_facts_live (all)        112,261 ms

The host leaves are 151 ms of 112,261 ms. The interpreted composition was the entire term, and effectively the entire census. So this moves the composition into the host rather than caching the leaves — caching them recovers 151 ms and leaves the census over its envelope.

Four hypotheses refuted by execution first

Recorded so nobody re-runs them:

  • Duplicate (importer, target) work in the consuming index — raw == unique exactly, measured post-join at the module grain, with path→module injectivity measured rather than assumed and zero silently dropped edges. The facts are already deduped at the same triple, so a dedup there had no inputs to act on.
  • The interpreter Map primitive — map_insert 38 ms at n=2000, 68 ms at n=8000 against a take-only subtractive control: 1.8× for 4× the items. Not quadratic. Explicitly not claimed as a defect; one population point cannot establish asymptotics. Left as quarry.
  • The bucket's list_snoc_item — replaced with a cons, measured 105,007 ms against a 104,943 ms baseline. No change. Reverted.
  • The read-hoist recorded in stage0_partition_closure (173.6 → 177.6 ms, noise) — its stated explanation is corrected below.

Preserved, and why each matters

  • Exact identity on {path, import_module, target_declared} — an unresolved target is a target_declared: false row, and the census exists to judge those.
  • Reference-first order, first occurrence — order is observable: module_target_index_from conses targets in arrival order, and module_impact_query derives provenance from which copy of a dual-backed fact survives.
  • The two-producer seam — the leaves remain the swappable pair dependency_edge_source_migration_note designates for the namespace terminal step. The native arm composes them; it does not replace them with a black box.

Evidence

  • Live-corpus ordered parity, row for row, against the interpreted spelling, plus an independent target_declared distribution check — banked at 7827977bd4 before that spelling was deleted, so "faster" provably is not "different".
  • 5 synthetic Rust controls on the dedup law (cargo test union_dedup_import_facts_law, 5 passed). The load-bearing one is declaredness_variants_of_one_pair_are_distinct_facts: a dedup keyed on (path, module) alone passes every other assertion and silently destroys the census's subject.
  • The enrolled control already existed — module_graph_edge_source_witness_test points the production function at a four-module fixture and asserts the union at identity grain in both directions, so it now exercises the native arm and reds on a dropped reference-only edge or a surplus.

Deleted rather than kept dormant

union_import_resolution_fact_lists, dedup_import_resolution_facts, add_unseen_import_fact_indexed, import_facts_bucket, ImportFactDedupAcc, import_resolution_fact_eq. A dead 105-second implementation beside the live one is future ambiguity, not authority. dependency_edges_from_import_facts and import_fact_to_dependency_edge are kept — still consumed by reference_derived_resolution_facts_flagged_pending_reference_producer, the latter also named by module_impact_query as the future provenance-stamp site.

Notes corrected, because they were actively misleading

stage0_partition_closure explained its null hoist result by asserting "the live fact producers dominate and are already memoized host-side". Never measured, and false — that sentence pointed the next reader at the one part of the pipeline that was already cheap. The live-census row's own 1.3%-margin warning came true at 180,600 ms and is now closed with the measurement. dedup_import_resolution_facts_cost_note keeps its name deliberately: stage0_partition_closure and repo_atlas_projection cite it by symbol, and deleting it would rot both citations. module_impact_provenance_derivation_note is repointed — it depends on the reference-first law, which changed home, not content.

Explicitly out of scope

The selection-graph twin is untouched. Sharing the helper is correct under §3, but selection_edges feeds selection_adjacency, which decides which witnesses run; reordering an input without proving order-insensitivity risks the empty-observation-narrow class on the affected-set path. That consolidation needs its own equivalence PR. Also excluded: floor-snapshot reuse, cache-key design, Map-primitive investigation, budget changes, other falsifier failures, G2.

Bootstrap ordering, recorded because nothing in the tree states it

Regen runs the existing binary, so a .dag already calling a new builtin fails regen with function not found in scope; but the new binary cannot compile until the generated dispatch carries the arm, since the macro rejects an unrostered arm. The order is forced: Rust + roster → regen → build → .dag adoption.

Brian Searls and others added 4 commits August 11, 2026 23:37
The partition-closure census exceeds its 180000ms long-lane ceiling
(measured 180600ms). Localizing the cost by execution, one process, live
corpus, per term:

  import_resolution_facts_live                      146ms
  reference_derived_import_resolution_facts_live      5ms
  union + dedup of the two                      104,943ms
  dependency_resolution_facts_live (all of it)  112,261ms

The two host leaves are 151ms of 112,261ms. The interpreted composition
is the whole cost, and it is effectively the whole census. So the repair
moves the COMPOSITION into the host rather than caching the leaves: a
snapshot join over the producers recovers 151ms and leaves the census
over its envelope.

Three hypotheses were tested and refuted before this one, each by
execution, and they are why the change is this narrow rather than
broader:

  - duplicate (importer, target) work in module_target_index_from:
    raw == unique EXACTLY, measured post-join at the module grain with
    path->module injectivity measured rather than assumed, and zero
    silently dropped edges. The facts are already deduped at the same
    triple, so a dedup there had no inputs to act on.
  - the interpreter Map primitive: map_insert measured 38ms at n=2000
    and 68ms at n=8000 against a take-only subtractive control, i.e.
    1.8x for 4x the items. Not quadratic. Left as quarry; the claim that
    each insert copies the map is NOT supported and is not made here.
  - the bucket's list_snoc_item: replaced with a cons and measured
    105,007ms against a 104,943ms baseline. No change. Reverted.

WHAT IS PRESERVED, and why each matters rather than being ceremony.
Exactness is the contract: the census judges unresolved targets, so
target_declared: false rows must survive, and equality stays the same
three-field identity import_resolution_fact_eq decides. Order is
observable, because module_target_index_from conses targets in arrival
order and downstream confirmation walks those lists, so reference-first
is kept. The two-producer seam is untouched: the leaves remain the
swappable pair dependency_edge_source_migration_note designates for the
namespace terminal step, and the native arm composes them rather than
replacing them with a black box.

ONE DEDUP AUTHORITY. union_dedup_import_facts_reference_first is the
single spelling of a union that had two: the .dag one and the host
selection-graph twin's import-first selection_edges. The twin is
deliberately NOT moved onto it in this change. Its output feeds
selection_adjacency, which decides which witnesses run, and reordering
an input to it without proving order-insensitivity would risk the
empty-observation-narrow class on the affected-set path. That
consolidation needs its own equivalence proof.

The interpreted spelling is RETAINED as
dependency_resolution_facts_interpreted_reference, not deleted: it is
the executable statement of what the native arm must agree with, and the
witness holds the two to a row-for-row ordered match over the same live
inputs, plus an independent assertion on the target_declared
distribution so collapsing the two declared-ness variants of one pair
cannot hide behind a path/target comparison.

BOOTSTRAP ORDERING, recorded because the next person adding a primitive
hits it identically and nothing in the tree states it: regen runs the
EXISTING binary, so a .dag already calling the new builtin fails regen
with 'function not found in scope'; but the new binary cannot compile
until the generated dispatch carries the arm, since the macro rejects an
unrostered arm outright. The order is forced -- Rust and roster, then
regen, then build, then .dag adoption.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XxLzhuMisV3GAPpP2mBtSG
Remote build+execute at 7827977 (one ctrl-build invocation, cold
checkout) proved the native arm before this deletion:

  native_matches_interpreted_row_for_row_in_order  PASS   89,514ms
  native_population_is_nonempty                    PASS      216ms
  native_preserves_target_declared_distribution    PASS   85,570ms

The middle row calls ONLY dependency_resolution_facts_live: 216ms
against the 112,261ms measured for the interpreted composition, so the
~105s union/dedup phase is ABSENT rather than reduced -- 520x. The other
two are expensive precisely because they still drive the interpreted
side to compare against it, row for row in order on the LIVE corpus,
plus an independent target_declared distribution check so a collapse of
the two declared-ness variants of one pair cannot hide behind a
path/target comparison.

With parity established as a receipt, the interpreted spelling is now
DELETED rather than kept dormant: union_import_resolution_fact_lists,
dedup_import_resolution_facts, add_unseen_import_fact_indexed,
import_facts_bucket, ImportFactDedupAcc, import_resolution_fact_eq, and
the temporary comparison witness. A dead 105-second implementation
beside the live one is future ambiguity, not authority. The comparison
witness could not have been enrolled in any case: it drives both sides,
so it costs ~89s, over the fast lane, and the long lane couples its
budget to expected-red rows.

KEPT: dependency_edges_from_import_facts and
import_fact_to_dependency_edge, still consumed by
reference_derived_resolution_facts_flagged_pending_reference_producer,
the latter also named by module_impact_query as the future
provenance-stamp construction site.

THE ENROLLED CONTROL IS THE ONE THAT ALREADY EXISTED:
test.claim.module_graph_edge_source_witness_test points the PRODUCTION
function at a four-module fixture and asserts the union at identity
grain in both directions, so it now exercises the native arm and reds on
a dropped reference-only edge or a surplus.

NOTES CORRECTED, because they would otherwise keep directing the next
reader at the wrong term. dedup_import_resolution_facts_cost_note is
rewritten to the measured truth and KEEPS ITS NAME, because
stage0_partition_closure and repo_atlas_projection cite it by symbol and
deleting it would rot both citations.
module_impact_provenance_derivation_note is repointed: it depends on the
reference-first law, which changed home, not content.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XxLzhuMisV3GAPpP2mBtSG
The rewritten note wrote the identity triple as a bare {path, ...},
which begins string interpolation - the class DESIGN records at the
2026-07-24 receipt, whose ruling is to use the documented \{ escape
rather than respell around the brace. Caught by execution: the census
run panicked in for_each_parsed_module_binding before any witness ran.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XxLzhuMisV3GAPpP2mBtSG
Five synthetic controls for union_dedup_import_facts_reference_first,
decided by a controlled fixture rather than by whatever the live corpus
happens to hold: the duplicate collapses, order is reference rows then
import rows each in input order, first occurrence wins without later
duplicates reordering, and the empty case is empty.

The load-bearing one is declaredness_variants_of_one_pair_are_distinct_
facts. An unresolved target IS a target_declared: false row, so the two
declaredness variants of one (path, module) pair are different facts and
both must survive. A dedup keyed on (path, module) alone passes every
other assertion in this module and silently destroys the partition
census's subject.

NOTES CORRECTED to the measurement. stage0_partition_closure explained
the null result of its read-hoist by saying the live fact producers
dominate and are already memoized host-side; that was never measured and
is false - the host leaves are 151ms of 112,261ms and the interpreted
composition was the whole term, so the stated explanation pointed the
next reader at the part of the pipeline that was already cheap. The
live-census row recorded a 1.3 percent margin and warned the ceiling
might be exceeded on a slower runner; it was, at 180,600ms, and now
measures 68,224ms against the same unchanged 180,000ms ceiling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XxLzhuMisV3GAPpP2mBtSG
@gunbai-bot gunbai-bot Bot changed the title main is red Native-realize the dependency-fact composition: census 180,600ms failing to 68,224ms passing Aug 12, 2026
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review August 12, 2026 00:14
…d registry

v1_compiler_infer_method.rs is a generated artifact. The registry row for
the new native producer was hand-written directly into the emitted Rust, so
a fresh self-compile did not produce it and the regen gate correctly refused
with "Stage0 is stale".

Hand-authoring a generated file is manual application committed as source:
the bytes were right, but no authority stood behind them, so the next regen
would have deleted the row and taken the primitive's type with it. The row
now lives in src/v1/04_method.dag beside module_declaration_facts, which is
the single authority the emitter reads.

Verified by execution: regen_stage0 built from current source and run in
this worktree rewrote all 141 generated files, and v1_compiler_infer_method.rs
came back byte-identical to the committed content.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XxLzhuMisV3GAPpP2mBtSG
@briansrls
briansrls merged commit 0269d97 into main Aug 12, 2026
5 checks passed
@briansrls
briansrls deleted the session/gentle-dove-143-native-dep-facts branch August 12, 2026 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant