PR-2 (Fix-1): reconcile-first — kill the merge-side O(M²) parent-surface copy (byte-identical)

[gunbai-bot]

PR-2 (Fix-1) — reconcile-first: kill the merge-side O(M²) parent-surface copy

Deliverable: Fix-1 — the merge quadratic is dead, byte-identically (union_parent_type_env_caches first-parent-share; 14.4×→4.0×, 103× fewer entries, one-function seed diff). Plus the docs/plans/type-env-single-authority-design.md §3.1 correction and both profile tables. Realization lane lively-raven-355.

SCOPE DECISION — wall-flattening moved to a named follow-up (a decision, review may veto)

This PR is re-scoped from "flatten the reconcile wall" to "kill the merge bucket." Wall-flattening (Fix-2, the source_visible_names/rewire iteration bucket) moves OUT to a named follow-up lane, for three reasons discovered by profiling (all validated by execution, none by argument):

• The iteration bucket's byte-identity oracle is diagnostic-parity (UnlistedImportUse), not regen --verify — a different receipt than Fix-1's.
• It is multi-site (svn:5750 + rewire:6732) and needs a per-site split before targeting.
• Its consumer is scheduled for redesign (family-closure SVN, 04_resolve.dag:377) — heavy optimization now risks pre-paying a redesign.

Fix-1 is independently valuable and fully validated; holding it hostage to a Fix-2 whose scope just grew serves nobody. The follow-up brief (per-site profile → targeted incremental set or rewire analysis, diagnostic-parity oracle) is filed separately. The SymbolIndex fill/consume is PR-3's consumer-side reform — this PR adds no inert fill code (unconsumed scaffold would be parallel-representation debt; Fix-1 is independent of it).

Plan map (for readers of the older combined-vehicle docs): #6360 = profile receipts + Fix-1 (merge bucket); PR-3 = SymbolIndex fill + ResolutionPolicy + first consumer; Fix-2 lane = iteration bucket (per-site split → targeted set, diagnostic-parity oracle); PR-4 = namespace-only pivot.

The measured root (three plausible mechanisms refuted by execution, not argument)

The reform's target evolved through disciplined prove-by-execution — each refutation prevented a wrong implementation:

1. Direct-import consume — refuted by the fixpoint oracle → root was re-export transitivity (§3.1 correction).
2. Persistent-map ("O(M²) is the eager copy") — refuted by reading the seed: the carrier is already im_rc::HashMap (v1_rt.rs:6), so the copy already structurally shares. A no-op, caught before implementing.
3. "map_merge is flat" — refuted by the profile's own instrument (my pre-registered bucket-b-flat prediction died on contact with data).

Profile (throwaway entry-count probe, reverted; 4× depth steps so O(M²)≈16×, O(M)≈4×/step):

depth	map_keys ENTRIES (iterate)	map_merge ENTRIES	map_get calls
50	14,446	4,047	19,731
200	147,796 (10.2×)	46,197 (11.4×)	154,056 (7.8×)
800	2,031,196 (13.7×)	664,797 (14.4×)	1,816,356 (11.8×)

Root (accepted): reprocess the parent's full surface per module, in three forms — iterate (map_keys), merge (map_merge with large parent-cache overlay), lookup (map_get). Residue check clean: reconcile-wall 4×-ratio (~11.6) ≤ every op's ratio, so the named ops explain the wall — no faster-growing unnamed site.

Fix-1 — merge bucket (LANDED, f4c37e5a48)

union_parent_type_env_caches now uses the first parent's cache as the fold base (structurally shared) instead of merge(empty_acc, parent.cache) — the large-overlay copy. Single-import → returns the parent cache directly (O(1) share); multi-import → first shared, rest merged. Same precedence (later overlays earlier) → byte-identical. This makes 04_env.dag:68's described-but-unimplemented ancestry_cache_sharing claim TRUE — a lie in the tree becoming a fact.

Receipt A — surgical-confinement (gold standard): the whole compiler corpus recompiled, and the emitted-seed diff vs main is exactly one function body (union_parent_type_env_caches, v1_compiler_infer.rs, +35/−10) — zero ripple across the other 91 seed files. Corpus-wide behavior preservation by construction.

Receipt B — byte-identity: regen_divergence_count=0 (committed stage0 == fresh self-compile).

Receipt C — sequential attribution (the merge bucket collapsed, iteration untouched):

depth	map_merge ENTRIES before	after	map_keys ENTRIES (unchanged)
50	4,047	421	14,446
200	46,197 (11.4×)	1,621 (3.9×)	147,796
800	664,797 (14.4×)	6,421 (4.0× = LINEAR)	2,031,196 (13.7×)

Merge doubling ratio 14.4× → 4.0×, a 103× reduction in entries at d800. The iteration bucket is identical (13.7×) — proving Fix-1 was confined to the merge; Fix-2 owns iteration. Exactly the pre-registered prediction.

Receipt D — RSS/memory (mechanism; magnitude lands with Fix-2): 658K fewer map inserts per compile at d800 — the memory the merge quadratic was ballooning; a direct mechanism-corroboration of neat-stag's whole-tree-emit OOM. Peak-RSS number (getrusage//proc) lands with Fix-2's whole-corpus re-measure.

Follow-up lane (Fix-2 — iteration bucket, NOT in this PR)

source_visible_names (04_infer.dag:5750) re-enumerates map_keys(parent.ancestry_str_bindings) per is_all import → O(M²) (the 2.0M-entry map_keys bucket, still 13.7×). The follow-up brief: per-site split first (svn:5750 vs rewire:6732 vs residue) → then either a dedicated incrementally-built reexport-name-set (if svn dominates, minimum-viable-size + update the 04_resolve.dag:377 scaffold note) or a separate rewire analysis (if rewire dominates). Oracle: diagnostic-parity on UnlistedImportUse, sharpened — verify it fires in the corpus today (else add a discriminating synthetic fixture so parity isn't vacuous). RSS peak number (getrusage//proc) lands with that lane's whole-corpus re-measure.

Atomicity-hazard receipt (recurrence)

Mid-surgery the auto-committer landed 04_infer.dag (Fix-1) without its regenerated seed (e1674c0), producing a dag/seed drift the regen gate caught loudly (CI red @ e1674c0) — the fail-closed system working. f4c37e5a48 committed the matching seed (regen_divergence_count=0). Recorded as a new instance of the documented multi-file-snapshot atomicity hazard (defork-audit §3): the fix (.dag + regenerated seed) must land as one atomic commit.

Day-0 receipt (settled reconcile-first)

Whole-corpus --target dag cold baseline is reconcile-bound (not emit, as §7.5's --target rust profile assumed): reconcile 89.1s / 87%, emit(dag) 8.4s / 8%. Synthetic resolve curve d50→d800 climbs monotonically (2.47→3.41) — genuine super-linear. → reconcile-first.

Acceptance criteria (this PR — Fix-1 only)

1. Merge bucket linear — map_merge doubling ratio 14.4×→4.0× ✓ (Receipt C).
2. Byte-identity — regen_divergence_count=0 ✓ (Receipt B) + one-function seed diff ✓ (Receipt A).
3. §7.6 invariants 1–6 green as red controls — import-DAG cycle exclusion, intra-module forward refs, multi-import overlay-wins, std.types filter, single-exporter canonical pick, variant_surfaces re-export chain.

(Wall-flattening / synthetic-curve-toward-2.0 is the follow-up lane's acceptance, not this PR's — see the Scope Decision above.)

Isolation note

Branched off main. Independent of the CI double-compile fix (Lever A, quick-tern-682, #6361) — that's the gate-shell missing-Share; this is the algorithmic reconcile cut. Both attack the --target dag wall.

[gunbai-bot]

Thanks — approval confirmed, and one strengthening for the record on the invariant you (correctly) flagged as the crux.

You noted byte-identity holds iff merge_type_env_cache(empty, X) ≡ X, calling it a reasonable invariant. It is not assumed here — it is proven by execution: the whole compiler corpus was recompiled and the emitted-seed diff vs main is confined to exactly one function body (union_parent_type_env_caches, +35/−10), with zero change across the other 91 seed files, and regen_divergence_count=0. If merge(empty, X) ≢ X at any live call site, some other emitted function would have resolved differently and the diff would not be one-function. So the invariant is not just plausible for a map-overlay merge — it is empirically closed corpus-wide, which is the strongest form of the confinement you described.

Two orientation notes for the merge decision: (1) the ci red is the known era-4 double-compile cap-kill (expected/non-blocking); ci_regen — the real byte-identity gate — is green, and ci clears for a true green once #6361 lands on main and I merge it in. (2) The wall-flattening (iteration bucket) is explicitly re-scoped to a named follow-up lane per the Scope Decision section, not silently dropped. — sent from lively-raven-355