Skip to content

Share one empty Rc<Vec<Rc<Node>>> singleton across Node list-fields (-43.7 MiB / -11.7% whole-tree compile RSS) - #5878

Merged
briansrls merged 1 commit into
mainfrom
fix/node-empty-list-singleton
Jun 27, 2026
Merged

Share one empty Rc<Vec<Rc<Node>>> singleton across Node list-fields (-43.7 MiB / -11.7% whole-tree compile RSS)#5878
briansrls merged 1 commit into
mainfrom
fix/node-empty-list-singleton

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

What

Node has four Rc<Vec<Rc<Node>>> list fields (children/params/uses/properties). Every empty one was built with Rc::new(vec![]) — a fresh ~40 B heap allocation (16 B Rc control block + 24 B Vec header) even though it holds nothing. Most leaf nodes use none of the four, so each carried up to ~160 B of pure per-node overhead.

This shares one thread_local empty Rc<Vec<Rc<Node>>> via an empty_node_list() helper and repoints the 506 empty node-field constructions at it. Cloning the shared Rc is a refcount bump, not an allocation — every empty list now aliases one heap object per thread instead of one per field per node. Same share-one-authority shape as the #5867 intern_table fix.

Measured (matched A/B)

Same base commit + same dsl/ tree, sole delta = this change; VmHWM via /proc sampler, 3 runs each, gunbc compile --source-root dsl --target rust:

VmHWM (avg)
before 381,938 KB
after 337,236 KB
reclaim ~44,703 KB = ~43.7 MiB (11.7%)

610 modules / 617 files / 0 diagnostics, identical both sides. The floor resolves dsl+src/v2 (larger), so absolute reclaim there is bigger; batch-2 per-shard resolves benefit too.

Correctness — green by execution, NOT emit-diff

This touches node construction only, not typecheck. Proven semantic no-op:

  • Floor-green oracle: 1101/1101 discovery witnesses pass with the patched claim_executor.
  • The failing witness v1_dag_parse_witnesses returns identical (false without the v1_src_dag_parse sub-binary built, true with it) on both base and patched binaries — a worktree-setup artifact, not this change.

⚠️ The Rust emitter is non-deterministic across runs — variant_to_enum: HashMap<String,String> resolves the owning enum of a variant shared across aliased coproducts (e.g. WriteThenCommit in both CacheWriteSemantics and AtomicityModel) by unordered iteration, so ~39 emitted files churn between two runs of the same binary. Therefore emit / regen_stage0 bit-identity is not a usable oracle here (it's already non-reproducible before any change), and this is hand-edited into the stage0 .rs seed (not a regen) per established practice — the seed is hand-maintained, regen is not run by build.rs/ci.yml, and a full regen would reintroduce that churn. (Filed separately as a §7 root-cause lead; out of scope here.)

The build fail-closed caught 3 ParserParam-typed params: sites (Rc<Vec<Rc<ParserParam>>>, not Node) — left as Rc::new(vec![]), correct.

Deliberately NOT bundled (verified)

  • Node.ident delete — REFUTED. It is set to Some on module/import nodes (parse.rs mod_ir.id/imp_ir.id) and read in 3 live type-lookup fast-paths (lookup_type_for, is_recursive_type_for, dag_node_surface_fingerprint). It is a live ident-keyed fast-path, not dead — deleting it is a behavior change, not a no-op. Don't re-chase the ~9.6 MiB.
  • Synthetic :0..0 span sentinel (make_span(0,0) → shared ZERO_SPAN): same pattern, but measured ~0.58 MiB — sub-MiB hygiene, left for a follow-up.

🤖 Generated with Claude Code

@briansrls @claude
Share one empty Rc<Vec<Rc<Node>>> singleton across Node list-fields (…
b1e7da2 
…-43.7 MiB / -11.7% whole-tree compile RSS)

Node has four Rc<Vec<Rc<Node>>> list fields (children/params/uses/properties).
Every empty one was built with Rc::new(vec![]) — a fresh ~40 B heap allocation
(16 B Rc control block + 24 B Vec header) even though it holds nothing. Most leaf
nodes use none of the four, so each carried up to ~160 B of pure per-node overhead.

Fix (same share-one-authority shape as the #5867 intern_table fix): one shared
thread_local empty Rc<Vec<Rc<Node>>> + an empty_node_list() helper; repoint the 506
empty node-field constructions at it. Cloning the shared Rc is a refcount bump, not
an allocation, so every empty list now aliases one heap object per thread instead of
one per field per node.

MEASURED (matched A/B, same base commit + same dsl/ tree, sole delta = this change;
VmHWM via /proc sampler, 3 runs each, gunbc compile --source-root dsl --target rust):
  before: 381,938 KB avg   after: 337,236 KB avg   reclaim: ~44,703 KB = ~43.7 MiB (11.7%)
610 modules / 617 files / 0 diagnostics, identical both sides.

Semantic no-op: touches node construction only, not typecheck. The Rust emitter is
non-deterministic across runs (variant_to_enum HashMap owner-selection for variants
shared across aliased coproducts — ~39 files churn between two runs of the SAME
binary), so emit bit-identity is NOT a usable oracle here; correctness is proven by
the floor witnesses staying green by execution. Hand-edited the stage0 .rs seed (not
a regen) per established practice — the seed is hand-maintained, regen is not run by
build.rs/ci.yml, and a full regen would reintroduce the emitter's nondeterministic churn.

The build fail-closed caught 3 ParserParam-typed `params:` sites (Rc<Vec<Rc<ParserParam>>>,
not Node) — left as Rc::new(vec![]), correct.

Not bundled (verified, deliberately out of scope):
- Node.ident delete: REFUTED — ident is set to Some on module/import nodes
  (parse.rs mod_ir.id / imp_ir.id) and read in 3 live type-lookup fast-paths
  (lookup_type_for, is_recursive_type_for, dag_node_surface_fingerprint). It is a
  live ident-keyed fast-path, not dead; deleting it is a behavior change, not a no-op.
- Synthetic :0..0 span sentinel (make_span(0,0) -> shared ZERO_SPAN): same pattern,
  but measured ~0.58 MiB — sub-MiB hygiene, left for a follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@briansrls
briansrls merged commit be26e1c into main Jun 27, 2026
2 checks passed
@briansrls
briansrls deleted the fix/node-empty-list-singleton branch June 27, 2026 22:56
briansrls pushed a commit that referenced this pull request Jun 28, 2026
Brian Searls @claude
Merge origin/main (#5874/#5878/#5880) into cutover branch; seed reset…
3610179 
… to main (re-regen reconciles), Path-B HAND_MAINTAINED re-applied for main.rs/parse.rs

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gunbai-bot Bot pushed a commit that referenced this pull request Jun 28, 2026
Brian Searls @claude
Restore compilable pre-regen interim: all source authoring (reg-2 A1+…
e72f61a 
…S5, reg-3, both grounded witnesses, extdeps_uri_path reg) on main seed; regen HELD pending #5878 empty_node_list ruling

The faithful regen is blocked: #5878's empty_node_list thread_local singleton is an
ungrounded seed-cement (hand-edited into v1_std_core.rs seed, not in any .dag); a regen
drops the definition while 2 HAND_MAINTAINED files (parse.rs/dag_collect_support.rs)
still call it. Escalated to manager (empty_node_list grounding option + #5878-documented
emitter non-determinism vs RegenVerifyGate bit-identity premise). Seed kept = origin/main
(compilable) until the ruling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
briansrls added a commit that referenced this pull request Jun 28, 2026
@briansrls Brian Searls @claude
Two-generation regen cutover + dissolve parked regen_verify_gate plan…
09014de 
… into executable RegenVerifyGate (cargo-green self-host fixed point, ONE PR) (#5873)

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* Part B: dissolve parked regen_verify_gate plan into executable RegenVerifyGate

Adds RegenVerifyGate to the CI floor (ci_spec type Gate + gunbc_ci_floor_gates),
wires all exhaustive Gate matches (ci_floor_plan gate_node/gate_runnable/
gate_is_heavy_resolve/gate_spawns_host_compiler, ci_gates.run_spec_gate,
floor_effect_gate_witness). New dsl/tools/regen_verify_{gate,transport}.dag run
regen_stage0 --verify (the §7 self-host fixed-point wall) via a new
ensure_regen_stage0_built host-prelude helper. Deletes the parked plan, its
empty plan_registry_batch_a, the generated required-facts md, and repoints the
roadmap carrier.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* Part A: two-generation cutover regen to the cargo-green self-host fixed point

Applies #5865's casing fix (05_emit_rust.dag) and runs the 2-pass cutover regen:
pass-1 advances the self-host blob + emitter (gen-1 drops v1_rt starts_with/
ends_with/trim, the one-generation-ahead seed-cement), rebuild, pass-2 re-emits
them fresh (PascalCase). regen_stage0 --verify now exits 0 (a 3rd pass is
byte-identical = THE fixed point). Registers extdeps_uri_path.rs in
GENERATED_STAGE0_FILES (the legit unregistered-emit gap). Regenerates ROADMAP.md.
Drift checks: cargo-header reproduced via emit_cargo_toml, wire_value_serialize.rs
unchanged (hand-maintained, copy-preserved), v1_rt.rs emitted-fresh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* Path B: parse.rs hand-maintained + re-regen against fresh main (#5818/#5819)

Operator ruled Path B for the #5864 parser seed-cement: v1_compiler_parse.rs
moves to HAND_MAINTAINED_STAGE0_FILES (copy-preserved, so #5864's O(N) cursor
optimization survives the regen instead of being reverted to the O(N^2) baseline
that 02_parse.dag still models). Named dissolution trigger marked on the carrier.
Re-regenerated the two-generation cutover against fresh main (incl #5865 casing,
#5818 Uri grounding, #5819 ci.yml thin-shim). regen_stage0 --verify exit 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* Fix stage0_core crate membership + hand-test for the faithful fixed point

The full-workspace build (the broadened oracle CI rust_tests uses) exposed two
more pre-existing main non-fixed-points that the cutover surfaces:

1. stage0_core E0432/E0433: the faithful regen emits use crate::extdeps_uri /
   extdeps_external_authority in the extdeps_languages_* modules and
   use crate::std_realization_schedule/std_decl_ref in std_effects/std_emit_model,
   but stage0_crates.dag's CoreCrate modules list omitted them. Added the full
   transitive closure (9 modules: extdeps_external_authority, extdeps_uri,
   std_decl_ref, std_lens_verdict, std_magnitude, std_measure, std_nat,
   std_pareto, std_realization_schedule) so stage0_core is closed under crate:: deps.

2. effects.rs (hand-test) imported parse_path_template/PathTemplateParseResult from
   std_http_path, but #5818 moved them to extdeps_uri_path in the authority (main
   never regenerated, so its stale seed + test still pointed at std_http_path).
   Repointed the imports to the fixed-point location.

regen_stage0 --verify exit 0; cargo build --workspace --all-targets -D warnings 0/0;
parse.rs still byte-identical to origin/main (#5864 preserved).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* Merge origin/main (#5874/#5878/#5880) into cutover branch; seed reset to main (re-regen reconciles), Path-B HAND_MAINTAINED re-applied for main.rs/parse.rs

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Restore compilable pre-regen interim: all source authoring (reg-2 A1+S5, reg-3, both grounded witnesses, extdeps_uri_path reg) on main seed; regen HELD pending #5878 empty_node_list ruling

The faithful regen is blocked: #5878's empty_node_list thread_local singleton is an
ungrounded seed-cement (hand-edited into v1_std_core.rs seed, not in any .dag); a regen
drops the definition while 2 HAND_MAINTAINED files (parse.rs/dag_collect_support.rs)
still call it. Escalated to manager (empty_node_list grounding option + #5878-documented
emitter non-determinism vs RegenVerifyGate bit-identity premise). Seed kept = origin/main
(compilable) until the ruling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

* WIP: Two-generation regen cutover + dissolve parked regen_verify_gate plan in

---------

Co-authored-by: Brian Searls <briansearls1@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@gunbai-bot gunbai-bot Bot mentioned this pull request Jun 28, 2026
Merged
4 of 5 tasks
briansrls added a commit that referenced this pull request Jun 28, 2026
@briansrls @claude
Enrich Lever B in representation_minimization.dag with arena/u32-inde…
f32054c 
…x deepening.

The arena/u32-index analysis is a deepening of Lever B (same lever: bytes-per-node;
same gate: #5879), not a separate plan. Single-authority in representation_minimization.dag.

- Lever B item in sequencing ol() updated to name two sub-moves: (i) minimal
  per-kind sum type [existing] + (ii) arena/index container [new], both gated on
  #5879, both landing in v2/.dag realization.
- Added two-paired-axes note: Lever B attacks BYTES-PER-NODE; Lever C attacks
  MODULES-HELD-AT-ONCE; orthogonal, compose multiplicatively, neither blocks the other.
- Added "Lever B (item 3) — arena/index deepening" section with: verified Node anatomy
  (size_of=144 B, Rc-wrapped 160 B/node, 12 pointer fields) reconciled with the
  existing 18-field/627k-nodes framing; arena/u32-index insight (ACYCLIC DAG = Rc
  pays for unused cycle-safety; u32 arena index = 4 B vs 16 B Rc ctrl, halves 12
  ptr fields, ~40-55% node-graph cut); cheap recoverables (.dag-first, independent
  of emitter gate) with corrections: empty-Vec singleton win already landed (#5878),
  Node.ident is LIVE (u32-shrink fine, deletion refuted by execution).
- Reverted incorrectly-created arena_node_representation.dag (§3 fork) and registry
  entry.
- Regenerated docs/plans/representation-minimization.md via wet gate (drift-clean).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
briansrls added a commit that referenced this pull request Jun 29, 2026
@gunbai-bot @briansrls @claude
Enrich Lever B (representation-minimization) with arena/u32-index dee…
2d190b6 
…pening (#5923)

* WIP: Write up arena/u32-index Node representation as a design note (plan proj

* Enrich Lever B in representation_minimization.dag with arena/u32-index deepening.

The arena/u32-index analysis is a deepening of Lever B (same lever: bytes-per-node;
same gate: #5879), not a separate plan. Single-authority in representation_minimization.dag.

- Lever B item in sequencing ol() updated to name two sub-moves: (i) minimal
  per-kind sum type [existing] + (ii) arena/index container [new], both gated on
  #5879, both landing in v2/.dag realization.
- Added two-paired-axes note: Lever B attacks BYTES-PER-NODE; Lever C attacks
  MODULES-HELD-AT-ONCE; orthogonal, compose multiplicatively, neither blocks the other.
- Added "Lever B (item 3) — arena/index deepening" section with: verified Node anatomy
  (size_of=144 B, Rc-wrapped 160 B/node, 12 pointer fields) reconciled with the
  existing 18-field/627k-nodes framing; arena/u32-index insight (ACYCLIC DAG = Rc
  pays for unused cycle-safety; u32 arena index = 4 B vs 16 B Rc ctrl, halves 12
  ptr fields, ~40-55% node-graph cut); cheap recoverables (.dag-first, independent
  of emitter gate) with corrections: empty-Vec singleton win already landed (#5878),
  Node.ident is LIVE (u32-shrink fine, deletion refuted by execution).
- Reverted incorrectly-created arena_node_representation.dag (§3 fork) and registry
  entry.
- Regenerated docs/plans/representation-minimization.md via wet gate (drift-clean).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix gating contradiction: Lever C has no emitter-gate dependency.

The two-axes note incorrectly said both levers are gated on #5879; Lever C
explicitly has NO dependency on the emitter gate (sequencing item 2 and the
A3 disposition both state this). Corrected paragraph now precisely states:
Lever B gated on #5879; Lever C deferred to v2 streaming-infer, no emitter
gate. Regen'd representation-minimization.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix stale kernel-binding citations: infer.rs -> v1_compiler_infer.rs, line 13854->13998.

The live seed file is src/v1/stage0/src/v1_compiler_infer.rs, not infer.rs.
seed_kernel_intern_table is at line 13998 (not 13854 as cited).
kernel_bindings_base at 11695 and intern TABLE at 11641 are correct lines.
Regen'd representation-minimization.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* CI retry (OOM on srv1-01 in batch 2, runner-level flake)

* Rebucket cheap-recoverables: correct gating claim, separate v1-seed from .dag-path.

The 'independent of emitter gate / .dag-first' framing was contradictory: kernel-slice
sharing targets v1_compiler_infer.rs (doomed v1 seed, bank-if-cheap per plan's
discriminator); ident u32-shrink and SourceSpan file-id are Node/SourceSpan
representation changes whose .dag-authority path IS gated on #5879 like Lever B.

Per-item gating now stated explicitly:
- Empty-Vec singleton: already landed, remaining is bank-if-cheap v1-seed work
- Ident u32-shrink / file-id side table: .dag path gated on #5879; v1-seed is bank-if-cheap
- Kernel-slice sharing: v1-seed work, bank-if-cheap, NOT a root item
- Node.name interning: item 5 bank-if-cheap, unchanged

Decidability section corrected: sub-moves i/ii are RATCHETS gated on #5879;
ident/file-id are representation changes gated on #5879 on the .dag path;
kernel-slice is v1-seed bank-if-cheap (not WALL-NOW in .dag).
Regen'd representation-minimization.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Brian Searls <briansrls@gunb.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@briansrls