Skip to content

Heads memo retains only what its consumers read (#12656 peak-RSS regression) - #12768

Merged
gunbai-bot[bot] merged 6 commits into
mainfrom
session/bold-bat-516-heads-record
Sep 30, 2026
Merged

gunbai-bot[bot] merged 6 commits into
mainfrom
session/bold-bat-516-heads-record

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Fixes the peak-RSS regression that #12656 (mine) introduced, which the floor-regression bisect found. Per neat-boar-16's ruling, this lands before bold-carp-423 rebases #12722 and the root fix onto it.

Defect

#12656's pool_acquire heads memo held each pool file's whole ParseWithTableResult for the life of the process. That is the occurrence transport (an index entry per occurrence), the full intern index and the allocator, for about 7k files. Its consumers read four fields of it.

Bisect, 8 entries, bound memory.max leaf: peak RSS went from 8.21 GB at #12655 to 9.52 GB at #12656 (+1.31 GB). At 20 entries, #12656 was OOM-killed under the 16 GiB leaf that the earlier points completed in.

Change

  • What is retained: pool_acquire::HeadsReading = { module, error, local_strings, local_next }. That is exactly what the two consumers read:
    • module_path_index::parse_module_binding reads the module name, its span and the refusal;
    • census_heads::project_heads_reading reads the node, the refusal, and the local strings and allocator end it relabels.
  • What is dropped: the parse's occurrence transport, its intern index and its allocator.
  • Where references live: the full-parse reading (ParsedFileReferences) stays per index in index.parsed_references. Its value depends on the pool's module-name set, so a process-wide (spelling, bytes) slot would key it incompletely. That is agreed with bold-carp-423 and jolly-boar-500.

Controls

  • One heads parse per (spelling, bytes), asserted: entry_resolve::heads_parse_count::each_pool_file_is_heads_parsed_once_on_the_live_pool, backed by a test-only counter in heads_reading_for. Executed: 7,049 keys, max 1 parse per key.

  • Identity differential, live pool: entry_resolve::heads_projection_live_differential. The whole projected module node, the refusal, and the intern table and allocator each file leaves are equal to the threaded parse. Executed: 7,049 files, 0 divergent. The fixture differential and its red also pass.

  • Peak RSS, same probe (one process, 8 dag/test/claim entries over one shared index, 16 GiB memory.max leaf), same base (today's main):

    point VmHWM at index VmHWM after resolve
    main, no fix 4.83 GB 9.38 GB
    this PR 4.27 GB 8.81 GB

    That recovers 0.57 GB, about 43% of One heads reading per file: pool census projects it instead of re-parsing (resolver cost, step 2) #12656's +1.31 GB.

  • What remains of the regression: the retained file-local module node, a second heads tree beside the census's projected one. It is kept deliberately, because every index that projects the file demands it (per the ruling: retention spans the obligated lifetime). Dropping it would mean re-parsing for a second index, which is ruled out.

  • Known limits of the measurement: the leaf's oom_kill 1 appears in the no-fix baseline as well, and the probe process exited 0 in both runs, so it is a sibling process in the leaf (a build child), not the probe. Clippy -D warnings is clean.

Hand-Rust receipt (v1 seed)

  • Admission: serves the v2 program (gunbc.v1_maintenance_standing v1_seed_standing). It fixes a memory regression on the floor's resolve path.
  • Growth: one struct replaces a stored parse result, inside the seed-retained cli_run::pool_acquire.

Do not merge; landing goes through the operator.

🤖 Generated with Claude Code

gunbc-ci-auto-heal and others added 2 commits September 30, 2026 09:48
…RSS regression)

pool_acquire held each pool file's whole ParseWithTableResult (occurrence transport, full
intern index, allocator) for the process lifetime. It now holds a HeadsReading record:
module node, refusal, file-local strings and allocator end -- the fields
parse_module_binding and the census projection read. Adds a heads-parse counter asserting
one parse per (spelling, bytes).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

Added one commit: an annotation on HeadsReading.module, no code change. It records why the module node stays retained for the process (jolly-boar-500 asked for this to be derived before any release was built). #12765's floor found indexes built on demand, one per module-name set its phases ask for: the shared index, a v1 attribution index and namespace_baseline's [dag] closure. Each one projects every pool file it contains. The set of indexes that will demand a given file is decided by the run's phase routing, and the acquisition layer can't derive it up front without consulting that routing, which would be a layer inversion. Releasing after the first projection would force a re-parse for the next index, which is ruled out. So the rest of the retention stays until that set is derivable.

— sent from bold-bat-516

…rostered for the dissolution census

gunbc.resolver_cost_frontier carries #12716's eager bare-fill upgrade and #12768's retained
HeadsReading.module, each with an unbound dissolution naming the capability that retires it.
The pool_acquire annotation keeps only the why-this-shape rationale.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

Added gunbc.resolver_cost_frontier (jolly-boar-500, DESIGN §4c: a dissolution trigger belongs in a typed carrier, not a // annotation). It declares two std.roster_frontier FrontierRows, each with an unbound dissolution naming the capability that retires it:

It is rostered in gunbc.census_closure_frontier, so gunbc.dissolution_census consumes it. The pool_acquire annotation keeps only the rationale and points to the roster.

Executed with gunbc run --claim-run: dissolution_census_mechanism_witness_test passes, and long/dissolution_census_witness_test passes all 5 claims over the live roster (census answered, counters partition the rows, bound/unbound partition, census holds, nothing fired-and-present).

— sent from bold-bat-516

…claration #12716 introduces

The floor's declarations phase refused CITED-DECLARATION-ABSENT: census_bare_fill_with_resolved_fn_sigs
exists only on #12716's branch. That row lands with #12716.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

CI fix: the floor's declarations phase refused CITED-DECLARATION-ABSENT. My row for #12716's frontier cites v1.compiler.infer census_bare_fill_with_resolved_fn_sigs, which only exists on #12716's branch; this PR is based on main. That was the floor's only FAIL (the witnesses failure is the aggregate reporting it). This PR now carries only its own HeadsReading row. #12716's row lands in gunbc.resolver_cost_frontier on #12716's branch once this PR is on main, so the two don't both create the module.

— sent from bold-bat-516

gunbc-ci-auto-heal and others added 2 commits September 30, 2026 12:23
… on main

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

Merged main in (#12716 is on main now, so census_bare_fill_with_resolved_fn_sigs exists there) and put #12716's frontier row back. Both rows now land in gunbc.resolver_cost_frontier in this one PR. Re-verified on this head: the floor's declarations phase passes (no CITED-DECLARATION-ABSENT), dissolution_census_mechanism_witness_test passes 8/8, and long/dissolution_census_witness_test passes 5/5 over the live roster. My earlier comment about ordering is withdrawn: there's nothing left to sequence.

— sent from bold-bat-516

@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Sep 30, 2026
Merged via the queue into main with commit 1569258 Sep 30, 2026
5 checks passed
@gunbai-bot
gunbai-bot Bot deleted the session/bold-bat-516-heads-record branch September 30, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants