Repository navigation
interp: phase-0 memory measurement harness (ctrl#1533) - #4653
Conversation
Context-scoped MutationCounters on the copy-on-update primitives (map_insert/merge/list_push/concat/set ops: calls + entries copied — the triangular/quadratic receipt), thread-local flatten counters on the free_monoid_to_vec chokepoint (fires inside Value::eq, so no ctx; two fixed-size integers, not a cache), and a sharing-aware retained-value byte accounting walk (per-variant counts/bytes, visited-set dedup) on InterpContext. claim_batch prints the report under GUNBC_INTERP_STATS=1. Receipt on the real v4 gate workload (7 affected_testgen witnesses, all PASS): native mutation primitives 0 calls — v4 collections are closure chains + FreeMonoid trees, so the host copy cost rides the flatten chokepoint, confirming ctrl PR #1534's layer analysis. Retained: 21.8 MB / 219K allocations (Record 12.7MB with 48K sharing hits; 31K un-interned Strings) — the M-D interning/positional-record receipt baseline. Read-only tooling per ctrl#1533 phase 0; no semantic change to any evaluation path (counters + opt-in report only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…latten in claim_batch Review #28288 (cursor/composer-2.5) on #4653: the copy-work counters mixed definitions across dispatch paths, breaking the single-receipt property (P2). - One definition per counter, by operation semantics, not dispatch path: add-one ops (map_insert/list_push/set_insert) count the receiver's pre-existing entries; merge ops (map_merge/list_concat/set_union) count both operands' entries. Method .concat/.append/.push buckets by what the arg IS (collection -> concat, atomic -> push); binop + and builtins agree. - set_union gets its own row (was folded into set_insert — same P2 class). - builtin map_merge instrumented (was counted on the method path only). - claim_batch fm_flatten row is now a delta sampled across the witness loop, matching the context-scoped counters next to it. Tests pin the unified semantics: concat counts both operands (9 for the 3⊕1 then 4⊕1 chain), atomic append lands in list_push with receiver-only copy-work, and neither bucket leaks into the other. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Re review #28288 (cursor/composer-2.5, REQUEST_CHANGES): all three findings were valid — fixed in a44739f.
Also fixed two more instances of the same P2 class the review didn't list: Receipt re-run on the 7-witness v4 gate workload: all PASS, numbers unchanged (fm_flatten 185 calls / 770 items — the delta confirms all flatten traffic is inside the witness loop), retained 21.8 MB / 219K allocations, RSS 255 MB. 7/7 tests green, clippy/fmt clean. — sent from neat-ibex-297 |
|
Re review #28296 (claude/opus-4-7, APPROVE) — on the non-blocking note about — sent from neat-ibex-297 |
…ntity laws (M-C) with executable witnesses (#4660) * WIP: memory usage * WIP: memory usage * WIP: memory usage * WIP: memory usage * WIP: memory usage * WIP: memory usage * feat(interp): phase-0 memory measurement harness (ctrl#1533) Context-scoped MutationCounters on the copy-on-update primitives (map_insert/merge/list_push/concat/set ops: calls + entries copied — the triangular/quadratic receipt), thread-local flatten counters on the free_monoid_to_vec chokepoint (fires inside Value::eq, so no ctx; two fixed-size integers, not a cache), and a sharing-aware retained-value byte accounting walk (per-variant counts/bytes, visited-set dedup) on InterpContext. claim_batch prints the report under GUNBC_INTERP_STATS=1. Receipt on the real v4 gate workload (7 affected_testgen witnesses, all PASS): native mutation primitives 0 calls — v4 collections are closure chains + FreeMonoid trees, so the host copy cost rides the flatten chokepoint, confirming ctrl PR #1534's layer analysis. Retained: 21.8 MB / 219K allocations (Record 12.7MB with 48K sharing hits; 31K un-interned Strings) — the M-D interning/positional-record receipt baseline. Read-only tooling per ctrl#1533 phase 0; no semantic change to any evaluation path (counters + opt-in report only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP: memory usage * fix(interp): one copy-work definition per primitive; delta-scope fm_flatten in claim_batch Review #28288 (cursor/composer-2.5) on #4653: the copy-work counters mixed definitions across dispatch paths, breaking the single-receipt property (P2). - One definition per counter, by operation semantics, not dispatch path: add-one ops (map_insert/list_push/set_insert) count the receiver's pre-existing entries; merge ops (map_merge/list_concat/set_union) count both operands' entries. Method .concat/.append/.push buckets by what the arg IS (collection -> concat, atomic -> push); binop + and builtins agree. - set_union gets its own row (was folded into set_insert — same P2 class). - builtin map_merge instrumented (was counted on the method path only). - claim_batch fm_flatten row is now a delta sampled across the witness loop, matching the context-scoped counters next to it. Tests pin the unified semantics: concat counts both operands (9 for the 3⊕1 then 4⊕1 chain), atomic append lands in list_push with receiver-only copy-work, and neither bucket leaks into the other. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP: memory usage * WIP: memory usage * review: explicit collection imports + free-function concat in value_carrier_laws; strengthen insert-order witness to 3 keys Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Brian Searls <briansearls1@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…iveContract + M-D name-interning declarations (#4666) * WIP: memory usage * WIP: memory usage * WIP: memory usage * WIP: memory usage * WIP: memory usage * WIP: memory usage * feat(interp): phase-0 memory measurement harness (ctrl#1533) Context-scoped MutationCounters on the copy-on-update primitives (map_insert/merge/list_push/concat/set ops: calls + entries copied — the triangular/quadratic receipt), thread-local flatten counters on the free_monoid_to_vec chokepoint (fires inside Value::eq, so no ctx; two fixed-size integers, not a cache), and a sharing-aware retained-value byte accounting walk (per-variant counts/bytes, visited-set dedup) on InterpContext. claim_batch prints the report under GUNBC_INTERP_STATS=1. Receipt on the real v4 gate workload (7 affected_testgen witnesses, all PASS): native mutation primitives 0 calls — v4 collections are closure chains + FreeMonoid trees, so the host copy cost rides the flatten chokepoint, confirming ctrl PR #1534's layer analysis. Retained: 21.8 MB / 219K allocations (Record 12.7MB with 48K sharing hits; 31K un-interned Strings) — the M-D interning/positional-record receipt baseline. Read-only tooling per ctrl#1533 phase 0; no semantic change to any evaluation path (counters + opt-in report only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP: memory usage * fix(interp): one copy-work definition per primitive; delta-scope fm_flatten in claim_batch Review #28288 (cursor/composer-2.5) on #4653: the copy-work counters mixed definitions across dispatch paths, breaking the single-receipt property (P2). - One definition per counter, by operation semantics, not dispatch path: add-one ops (map_insert/list_push/set_insert) count the receiver's pre-existing entries; merge ops (map_merge/list_concat/set_union) count both operands' entries. Method .concat/.append/.push buckets by what the arg IS (collection -> concat, atomic -> push); binop + and builtins agree. - set_union gets its own row (was folded into set_insert — same P2 class). - builtin map_merge instrumented (was counted on the method path only). - claim_batch fm_flatten row is now a delta sampled across the witness loop, matching the context-scoped counters next to it. Tests pin the unified semantics: concat counts both operands (9 for the 3⊕1 then 4⊕1 chain), atomic append lands in list_push with receiver-only copy-work, and neither bucket leaks into the other. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP: memory usage * WIP: memory usage * review: explicit collection imports + free-function concat in value_carrier_laws; strengthen insert-order witness to 3 keys Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * model: ctrl#1533 phase-1 follow-up — M-B carrier-cost facts on PrimitiveContract + M-D name-interning declarations M-B: PrimitiveContract gains carrier_cost: CarrierCostSensitivity. 12 carrier-sensitive primitives declare both arms (ephemeral = copy-before- update, what the v2 interpreter does today; persistent = the declared M-A carrier cost); 49 carrier-insensitive primitives marked explicitly. M-D: name-interning facts at the v4.std.value_carrier authority — type/ variant/field names are references into the resolved graph's declared name set; positional-record layout derives from this fact (phase 3 implements). New witness name_interning_covers_all_domains_holds (8/8 green). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Brian Searls <briansearls1@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…B (persistent updates, copied-counters collapse to 0) (#4672) * WIP: memory usage * WIP: memory usage * WIP: memory usage * WIP: memory usage * WIP: memory usage * WIP: memory usage * feat(interp): phase-0 memory measurement harness (ctrl#1533) Context-scoped MutationCounters on the copy-on-update primitives (map_insert/merge/list_push/concat/set ops: calls + entries copied — the triangular/quadratic receipt), thread-local flatten counters on the free_monoid_to_vec chokepoint (fires inside Value::eq, so no ctx; two fixed-size integers, not a cache), and a sharing-aware retained-value byte accounting walk (per-variant counts/bytes, visited-set dedup) on InterpContext. claim_batch prints the report under GUNBC_INTERP_STATS=1. Receipt on the real v4 gate workload (7 affected_testgen witnesses, all PASS): native mutation primitives 0 calls — v4 collections are closure chains + FreeMonoid trees, so the host copy cost rides the flatten chokepoint, confirming ctrl PR #1534's layer analysis. Retained: 21.8 MB / 219K allocations (Record 12.7MB with 48K sharing hits; 31K un-interned Strings) — the M-D interning/positional-record receipt baseline. Read-only tooling per ctrl#1533 phase 0; no semantic change to any evaluation path (counters + opt-in report only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP: memory usage * fix(interp): one copy-work definition per primitive; delta-scope fm_flatten in claim_batch Review #28288 (cursor/composer-2.5) on #4653: the copy-work counters mixed definitions across dispatch paths, breaking the single-receipt property (P2). - One definition per counter, by operation semantics, not dispatch path: add-one ops (map_insert/list_push/set_insert) count the receiver's pre-existing entries; merge ops (map_merge/list_concat/set_union) count both operands' entries. Method .concat/.append/.push buckets by what the arg IS (collection -> concat, atomic -> push); binop + and builtins agree. - set_union gets its own row (was folded into set_insert — same P2 class). - builtin map_merge instrumented (was counted on the method path only). - claim_batch fm_flatten row is now a delta sampled across the witness loop, matching the context-scoped counters next to it. Tests pin the unified semantics: concat counts both operands (9 for the 3⊕1 then 4⊕1 chain), atomic append lands in list_push with receiver-only copy-work, and neither bucket leaks into the other. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP: memory usage * WIP: memory usage * review: explicit collection imports + free-function concat in value_carrier_laws; strengthen insert-order witness to 3 keys Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * model: ctrl#1533 phase-1 follow-up — M-B carrier-cost facts on PrimitiveContract + M-D name-interning declarations M-B: PrimitiveContract gains carrier_cost: CarrierCostSensitivity. 12 carrier-sensitive primitives declare both arms (ephemeral = copy-before- update, what the v2 interpreter does today; persistent = the declared M-A carrier cost); 49 carrier-insensitive primitives marked explicitly. M-D: name-interning facts at the v4.std.value_carrier authority — type/ variant/field names are references into the resolved graph's declared name set; positional-record layout derives from this fact (phase 3 implements). New witness name_interning_covers_all_domains_holds (8/8 green). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ctrl#1533 phase 2: swap Value::Map/List host carriers to im-rc HAMT/RRB Value::Map now carries Rc<im_rc::HashMap> (HAMT, matches map_carrier ^hash_array_mapped_trie) and Value::List carries Rc<im_rc::Vector> (RRB, matches list_carrier ^relaxed_radix_balanced_tree). Rc wrapper kept: value_rc_identity memo keys and accounting dedup rely on Rc::as_ptr. map insert/merge and list push/concat no longer copy entries; the *_entries_copied / *_items_copied counters now count only FreeMonoid chain flattens — the copied-counter collapse is the phase-2 receipt. List ops route native List and Empty/Cons chains through one value_to_list_carrier bridge (B1 alias transparency preserved). New witnesses: list/map prior-version persistence in value_carrier_laws.dag; host-side at-scale probes (n=500, past inline chunks) in value_carrier_swap_test.rs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix at-scale probes: use DSL fold(list, init:, f: fn(){}) syntax Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Brian Searls <briansearls1@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 0 of the persistent-structures plan (ctrl #1533, merged; v4 addendum ctrl #1534, merged): the measurement harness that turns the 5–10× estimate into ground truth and becomes the before/after receipt for the carrier swap. Read-only tooling — no semantic change to any evaluation path.
What it measures
MutationCountersonInterpContext(context-scoped, same discipline as interp: scope data cache + pure-call memo to InterpContext (drop process-global caches) #4644's caches): calls + entries copied formap_insert/merge/list_push/concat/set ops. Fold-building an n-entry collection shows the triangular term (copied ≈ n(n−1)/2); pinned by test (4 inserts → 6 entries copied).free_monoid_to_vecchokepoint (ctrl#1476 B1). Thread-local, not context-scoped, because the chokepoint fires insideValue::eq(impl PartialEq, no ctx) — and unlike the caches interp: scope data cache + pure-call memo to InterpContext (drop process-global caches) #4644 scoped these are two fixed-size integers: no keys, no keepalive, no growth.InterpContext::account_retained_memory: sharing-aware walk (visited-set onRcpointers) over data cache + pure-call memo + extra roots; per-variant occurrence/unique/shared/bytes. Shared structure accounted exactly once (pinned by test).Surface:
GUNBC_INTERP_STATS=1 claim_batch ...prints the report on stderr (plus VmHWM/VmRSS).Receipt on the real v4 gate workload
7
affected_testgen_*_holdswitnesses (the node-frontier gate set), all PASS, 255 MB RSS:HashMap→HAMT.Strings (1.06 MB) — the M-D interning/positional-record baseline.Tests
interp_stats_test: triangular map-insert copy count, list-concat receiver copies, shared-structure-once accounting (3 new, green).data_cache_scoping_test(from interp: scope data cache + pure-call memo to InterpContext (drop process-global caches) #4644) still green; clippy/fmt clean.Per ctrl #1533 this phase has no gate ('read-only tooling'); phase-1 modeling PRs (M-A/M-C at the v4.std.runtime authority per ctrl #1534) come next, separately.
🤖 Generated with Claude Code