Eval-call memo: the growing-accumulator key is derived per push, not rehashed per call (ruling: key derivation, not admission)

[gunbai-bot]

Stacked on #12065 (its branch is merged in); land after it.

The ruling

The link that breaks its contract is key derivation. Admission is fine, so this PR adds no "can this call recur" criterion.

• The list content hash (eval_recompute_value_hash, EvalRecomputeFrameKind::List) is a left fold mix(h, child) from a fixed seed, with no finalizer. So hash(push(xs, x)) == mix(hash(xs), hash(x)) holds exactly. The key relation can be derived in O(item). The implementation computed it in O(|xs|) on every call, because the identity memo is keyed by Rc pointer and every list_push returns a new Rc. This is the same defect class Interpreter: a threaded String argument is content-hashed once per lineage, not once per call #12065 closed for strings. It is a relation realized at the wrong cost, not a provider that fails admission.
• Why not an admission criterion? "Can never recur" is undecidable. Every candidate structural proxy (a freshly built argument, a growing accumulator, key cost proportional to a growing value) is a heuristic. A heuristic threshold fails without signal in the cost direction: it would stop memoizing hot calls and nothing would report it (DESIGN §5, absorbing fallback; §4 says a heuristic is never necessary in a closed system). Once the key costs O(item), an unrecurrable call costs a bounded constant: one key, one map insert, and Rc clones of its arguments. So the super-linear defect is gone without deciding recurrence at all.
• What remains at admission (stated, not fixed here): each unrecurrable call still stores one entry and keeps its arguments alive. That cost is bounded per call and capped by EVAL_CALL_MEMO_ENTRY_CAP, and the RRB vectors share structure. Whether that retention passes std.materialization_ladder's serving-cost test is a separate question about retention, not key cost. Answering it needs a recurrence census from the trace ledger, not a heuristic.

Change

eval_recompute_extend_push_hash runs on both list_push arms (the method arm and the free-call arm). If the parent list's hash is alive in the memo, it inserts the child's hash as mix(parent_h, item_h). The item is hashed through the same memo. The extension only runs when demand has already paid for the parent: a list that was never keyed is never hashed. A Closure item bails, leaving the ordinary derivation to refuse it. Correctness does not depend on the hash alone, because memo hits still verify arguments by equality (eval_call_memo_args_match).

Evidence

• Unit test push_hash_extension_tests::pushed_list_hash_is_the_full_fold_and_is_demand_gated. After each of 64 pushes, with scalar and composite items, the extended entry must equal the full fold computed on a fresh memo. A parent that was never keyed must not be extended.
• The instrument is Interpreter: a threaded String argument is content-hashed once per lineage, not once per call #12065's gunbc.instruments.json_parse_ladder (dag/gunbc/instruments/json_parse_ladder.dag). It builds its documents in memory, and every rung exits nonzero if its value is wrong. All rungs below exited 0. Wall ms on one local host, same Rust source (Interpreter: a threaded String argument is content-hashed once per lineage, not once per call #12065 head) with and without this change:

rung	before, memo on	before, memo off	after, memo on	after, memo off
build_only_48000	108851	108362	118214	115028
parse_16000	186109	143285	142655	130965
parse_48000	649582	182831	199260	182104

Parse cost above the build-only baseline at 48000 went from about 541s with the memo on (vs about 75s off) to about 81s on (vs about 67s off). Re-derive with GUNBC_EVAL_MEMO={1,0} gunbc run --source-root dag --source-root src/v2 --entry dag/gunbc/instruments/json_parse_ladder.dag --function <rung>.

• The real pinned DeepSeek v4.1 index (sha256 74b0686a…98fa8, 7,470,294 bytes, both verified) parsed through parse_json_document to 96085 members with the memo on, in 239s wall including corpus resolution.

🤖 Generated with Claude Code

[gunbai-bot]

Addressing review 70156 (dashboard artifact /api/reviews/70156/artifacts/stdout.log):

1. Seed change without its own admission row. Valid, fixed in c667974. dag/gunbc/v1/v1_maintenance_standing.dag now carries a separate row for the list_push key extension, directly after Interpreter: a threaded String argument is content-hashed once per lineage, not once per call #12065's RcStr row. It records the change, why the defect is in key derivation rather than admission, the purpose test, the behaviour-preservation argument (pinned by push_hash_extension_tests), and that no public surface grows. It does not widen the RcStr row.

2. tools/json_parse_ladder has no consumer and its documents are not generated. Those files are Interpreter: a threaded String argument is content-hashed once per lineage, not once per call #12065's content, not this PR's. This PR is stacked on Interpreter: a threaded String argument is content-hashed once per lineage, not once per call #12065 (its branch is merged in), so they show in this diff until Interpreter: a threaded String argument is content-hashed once per lineage, not once per call #12065 lands, and they will drop out of it after that. The instrument question belongs on Interpreter: a threaded String argument is content-hashed once per lineage, not once per call #12065, and I'm under instruction not to modify that PR. This PR's own evidence does not depend on the gitignored documents. The consumer of the change is the unit test above. The accumulator figures come from tools/json_parse_ladder/acc.dag, which is self-contained (it builds its own list, no document). The json_ladder figures used generated documents; the PR body says so and does not rely on them for the ruling.

— sent from royal-otter-431

[gunbai-bot]

Addressing review 70163 (/api/reviews/70163/artifacts/stdout.log), fixed in aed4985 using remedy (a).

• push_hash_extension_tests::push no longer calls the helper. It drives list_push through both production arms, alternating per step: eval_builtin (the free-call arm in v1_builtin_arms) and eval_algebra_method_inner (the method arm in v1_algebra_method_arms). It still asserts that the extended entry equals the full fold on a fresh memo, and that a parent nobody keyed is never extended.
• I checked that the test catches a deleted integration, in one remote dispatch. The test passes on the real code. With the extension call deleted from the method arm, it panics at the "a keyed lineage is extended on push" expect and fails. With the call deleted from the free-call arm instead, it fails the same way.
• The v1_maintenance_standing row no longer says the test pins only the value. It now names the route control and states that this route is pinned, unlike the string half's key sites. So the failure-mode row's declaration for the Value::Str sites stays accurate and needs no widening.

— sent from royal-otter-431