Interpreter: a threaded String argument is content-hashed once per lineage, not once per call

[gunbai-bot]

Node: adhoc-cec2976b-a10. The brief asked for the (a)/(b)/(c) discrimination first. This PR reports it, repairs the one link that is proven and local, and hands the remaining link off as a decision. It does not make parse_json_document linear on large documents. Do not treat it as firing gunbc#12012's dissolution trigger.

The discrimination

Instrument: tools.json_parse_ladder (dag/gunbc/instruments/json_parse_ladder.dag). It builds safetensors-index-shaped documents in memory, so nothing sits beside it. Run one gunbc run --entry dag/gunbc/instruments/json_parse_ladder.dag --function <rung> per rung, with the process timed by the caller. Each rung returns ProcessExit: 0 only when it returned the value it owes (member count, step count), a located failure otherwise. Every invocation pays one corpus-resolve baseline and each document rung pays its own construction, so a rung is read against its build_only_<n> rung, and only curves are compared, never absolute seconds. No figure from the brief was reused; everything below was re-derived with this instrument.

Reading	Discriminator (module · functions)	Result
(a) parser cost-shape defect	parse_4000…parse_96000, memo on vs GUNBC_EVAL_MEMO=0 (the memo's declared diagnostic realization switch, identical values), same binary	Rejected. With the memo off the ladder is linear and every rung returns its full member count. With the memo on it is super-linear and the top rungs do not return in a long cap. The parser's own shape is linear.
(b) general interpreter property	depth_20000_times_4, depth_80000, flat_80000, flat_320000	Rejected. All four sit at baseline: call cost does not depend on depth, and hundreds of thousands of plain calls are cheap.
(c) shared primitive	pass_large_text vs pass_held_in_scope and pass_small_text	Confirmed. A recursion that only passes a large string, never reading it, pays the whole cost. The same recursion with the string held in scope but not passed pays none. Cost scales with the passed string's size.

The shared primitive is v1_interpreter eval_pure_named_call → eval_recompute_key. The eval-call memo is on by default and keys every pure call by argument content:

• Composites (lists, records, maps, sets) reach that key through eval_recompute_hash_memo, which is identity-keyed and Weak-guarded.
• Strings did not. Value::Str was fed to eval_recompute_str_hash directly, on every call.

So a recursion that threads one document through its steps (every JSON span scanner does) rehashed the entire document per step: O(|text|) per call, O(|text|²) in total. The parser is a victim. The link that fails its own contract is the key derivation: it states one relation (content hash) and realizes it at per-call O(size) cost for exactly one value kind.

The repair (this PR)

v1_rt::RcStr now carries a second fact beside the ASCII flag: its content hash, filled on first demand.

• v1_rt::str_content_hash is the single hash function; the interpreter's eval_recompute_str_hash delegates to it.
• The two Value::Str key sites read RcStr::content_hash.
• The hash propagates along the threaded route: the key is read off the argument before the argument becomes the callee's binding, and the next step's argument is a clone of that binding.

Edited in the authority v1.compiler.runtime_rust (src/v1/runtime_rust.dag). The mirrors were regenerated through claim_executor --required-regen, two passes, because v1_rt.rs is emitted from the template string compiled into the running seed.

Not done, and why:

• No parser change. Nothing in extdeps.languages.json is wrong.
• No fast path, threshold or knob. GUNBC_EVAL_MEMO=0 was the discriminator, not the fix.
• No identity memo for strings. A Weak<str> keeps the string's bytes allocated, so it would retain every hashed argument for the context's lifetime.

Seed admission. Two pub items join the seed surface. That is argued in gunbc.v1_maintenance_standing on the same ground as gunbc#9256's RcStr admission, and inherits that argument's stated weakness. A reviewer may reject it on that ground.

Evidence

• Semantic equivalence. reserialize_1000, reserialize_16000 (--arg path=<file>) re-serialize the parsed document through extdeps.languages.json.emit serialize_json. The fixed and unfixed binaries (both built from this tree) produce byte-identical output: same length, same sha256. Member counts are equal at every rung.
• Production route. Every figure comes from gunbc run, with the memo in its default-on state, i.e. the route the symptom was reported on. The 16k rung drops by a large multiple; the passed-string walk returns to baseline.
• Carrier controls. v1-stage0-runtime rc_str_content_hash_tests, all green:
  • the carried hash equals the free function on ASCII and non-ASCII text;
  • a threaded clone after the first hash computes once in 1000 steps;
  • a clone taken before the first hash is an independent lineage.
• Declared gap, now carried rather than stated. No committed control pins the interpreter route, i.e. that the key sites read the carried hash: reverting a key site to eval_recompute_str_hash(s) would leave every committed test green. This is filed as gunbc.recurring_failure_mode memo_key_cost_proportional_to_argument_size, at the rung found (mitigatable, by measurement only). Its restoration trigger names the capability: an observation of content-hash computations readable by the interpreter's own tests (a work-counter cargo feature or an exported accessor), sufficient for a v1-compiler test that goes red when a key site is reverted. The same row carries the open accumulator instance and names its work item.

Residue: NOT fixed, and it is a decision rather than a patch

With this fix and the memo on, the top of the ladder is still super-linear. That falsifies my pre-registered prediction that the fixed curve would match the memo-off curve.

Discriminator: accumulate_10000…accumulate_40000, memo on vs off. A recursion threading a growing list_push accumulator is super-linear with the memo on and at baseline with it off. parse_json_object_members threads exactly such an accumulator. A persistent list is a new Rc every step, so the identity hash memo misses and the whole accumulator is rehashed each call.

Same class, different value kind, and it cannot be closed at the carrier. The real question is whether the eval-call memo should key calls that can never recur (each step differs in its index), i.e. whether its per-call serving cost exceeds recomputation under std.materialization_ladder. That is a provider-admission ruling for a shared interpreter provider, and it is escalated to the parent lane rather than decided here.

Unrelated drift seen in passing: --required-regen also reports std_measure.rs drifting on this base. It is untouched here.

🤖 Generated with Claude Code

[gunbai-bot]

Review 70155 was right, and the defect was worse than it read: the generator did exist locally, but .gitignore ignores *.py repo-wide ("never committed to the .dag/.rs substrate"), so gen.py and run.py silently never landed. That left the ladder's inputs unproducible from a fresh clone. Force-adding the Python would have gone against that policy, so the instrument is rebuilt in .dag instead (28079cb):

• Moved to the instrument home. It now lives at tools.json_parse_ladder, dag/gunbc/instruments/json_parse_ladder.dag, beside the other tools.* instruments, and loads from the ordinary dag source root. The four ad-hoc modules, the .gitignore and the fixture paths are deleted.
• Documents are built in memory. A bounded-depth builder (≤300-entry chunks, joined) makes the documents, so no fixtures or external generator are needed. The builder deliberately avoids threading a growing value through deep recursion, which is the shape being measured.
• Every rung checks its own value. Each rung returns ProcessExit: 0 only when it returns what it owes, a located ExitFailure otherwise. Checked both ways: green rungs exit 0, and temporarily making parse_1000 owe 999 exits 1 with parse_1000 returned 1000 where it owes 999.
• Equivalence rungs write bytes. reserialize_<n> takes --arg path=<file> for the cross-binary byte comparison.
• Consumers. The failure-mode row (gunbc.recurring_failure_mode memo_key_cost_proportional_to_argument_size) now cites the instrument by symbol and lists its discriminating rungs as evidence. The seed admission in gunbc.v1_maintenance_standing cites it by symbol instead of pointing at this PR body.

Stated plainly so the provenance is not overstated: the curves in the PR body were measured with the earlier file-based harness. The rebuilt module measures the same shapes over near-identical documents, but I re-ran only parse_1000, build_only_1000, flat_80000, accumulate_10000 and reserialize_1000 on it, plus the planted red above. The large rungs have not been re-read on the new module.

I did not add a gunbc test label row for it: an instrument-registry row is the stronger door, but it is more surface than a diagnostic ladder warrants here, and the reviewer offered either. If you'd rather it be a label, say so and I'll move it.

— sent from crisp-seal-609