Skip to content

Remove quadratic remainder copying from code-point/octet slicing in both realizations (skip/take/get/count to their contracts; Value drop no longer walks a shared slice; cost controls enrolled) - #13346

Closed
gunbai-bot[bot] wants to merge 30 commits into
mainfrom
session/swift-ibex-835

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

What

Removes the quadratic remainder copying from slicing a code-point or octet list, in both realizations. The interpreter and the emitted Rust runtime (v1_rt) both change, plus the emit bridge. No new primitive. The existing slice surface already declares the right cost; its realizations didn't meet it.

The surface a decoder uses (agreed with proud-wren-11 for #13214's std.checkpointed_workspace entry decoder):

let cps = unicode_scalar_unfold(s: body)                 // once, linear
get(xs: cps, index: i)                                    // one member, Absent past the end
count(cps)                                                // length
unicode_scalar_fold(xs: cps |> skip(n: a) |> take(n: b - a))   // a field, as host text

dag/test/claim/code_point_slice_witness_test.dag decodes <len>:<code points> fields exactly this way. It is the worked example #13214 mirrors.

What this establishes, and what it does not

Both realizations already hold a list as a persistent RRB vector: the interpreter's Value::List is Rc<im::Vector>, and v1_rt's Vec is im::Vector. Splitting one is O(log n) and shares every node except one boundary path, so a slice now costs log len to form and its own length to read. That removes the quadratic remainder copying. It does not make an offset walk linear. Every member read is get_contract's log n on this carrier, so a walk that reads each offset is O(n log n). This PR does not discharge #13214's linear obligation; per the side-chat ruling, #13214 gets a sequential fold decoder instead.

Counts are clamped exactly. With k = if n < 0 then len else min(n, len), skip keeps len - k members and take keeps k. That is the copying forms' reading, now written into skip_contract and take_contract.

The copies, and the earliest boundary for each (§6b)

The std.primitives contracts are the authority. Each defect below is a realization breaking its own declared contract, so each repair is local to its owning link.

link contract realization before now
interpreter get(list, i) get_contract work 1 / log n free_monoid_to_vec copied all n members as a type guard indexes the RRB directly
interpreter count(list) count_contract work 1 flattened all n len()
interpreter chars_to_string(cps, s, e) slice writer flattened all n, then sliced skip/take on the RRB, then reads only the slice
interpreter skip / take skip_contract (now carrier-sensitive) and a new take_contract copied the remainder rrb_skip / rrb_take: O(log n), shared
emitted skip / take (extdeps.languages.rust.emit rust_simple_method_specs) same .iter().cloned().skip(n).collect() copied the remainder v1_rt::list_skip / list_take: O(log n), shared
emitted xs.skip(n).first() (v1.compiler.emit_rust emit_rust_first_method_call) get .iter().cloned().skip(n).next() walked n .get(n as usize).cloned() (same answer, including negative n)
interpreter impl Drop for Value dropping a value must not cost more than the value owns consumed every uniquely-owned list or map into the iterative-drop worklist. On an RRB that shares nodes with its source, that clones every member, so dropping a slice cost the source's length detaches a persistent carrier only beyond 64 nested drops. Below that, the carrier is released by refcount inside drop, while the depth counter is raised

The last row was found by measurement, not by the brief. After the first six fixes, an end-to-end walk was still quadratic. Decomposing it showed get and count scaled with eval steps and only skip did not. GUNBC_FLATTEN_SITE_DUMP_SECS builtin timings then put 12,000 m:count calls at 13,247 ms and the skip arms at 207 ms. The time was the drop of each slice, which happens inside the next call that consumes it. Memo-off A/B (GUNBC_EVAL_MEMO=0) ruled out the eval memo first. The drop gate keeps the totality that the iterative drop exists for: value_depth_walker_tests::a_deep_list_drops (a 262,144-deep nested list on the default 2 MiB test stack) and the rest of that module, 11 tests, pass.

Not changed, and why: char_at/substring over host text in v1_rt stay O(offset) on a bare &str; their own notes name the cursor as that residual's next rung. A decoder that wants linear reads uses the code-point surface above instead.

Evidence

Cost controls: RED on the old form, GREEN on the new, deterministic (counts, not clocks), non-ASCII input. All run on the merge path:

  • Emitted runtime. The new Run the slice cost controls step (slice_cost_controls) runs v1-stage0-runtime list_slice_tests on the existing generated job, beside the lint and under the same status guard. Its command is authored at gunbc.repo_self_build repo_self_slice_cost_control_command, and the workflow is regenerated from gunbc.compiler_gate_workflow. No new job. That crate is outside the package the unit-test lane runs, so without this step its controls would only compile.
  • Interpreter. slice_cost_tests and the deep-value drop tests are v1-compiler unit tests. They run in the existing rust-unit-tests lane (repo_self_test_command), not in this step, so they aren't run twice.
  • Emitted realization. v1_rt::list_slice_tests, in crate v1-stage0-runtime:
    • a_slice_from_the_middle_copies_only_boundary_chunks checks a clone-counting code point (é, 中, 😀, a) at N = 200,000: a mid-list skip+take copies ≤ 2,048 members.
    • the_copying_template_form_exceeds_the_budget runs the old emit templates, spelled exactly as they emitted, on the same input and bound. They must exceed it (the RED).
  • Interpreter. v1_interpreter::slice_cost_tests:
    • The same pair for rrb_skip/rrb_take.
    • reading_a_list_value_does_not_flatten_it (with the old flatten as its RED).
    • dropping_a_slice_of_a_shared_list_moves_no_member, with the_detaching_drop_moves_the_slice_members as its RED.
  • Positive controls. negative_and_overlong_counts_keep_the_copying_forms_reading (both realizations): n ∈ {-3, 0, 4, 10, 11, i64::MAX} gives the same answer as the copying form. The witness code_point_slice_decodes_non_ascii_fields covers astral code points, an empty field and a two-digit prefix. code_point_slice_refuses_an_overlong_field covers refusal rather than truncation.

End to end (interpreter), claim_batch CPU ms at identical eval steps. This is a probe walking 2^9 / 2^11 / 2^13 three-code-point non-ASCII fields, i.e. 4× per step:

3,072 cps 12,288 cps 49,152 cps growth per 4×
main 113 1,379 20,206 ~12–15× (quadratic)
this PR 48 197 856 ~4.1–4.3× (consistent with O(n log n) at these sizes)

The probe is the witness's decode_fields at three sizes. It isn't committed, because a 49k walk is a measurement, not a floor claim: the floor's new-witness budget is 72,300 eval steps, and the committed many-fields claim walks 3,072 code points (50,828 steps). Eval steps cannot see a host copy, so the floor claim checks correctness and the enrolled Rust controls check cost. To re-derive the table, run decode_fields(body: doubled(s: <the witness field>, times: t)) at t = 9 / 11 / 13 through claim_batch, against each binary.

Regen. The stage0 mirrors are regenerated through the documented route (gunbc.generated_artifact_merge_driver repair steps). On the merged tree, required-regen converged to first_generation_equal=true. Conflicted mirrors were taken from main and regenerated. This head's evidence is one emission, not a fixed point: the generated job's step Stage0 mirrors match one emission by this seed checks that the committed mirrors equal what this seed emits. No two-generation fixed point is claimed for this head. My local --required-regen-fixed-point run ended with no observation (rc 2), so it is not evidence.

Admission (v1 seed)

This is admitted under gunbc.v1_maintenance_standing v1_seed_standing's purpose test as SupportsV2SelfHostProgram { BehaviorPreservingRedundancyRemoval }. Every change removes copies while keeping answers identical; the positive controls above pin that.

It serves v2. The emitted runtime is what v2's self-emitted crates run on, and slicing there was a remainder copy on every target. Any emitted decoder walking text by offset was quadratic, ASCII or not. The interpreter is v2's measurement substrate.

Refused classes:

  • No language behaviour, no compatibility obligation, no escape hatch.
  • PublicSurfaceGrowth, stated rather than claimed absent: v1_rt gains two pub fns (list_skip, list_take). They realize two existing methods and add no language-level name. This sits on the same boundary v1_maintenance_typecheck_progress_admission_note says is undrawn, so a reviewer should judge it rather than take my classification.

Operator decision msg_6e6044ca (2026-10-04, option A) authorized the lane.

🤖 Generated with Claude Code

gunbc-ci-auto-heal and others added 8 commits October 4, 2026 23:52
…ier in both realizations

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…he slice

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…share the carrier; regen fixed point holds

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…threshold

Dropping a slice of a shared RRB list consumed it into the iterative-drop worklist,
cloning every member out of nodes it shared with its source: O(len) per drop, so an
offset walk was quadratic in drop alone (measured: 12000 m:count calls = 13.2s at
49k members; the skip arms themselves 207ms). Detach only past 64 nested drops.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n follows)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d; fixed_point_equal=true)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gunbai-bot Bot pushed a commit that referenced this pull request Oct 5, 2026
…ice surface (#13346)

The body is unfolded to code points once; delimiters are found by get, the length is count, and
each field is skip/take folded back with unicode_scalar_fold, so the decode is linear on non-ASCII
text and on the emitted runtime, not only on ASCII in the interpreter. Adds a non-ASCII witness:
accented, CJK and astral paths containing the body's own delimiters round-trip in order.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… enrol the slice cost controls on the generated job

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gunbai-bot Bot pushed a commit that referenced this pull request Oct 5, 2026
…d over its code points

Per the side-chat ruling: no get-by-index and no re-slicing; a state machine folds the unfolded
code points once, collecting each field by prepend and reversing it once when it closes, so the
decode is linear on any text in either realization. Length prefixes keep code-point semantics.
Unstacked from #13346 (its merge reverted above). The cost control is
gunbc.instruments.workspace_entry_decode_scaling decode_scaling, whose verdict is claimed with a
quadratic red.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot gunbai-bot Bot changed the title Linear code-point/octet slicing in both realizations (skip/take/get/count to their contracts; Value drop no longer walks a shared slice) Remove quadratic remainder copying from code-point/octet slicing in both realizations (skip/take/get/count to their contracts; Value drop no longer walks a shared slice; cost controls enrolled) Oct 5, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Oct 5, 2026
Every file this PR does not own is taken from main, so the reverted #13346 merge leaves no residue.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gunbc-ci-auto-heal and others added 2 commits October 5, 2026 05:19
…e controls

The portable round-trip deep test overflowed the fleet runner's test stack on the
first CI run that executed it; it passes on main and this branch alike off the fleet
(3/3 each), so it is a separate finding about the portable form.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ness eval-step budget

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

On the advisory about the new hand-written Rust in v1_interpreter.rs (review 76272): here is the receipt.

What it displaces. Each addition replaces a copying path that this diff deletes in the same arm:

addition deleted path it replaces
rrb_skip / rrb_take items.iter().skip/take(n).cloned().collect() in method_call.skip / method_call.take
free_monoid_items the whole-list free_monoid_to_vec read in free_call.chars_to_string; it is also the read free_call.get and free_call.count now take for a Value::List
Value::drop threshold pending.extend(std::mem::take(items)) for every list and map below the threshold

What does not shrink. No seed feature, language behaviour or escape hatch is added. The admission class is SupportsV2SelfHostProgram { BehaviorPreservingRedundancyRemoval } under gunbc.v1_maintenance_standing v1_seed_standing, as the PR body states.

Line count, stated honestly. Most of the line growth is test code (the RED/GREEN cost pairs) and doc comments, not production arms. The net production growth is the two helpers, the read helper, and the drop threshold with its two empty-carrier thread-locals. There is no separate seed census row for it, because nothing here is retained-for-later. The bodies retire with the interpreter they live in.

— sent from swift-ibex-835

gunbc-ci-auto-heal and others added 3 commits October 5, 2026 07:10
…gen follows)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…_size

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…red-regen converged)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Changed since be51a9ff (for the side chat, which saw ddb5b64 and f70eb02). Head is now 91f195d084.

  1. ddb5b64: witness sized under the floor budget. code_point_slice_walk_is_correct_at_scale now walks 512 fields (3,072 code points, 50,828 eval steps) instead of 2,048 fields (209,060 steps). That is under the 72,300-step new-witness budget that refused it on be51a9ff. Its comment says the floor claim checks correctness only, because eval steps cannot see a host copy, and that the cost is checked by the enrolled Rust controls.
  2. f70eb02: merged origin/main. Six generated artifacts conflicted (witnesses.yml and five stage0 mirrors). I took main's copy of each, then regenerated them.
  3. e68e82d: contract fix (the side chat's remaining item). skip_contract and take_contract now derive ephemeral_work from the same clamped k as output_size: len - k and k, with k = if n < 0 then len else min(n, len).
  4. 91f195d: regenerated on the merged tree. required-regen converged, and the registry regen re-emitted witnesses.yml. Against origin/main, the regenerated files differ only by this PR's content: the slice_cost_controls step, and the emitted skip/take and skip().first() lowering. This head's evidence is one emission, not a two-generation fixed point: the generated job's step Stage0 mirrors match one emission by this seed checks the committed mirrors against what this seed emits. My local --required-regen-fixed-point run ended with no observation (rc 2) and is not evidence.

Nothing else changed.

— sent from swift-ibex-835

@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 5, 2026
…gen follows)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 5, 2026
…red-regen converged)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 5, 2026
@gunbai-bot
gunbai-bot Bot removed this pull request from the merge queue due to a manual request Oct 5, 2026
gunbc-ci-auto-heal and others added 2 commits October 5, 2026 19:32
…threshold; the slice step now runs only the runtime crate's controls (rust-unit-tests runs v1-compiler whole)

Generated artifacts conflicted and were taken from main; regen follows.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…me_rust.dag block); regen verifies

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gunbc-ci-auto-heal and others added 5 commits October 5, 2026 19:59
…s restored, regen verifies)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…converged, v1_rt.rs equals its emission)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ane off the merge path

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rged)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 6, 2026
@gunbai-bot
gunbai-bot Bot removed this pull request from the merge queue due to a manual request Oct 6, 2026
@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 6, 2026
@gunbai-bot
gunbai-bot Bot removed this pull request from the merge queue due to a manual request Oct 6, 2026
@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 6, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 7, 2026
gunbc-ci-auto-heal and others added 2 commits October 7, 2026 05:59
…s the runtime_rust.dag list_skip/list_take block; regen follows)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…red-regen converged; v1_rt.rs equals its emission)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 7, 2026
@gunbai-bot
gunbai-bot Bot removed this pull request from the merge queue due to a manual request Oct 8, 2026
gunbc-ci-auto-heal and others added 2 commits October 8, 2026 03:44
…regen follows)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…deps_version_semver.rs re-emitted)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 8, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 9, 2026
gunbc-ci-auto-heal and others added 2 commits October 9, 2026 02:03
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…_tests_claim_interpolation_hole_census.rs)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 9, 2026
@gunbai-bot
gunbai-bot Bot removed this pull request from the merge queue due to a manual request Oct 9, 2026
@gunbai-bot

gunbai-bot Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #13641 at 634453d: this PR's head is an ancestor of integration/v1-closeout. The source branch is kept for archaeology; this PR is no longer an independent merge authority. — sent from neat-wolf-604

@gunbai-bot gunbai-bot Bot closed this Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants