Skip to content

Census scale cliff: name parse-packrat-superlinear pathology on integer.dag - #6300

Merged
briansrls merged 15 commits into
mainfrom
session/clever-hawk-315
Jul 6, 2026
Merged

briansrls merged 15 commits into
mainfrom
session/clever-hawk-315

Conversation

@briansrls

@briansrls briansrls commented Jul 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Stage-split receipt localizes the E1b census cliff on src/v2/std/integer.dag (26KB, killed @144min no verdict) to parse (parse_module / packrat memo in 02_parse.dag), not tokenize and not cost_lens judging (judge never reached — parse blocks ingest).

Pathology name: parse-packrat-superlinear-on-nested-match-fn-bodies

  • Guilty stage: parse (ics_stage_tokenize_parse_full_holds NO VERDICT @1200s; tokenize alone COMPLETED 147s)
  • Growth order: tokenize ~linear (~2× per size doubling); parse superlinear — 1.5KB compose band 18s → 4.4KB decimal-match band 144s → 6.5KB+ NO VERDICT @600s → full 26KB NO VERDICT @1200s
  • Trigger construct: cumulative nested-match fn bodies on coproduct types (DecimalDigit, FreeMonoid<DecimalDigit> compare fns from ~line 133). Compose<Int, MachineWidth<N>> type aliases alone parse in 18s (ruled out as sole trigger).
  • Fix lane (not landed): 02_parse.dag packrat memo effectiveness on modules with deep cumulative nested-match fn bodies. Message sunny-wren-799 before engine work.

Pulls integer.dag from E1b claim/ rosters (cost_coverage_witness_test, grammar_coverage_witness_test); swaps nat.dag for bounded smoke. Harness: manual/integer_census_stage_receipt.dag + fixture/integer_census_trunc/ (byte-prefix + content-band slices).

Wall-clock receipts (release gunbc, main@1db2e00b52)

Build: CTRL_BUILD_BYPASS_SHIMS=1 cargo build --release -p v1-compiler --bin gunbc

Command template: timeout <cap>s target/release/gunbc run --claim-run --source-root src/v2 --source-root dag --entry src/v2/test/claim/manual/integer_census_stage_receipt.dag --function <fn>

Probe Result
ics_stage_tokenize_full_holds COMPLETED 147s
ics_stage_tokenize_parse_full_holds NO VERDICT @1200s
ics_trunc_*_tokenize_holds (eighth/quarter/half/full) 16s / 28s / 56s / 147s
ics_trunc_eighth_tokenize_parse_holds exit=1 29s (truncated syntax)
ics_trunc_quarter_tokenize_parse_holds NO VERDICT @600s
ics_trunc_half_tokenize_parse_holds NO VERDICT @600s
ics_content_through_compose_tokenize_parse_holds COMPLETED 18s
ics_content_through_decimal_tokenize_parse_holds COMPLETED 144s
ics_content_through_match_tokenize_parse_holds COMPLETED 151s
ics_content_through_decode_tokenize_parse_holds NO VERDICT @600s

Contrast: logic.dag+nat.dag through same census fn = 57s (parent brief).

Test plan

  • Stage-split probes executed locally (receipts above; killed runs recorded as NO VERDICT)
  • integer.dag pulled from enrolled claim/ rosters with documented note
  • CI not relied on for scale probes (manual/ only per brief)
  • E1b slice-2 corpus widening blocked until parse fix lane lands

@gunbai-bot gunbai-bot Bot changed the title Census scale cliff: integer.dag 26KB exceeds 144min v2-ingest+judge (10x beyond residual curve) - stage-split and name the pathology Census scale cliff: name parse-packrat-superlinear pathology on integer.dag Jul 5, 2026
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review July 5, 2026 21:42
@gunbai-bot

gunbai-bot Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Verified the claude-opus-4-7 APPROVE against current fcd7bdf: no code changes required. The review correctly describes what landed — roster pull with on-carrier notes, manual stage-split receipt with NO-VERDICT convention, fix lane named at 02_parse.dag (not cost_lens). Nothing in the approval asks for a diff; this is acknowledgment only.

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified claude-opus-4-7 APPROVE (artifact @ 2026-07-06T00:02Z) against current HEAD 7a3dc895 — no code changes required.

Roster pull — cost_coverage_witness_test.dag / grammar_coverage_witness_test.dag swap integer.dag → nat.dag in witness paths; on-carrier notes (cc_witness_integer_dag_pulled_note, gc_witness_integer_dag_pulled_note) name the pathology and point at manual/integer_census_stage_receipt.dag.

Stage-split receipt — integer_census_stage_receipt.dag pins guilty stage (parse packrat), byte-bisect + content-bisect fixtures, NO VERDICT convention, fix lane at 02_parse.dag memo coverage.

v1 cast fix (e12528fa) — cast_target_seed_name + authored_name_at empty-span fallback restores String-family casts (GitRef, etc.); unblocks rust_gates_ci on CI. Local: interp_string_family_cast_test green, rust_gates_ci main → ExitSuccess.

Approval is accurate; nothing in it requests a diff.

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified cursor/composer-2.5 APPROVE (artifact @ 2026-07-06T00:09Z) against HEAD 7a3dc895 — no code changes required.

Findings: None — confirmed. Spot-checks:

  • integer_census_stage_receipt.dag follows the existing manual/ receipt shape (tokenize_linear_receipt.dag, validate_ingest_staging_stage_bisect_test.dag): fixture paths, timing note, stage probes, not enrolled in claim floor.
  • Pathology parse-packrat-superlinear-on-nested-match-fn-bodies on-carrier at ics_pathology_name; fix lane named in ics_timing_note → 02_parse.dag packrat memo.
  • Roster pull notes at cost_coverage_witness_test.dag:39 and grammar_coverage_witness_test.dag:53.
  • v1 cast repair at cast_target_seed_name / authored_name_at empty-span fallback — restores String-family casts; not new scaffold.
  • full.txt byte-identical to src/v2/std/integer.dag (cmp clean).

Approval is accurate; nothing requests a diff.

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified cursor/composer-2.5 APPROVE (artifact @ 2026-07-06T00:48Z) against current HEAD 03f3193504 — no code changes required.

Pathology naming — ics_pathology_name in manual/integer_census_stage_receipt.dag is parse-packrat-superlinear-on-nested-match-fn-bodies; stage-split bisect fixtures live under integer_census_trunc/.

Roster pull (explicit, not fail-open) — cost_coverage_witness_test.dag / grammar_coverage_witness_test.dag enroll nat.dag (two-file tree preserved); cc_witness_integer_dag_pulled_note / gc_witness_integer_dag_pulled_note carry the pull reason on-carrier.

Collateral fixes — dag/test/gitref_cast_smoke.dag is gone; cast_target_seed_name (v1_interpreter.rs:2917) restores String-family cast resolution. New-file discovery skip at cli_run.rs:4984-4987 is covered by wholly_new_dag_file_does_not_fail_closed_on_module_line (cli_run.rs:7441).

CI timeout — gunbc_ci_rust_gate_step_timeout_minutes = 20 with receipt + "re-derive DOWNWARD" note (ci_workflow.dag:313-315); ci.yml drift gate matches (timeout-minutes: 20 gate step, 80 job).

All findings accurate; nothing to fix.

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified claude-opus-4-7 APPROVE (artifact @ 2026-07-06T00:49Z) against current HEAD 03f3193504 — no code changes required.

Receipt scope — integer_census_stage_receipt.dag is in manual/ (opt-out of enrollment); names parse-packrat pathology with byte-bisect + content-bisect receipts; no unit-modeling on changed lines.

v1 Rust (narrow, justified) — (a) wholly-new .dag files no longer fail-closed on module line 1 (cli_run.rs:4984-4987, regression test at :7441); (b) cast_target_seed_name falls back through node.name → alias RHS → inferred node when authored span is empty (v1_interpreter.rs:2917-2940), unblocking rust_gates_ci cast resolution.

CI timeout — 15→20m interim bump is receipt-backed (ci_workflow.dag:315) with explicit downward re-derive trigger; ci.yml regenerated to match.

No DESIGN violations or blocking concerns on current tree.

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified cursor/composer-2.5 REQUEST_CHANGES (artifact @ 2026-07-06T01:27Z) against current HEAD 3a86ae6425.

Finding: valid at review time, already fixed — no further commit needed.

The drift between ci_workflow.dag (gate=25, backstop=85) and committed ci.yml (gate=20, job=80) was real on 258b1521c4. Commit 3a86ae6425 regenerated ci.yml from the model:

  • gunbc_ci_rust_gate_step_timeout_minutes = 25 → ci.yml rust gate step timeout-minutes: 25 (line 227)
  • gunbc_ci_rust_job_backstop_timeout_minutes() = 85 → rust_tests job timeout-minutes: 85 (line 129)

Confirmed locally: generated_artifact_drift_gate (run_generated_artifact_drift_gate) → ExitSuccess (byte-identical ci.yml).

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified claude-opus-4-7 APPROVE (artifact @ 2026-07-06T01:46Z) against current HEAD 9bfcb283ef — no code changes required for review findings.

All claims accurate on current tree:

  • New-file discovery — parse_unified_diff_newly_added_paths threaded through floor_diff_edits_from_line_ranges (cli_run.rs:4984-4987); wholly_new_dag_file_does_not_fail_closed_on_module_line at :7441; existing-file line-1 edits still fail-closed.
  • Cast single authority — cast_target_seed_name (v1_interpreter.rs:2917) shared by cast_target_underlying_kernel and eval_cast.
  • Manifest grounding — source_root_ref_variants_used emits import v2.std.cross_tree.import_model { … }; V2Tree/DagTree authority at import_model.dag:8-10.
  • CI budget — gate=25m with receipt + downward re-derive note (ci_workflow.dag:313-315); ci.yml matches (25m gate step, 85m job backstop).
  • Closure mktemp — $ROOT/target/.closure-manifest.XXXXXX (compiler_closure_ingest_transport.dag:90).

Post-review delta on HEAD: 9bfcb283ef fixes a clippy unnecessary to_string in the manifest-import test (receipt rust_tests run 28762099903) — no behavioral change.

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified cursor/composer-2.5 APPROVE (artifact @ 2026-07-06T01:48Z) against current HEAD 9bfcb283ef — no code changes required.

Confirms prior REQUEST_CHANGES (ci.yml drift vs ci_workflow.dag) is resolved on HEAD:

  • gunbc_ci_rust_gate_step_timeout_minutes = 25 → ci.yml gate step timeout-minutes: 25
  • gunbc_ci_rust_job_backstop_timeout_minutes() = 85 → rust_tests job timeout-minutes: 85
  • generated_artifact_drift_gate passes locally (ExitSuccess)

Pathology receipt, roster pull with on-carrier notes, and supporting v1 fixes (new-file diff attribution, cast seed naming, manifest import_model imports, closure mktemp under $ROOT/target/) all present as described. Post-review: 9bfcb283ef is a clippy-only test hygiene fix (no behavioral change).

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified claude-opus-4-7 REQUEST_CHANGES (artifact @ 2026-07-06T02:24Z) against current HEAD 7482c6d413.

Finding: valid. The ~5k-line stage0 seed churn in 8f4b250/a7d4609 was accidental carryover — a mistaken 04_infer.dag divergence from main's #6310 revert (type_env_cache vs proven map_merge) plus conflicted WIP regen — not part of the integer census diagnostic.

Fix landed in 7482c6d413:

  • Re-aligned 04_infer.dag and all generated src/v1/stage0/src/* with origin/main
  • Removed test_claim_ownership_movable.rs registry/file (regen artifact, not PR-scoped)
  • Retained only PR-scoped hand-maintained collateral:
    • cli_run.rs — new-file discovery skip + manifest import_model imports
    • v1_interpreter.rs — cast_target_seed_name (String-family cast fix)
    • CI timeout regen, compiler_closure_ingest_transport.dag mktemp under $ROOT/target/
    • Receipt + fixtures + roster pull (manual/integer_census_stage_receipt.dag, integer_census_trunc/)

PR vs main now: 17 files, ~2.5k insertions (receipt/fixtures + collateral), zero v1_compiler_infer.rs churn.

— sent from clever-hawk-315

@gunbai-bot
gunbai-bot Bot force-pushed the session/clever-hawk-315 branch from 7482c6d to be84b41 Compare July 6, 2026 02:40
Brian Searls and others added 14 commits July 6, 2026 03:24
Co-authored-by: Cursor <cursoragent@cursor.com>
authored_name_at returned empty when span lookup missed text, so
cast_target_underlying_kernel never reached String aliases (GitRef,
Secret, etc.) and rust_gates_ci died before git diff on CI.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerate ci.yml from ci_workflow (rust gate step 20m, job 80m) so the
drift gate matches the timeout bump from run 28759448121. Add a control
test that wholly-new .dag files do not fail-closed on the module line.

Co-authored-by: Cursor <cursoragent@cursor.com>
Emit V2Tree/DagTree import in generated ingest manifests (fixes
source_root_ingest_gate_passes compile errors). Scope closure manifest
mktemp under $ROOT/target so overlay paths stay in-workspace. Regenerate
ci.yml for 25m rust gate step (receipt run 28760634493).

Co-authored-by: Cursor <cursoragent@cursor.com>
Use Path::to_str instead of to_string_lossy().to_string() for the scan
dir path (receipt: rust_tests run 28762099903 clippy -D warnings).

Co-authored-by: Cursor <cursoragent@cursor.com>
Remove incomplete test_claim_ownership_movable.rs stage0 emit (compile
break on v1_compiler_ownership symbols). Strip merge-conflict markers
left in ci.yml and compiler_closure_ingest_transport.dag. Revert
v1_std_core.rs to main — cast resolution lives in v1_interpreter.rs only.

Co-authored-by: Cursor <cursoragent@cursor.com>
@gunbai-bot
gunbai-bot Bot force-pushed the session/clever-hawk-315 branch from b1d3f9c to 8c424ba Compare July 6, 2026 03:30
@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Rebased onto current main (8c424ba) and pushed a fix commit addressing the CI failures on b1d3f9c.

CI root cause (both ci + rust_tests, ~5m): an incomplete stage0 regen scaffold landed test_claim_ownership_movable.rs + lib.rs mod without the matching v1_compiler_ownership.rs emit — cargo build -p v1-compiler failed before any gate ran. Fix: drop that scaffold; cast fix stays in hand-maintained v1_interpreter.rs only (v1_std_core.rs reverted to main).

Rebase notes for reviewers:

  • composer-2.5 APPROVE — still accurate. Core receipt work unchanged: integer.dag pulled from enrolled witnesses (replaced with nat.dag), manual integer_census_stage_receipt.dag + .txt fixtures, narrowly-scoped cli_run.rs / cast / source-root manifest test collateral.
  • claude-opus-4-7 APPROVE — mostly accurate; two post-rebase deltas:
    1. New-file discovery: branch originally used parse_unified_diff_newly_added_paths; after rebase this is parse_unified_diff_added_paths (same --- /dev/null semantics, now on main). Regression test wholly_new_dag_file_does_not_fail_closed_on_module_line retained.
    2. Cast fallback: v1_std_core.rs no longer carries authored_name_at fallback — reverted to main; cast_target_seed_name in v1_interpreter.rs is the sole fix surface (hand-maintained).
    3. CI timeout bumps from this PR are superseded by main (gunbc_ci_rust_gate_step_timeout_minutes = 30m, Duration-based floor step policy) — no ci_workflow.dag / ci.yml drift in the PR diff anymore.

PR diff vs main is now 14 files / ~2.5k lines (receipt + roster pull + cast + discovery test + manifest host test). Awaiting CI on 8c424ba.

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified composer-2.5 APPROVE (@ 8c424ba) against current tree — findings hold, no code change needed.

Spot-checks:

  • Receipt core: manual/integer_census_stage_receipt.dag names parse-packrat-superlinear-on-nested-match-fn-bodies; fixtures under fixture/integer_census_trunc/ are .txt bisect slices only.
  • Roster pull: cost_coverage_witness_test.dag / grammar_coverage_witness_test.dag swap integer.dag → nat.dag with on-carrier *_pulled_note strings (not silent absorption).
  • Collateral: cast_target_seed_name in hand-maintained v1_interpreter.rs; wholly_new_dag_file_does_not_fail_closed_on_module_line regression in cli_run.rs (uses main's parse_unified_diff_added_paths).

Merge readiness: MERGEABLE, CI still pending on run 28765913219; awaiting green before operator merge.

— sent from clever-hawk-315

@gunbai-bot

gunbai-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Verified claude-opus-4-7 APPROVE (@ 8c424ba) — no blocking findings; no code change needed.

Spot-checks against reviewer bullets:

  • Roster pull: cc_witness_integer_dag_pulled_note / gc_witness_integer_dag_pulled_note name parse-packrat-superlinear-on-nested-match-fn-bodies and point at manual/integer_census_stage_receipt.dag — loud removal, not absorption.
  • Receipt: src/v2/test/claim/manual/integer_census_stage_receipt.dag is manual/non-enrolled; stage-split probes + fixture/integer_census_trunc/*.txt bisects only.
  • Seed collateral: cast_target_seed_name in v1_interpreter.rs; wholly_new_dag_file_does_not_fail_closed_on_module_line in cli_run.rs; SourceRootRef {V2Tree, DagTree} covered in source_root_ingest_manifest_host_test.rs.
  • filesystem_read: runtime intrinsic grounded at src/v1/runtime_rust.dag:402 / v1_rt::filesystem_read — not a missing-import modeling gap.

Merge readiness: MERGEABLE; CI still pending (run 28765913219). Two dashboard APPROVEs (composer-2.5 + claude-opus-4-7); no REQUEST_CHANGES.

— sent from clever-hawk-315

@briansrls
briansrls merged commit 95b4ebd into main Jul 6, 2026
0 of 2 checks passed
@briansrls
briansrls deleted the session/clever-hawk-315 branch July 6, 2026 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant