Repository navigation
Interpreted-parse class break closed (DNF→88s) + S1 closure receipt harness - #6371
Merged
Merged
Conversation
…+ empty-list semantics pin (fix 2) Manager-mandated merge-bar demands on #6362, addressed with durable, re-runnable witnesses rather than a one-off manual diff: - parse_tree_content_hash_witness.dag: content_hash(ParseTree) over the tiny/small corpus sample, computed once against the true PR base (0f98a26, built as its own release binary in an isolated worktree) and once against HEAD. Both hashes are byte-identical (witness.dag=9afb740ee43f9e61, execution_mode.dag=ed071cb2baeaeb61), proving fix 2 (list_at_optional -> list_head) and fix 3 (fold_list -> parse_match_arm_stmt_step) preserve parse semantics exactly, not just "no test regressions." - parse_token_first_empty_semantics_test.dag: two `test fn` claims pinning parse_token_first (post-fix, list_head-based) equal to list_at_optional(xs, index: 0) (pre-fix arm) on both the empty-list boundary case that motivated the rewrite and the ordinary present case. Both PASS via claim_batch. Note: the local `main` ref was stale (21fb5d1); the true PR base is origin/main's 0f98a26 (merge-base origin/main HEAD) — using the stale ref for the pre-fix binary produced a false "grammar_invalid" rejection unrelated to this PR (fixed upstream by #6337, landed after 21fb5d1 but before 0f98a26).
…de/roadmap-strategy-discussion-313pvj
…s+) -> 88s Continuation of neat-stag's adhoc-c328b166-bca forensics (fixes 1-3 merged from session/neat-stag-780). Three new env-gated instruments in the v1 interpreter named each residual driver; each was then killed at its single authority in v2 source: - 02_parse: memo position key O(1) via head-token start+1 (was length(all) - length(remaining) per nonterminal attempt; v2's length() is a .dag fold_list body, so the seed's native_len fast path never fired) - 01_tokenize: lex_repeat_loop + delimited combinator short-circuit recursion (were done-flag no-op folds over the entire remaining source per attempt); lex rule matcher thunks compiled once per tokenize (CompiledLexRule) instead of fold_lex_pattern per (rule, position) - std/nat: nat_compare grounded on the realization's native < / == (ordering side of the #5428 numeric-tower grounding; the Peano walk cost ~66 frames per char-class range check, 5.2M frames/min) - std/provenance: SpanIndex assoc list -> Map<OccurrenceId, OriginEvent> (ParseTable.entries carrier pattern; lookup was 13.4M frames / 242s self-time on one medium parse; merge stays base-wins) - v1 interpreter instruments (recording gated on GUNBC_FLATTEN_SITE_DUMP_SECS, off = one atomic load): big-fold .dag-site attribution, per-builtin inclusive wall time, per-.dag-fn self-time profile Receipts: datetime.dag DNF>1200s/4.3GB RSS -> 88s; node.dag 116s; grammar.dag 201s (class break closed, ~linear in content); tiny/small packrat stats byte-identical (0/43/43, 0/200/200); 13 targeted claims green; compile --entry 0 diagnostics; regen divergence 0; fmt/clippy green. P8 wall-clock blocker ledger updated with the closure and the two named, priced residuals (delimited lexeme O(k^2) accumulation, per-file grammar-analysis base). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
…struct in the parser's closure - dag/std/algebra.dag: kernel_algebra_profile's map literal (the only string-keyed brace literal in the parse pipeline's transitive closure) becomes an ordinary builder fn over empty_map/map_insert -- same map value, one dialect construct fewer; std_algebra.rs regenerated (regen_stage0, divergence 0 after) - s1_closure_receipt_test.dag: S1 self-hosting receipt harness. Subject = the parse pipeline's transitive import closure (40 files rooted at 01_tokenize/02_parse/03_normalize/extdeps-languages-dag), which is "v2's parser parsing itself"; the harness's own filesystem plumbing is test infrastructure and deliberately outside the subject (note in file). Grammar validated once per run; per-file tokenize -> parse_production -> normalize; failures collected by path, claim = failures == []. Full-closure census run in flight; receipt lands with its result. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
- docs/plans/s2-v2-self-emit-brief.md: hand-off brief for the parallel v2-emits-v2 lane (receipt ladder, measured landmines incl. the Symbol carrier seam = 1,790 of 3,560 E0308s on fresh v1-emit of the closure, file-ownership split vs the tactical lane) - tm_probe.dag: profiled single-file parse probe for target_model.dag (census residency diagnosis; forensics-enabled run) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
…t diagnosis probe - target_model.dag: the closure's only live |> uses migrate to the v2 surface (length(xs:), any(xs:, predicate: fn)) -- v1 frontend 0 diags; these were two of the census-named parse failures' construct class - s1_stall_probe.dag: parses a file at the start production via parse_expr directly and reports the byte offset of the first token the parse could not consume (greedy repeat => Accepted-with-remaining pinpoints the first uncovered construct); batch fns for the remaining census failures (runtime, refinement, node, grammar, languages/dag, algebra post-migration re-verify) Census receipt (interpreted, 68 min, pre-fix tree): 35/45 parsed clean; subject-relevant failures under diagnosis: runtime, refinement, node, grammar, target_model (pipes, fixed here), languages/dag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
- languages/dag.dag: the postfix projection name after '.' uses dag_grammar_binding_name_terminal() (the soft-keyword choice already used for field/param names), so interpretation.loop / .match parse -- the runtime.dag census failure's construct class - refinement.dag, grammar.dag: the subject's three negative int literals (-1/-2) become parenthesized binary minus (0 - 1); the wave1 grammar has no negative-literal lexeme and three sites do not justify the lexer/normalize lever yet -- class noted for the tree-wide census All three v1-frontend green (0 diagnostics). Stall probes re-verifying under the new grammar; node.dag + languages/dag.dag offsets pending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
The content-hash canonicalizer block used v1 arrow lambdas
(e => match ...) with comma-separated inline match arms at two sites --
node.dag's census failure class (stall byte 13535 = the first such fn).
All nine sites become fn(e) { ... } with newline arms; pure syntax
migration, identical behavior, v1 frontend 0 diagnostics.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
- languages/dag.dag: the string-literal rule's own delimiters were
LiteralPattern { text: "\"" } -- an escaped-quote string the wave1
lexer cannot tokenize when reading this file itself (the stream
garbles silently and the parse dies ~2,400 lines later; stall probe
receipt). Now CharPattern { char: 34 }, the exact idiom the
string-BODY escape element two rules up already uses.
- node.dag: sort_by(xs, h => h) -> fn(h) { h } (the census's ninth-plus
arrow site, hidden earlier by a head-truncated grep).
Both v1-frontend 0 diagnostics. Full 40-file receipt run in flight.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
…6369/#6370) Resolution choices: - 02_parse diag accumulation: main's O(1) batch-cons + single flatten wins over per-step list_append (kills the 252k-appends cost the self-time profile measured); my O(1) memo position key preserved - 01_tokenize: my superset kept (repeat short-circuit == main's #6368 semantics, plus delimited short-circuit + CompiledLexRule precompile) - v1 interpreter: both instrument sets kept (main's fold-caller/memo discriminator + my big-fold site/builtin-time/self-time profiler); main's Map-grounding fast path moved into call_function_inner so both profiled and unprofiled paths hit it; env gate consolidated on main's residual_hunt_forensics_enabled (same var, one authority) - grammar_coverage ledger: main's fixes (1)-(4) text + my RESIDUAL CLOSED continuation renumbered (5)-(9) Gates: build green, fmt/clippy green, regen divergence 0, tiny/small packrat stats byte-identical (0/43/43, 0/200/200). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
Merged-tree receipt: 38/40 subject files parse clean; the two giants (target_model.dag, languages/dag.dag) each have a residual construct behind the first fix -- stall probes running to name them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
…get_model
- languages/dag.dag: `let module = ...` used the bare `module` keyword as
a let-binding name (lexes as kw_module, so the whole
dag_wave1_grammar_root decl failed); renamed to module_prod -- a local
binding, so a rename beats extending keywords into let-name/expr
positions where they would collide with the match/if productions
- target_model.dag: NamedMetadataFoldOk field puns made explicit
(name: name); NOTE the sweep shows shorthand patterns parse fine in
green files (integer.dag's Cons { head, tail }), so this may be
hygiene rather than the poison -- fresh stall probes running to
arbitrate
Both v1-frontend 0 diagnostics.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two lanes, continuing from #6359/#6364:
1. Interpreted-parse superlinear drivers closed. Merges
session/neat-stag-780(that branch's WIP forensics + snoc→Cons parse fixes; the remote branch was deleted, so this PR is now the only carrier of that work) and then closes the residual the neat-stag receipt named honest-but-unidentified. Three new env-gated instruments in the v1 interpreter (big-fold.dag-site attribution keyed on the fold closure's source span, per-builtin inclusive wall time, per-.dag-fn self-time profile — all recording-gated behindGUNBC_FLOOR-style envGUNBC_FLATTEN_SITE_DUMP_SECS, one atomic load when off) named five drivers, each killed at its single authority in v2 source:02_parse: memo position key O(1) via head-tokenstart + 1(waslength(all) − length(remaining)per nonterminal attempt; v2'slength()is a.dagfold_listbody, so the seed'snative_lenfast path never fired on this path)01_tokenize:lex_repeat_loop+ delimited combinator → short-circuiting recursion (were done-flag no-op folds over the entire remaining source per attempt); rule matcher thunks compiled once per tokenize (CompiledLexRule) instead offold_lex_patternper (rule, position)std/nat:nat_comparegrounded on the realization's native</==— the ordering side of the Numeric-tower grounding: ground Nat construction-side so native form == modeled form (§0) #5428 numeric-tower grounding (the Peano walk cost ~66 frames per char-class range check, 5.2M frames/min)std/provenance:SpanIndexassoc list →Map<OccurrenceId, OriginEvent>(theParseTable.entriescarrier pattern; lookup was 13.4M frames / 242s self-time on one medium parse; merge stays base-wins)2. S1 groundwork.
s1_closure_receipt_test.dag: the S1 self-hosting receipt harness — subject = the parse pipeline's transitive import closure (40 files rooted at01_tokenize/02_parse/03_normalize/extdeps/languages/dag), grammar validated once per run, per-file tokenize →parse_production→ normalize, failures collected by path, claim =failures == []. Plus the one v1-only construct in that closure dissolved:kernel_algebra_profile's string-keyed map literal → ordinaryempty_map/map_insertbuilder (same value;std_algebra.rsregenerated, divergence 0).Receipts
compile --entryon the harness closure: 0 diagnostics ·regen_stage0 --verify: divergence 0 · fmt/clippy greenphase_profile_sigtermtest verified green in CI's (debug) profile — its release-profile fallback path is a pre-existing latent issue, untouched heregrammar_coverage.dag) updated: residual closed, two named-and-priced leftovers (delimited lexeme O(k²) accumulation on multi-KB literals; per-file grammar-analysis base, amortized by this harness)In flight on this branch
The full 40-file S1 census is running now (interpreted, ~45-60 min); the receipt commit (census result + any grammar fixes it names) lands on this branch.
🤖 Generated with Claude Code
https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX
Generated by Claude Code