Admit raw newlines in v2 string lexemes and locate the unmatched character

[gunbai-bot]

Summary

• Root is dag_string_literal_lex_rule / dag_string_body_element_pattern: a raw line feed is now a body element, matching v1.compiler.tokenize scan_string_body. StringTextChar still excludes LF so TypeScript/Swift/Kotlin do not pick up a second meaning.
• Unrecognized-char Textual extent is [pos, pos+1] when remaining is nonempty, so an unterminated literal locates the opening quote instead of a zero-width caret.

Enrolled controls (v2.test.tokenize.string_literal_raw_newline)

• string_literal_with_raw_newline_lexes_holds — raw LF in a string accepts
• escaped_newline_in_string_still_lexes_holds — escaped \n still accepts
• unterminated_string_locus_covers_the_opening_quote_holds — "oops refuses with extent covering the opening quote
• em_dash_in_identifier_position_still_refuses_holds — unrelated invalid scalar still refuses

Acceptance

Required CI is build + floor. The head-bound srv2 closure receipt is the required native semantic evidence for the D train. It establishes the compiler closure and this PR's cause-relative delta; it is not whole-v2.test.* native admission.

The posted receipt at c5968bcf60 (68 → 67, 1 RefusalCleared + 7 AdvancedToNewCause, zero regressions) is interim. The binding receipt is retaken after D2 lands, at this PR's final integrated head.

Landing order: #10988 → D2 → D3 → D1 → G → D4.

Test plan

• Four controls above as test fn
• Required CI: build + floor
• Interim srv2 closure overlay at c5968bcf60 (68 → 67)

[gunbai-bot]

Draft until srv2 closure receipt (parent addendum).

• Base: main (merge-base 0db47c5)
• Head: 7cbe4ac

Please local-merge #10882-head + this head and post before/after: the 8 lexer-class files should leave tokenize_lex_e1_unrecognized_char; files that advance to a masked later cause are progress, not green.

[gunbai-bot]

Closure measurement at head 8e773d2236 overlaid on main 0006e0aabd (srv2): refused at the EMIT stage, before any fold ran. The v2-native lane's seed emission of src/v2/compiler/00_compile.dag produced three hard diagnostics, all in the file this PR edits:

source annotation sits inside a declaration body. Only module-item grain is modeled; move it above the declaration it describes. (src/v2/compiler/01_tokenize.dag:21502-21576)
source annotation sits inside a declaration body. Only module-item grain is modeled; move it above the declaration it describes. (src/v2/compiler/01_tokenize.dag:21579-21655)
source annotation sits inside a declaration body. Only module-item grain is modeled; move it above the declaration it describes. (src/v2/compiler/01_tokenize.dag:21658-21729)

Per DESIGN §4c the .dag realization admits only standalone leading // blocks attached to module-scope declarations; the three annotation blocks this PR adds inside a declaration body in 01_tokenize.dag (byte ranges above) are refused by the emitter, so the emitted compiler is never built. Hoist each block above the declaration it describes (or fold the rationale into the module-item annotation) and I rerun. No closure before/after is reportable for this head.

[gunbai-bot]

Closure before/after for #10969 at head ef78fe8108, overlaid on main 0006e0aabd (PR diff from merge-base applied, no unmerged paths; srv2, emitted driver over the 167-member closure root, controls-only universe, {"_terminal":"complete","file_refusals":67}, wall 279 s, rss 0.87 GB). Baseline: the #10943 carrier at 2fb67b411a, 68 refusals.

68 → 67. Zero regressions. No previously accepted module refuses; no non-lexer file changed cause; nothing moved to an earlier stage.

Cause C (tokenize_lex_e1_unrecognized_char), 8 attributed files, all discharged:

• RefusalCleared 1: dag/std/witness_admission.dag
• AdvancedToNewCause 7: dag/std/primitive_identity.dag, dag/std/primitive_projection.dag, dag/std/realization_schedule.dag, dag/std/types.dag, src/v2/compiler/source_authority.dag, src/v2/extdeps/languages/typescript.dag → parse_g0_tokens_remain; src/v2/compiler/02_parse.dag → body_lowering_reason_match_arm_navigation_refused (cause D, D4: single-arm match, field named match, data-initializer lambdas #10967)
• StillBlockedByTargetCause 0

The lexer cause no longer occurs anywhere in the closure; the seven advanced files were masked at the lexer and now speak at parse (their next cause, owned by the D5 expression-grammar lane) or body lowering. Histogram after: parse 32, graft 25, match-arm 9, wrapper-retention 1. Meets the cause-relative bar. Ready to leave draft.

[gunbai-bot]

Refreshed closure receipt for #10969 at head 320b068eb1 (PR diff from merge-base 90c6041a4b overlaid on main 0423c33770; srv2 private base clone; emitted driver over the 167-member closure root, controls-only universe; {"_terminal":"complete","file_refusals":67}, wall 274 s, rss 0.87 GB; instrument digest 815a9684d69a0651). Baseline: #10943 carrier at 2fb67b411a, 68 refusals.

Identical to the ef78fe81 result: 68 → 67, zero regressions. Cause C, 8 files: dag/std/witness_admission.dag RefusalCleared; primitive_identity, primitive_projection, realization_schedule, types (all dag/std), src/v2/compiler/source_authority.dag, src/v2/extdeps/languages/typescript.dag AdvancedToNewCause → parse_g0_tokens_remain; src/v2/compiler/02_parse.dag AdvancedToNewCause → body_lowering_reason_match_arm_navigation_refused. No accepted module regressed; nothing earlier-stage. Histogram after: parse 32, graft 25, match-arm 9, wrapper-retention 1.

This receipt binds to 320b068eb1. Landing position: second, after #10963.

[gunbai-bot]

review 63330 is right: the long-home rename was the #7762 relocation discharge (long_lane_exclusion_note / witness_deferral_freeze). Undone. The four identities stay at src/v2/test/native_decl_selection_test.dag and are withheld in place on v2.workflow.floor_cost_debt (censored class: required-floor run 34531151284, interrupted_before_verdict at 502-508ms CPU). That roster is the counted shrink-only withhold the cost-debt header already distinguishes from a path move.

Lex / locus work unchanged.

— sent from witty-crab-528

[gunbai-bot]

Closure before/after for #10969 at head c5968bcf60, overlaid on main b0a6d53c35 (PR diff from merge-base applied, no unmerged paths; srv2, pinned instrument sha256 815a9684d69a0651, emitted driver over the 167-member closure root, controls-only universe, {"_terminal":"complete","file_refusals":67}). Baseline: the #10943 carrier at 2fb67b411a, 68 refusals. Target cause: tokenize_lex_e1_unrecognized_char (8 baseline rows).

68 → 67. Zero regressions. Cause-relative outcome for the 8 target rows: 1 RefusalCleared, 7 AdvancedToNewCause, 0 StillBlockedByTargetCause, 0 RegressedToEarlierCause — all 8 discharge.

• RefusalCleared: dag/std/witness_admission.dag
• AdvancedToNewCause → parse_g0_tokens_remain: dag/std/primitive_identity.dag, dag/std/primitive_projection.dag, dag/std/realization_schedule.dag, dag/std/types.dag, src/v2/compiler/source_authority.dag, src/v2/extdeps/languages/typescript.dag
• AdvancedToNewCause → body_lowering_reason_match_arm_navigation_refused: src/v2/compiler/02_parse.dag

Non-target files whose cause changed: 0. Previously accepted members now refusing: 0. After-histogram: parse_g0_tokens_remain 32, namespace_graft_body_dissolved_refused 25, body_lowering_reason_match_arm_navigation_refused 9, normalized_tree_reason_wrapper_retention_not_normalized 1.

Identical outcome to the receipt at 320b068e; this receipt supersedes it and is bound to c5968bcf60. Any later push touching src/v2/{compiler,std,lens,extdeps}, dag/std or dag/extdeps voids it.

[gunbai-bot]

review 63512: verified. ea674a6d03d (#10992) is an ancestor of this head; the four native_decl_selection identities are no longer over-ceiling. Removed floor_cost_debt_censored_chunk_03 rather than adding a restoration trigger — the roster shrinks; coverage is not left gone for a cause that does not exist.

— sent from witty-crab-528