Implement Unicode string escapes

[gunbai-bot]

Summary

Implement Rust-style \u{H...} string escapes in the authoritative v1 tokenizer, accepting one to six hex digits only when they satisfy the std.unicode.types.unicode_scalar authority denoted by the Char refinement. The scalar boundaries are named and grounded in Unicode Standard §3.9 D76. The scanner keeps the opening brace inside the escape even when the first digit is A-F, so interpolation cannot steal the production.

Escape parsing is now closed and fail-closed: malformed \x/\u productions and unsupported escapes such as \q produce one located ShUnknown token spanning the complete literal. They can no longer succeed as a prefix or silently fall back to printable text. The recognized vocabulary also makes the corpus's existing \0 and \r spellings explicit; the strict whole-tree pass identified those as intended NUL and CR inputs rather than unknown syntax.

This closes a silent language-layer defect that left all 32 ECMA-48 C1 controls as printable escape text and made the Bash materializer's NUL guard fail open. The NUL witness constructs its poison value independently with from_code_point(0) and proves the printable literal escape spelling is accepted rather than mistaken for NUL. That .dag witness is the CI-enforcing receipt: its production pattern remains authored as "\u{0000}", so a tokenizer regression would make the pattern printable and red the independently constructed real-NUL claim.

Consumer receipts now execute both sides of the ANSI guard (real U+0080/U+009F versus literal escape text), and the existing Bash, GHA, and SQL fold receipts assert that every sentinel constant evaluates to its intended control character.

The six required refusal forms are discovery-enrolled through the production v1 compile path: each .dag witness builds malformed source at runtime and requires a hard compile refusal. Literal passthrough would make that virtual module compile and red the witness. The local Rust receipts additionally pin the single ShUnknown token's spelling, file identity, and full-literal span.

Validation

• cargo test -p v1-compiler --test tokenize_escape_receipt -- --nocapture — 4 passed, 1 intentional benchmark ignored.
• cargo run -p v1-compiler --bin regen_stage0 -- --verify — regen_divergence_count=0.
• Rebuilt gunbc compile over both src/v2 and dag — no unknown-escape parse diagnostics.
• Discovery-enrolled ANSI, NUL materializer, Bash, GHA, and SQL consumer receipts — all pass, including exact sentinel character assertions.
• Discovery-enrolled runtime-source escape wall — positive Unicode control plus all six malformed/unsupported forms pass (11/11 focused file witnesses).
• Shard-completed discovery corpus — every enrolled witness passed; the monolithic process exceeded the host memory envelope near the tail, so the exact discovered roster was resumed in bounded fresh processes, with no witness red in any segment.
• Direct C1 roster audit — all 32 entries evaluate to exact scalars U+0080 through U+009F.
• cargo fmt --all -- --check and git diff --check — clean.

The earlier CI failures were on superseded WIP SHAs: first before generated stage-0 outputs were committed, then before the corpus's intended \r production was made explicit. The current branch includes both corrections and passes the local fixed-point check.