fix(frontend): accept a UTF-8 BOM in grammar sources - #215
Merged
Conversation
A `.g4` file saved with a UTF-8 byte order mark failed to compile:
error[G4F003]: Hello.g4:1:0: mismatched input '\u{feff}grammar'
expecting {'lexer', 'parser', 'grammar'}
The cause is in the pinned meta-grammar rather than the runtime. antlr-ng
collapsed Java ANTLR's two trailing `NameStartChar` ranges
(`'ﷰ'..''` and `''..'�'`) into a single
`'ﷰ'..'�'`, which covers U+FEFF. That made the byte order mark a
legal identifier-start character, so the lexer produced one `ID` token
spanning the mark and the following keyword. A `UnicodeBOM` rule alone
cannot fix this, because `ID` still matches longer and the lexer takes the
longest match.
Restore the split so `NameStartChar` excludes U+FEFF, and add `` to
`WS` so the mark lands off the default channel.
Java spends a dedicated `UnicodeBOM : '' -> skip;` rule on this.
Folding it into `WS` instead is observationally identical and avoids
allocating a token type: a new rule renumbers every token after it
(`END_ARGUMENT` 75 -> 76, ...), which breaks the `frontend-snapshots.tsv`
oracles for the one corpus grammar that uses Argument-mode tokens. Those
oracles are recorded against antlr-ng, which has no such rule, so they
cannot be faithfully regenerated. Max token type stays 78.
The mark is skipped rather than stripped, so it keeps occupying a column
and every `SourceSpan` stays anchored to the real file offsets. Verified
against the 4.13.2 jar: a BOM'd source reports 1:18 where the unmarked
twin reports 1:17, a mid-file mark errors at 2:2, and a mark inside a
STRING_LITERAL or LEXER_CHAR_SET remains content.
`.tokens` vocabularies need a separate fix. They are generated sidecars
parsed line by line, so they never reach the grammar lexer, and U+FEFF is
not `char::is_whitespace`, so `trim` left the mark glued to the first token
name and generation failed with G4S029. Strip a leading mark there too.
Upstream's Java regex instead accepts `"ID"` as the token name and
silently imports a wrong vocabulary; failing that way is not worth
reproducing.
Also regenerates the self-hosted frontend, which had drifted: the
checked-in artifacts carried a `v0.15.2` header against crate 0.19.0 and
predated the `add_parse_listener` facade from #204, so
`update-stage0.sh --check` was already red before this change.
Verified with the full suite (1084 tests), CI clippy, the conformance
sweep (357 passed, 0 failed, 0 skipped), and
`tools/grammar-frontend/update-stage0.sh --check` (Stage 1 == Stage 2).
Closes #212
📝 WalkthroughWalkthroughGrammar processing now supports UTF-8 BOMs and LF, CRLF, and CR line endings. Frontend and CLI tests cover grammar and ChangesBOM and line-ending support
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant CLI as antlr4-rust-gen
participant Frontend as Grammar frontend
participant Lexer as ANTLRv4Lexer
participant Vocabulary as Token vocabulary loader
participant Output as Generated Rust
CLI->>Frontend: Read grammar with BOM or CRLF/CR
Frontend->>Lexer: Tokenize grammar text
Lexer-->>Frontend: Tokens with normalized positions
CLI->>Vocabulary: Read .tokens sidecar
Vocabulary-->>CLI: Token mappings without leading BOM
CLI->>Output: Generate Rust source
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Claude encountered an error after 2s —— View job I'll analyze this and get back to you. |
📊 Source Code Metrics (this PR vs
|
| File | Cyclomatic | Cognitive | Functions | LLOC | MI |
|---|---|---|---|---|---|
| src/bin_support/grammar/semantics.rs | 654 ⚪ | 511 ⚪ | 124 ⚪ | 889 (main: 888) 🔴 | 0 ⚪ |
| src/bin_support/grammar/frontend.rs | 225 (main: 216) 🔴 | 91 (main: 90) 🔴 | 66 (main: 63) 🔴 | 302 (main: 280) 🔴 | 0 ⚪ |
Generated by mehen v1.7.0 — the code quality watcher.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/bin_support/grammar/frontend.rs`:
- Around line 1127-1168: The test carriage_returns_are_whitespace_and_end_lines
currently compares coordinates only for CRLF input; add a parsed bare-CR fixture
and assert its second-semicolon line_column matches the LF fixture, preserving
the existing CRLF assertion. Keep the coverage focused on lone-CR line_starts
behavior and follow the documented cargo-llvm-cov workflow.
In `@src/bin_support/grammar/semantics.rs`:
- Around line 822-826: Normalize token-sidecar text in the parsing flow around
the BOM handling in src/bin_support/grammar/semantics.rs:822-826 by converting
CRLF and lone CR line endings to LF before iterating with str::lines(), while
preserving leading BOM removal. Add regression coverage in
tests/antlr4_rust_gen_cli.rs:332-337 within
byte_order_mark_and_crlf_token_vocabularies_are_honored for both CR-only and
BOM-plus-CR vocabularies.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 31a7704b-5a30-425b-9b14-78eb0fa09716
⛔ Files ignored due to path filters (2)
src/bin_support/grammar/generated/antlr_v4_lexer.rsis excluded by!**/generated/**src/bin_support/grammar/generated/antlr_v4_parser.rsis excluded by!**/generated/**
📒 Files selected for processing (6)
README.mdsrc/bin_support/grammar/frontend.rssrc/bin_support/grammar/semantics.rstests/antlr4_rust_gen_cli.rsthird_party/antlr-v4-grammar/ANTLRv4Lexer.g4third_party/antlr-v4-grammar/self-hosted.sha256
Comment on lines
+1127
to
+1168
| /// Windows and classic-Mac line endings are plain whitespace, and the line | ||
| /// map counts them the way Java ANTLR does. | ||
| #[test] | ||
| fn carriage_returns_are_whitespace_and_end_lines() { | ||
| for (name, text) in [ | ||
| ("crlf", "grammar A;\r\na: 'x';\r\n"), | ||
| ("cr", "grammar A;\ra: 'x';\r"), | ||
| ] { | ||
| let file = parse_source(SourceId::new(0), name, text) | ||
| .unwrap_or_else(|error| panic!("{name}: {:?}", error.diagnostics())); | ||
| assert_eq!( | ||
| file.tokens() | ||
| .iter() | ||
| .filter( | ||
| |token| token.channel == antlr4_runtime::token::DEFAULT_CHANNEL | ||
| && token.token_type != RUNTIME_TOKEN_EOF | ||
| ) | ||
| .map(|token| file.token_text(token)) | ||
| .collect::<Vec<_>>(), | ||
| ["grammar", "A", ";", "a", ":", "'x'", ";"], | ||
| "{name}", | ||
| ); | ||
| } | ||
|
|
||
| // A CRLF pair must advance the line once, not twice, so diagnostics on | ||
| // the second line report the same column as the LF-only spelling. | ||
| let crlf = parse_source(SourceId::new(0), "crlf-lines", "grammar A;\r\na: 'x';\r\n") | ||
| .expect("CRLF grammar should parse"); | ||
| let lf = parse_source(SourceId::new(1), "lf-lines", "grammar A;\na: 'x';\n") | ||
| .expect("LF grammar should parse"); | ||
| let semicolon = |file: &SourceFile| { | ||
| let token = file | ||
| .tokens() | ||
| .iter() | ||
| .filter(|token| token.token_type == SEMI) | ||
| .nth(1) | ||
| .expect("both grammars have two semicolons"); | ||
| file.line_column(token.span.bytes.start) | ||
| }; | ||
| assert_eq!(semicolon(&crlf), Some((2, 6))); | ||
| assert_eq!(semicolon(&crlf), semicolon(&lf)); | ||
| } |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Assert bare-CR coordinates.
The lone-CR case is parsed, but Lines 1151-1167 only verify CRLF coordinates. Assert the second semicolon’s position for \r against the LF case too, so a lone-CR line_starts regression cannot pass.
As per coding guidelines, maintain coverage using the documented cargo-llvm-cov workflows.
Suggested test addition
let lf = parse_source(SourceId::new(1), "lf-lines", "grammar A;\na: 'x';\n")
.expect("LF grammar should parse");
+ let cr = parse_source(SourceId::new(2), "cr-lines", "grammar A;\ra: 'x';\r")
+ .expect("CR grammar should parse");
let semicolon = |file: &SourceFile| {
// ...
};
assert_eq!(semicolon(&crlf), Some((2, 6)));
assert_eq!(semicolon(&crlf), semicolon(&lf));
+ assert_eq!(semicolon(&cr), semicolon(&lf));📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| /// Windows and classic-Mac line endings are plain whitespace, and the line | |
| /// map counts them the way Java ANTLR does. | |
| #[test] | |
| fn carriage_returns_are_whitespace_and_end_lines() { | |
| for (name, text) in [ | |
| ("crlf", "grammar A;\r\na: 'x';\r\n"), | |
| ("cr", "grammar A;\ra: 'x';\r"), | |
| ] { | |
| let file = parse_source(SourceId::new(0), name, text) | |
| .unwrap_or_else(|error| panic!("{name}: {:?}", error.diagnostics())); | |
| assert_eq!( | |
| file.tokens() | |
| .iter() | |
| .filter( | |
| |token| token.channel == antlr4_runtime::token::DEFAULT_CHANNEL | |
| && token.token_type != RUNTIME_TOKEN_EOF | |
| ) | |
| .map(|token| file.token_text(token)) | |
| .collect::<Vec<_>>(), | |
| ["grammar", "A", ";", "a", ":", "'x'", ";"], | |
| "{name}", | |
| ); | |
| } | |
| // A CRLF pair must advance the line once, not twice, so diagnostics on | |
| // the second line report the same column as the LF-only spelling. | |
| let crlf = parse_source(SourceId::new(0), "crlf-lines", "grammar A;\r\na: 'x';\r\n") | |
| .expect("CRLF grammar should parse"); | |
| let lf = parse_source(SourceId::new(1), "lf-lines", "grammar A;\na: 'x';\n") | |
| .expect("LF grammar should parse"); | |
| let semicolon = |file: &SourceFile| { | |
| let token = file | |
| .tokens() | |
| .iter() | |
| .filter(|token| token.token_type == SEMI) | |
| .nth(1) | |
| .expect("both grammars have two semicolons"); | |
| file.line_column(token.span.bytes.start) | |
| }; | |
| assert_eq!(semicolon(&crlf), Some((2, 6))); | |
| assert_eq!(semicolon(&crlf), semicolon(&lf)); | |
| } | |
| /// Windows and classic-Mac line endings are plain whitespace, and the line | |
| /// map counts them the way Java ANTLR does. | |
| #[test] | |
| fn carriage_returns_are_whitespace_and_end_lines() { | |
| for (name, text) in [ | |
| ("crlf", "grammar A;\r\na: 'x';\r\n"), | |
| ("cr", "grammar A;\ra: 'x';\r"), | |
| ] { | |
| let file = parse_source(SourceId::new(0), name, text) | |
| .unwrap_or_else(|error| panic!("{name}: {:?}", error.diagnostics())); | |
| assert_eq!( | |
| file.tokens() | |
| .iter() | |
| .filter( | |
| |token| token.channel == antlr4_runtime::token::DEFAULT_CHANNEL | |
| && token.token_type != RUNTIME_TOKEN_EOF | |
| ) | |
| .map(|token| file.token_text(token)) | |
| .collect::<Vec<_>>(), | |
| ["grammar", "A", ";", "a", ":", "'x'", ";"], | |
| "{name}", | |
| ); | |
| } | |
| // A CRLF pair must advance the line once, not twice, so diagnostics on | |
| // the second line report the same column as the LF-only spelling. | |
| let crlf = parse_source(SourceId::new(0), "crlf-lines", "grammar A;\r\na: 'x';\r\n") | |
| .expect("CRLF grammar should parse"); | |
| let lf = parse_source(SourceId::new(1), "lf-lines", "grammar A;\na: 'x';\n") | |
| .expect("LF grammar should parse"); | |
| let cr = parse_source(SourceId::new(2), "cr-lines", "grammar A;\ra: 'x';\r") | |
| .expect("CR grammar should parse"); | |
| let semicolon = |file: &SourceFile| { | |
| let token = file | |
| .tokens() | |
| .iter() | |
| .filter(|token| token.token_type == SEMI) | |
| .nth(1) | |
| .expect("both grammars have two semicolons"); | |
| file.line_column(token.span.bytes.start) | |
| }; | |
| assert_eq!(semicolon(&crlf), Some((2, 6))); | |
| assert_eq!(semicolon(&crlf), semicolon(&lf)); | |
| assert_eq!(semicolon(&cr), semicolon(&lf)); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/bin_support/grammar/frontend.rs` around lines 1127 - 1168, The test
carriage_returns_are_whitespace_and_end_lines currently compares coordinates
only for CRLF input; add a parsed bare-CR fixture and assert its
second-semicolon line_column matches the LF fixture, preserving the existing
CRLF assertion. Keep the coverage focused on lone-CR line_starts behavior and
follow the documented cargo-llvm-cov workflow.
Source: Coding guidelines
Comment on lines
+822
to
+826
| // `.tokens` files are generated sidecars rather than grammar | ||
| // source, so the frontend's byte order mark handling never sees | ||
| // them. Drop a leading mark here: U+FEFF is not `char::is_whitespace`, | ||
| // so `trim` would otherwise leave it glued to the first token name. | ||
| let text = text.strip_prefix('\u{feff}').unwrap_or(&text); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
In Rust's standard library, does str::lines() split on a lone carriage return (\r), or only on \nand\r\n?
💡 Result:
In Rust's standard library, str::lines() splits strings only on newlines (\n) and carriage return followed by a newline (\r\n) [1][2]. According to the official documentation, a "bare" carriage return (\r) that is not immediately followed by a newline does not split the string [1][2]. Such carriage returns are intended to be included in the produced string slices [1][2]. However, there is a known, long-standing discrepancy between this documented behavior and the actual implementation [3][4]. In practice, str::lines() often consumes a trailing bare carriage return at the very end of a string, even though it does not treat it as a line separator elsewhere [3][4]. This behavior has been identified in issue reports as undocumented and inconsistent with the stated rule that only \n and \r\n serve as line endings [3][4].
Citations:
- 1: https://doc.rust-lang.org/stable/std/primitive.str.html
- 2: https://doc.rust-lang.org/std/primitive.str.html
- 3: str::lines consumes carriage returns at the end of a line which is undocumented rust-lang/rust#94435
- 4: std::str::Lines strips bare carriage return from end of string rust-lang/rust#106258
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
echo "== Rust version if cargo/rustc present =="
if command -v rustc >/dev/null 2>&1; then
rustc --version
else
echo "rustc not installed"
fi
if command -v cargo >/dev/null 2>&1; then
cargo --version
else
echo "cargo not installed"
fi
echo
echo "== Locate relevant files =="
git ls-files | rg '(^|/)src/bin_support/grammar/semantics\.rs$|(^|/)tests/antlr4_rust_gen_cli\.rs$' || true
echo
echo "== Inspect semantics .tokens handling section =="
sem_file="$(git ls-files | rg '(^|/)src/bin_support/grammar/semantics\.rs$' | head -n1 || true)"
if [ -n "${sem_file:-}" ]; then
wc -l "$sem_file"
sed -n '780,860p' "$sem_file" | cat -n
fi
echo
echo "== Inspect test around line 332 =="
test_file="$(git ls-files | rg '(^|/)tests/antlr4_rust_gen_cli\.rs$' | head -n1 || true)"
if [ -n "${test_file:-}" ]; then
wc -l "$test_file"
sed -n '300,370p' "$test_file" | cat -n
fi
echo
echo "== Search for lines() and line-ending code in .tokens/vocabulary contexts =="
rg -n '\.lines\(|str::lines|line-end|vocabulary|\.tokens|FEFF|bom|bare CR|CR' -S src tests 2>/dev/null | head -n 200
echo
echo "== Behavioral probe for Rust lines() if toolchain exists =="
if command -v rustc >/dev/null 2>&1; then
cat >/tmp/lines_probe.rs <<'RS'
fn main() {
for s in [
"ID=1\nNUM=2\n",
"ID=1\r\nNUM=2\r\n",
"ID=1\rNUM=2\r",
"\u{feff}ID=1\r\nNUM=2\r\n",
"\u{feff}ID=1\rNUM=2\r",
] {
let parts: Vec<_> = s.lines().collect();
println!("{:?} => {:?}", s, parts);
}
}
RS
rustc /tmp/lines_probe.rs -o /tmp/lines_probe
/tmp/lines_probe
else
echo "rustc not available; cannot run Rust lines() probe"
fiRepository: ophi-dev/antlr-rust-runtime
Length of output: 23801
Handle lone-CR .tokens sidecars.
str::lines() does not split ID=1\rNUM=2\r into two definitions, so bare-CR vocabulary sidecars are parsed as a single malformed entry. Normalize CRLF and lone CR to LF before iterating entries, and add CR/BOM+CR regressions to byte_order_mark_and_crlf_token_vocabularies_are_honored.
📍 Affects 2 files
src/bin_support/grammar/semantics.rs#L822-L826(this comment)tests/antlr4_rust_gen_cli.rs#L332-L337
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/bin_support/grammar/semantics.rs` around lines 822 - 826, Normalize
token-sidecar text in the parsing flow around the BOM handling in
src/bin_support/grammar/semantics.rs:822-826 by converting CRLF and lone CR line
endings to LF before iterating with str::lines(), while preserving leading BOM
removal. Add regression coverage in tests/antlr4_rust_gen_cli.rs:332-337 within
byte_order_mark_and_crlf_token_vocabularies_are_honored for both CR-only and
BOM-plus-CR vocabularies.
Copy/Paste DetectionFound 1816 duplication(s) across 5 changed Rust file(s) (threshold: 100 tokens). Show duplicationsFound a 5 line (823 tokens) duplication in the following files:
&["DOC_COMMENT", "BLOCK_COMMENT", "LINE_COMMENT", "INT", "STRING_LITERAL", "UNTERMINATED_STRING_LITERAL", "BEGIN_ARGUMENT", "ACTION", "NESTED_ACTION", "OPTIONS", "TOKENS", "CHANNELS", "IMPORT", "FRAGMENT", "LEXER", "PARSER", "GRAMMAR", "PROTECTED", "PUBLIC", "PRIVATE", "RETURNS", "LOCALS", "THROWS", "CATCH", "FINALLY", "MODE", "COLON", "COLONCOLON", "COMMA", "SEMI", "LPAREN", "RPAREN", "RBRACE", "RARROW", "LT", "GT", "ASSIGN", "QUESTION", "STAR", "PLUS_ASSIGN", "PLUS", "OR", "DOLLAR", "RANGE", "DOT", "AT", "POUND", "NOT", "ID", "WS", "NESTED_ARGUMENT", "ARGUMENT_ESCAPE", "ARGUMENT_STRING_LITERAL", "ARGUMENT_CHAR_LITERAL", "END_ARGUMENT", "UNTERMINATED_ARGUMENT", "ARGUMENT_CONTENT", "LEXER_CHAR_SET_BODY", "LEXER_CHAR_SET", "UNTERMINATED_CHAR_SET", "ESC_SEQUENCE", "HexDigit", "UnicodeESC", "DoubleQuoteLiteral", "TripleQuoteLiteral", "BacktickQuoteLiteral", "NameChar", "NameStartChar"],
&[None, None, None, None, None, None, None, Some("\'=\'"), None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, Some("\'[\'"), None, None, None, Some("\'import\'"), Some("\'fragment\'"), Some("\'lexer\'"), Some("\'parser\'"), Some("\'grammar\'"), Some("\'protected\'"), Some("\'public\'"), Some("\'private\'"), Some("\'returns\'"), Some("\'locals\'"), Some("\'throws\'"), Some("\'catch\'"), Some("\'finally\'"), Some("\'mode\'"), Some("\':\'"), Some("\'::\'"), Some("\',\'"), Some("\';\'"), Some("\'(\'"), Some("\')\'"), Some("\'}\'"), Some("\'->\'"), Some("\'<\'"), Some("\'>\'"), Some("\'?\'"), Some("\'*\'"), Some("\'+=\'"), Some("\'+\'"), Some("\'|\'"), Some("\'$\'"), Some("\'..\'"), Some("\'.\'"), Some("\'@\'"), Some("\'#\'"), Some("\'~\'"), None, None, None, None, None],
&[None, None, None, None, Some("ACTION"), Some("ARG_ACTION"), Some("ARG_OR_CHARSET"), Some("ASSIGN"), Some("LEXER_CHAR_SET"), Some("RULE_REF"), Some("SEMPRED"), Some("STRING_LITERAL"), Some("TOKEN_REF"), Some("UNICODE_ESC"), Some("UNICODE_EXTENDED_ESC"), Some("WS"), Some("ALT"), Some("BLOCK"), Some("CLOSURE"), Some("ELEMENT_OPTIONS"), Some("EPSILON"), Some("LEXER_ACTION_CALL"), Some("LEXER_ALT_ACTION"), Some("OPTIONAL"), Some("POSITIVE_CLOSURE"), Some("RULE"), Some("RULEMODIFIERS"), Some("RULES"), Some("SET"), Some("WILDCARD"), Some("DOC_COMMENT"), Some("BLOCK_COMMENT"), Some("LINE_COMMENT"), Some("INT"), Some("UNTERMINATED_STRING_LITERAL"), Some("BEGIN_ARGUMENT"), Some("OPTIONS"), Some("TOKENS"), Some("CHANNELS"), Some("IMPORT"), Some("FRAGMENT"), Some("LEXER"), Some("PARSER"), Some("GRAMMAR"), Some("PROTECTED"), Some("PUBLIC"), Some("PRIVATE"), Some("RETURNS"), Some("LOCALS"), Some("THROWS"), Some("CATCH"), Some("FINALLY"), Some("MODE"), Some("COLON"), Some("COLONCOLON"), Some("COMMA"), Some("SEMI"), Some("LPAREN"), Some("RPAREN"), Some("RBRACE"), Some("RARROW"), Some("LT"), Some("GT"), Some("QUESTION"), Some("STAR"), Some("PLUS_ASSIGN"), Some("PLUS"), Some("OR"), Some("DOLLAR"), Some("RANGE"), Some("DOT"), Some("AT"), Some("POUND"), Some("NOT"), Some("ID"), Some("END_ARGUMENT"), Some("UNTERMINATED_ARGUMENT"), Some("ARGUMENT_CONTENT"), Some("UNTERMINATED_CHAR_SET")],
&[None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None],
&["DEFAULT_TOKEN_CHANNEL", "HIDDEN", "", "", "OFF_CHANNEL", "COMMENT"],
```rust
---
Found a 80 line (613 tokens) duplication in the following files:
* Starting at line 14 of src/bin_support/grammar/generated/antlr_v4_lexer.rs
* Starting at line 16 of src/bin_support/grammar/generated/antlr_v4_parser.rs
```rust
use std::sync::OnceLock;
pub const EOF: i32 = antlr4_runtime::TOKEN_EOF;
pub const ACTION: i32 = 4;
pub const ARG_ACTION: i32 = 5;
pub const ARG_OR_CHARSET: i32 = 6;
pub const ASSIGN: i32 = 7;
pub const LEXER_CHAR_SET: i32 = 8;
pub const RULE_REF: i32 = 9;
pub const SEMPRED: i32 = 10;
pub const STRING_LITERAL: i32 = 11;
pub const TOKEN_REF: i32 = 12;
pub const UNICODE_ESC: i32 = 13;
pub const UNICODE_EXTENDED_ESC: i32 = 14;
pub const WS: i32 = 15;
pub const ALT: i32 = 16;
pub const BLOCK: i32 = 17;
pub const CLOSURE: i32 = 18;
pub const ELEMENT_OPTIONS: i32 = 19;
pub const EPSILON: i32 = 20;
pub const LEXER_ACTION_CALL: i32 = 21;
pub const LEXER_ALT_ACTION: i32 = 22;
pub const OPTIONAL: i32 = 23;
pub const POSITIVE_CLOSURE: i32 = 24;
pub const RULE: i32 = 25;
pub const RULEMODIFIERS: i32 = 26;
pub const RULES: i32 = 27;
pub const SET: i32 = 28;
pub const WILDCARD: i32 = 29;
pub const DOC_COMMENT: i32 = 30;
pub const BLOCK_COMMENT: i32 = 31;
pub const LINE_COMMENT: i32 = 32;
pub const INT: i32 = 33;
pub const UNTERMINATED_STRING_LITERAL: i32 = 34;
pub const BEGIN_ARGUMENT: i32 = 35;
pub const OPTIONS: i32 = 36;
pub const TOKENS: i32 = 37;
pub const CHANNELS: i32 = 38;
pub const IMPORT: i32 = 39;
pub const FRAGMENT: i32 = 40;
pub const LEXER: i32 = 41;
pub const PARSER: i32 = 42;
pub const GRAMMAR: i32 = 43;
pub const PROTECTED: i32 = 44;
pub const PUBLIC: i32 = 45;
pub const PRIVATE: i32 = 46;
pub const RETURNS: i32 = 47;
pub const LOCALS: i32 = 48;
pub const THROWS: i32 = 49;
pub const CATCH: i32 = 50;
pub const FINALLY: i32 = 51;
pub const MODE: i32 = 52;
pub const COLON: i32 = 53;
pub const COLONCOLON: i32 = 54;
pub const COMMA: i32 = 55;
pub const SEMI: i32 = 56;
pub const LPAREN: i32 = 57;
pub const RPAREN: i32 = 58;
pub const RBRACE: i32 = 59;
pub const RARROW: i32 = 60;
pub const LT: i32 = 61;
pub const GT: i32 = 62;
pub const QUESTION: i32 = 63;
pub const STAR: i32 = 64;
pub const PLUS_ASSIGN: i32 = 65;
pub const PLUS: i32 = 66;
pub const OR: i32 = 67;
pub const DOLLAR: i32 = 68;
pub const RANGE: i32 = 69;
pub const DOT: i32 = 70;
pub const AT: i32 = 71;
pub const POUND: i32 = 72;
pub const NOT: i32 = 73;
pub const ID: i32 = 74;
pub const END_ARGUMENT: i32 = 75;
pub const UNTERMINATED_ARGUMENT: i32 = 76;
pub const ARGUMENT_CONTENT: i32 = 77;
pub const UNTERMINATED_CHAR_SET: i32 = 78;
pub const CHANNEL_COMMENT: i32 = 3;Found a 1 line (391 tokens) duplication in the following files:
static LEXER_DFA_DATA: &[u32] = &[1280852999,3,0,4096,4133,4138,0,0,65535,4294967295,1,1,65535,0,2,2,65535,1,2,2,65535,2,3,3,65535,3,2,2,65535,4,2,2,65535,5,2,2,65535,6,4,2,65535,7,2,2,65535,8,5,2,65535,4294967295,6,2,65535,9,7,2,65535,4294967295,2,2,65535,10,8,2,65535,11,9,2,65535,12,2,2,65535,13,2,2,65535,14,2,2,65535,15,2,2,65535,16,2,2,65535,17,2,2,65535,18,10,4,65535,19,2,2,65535,20,11,4,65535,21,12,4,65535,22,13,4,65535,23,14,4,65535,24,15,4,65535,25,16,4,65535,26,17,4,65535,27,18,4,65535,28,19,4,65535,29,20,4,65535,30,21,5,65535,4294967295,2,2,65535,31,2,2,65535,32,2,2,65535,33,2,2,65535,34,22,3,68,4294967295,2,2,65535,35,2,2,65535,36,2,2,65535,37,23,6,70,4294967295,24,7,65535,38,2,2,65535,39,25,4,65535,40,26,4,65535,41,27,4,65535,42,28,4,65535,43,29,4,65535,44,30,4,65535,45,31,4,65535,46,32,4,65535,47,33,4,65535,48,34,4,65535,49,35,4,65535,50,36,4,65535,51,37,4,65535,52,38,4,65535,53,39,4,65535,54,40,4,65535,55,41,8,65535,4294967295,42,9,65535,4294967295,43,5,65535,4294967295,44,5,65535,4294967295,45,10,65535,4294967295,2,2,65535,56,46,11,65535,57,47,3,65535,58,2,2,65535,59,48,6,70,4294967295,49,12,102,4294967295,50,4,65535,60,51,4,65535,61,52,4,65535,62,53,4,65535,63,54,4,65535,64,55,4,65535,65,56,4,65535,66,57,4,65535,67,58,4,65535,68,59,4,65535,69,60,4,65535,70,61,4,65535,71,62,4,65535,72,63,4,65535,73,64,4,65535,74,65,4,65535,75,66,4,65535,76,67,8,65535,4294967295,68,5,65535,4294967295,69,8,124,4294967295,70,9,126,4294967295,71,13,65535,4294967295,72,14,65535,4294967295,73,10,142,4294967295,74,15,65535,4294967295,2,2,65535,77,75,11,68,4294967295,76,3,65535,78,77,6,70,4294967295,2,2,65535,79,78,12,102,4294967295,79,12,102,4294967295,80,16,152,80,81,4,65535,81,82,4,65535,82,83,4,65535,83,84,4,65535,84,85,4,65535,85,86,4,65535,86,87,4,65535,87,88,4,65535,88,10,4,65535,89,89,4,65535,90,90,4,65535,91,91,4,65535,92,92,4,65535,93,93,4,65535,94,94,4,65535,95,95,4,65535,96,96,4,65535,97,97,17,65535,4294967295,98,18,65535,4294967295,99,8,65535,4294967295,100,19,65535,4294967295,101,9,65535,4294967295,102,20,65535,4294967295,103,21,65535,4294967295,104,13,65535,4294967295,105,13,65535,4294967295,106,13,65535,4294967295,107,22,65535,4294967295,108,23,65535,98,109,14,65535,4294967295,110,24,65535,4294967295,111,25,65535,4294967295,112,26,65535,4294967295,109,14,65535,4294967295,113,27,65535,4294967295,109,14,65535,99,114,28,65535,4294967295,115,10,65535,4294967295,116,29,65535,4294967295,117,15,65535,4294967295,118,15,142,4294967295,119,30,65535,4294967295,120,11,65535,100,121,3,65535,101,2,2,65535,102,2,2,65535,103,2,2,65535,104,80,16,152,4294967295,122,16,152,4294967295,10,4,65535,105,123,4,65535,106,124,4,65535,107,125,4,65535,108,126,4,65535,109,127,4,65535,110,10,4,65535,111,128,4,65535,112,129,4,65535,113,130,4,65535,114,131,4,65535,115,132,4,65535,116,133,4,65535,117,134,4,65535,118,135,4,65535,119,136,4,65535,120,137,17,65535,4294967295,138,31,65535,4294967295,139,32,65535,4294967295,140,17,249,4294967295,141,33,65535,4294967295,142,18,124,4294967295,143,8,65535,4294967295,144,19,126,4294967295,145,9,65535,4294967295,146,20,65535,4294967295,108,23,65535,4294967295,147,13,65535,4294967295,148,20,65535,4294967295,149,20,124,4294967295,150,21,65535,4294967295,151,21,126,4294967295,152,34,65535,4294967295,153,35,65535,4294967295,154,22,65535,4294967295,155,22,142,4294967295,156,36,65535,4294967295,157,23,65535,4294967295,158,24,65535,4294967295,159,24,65535,4294967295,160,37,65535,4294967295,161,24,65535,4294967295,162,24,124,4294967295,163,38,65535,4294967295,164,39,65535,4294967295,165,25,65535,4294967295,166,25,126,4294967295,167,40,65535,4294967295,168,34,65535,4294967295,169,41,65535,4294967295,170,42,65535,4294967295,112,26,65535,4294967295,171,26,65535,4294967295,112,26,65535,4294967295,172,43,65535,4294967295,173,44,65535,4294967295,174,27,65535,4294967295,175,27,142,4294967295,176,28,142,4294967295,177,45,65535,4294967295,178,10,65535,4294967295,179,29,333,4294967295,180,46,65535,4294967295,181,15,65535,4294967295,182,47,65535,4294967295,183,30,65535,4294967295,184,30,142,4294967295,45,10,65535,121,185,11,65535,122,3,3,65535,123,2,2,65535,124,186,4,65535,125,187,4,65535,126,188,4,65535,127,189,4,65535,128,10,4,65535,129,10,4,65535,130,190,4,65535,131,10,4,65535,132,191,4,65535,133,192,4,65535,134,10,4,65535,135,193,4,65535,136,10,4,65535,137,194,48,65535,138,195,32,65535,4294967295,196,31,65535,4294967295,197,31,364,4294967295,198,32,65535,4294967295,199,17,65535,4294967295,200,49,65535,4294967295,201,32,65535,4294967295,202,32,364,4294967295,203,50,65535,4294967295,204,51,65535,4294967295,205,17,65535,4294967295,206,17,364,4294967295,207,17,65535,4294967295,208,18,65535,4294967295,209,33,65535,4294967295,210,33,65535,4294967295,2,2,65535,139,211,18,65535,4294967295,212,8,65535,4294967295,213,19,65535,4294967295,214,9,65535,4294967295,215,52,65535,4294967295,216,53,65535,4294967295,217,20,65535,4294967295,218,54,65535,4294967295,219,21,65535,4294967295,220,55,65535,4294967295,221,56,65535,4294967295,168,34,65535,4294967295,222,57,65535,4294967295,223,34,65535,4294967295,224,58,65535,4294967295,153,35,65535,4294967295,225,59,65535,4294967295,226,60,65535,4294967295,227,35,65535,4294967295,153,35,65535,4294967295,228,61,65535,4294967295,153,35,65535,140,229,62,65535,4294967295,230,22,65535,4294967295,231,63,65535,4294967295,232,36,65535,4294967295,233,36,65535,4294967295,234,36,142,4294967295,235,64,65535,4294967295,236,65,65535,4294967295,237,37,65535,4294967295,238,37,464,4294967295,239,66,65535,4294967295,240,67,65535,4294967295,241,68,65535,4294967295,158,24,65535,4294967295,242,24,65535,4294967295,243,69,65535,4294967295,244,38,65535,4294967295,245,38,484,4294967295,246,39,65535,4294967295,247,39,65535,4294967295,248,39,65535,4294967295,249,39,490,4294967295,250,70,65535,4294967295,251,71,65535,4294967295,252,72,65535,4294967295,111,25,65535,4294967295,253,25,65535,4294967295,254,40,65535,4294967295,255,40,507,4294967295,256,34,65535,4294967295,257,41,65535,4294967295,258,41,65535,4294967295,259,73,65535,4294967295,257,41,65535,4294967295,260,41,65535,4294967295,261,41,124,4294967295,262,74,65535,4294967295,263,75,65535,4294967295,170,42,65535,4294967295,264,42,65535,4294967295,265,42,126,4294967295,266,76,65535,4294967295,112,26,65535,4294967295,267,77,65535,4294967295,172,43,65535,4294967295,268,43,65535,4294967295,269,43,142,4294967295,270,44,65535,4294967295,271,44,333,4294967295,272,78,65535,4294967295,273,79,65535,4294967295,113,27,65535,4294967295,274,27,65535,4294967295,275,28,65535,4294967295,276,80,65535,4294967295,277,45,65535,4294967295,278,45,142,4294967295,279,81,65535,4294967295,280,10,65535,4294967295,281,29,65535,4294967295,282,82,65535,4294967295,283,83,65535,4294967295,284,46,65535,4294967295,285,84,333,4294967295,286,84,65535,4294967295,287,9,65535,141,288,15,65535,4294967295,289,47,507,4294967295,290,85,65535,4294967295,291,86,65535,4294967295,292,87,65535,4294967295,293,30,65535,4294967295,294,11,65535,142,295,4,65535,143,10,4,65535,144,296,4,65535,145,10,4,65535,146,297,88,65535,147,10,4,65535,148,298,4,65535,149,10,4,65535,150,299,89,65535,4294967295,2,2,65535,151,300,17,65535,4294967295,301,31,65535,4294967295,302,90,65535,4294967295,303,31,65535,4294967295,304,31,364,4294967295,305,31,65535,4294967295,300,17,65535,4294967295,306,32,65535,4294967295,307,49,65535,4294967295,308,49,610,4294967295,309,91,65535,4294967295,310,92,65535,4294967295,311,32,65535,4294967295,312,32,364,4294967295,313,32,65535,4294967295,314,50,629,4294967295,315,93,65535,4294967295,316,94,65535,4294967295,317,51,249,4294967295,318,32,65535,4294967295,319,17,65535,4294967295,320,17,65535,4294967295,321,33,65535,4294967295,322,95,65535,4294967295,323,96,65535,4294967295,324,18,65535,4294967295,67,8,65535,4294967295,325,19,65535,4294967295,42,9,65535,4294967295,326,52,65535,4294967295,327,97,65535,4294967295,328,91,65535,4294967295,329,52,65535,4294967295,330,52,249,4294967295,331,98,65535,4294967295,332,99,65535,4294967295,216,53,65535,4294967295,333,53,65535,4294967295,334,53,124,4294967295,335,100,65535,4294967295,336,20,65535,4294967295,337,101,65535,4294967295,338,102,65535,4294967295,218,54,65535,4294967295,339,54,65535,4294967295,340,54,126,4294967295,341,103,65535,4294967295,342,21,65535,4294967295,343,55,65535,4294967295,344,34,65535,4294967295,345,55,65535,4294967295,346,55,124,4294967295,347,56,65535,4294967295,348,56,126,4294967295,349,57,65535,4294967295,350,104,65535,4294967295,351,105,65535,4294967295,352,106,65535,4294967295,353,57,65535,4294967295,354,107,65535,4294967295,355,58,65535,4294967295,356,58,142,4294967295,357,108,65535,4294967295,358,59,65535,4294967295,359,59,65535,4294967295,360,109,65535,4294967295,361,59,65535,4294967295,358,59,65535,4294967295,362,59,124,4294967295,363,110,65535,4294967295,364,111,65535,4294967295,365,60,65535,4294967295,226,60,65535,4294967295,366,60,126,4294967295,367,112,65535,4294967295,349,57,65535,4294967295,368,113,65535,4294967295,369,61,65535,4294967295,228,61,65535,4294967295,370,61,142,4294967295,371,114,65535,4294967295,229,62,65535,4294967295,372,62,65535,4294967295,373,62,142,4294967295,374,22,65535,4294967295,375,63,65535,4294967295,376,63,333,4294967295,377,115,65535,4294967295,378,62,65535,4294967295,379,36,65535,4294967295,380,116,65535,4294967295,381,64,65535,4294967295,382,64,65535,4294967295,383,64,142,4294967295,107,22,65535,152,384,65,65535,4294967295,385,65,65535,4294967295,386,117,65535,4294967295,387,65,65535,4294967295,388,65,249,4294967295,389,118,65535,4294967295,390,119,65535,4294967295,391,120,65535,4294967295,392,121,65535,4294967295,160,37,65535,4294967295,393,37,65535,4294967295,394,66,65535,4294967295,395,66,829,4294967295,396,122,65535,4294967295,240,67,65535,4294967295,397,67,65535,4294967295,398,67,124,4294967295,399,123,65535,4294967295,400,124,65535,4294967295,401,82,65535,4294967295,402,68,65535,4294967295,403,68,124,4294967295,404,125,65535,4294967295,405,24,65535,4294967295,406,69,65535,4294967295,407,69,853,4294967295,408,126,65535,4294967295,409,127,65535,4294967295,410,128,65535,4294967295,163,38,65535,4294967295,411,38,65535,4294967295,412,129,65535,4294967295,413,130,65535,4294967295,414,131,65535,4294967295,415,132,65535,4294967295,246,39,65535,4294967295,416,39,65535,4294967295,417,70,65535,4294967295,418,70,887,4294967295,419,133,65535,4294967295,251,71,65535,4294967295,420,71,65535,4294967295,421,71,126,4294967295,422,134,65535,4294967295,423,135,65535,4294967295,424,136,65535,4294967295,425,72,65535,4294967295,426,72,126,4294967295,427,25,65535,4294967295,428,137,65535,4294967295,429,138,65535,4294967295,430,139,65535,4294967295,167,40,65535,4294967295,431,40,65535,4294967295,71,13,65535,4294967295,432,140,65535,4294967295,259,73,65535,4294967295,433,73,65535,4294967295,434,73,464,4294967295,435,141,65535,4294967295,240,67,65535,4294967295,436,142,65535,4294967295,257,41,65535,4294967295,437,41,65535,4294967295,438,143,65535,4294967295,262,74,65535,4294967295,439,74,65535,4294967295,440,74,484,4294967295,441,75,65535,4294967295,442,75,65535,4294967295,441,75,65535,4294967295,443,75,65535,4294967295,444,75,490,4294967295,445,144,65535,4294967295,251,71,65535,4294967295,446,145,65535,4294967295,170,42,65535,4294967295,447,42,65535,4294967295,266,76,65535,4294967295,448,76,65535,4294967295,449,76,507,4294967295,267,77,65535,4294967295,450,77,65535,4294967295,451,77,333,4294967295,272,78,65535,4294967295,452,146,65535,4294967295,172,43,65535,4294967295,453,43,65535,4294967295,454,147,65535,4294967295,455,148,65535,4294967295,173,44,65535,4294967295,456,44,65535,4294967295,457,149,65535,4294967295,272,78,65535,4294967295,458,78,65535,4294967295,459,78,142,4294967295,460,79,65535,4294967295,461,150,65535,4294967295,462,79,65535,4294967295,463,79,992,4294967295,464,27,65535,4294967295,465,28,65535,4294967295,466,80,333,4294967295,467,151,65535,4294967295,468,152,65535,4294967295,469,153,65535,4294967295,470,45,65535,4294967295,471,81,65535,4294967295,472,81,142,4294967295,114,28,65535,153,45,10,65535,4294967295,473,29,65535,4294967295,474,136,65535,4294967295,475,82,464,4294967295,476,83,65535,4294967295,477,83,333,4294967295,478,154,65535,4294967295,479,155,65535,4294967295,480,156,65535,4294967295,481,84,65535,4294967295,482,84,65535,4294967295,116,29,65535,154,483,15,65535,4294967295,484,47,65535,4294967295,485,85,65535,4294967295,486,85,507,4294967295,487,157,65535,4294967295,488,86,65535,4294967295,489,86,65535,4294967295,490,86,142,4294967295,224,58,65535,155,491,87,65535,4294967295,492,158,65535,4294967295,493,159,65535,4294967295,494,87,142,4294967295,495,160,65535,4294967295,491,87,65535,156,496,30,65535,4294967295,46,11,65535,157,497,161,65535,158,10,4,65535,159,498,162,65535,4294967295,2,2,65535,160,499,4,65535,161,500,32,65535,4294967295,501,32,65535,4294967295,502,90,65535,4294967295,503,90,364,4294967295,504,31,65535,4294967295,505,31,65535,4294967295,506,31,65535,4294967295,507,163,65535,4294967295,508,49,65535,4294967295,509,164,65535,4294967295,510,49,65535,4294967295,511,49,364,4294967295,512,49,65535,4294967295,513,52,65535,4294967295,514,165,65535,4294967295,515,91,65535,4294967295,516,91,65535,4294967295,517,91,364,4294967295,518,166,65535,4294967295,519,92,65535,4294967295,520,65,65535,4294967295,521,167,65535,4294967295,522,168,65535,4294967295,523,92,364,4294967295,524,169,65535,4294967295,300,17,65535,4294967295,525,32,65535,4294967295,526,32,65535,4294967295,527,170,65535,4294967295,528,50,65535,4294967295,529,50,364,4294967295,530,50,65535,4294967295,531,171,65535,4294967295,532,93,65535,4294967295,533,93,629,4294967295,534,172,65535,4294967295,535,94,65535,4294967295,536,51,65535,4294967295,537,94,65535,4294967295,538,94,364,4294967295,539,51,65535,4294967295,540,51,364,4294967295,541,51,65535,4294967295,542,163,65535,4294967295,543,17,65535,4294967295,544,17,65535,4294967295,545,51,65535,4294967295,546,173,65535,4294967295,547,174,65535,4294967295,548,95,65535,4294967295,549,95,65535,4294967295,550,95,65535,4294967295,551,175,65535,4294967295,552,176,65535,162,553,96,65535,4294967295,554,177,65535,4294967295,555,178,65535,4294967295,556,179,65535,4294967295,553,96,65535,4294967295,557,153,65535,4294967295,553,96,65535,163,558,18,65535,4294967295,559,19,65535,4294967295,560,91,65535,4294967295,561,97,65535,4294967295,562,97,65535,4294967295,563,97,364,4294967295,564,52,65535,4294967295,565,180,65535,4294967295,566,52,65535,4294967295,567,52,364,4294967295,568,52,65535,4294967295,569,98,65535,4294967295,570,53,65535,4294967295,571,101,65535,4294967295,572,98,65535,4294967295,573,98,65535,4294967295,574,98,124,4294967295,575,181,65535,4294967295,343,55,65535,164,576,99,65535,4294967295,577,182,464,4294967295,578,183,65535,4294967295,579,53,65535,4294967295,580,100,65535,4294967295,581,184,484,4294967295,582,185,65535,4294967295,583,20,65535,4294967295,584,54,65535,4294967295,585,101,65535,4294967295,586,186,490,4294967295,587,99,65535,4294967295,588,102,65535,4294967295,589,102,65535,4294967295,590,102,126,4294967295,221,56,65535,165,591,187,65535,4294967295,592,54,65535,4294967295,593,103,65535,4294967295,594,188,507,4294967295,595,189,65535,4294967295,596,21,65535,4294967295,597,190,65535,4294967295,598,191,65535,4294967295,599,55,65535,4294967295,600,192,65535,4294967295,601,56,65535,4294967295,349,57,65535,4294967295,602,104,65535,4294967295,603,193,65535,4294967295,604,194,65535,4294967295,605,104,65535,4294967295,602,104,65535,4294967295,606,104,124,4294967295,607,195,65535,4294967295,602,104,65535,166,608,196,65535,4294967295,609,197,65535,4294967295,610,105,65535,4294967295,351,105,65535,4294967295,611,105,126,4294967295,612,198,65535,4294967295,351,105,65535,167,352,106,65535,4294967295,613,199,65535,4294967295,614,200,65535,4294967295,615,106,65535,4294967295,616,106,65535,4294967295,617,106,65535,4294967295,618,201,65535,4294967295,619,193,65535,4294967295,349,57,65535,4294967295,620,202,65535,4294967295,349,57,65535,168,621,203,65535,4294967295,622,107,65535,4294967295,354,107,65535,4294967295,623,107,142,4294967295,624,204,65535,4294967295,354,107,65535,169,625,205,65535,4294967295,626,58,65535,4294967295,627,206,65535,4294967295,357,108,65535,4294967295,628,108,65535,4294967295,629,108,142,4294967295,630,207,65535,4294967295,631,109,65535,4294967295,360,109,65535,4294967295,632,109,464,4294967295,633,208,65535,4294967295,634,193,65535,4294967295,578,183,65535,4294967295,358,59,65535,4294967295,635,59,65535,4294967295,636,209,65535,4294967295,637,110,65535,4294967295,363,110,65535,4294967295,638,110,484,4294967295,639,111,65535,4294967295,640,111,65535,4294967295,641,111,65535,4294967295,639,111,65535,4294967295,642,111,490,4294967295,643,210,65535,4294967295,609,197,65535,4294967295,591,187,65535,4294967295,226,60,65535,4294967295,644,60,65535,4294967295,645,112,65535,4294967295,367,112,65535,4294967295,646,112,507,4294967295,647,113,65535,4294967295,368,113,65535,4294967295,648,113,333,4294967295,620,202,65535,4294967295,649,211,65535,4294967295,228,61,65535,4294967295,650,61,65535,4294967295,651,114,65535,4294967295,652,206,333,4294967295,156,36,65535,4294967295,653,62,65535,4294967295,654,22,65535,4294967295,655,212,65535,4294967295,656,63,65535,4294967295,657,213,65535,4294967295,658,214,65535,4294967295,659,115,65535,4294967295,660,115,65535,4294967295,661,215,333,4294967295,662,215,65535,4294967295,663,21,65535,170,664,202,65535,4294967295,665,36,65535,4294967295,666,116,65535,4294967295,667,116,507,4294967295,668,216,65535,4294967295,669,62,65535,4294967295,618,201,65535,4294967295,670,64,65535,4294967295,385,65,65535,4294967295,671,217,65535,4294967295,672,117,65535,4294967295,673,117,1429,4294967295,674,218,65535,4294967295,675,219,65535,4294967295,676,220,65535,4294967295,677,65,65535,4294967295,678,65,364,4294967295,679,65,65535,4294967295,680,221,65535,4294967295,681,118,65535,4294967295,682,118,1452,4294967295,390,119,65535,4294967295,683,119,65535,4294967295,684,119,464,4294967295,685,222,65535,4294967295,686,223,65535,4294967295,687,224,65535,4294967295,688,120,464,4294967295,689,225,65535,4294967295,690,226,65535,4294967295,691,121,65535,4294967295,692,121,464,4294967295,693,227,65535,4294967295,694,37,65535,4294967295,695,228,65535,4294967295,696,229,65535,4294967295,697,230,65535,4294967295,239,66,65535,4294967295,698,66,65535,4294967295,396,122,65535,4294967295,699,122,65535,4294967295,700,231,65535,4294967295,240,67,65535,4294967295,240,67,65535,4294967295,701,67,65535,4294967295,702,232,65535,4294967295,399,123,65535,4294967295,703,123,65535,4294967295,704,124,65535,4294967295,705,68,65535,4294967295,706,135,65535,4294967295,707,124,65535,4294967295,708,124,124,4294967295,709,233,65535,4294967295,67,8,65535,171,710,68,65535,4294967295,711,125,484,4294967295,712,24,65535,4294967295,713,234,65535,4294967295,714,235,65535,4294967295,715,236,65535,4294967295,243,69,65535,4294967295,716,69,65535,4294967295,717,237,65535,4294967295,408,126,65535,4294967295,718,126,65535,4294967295,719,126,484,4294967295,720,127,484,4294967295,721,238,65535,4294967295,722,239,65535,4294967295,723,128,65535,4294967295,724,128,484,4294967295,725,240,65535,4294967295,726,38,65535,4294967295,727,129,65535,4294967295,728,129,65535,4294967295,729,129,65535,4294967295,730,129,1539,4294967295,731,241,65535,4294967295,413,130,65535,4294967295,732,130,65535,4294967295,733,130,490,4294967295,734,242,65535,4294967295,735,243,65535,4294967295,736,244,65535,4294967295,737,131,490,4294967295,738,245,65535,4294967295,739,246,65535,4294967295,740,132,65535,4294967295,741,132,490,4294967295,742,247,65535,4294967295,743,39,65535,4294967295,744,248,65535,4294967295,745,249,65535,4294967295,746,250,65535,4294967295,250,70,65535,4294967295,747,70,65535,4294967295,748,133,65535,4294967295,749,133,65535,4294967295,748,133,65535,4294967295,750,133,65535,4294967295,751,251,65535,4294967295,251,71,65535,4294967295,251,71,65535,4294967295,752,71,65535,4294967295,422,134,65535,4294967295,753,134,65535,4294967295,754,72,65535,4294967295,755,135,490,4294967295,756,136,65535,4294967295,757,136,126,4294967295,42,9,65535,172,758,72,65535,4294967295,759,25,65535,4294967295,428,137,65535,4294967295,760,137,65535,4294967295,761,137,507,4294967295,762,138,507,4294967295,763,252,65535,4294967295,764,139,65535,4294967295,765,139,65535,4294967295,766,139,1604,4294967295,767,40,65535,4294967295,768,140,65535,4294967295,769,253,65535,4294967295,770,140,65535,4294967295,771,254,65535,4294967295,768,140,65535,4294967295,772,140,65535,4294967295,773,140,249,4294967295,774,255,65535,4294967295,390,119,65535,4294967295,775,256,65535,4294967295,259,73,65535,4294967295,776,73,65535,4294967295,435,141,65535,4294967295,777,141,65535,4294967295,778,141,829,4294967295,779,191,65535,4294967295,780,182,65535,4294967295,436,142,65535,4294967295,781,142,65535,4294967295,782,142,124,4294967295,783,184,65535,4294967295,784,41,65535,4294967295,438,143,65535,4294967295,785,143,65535,4294967295,786,143,853,4294967295,408,126,65535,4294967295,787,257,65535,4294967295,262,74,65535,4294967295,788,74,65535,4294967295,789,258,65535,4294967295,413,130,65535,4294967295,790,259,65535,4294967295,441,75,65535,4294967295,791,75,65535,4294967295,445,144,65535,4294967295,792,144,65535,4294967295,793,144,887,4294967295,794,186,65535,4294967295,446,145,65535,4294967295,795,145,65535,4294967295,796,145,126,4294967295,797,188,65535,4294967295,798,42,65535,4294967295,428,137,65535,4294967295,799,260,65535,4294967295,800,261,65535,4294967295,266,76,65535,4294967295,801,76,65535,4294967295,454,147,65535,4294967295,802,262,65535,4294967295,803,263,65535,4294967295,267,77,65535,4294967295,804,77,65535,4294967295,805,146,65535,4294967295,806,264,65535,4294967295,805,146,65535,4294967295,807,146,65535,4294967295,808,146,992,4294967295,809,43,65535,4294967295,454,147,65535,4294967295,810,147,65535,4294967295,811,147,333,4294967295,812,148,65535,4294967295,813,148,65535,4294967295,814,148,1707,4294967295,815,44,65535,4294967295,457,149,65535,4294967295,816,149,65535,4294967295,272,78,65535,4294967295,817,265,65535,4294967295,272,78,65535,4294967295,818,78,65535,4294967295,819,150,65535,4294967295,820,150,1719,4294967295,821,266,65535,4294967295,822,267,65535,4294967295,823,268,65535,4294967295,460,79,65535,4294967295,824,79,65535,4294967295,825,27,65535,4294967295,826,28,65535,4294967295,827,80,65535,4294967295,828,120,65535,4294967295,829,269,65535,4294967295,830,151,65535,4294967295,831,270,333,4294967295,832,270,65535,4294967295,833,19,65535,173,834,271,65535,4294967295,835,152,65535,4294967295,836,152,65535,4294967295,837,152,142,4294967295,838,272,65535,4294967295,839,273,65535,4294967295,840,274,65535,4294967295,841,153,142,4294967295,842,45,65535,4294967295,843,275,65535,4294967295,844,276,65535,4294967295,845,81,65535,4294967295,846,29,65535,4294967295,847,277,65535,4294967295,848,82,65535,4294967295,849,83,65535,4294967295,850,154,65535,4294967295,851,154,507,4294967295,182,47,65535,174,852,155,65535,4294967295,853,155,65535,4294967295,854,278,333,4294967295,855,279,65535,4294967295,856,56,65535,175,857,156,65535,4294967295,858,280,65535,4294967295,859,281,65535,4294967295,860,282,65535,4294967295,861,283,333,4294967295,862,284,65535,176,863,84,65535,4294967295,864,278,65535,4294967295,865,283,65535,4294967295,74,15,65535,4294967295,866,47,65535,4294967295,867,85,65535,4294967295,868,157,65535,4294967295,869,10,65535,177,870,205,65535,4294967295,871,86,65535,4294967295,872,158,65535,4294967295,873,285,507,4294967295,874,286,65535,4294967295,875,87,65535,178,876,287,65535,4294967295,493,159,65535,4294967295,877,159,65535,4294967295,878,159,142,4294967295,879,288,65535,4294967295,880,289,65535,4294967295,491,87,65535,4294967295,881,87,65535,4294967295,882,290,65535,4294967295,883,283,65535,4294967295,884,160,65535,4294967295,885,160,142,4294967295,886,291,65535,4294967295,887,30,65535,4294967295,888,292,65535,4294967295,2,2,65535,179,10,4,65535,180,889,163,65535,4294967295,890,293,65535,4294967295,891,90,65535,4294967295,892,90,65535,4294967295,893,90,364,4294967295,894,90,65535,4294967295,895,294,65535,4294967295,896,31,65535,4294967295,897,31,65535,4294967295,898,163,65535,4294967295,899,294,65535,4294967295,900,295,65535,4294967295,901,296,65535,4294967295,902,163,65535,4294967295,903,163,249,4294967295,904,297,65535,4294967295,137,17,65535,181,905,298,65535,4294967295,906,164,65535,4294967295,907,164,610,4294967295,908,49,65535,4294967295,909,49,65535,4294967295,910,49,65535,4294967295,911,91,65535,4294967295,912,165,65535,4294967295,913,165,65535,4294967295,914,165,610,4294967295,915,299,65535,4294967295,916,300,65535,4294967295,917,91,65535,4294967295,918,91,364,4294967295,919,91,65535,4294967295,920,166,65535,4294967295,921,166,629,4294967295,922,301,65535,4294967295,923,65,65535,4294967295,924,167,65535,4294967295,925,167,610,4294967295,926,302,65535,4294967295,927,299,65535,4294967295,928,303,65535,4294967295,929,304,65535,4294967295,522,168,65535,4294967295,930,168,65535,4294967295,931,168,364,4294967295,932,305,65535,4294967295,933,92,364,4294967295,934,92,65535,4294967295,935,306,65535,4294967295,936,169,65535,4294967295,937,169,629,4294967295,938,32,65535,4294967295,939,32,65535,4294967295,940,170,629,4294967295,941,307,65535,4294967295,942,50,65535,4294967295,943,50,65535,4294967295,944,171,1987,4294967295,945,308,65535,4294967295,946,93,65535,4294967295,947,93,364,4294967295,948,93,65535,4294967295,949,309,65535,4294967295,950,172,65535,4294967295,951,310,629,4294967295,952,10,65535,182,536,51,65535,4294967295,953,94,65535,4294967295,954,311,65535,4294967295,955,312,65535,4294967295,956,94,65535,4294967295,957,94,364,4294967295,958,94,65535,4294967295,959,94,65535,4294967295,960,51,65535,4294967295,961,51,65535,4294967295,962,295,65535,4294967295,963,17,65535,4294967295,137,17,65535,4294967295,964,94,65535,4294967295,965,173,65535,4294967295,552,176,65535,4294967295,966,95,65535,4294967295,967,173,65535,4294967295,968,173,124,4294967295,969,174,65535,4294967295,970,174,126,4294967295,971,313,65535,4294967295,972,314,65535,4294967295,973,175,65535,4294967295,974,175,142,4294967295,975,176,65535,4294967295,976,177,65535,4294967295,977,177,65535,4294967295,978,315,65535,4294967295,979,177,65535,4294967295,980,177,124,4294967295,981,316,65535,4294967295,982,317,65535,4294967295,983,178,65535,4294967295,984,178,126,4294967295,985,318,65535,4294967295,986,313,65535,4294967295,987,319,65535,4294967295,988,320,65535,4294967295,556,179,65535,4294967295,989,179,65535,4294967295,556,179,65535,4294967295,990,321,65535,4294967295,98,18,65535,4294967295,100,19,65535,4294967295,513,52,65535,4294967295,991,97,65535,4294967295,992,97,65535,4294967295,993,97,364,4294967295,994,97,65535,4294967295,995,91,65535,4294967295,996,322,65535,4294967295,997,323,65535,4294967295,565,180,65535,4294967295,998,180,65535,4294967295,999,180,249,4294967295,1000,324,65535,4294967295,1001,91,65535,4294967295,1002,52,65535,4294967295,1003,52,65535,4294967295,1004,53,65535,4294967295,1005,325,65535,4294967295,1006,191,65535,4294967295,1007,98,65535,4294967295,1008,181,65535,4294967295,1009,326,992,4294967295,1010,327,65535,4294967295,1011,328,65535,4294967295,1012,182,65535,4294967295,1013,182,65535,4294967295,1014,329,65535,4294967295,1015,213,65535,4294967295,1016,183,65535,4294967295,1017,183,65535,4294967295,1018,183,124,4294967295,1019,330,65535,4294967295,1020,53,65535,4294967295,1021,331,65535,4294967295,1022,184,65535,4294967295,1023,184,65535,4294967295,1024,332,65535,4294967295,582,185,65535,4294967295,1025,185,65535,4294967295,1026,257,484,4294967295,146,20,65535,4294967295,1027,333,65535,4294967295,1028,334,65535,4294967295,1029,186,65535,4294967295,1030,186,65535,4294967295,1031,186,65535,4294967295,1032,102,65535,4294967295,1033,192,65535,4294967295,1034,102,65535,4294967295,1035,335,65535,4294967295,1036,336,65535,4294967295,1037,187,65535,4294967295,1038,187,65535,4294967295,1039,187,126,4294967295,1040,54,65535,4294967295,1041,337,65535,4294967295,1042,188,65535,4294967295,1043,188,65535,4294967295,595,189,65535,4294967295,1044,189,65535,4294967295,1045,260,507,4294967295,103,21,65535,4294967295,1046,190,65535,4294967295,1047,338,65535,4294967295,1048,253,65535,4294967295,1049,190,65535,4294967295,1050,190,249,4294967295,1051,142,65535,4294967295,598,191,65535,4294967295,1052,191,65535,4294967295,1053,191,124,4294967295,1054,326,65535,4294967295,1055,55,65535,4294967295,1056,182,65535,4294967295,600,192,65535,4294967295,1057,192,65535,4294967295,1058,192,126,4294967295,1059,56,65535,4294967295,634,193,65535,4294967295,1060,339,65535,4294967295,1061,196,65535,4294967295,634,193,65535,4294967295,1062,193,65535,4294967295,1063,193,124,4294967295,1064,340,65535,4294967295,1065,341,65535,4294967295,1066,194,65535,4294967295,604,194,65535,4294967295,1067,342,490,4294967295,1068,343,65535,4294967295,1069,104,65535,183,1070,344,65535,4294967295,1071,104,65535,4294967295,602,104,65535,4294967295,1072,104,65535,4294967295,1073,345,65535,4294967295,1074,195,65535,4294967295,607,195,65535,4294967295,1075,346,992,4294967295,1076,104,65535,184,1077,196,65535,4294967295,1078,347,65535,4294967295,1079,196,65535,4294967295,1061,196,65535,4294967295,1080,348,464,4294967295,1081,349,65535,4294967295,1082,105,65535,185,1083,194,65535,4294967295,609,197,65535,4294967295,1084,197,65535,4294967295,1085,197,126,4294967295,1086,350,65535,4294967295,351,105,65535,4294967295,1087,105,65535,4294967295,1088,351,65535,4294967295,1089,198,65535,4294967295,612,198,65535,4294967295,1090,352,333,4294967295,1091,105,65535,186,1092,199,65535,4294967295,1093,353,65535,4294967295,1094,354,65535,4294967295,1095,199,65535,4294967295,1092,199,65535,4294967295,1096,199,124,4294967295,1097,355,65535,4294967295,1092,199,65535,187,1098,356,65535,4294967295,1099,357,65535,4294967295,1100,200,65535,4294967295,614,200,65535,4294967295,1101,200,126,4294967295,1102,358,65535,4294967295,614,200,65535,188,349,57,65535,4294967295,1103,353,65535,4294967295,352,106,65535,4294967295,352,106,65535,4294967295,1104,359,65535,4294967295,352,106,65535,189,1105,360,65535,4294967295,1106,201,65535,4294967295,1107,201,142,4294967295,1108,361,65535,4294967295,618,201,65535,190,1109,104,65535,4294967295,620,202,65535,4294967295,1110,202,142,4294967295,1111,203,65535,4294967295,621,203,65535,4294967295,1112,362,507,4294967295,1113,363,65535,4294967295,1114,107,65535,191,1115,364,65535,4294967295,1116,340,65535,4294967295,354,107,65535,4294967295,1117,107,65535,4294967295,1118,352,65535,4294967295,624,204,65535,4294967295,1119,204,65535,4294967295,1120,204,142,4294967295,1115,364,65535,4294967295,625,205,65535,4294967295,1121,205,65535,4294967295,1122,205,142,4294967295,1123,58,65535,4294967295,1124,206,65535,4294967295,1125,365,65535,4294967295,156,36,65535,4294967295,1126,108,65535,4294967295,1127,207,65535,4294967295,1128,207,65535,4294967295,1129,366,65535,4294967295,1130,207,65535,4294967295,1127,207,65535,4294967295,1131,207,249,4294967295,1132,367,65535,4294967295,1078,347,65535,4294967295,1133,368,65535,4294967295,360,109,65535,4294967295,1134,109,65535,4294967295,1135,208,65535,4294967295,633,208,65535,4294967295,1136,208,829,4294967295,1137,59,65535,4294967295,1138,209,65535,4294967295,636,209,65535,4294967295,1139,209,853,4294967295,1140,369,65535,4294967295,1141,370,65535,4294967295,363,110,65535,4294967295,1142,110,65535,4294967295,1143,371,65535,4294967295,1144,341,65535,4294967295,1145,372,65535,4294967295,639,111,65535,4294967295,1146,111,65535,4294967295,1147,210,65535,4294967295,643,210,65535,4294967295,1148,210,887,4294967295,1149,60,65535,4294967295,1150,373,65535,4294967295,1151,374,65535,4294967295,367,112,65535,4294967295,1152,112,65535,4294967295,1088,351,65535,4294967295,1153,375,65535,4294967295,368,113,65535,4294967295,1154,113,65535,4294967295,1155,211,65535,4294967295,1156,376,65535,4294967295,1157,211,65535,4294967295,1155,211,65535,4294967295,1158,211,992,4294967295,1159,61,65535,4294967295,1160,377,65535,4294967295,1161,206,65535,4294967295,1162,62,65535,4294967295,107,22,65535,4294967295,655,212,65535,4294967295,1163,212,65535,4294967295,1164,212,333,4294967295,1165,378,65535,4294967295,1166,63,65535,4294967295,1167,336,65535,4294967295,1168,213,65535,4294967295,1169,213,464,4294967295,1170,214,65535,4294967295,1171,214,65535,4294967295,1172,214,333,4294967295,1173,379,65535,4294967295,1174,212,65535,4294967295,1102,358,65535,4294967295,1175,215,65535,4294967295,1176,215,65535,4294967295,1177,215,65535,4294967295,231,63,65535,192,1104,359,65535,4294967295,1178,36,65535,4294967295,1179,380,65535,4294967295,1180,116,65535,4294967295,1181,216,65535,4294967295,1182,216,65535,4294967295,1183,216,507,4294967295,1184,381,65535,4294967295,1185,382,65535,4294967295,1186,64,65535,4294967295,1187,217,65535,4294967295,1188,217,65535,4294967295,1189,217,65535,4294967295,1190,217,2528,4294967295,1191,383,65535,4294967295,1192,384,65535,4294967295,1193,385,65535,4294967295,1194,386,65535,4294967295,1195,217,65535,4294967295,1196,117,364,4294967295,1197,117,65535,4294967295,1198,218,65535,4294967295,1199,218,2551,4294967295,1200,387,65535,4294967295,675,219,65535,4294967295,1201,219,65535,4294967295,1202,219,249,4294967295,1203,388,65535,4294967295,890,293,65535,4294967295,1204,277,65535,4294967295,1205,220,65535,4294967295,1206,220,249,4294967295,1207,389,65535,4294967295,923,65,65535,4294967295,1208,65,65535,4294967295,1209,65,65535,4294967295,1210,221,65535,4294967295,1211,221,2575,4294967295,1212,390,65535,4294967295,1213,391,65535,4294967295,1214,392,65535,4294967295,1215,393,65535,4294967295,1216,118,364,4294967295,1217,118,65535,4294967295,390,119,65535,4294967295,1218,394,65535,4294967295,390,119,65535,4294967295,1219,119,65535,4294967295,685,222,65535,4294967295,1220,222,65535,4294967295,1221,223,65535,4294967295,1222,223,126,4294967295,100,19,65535,193,1223,244,65535,4294967295,1224,224,65535,4294967295,1225,224,124,4294967295,1226,120,65535,4294967295,1227,225,65535,4294967295,1228,121,65535,4294967295,1229,245,65535,4294967295,1230,225,65535,4294967295,1231,225,490,4294967295,1232,395,65535,4294967295,706,135,65535,194,1233,246,65535,4294967295,1234,226,65535,4294967295,1235,246,464,4294967295,1236,9,65535,195,1237,121,65535,4294967295,1238,227,829,4294967295,1239,37,65535,4294967295,695,228,65535,4294967295,1240,228,65535,4294967295,1241,228,829,4294967295,1242,396,65535,4294967295,1243,229,829,4294967295,1244,397,65535,4294967295,1245,398,65535,4294967295,1246,230,65535,4294967295,1247,230,829,4294967295,1248,399,65535,4294967295,1249,66,65535,4294967295,390,119,65535,4294967295,700,231,65535,4294967295,1250,231,65535,4294967295,1251,67,65535,4294967295,702,232,65535,4294967295,1252,232,65535,4294967295,408,126,65535,4294967295,1253,68,65535,4294967295,1254,163,65535,4294967295,1255,290,65535,4294967295,1256,124,65535,4294967295,1257,233,992,4294967295,1258,400,65535,4294967295,1259,68,65535,4294967295,1260,125,65535,4294967295,158,24,65535,4294967295,713,234,65535,4294967295,1261,234,65535,4294967295,1262,234,853,4294967295,1263,235,853,4294967295,1264,401,65535,4294967295,1265,402,65535,4294967295,1266,236,65535,4294967295,1267,236,853,4294967295,1268,403,65535,4294967295,1269,69,65535,4294967295,717,237,65535,4294967295,1270,237,65535,4294967295,408,126,65535,4294967295,1271,404,65535,4294967295,408,126,65535,4294967295,1272,126,65535,4294967295,1273,127,65535,4294967295,1274,238,65535,4294967295,1275,238,484,4294967295,1276,239,853,4294967295,1277,128,65535,4294967295,1278,240,65535,4294967295,1279,405,484,4294967295,1280,10,65535,196,1281,38,65535,4294967295,1282,129,65535,4294967295,728,129,65535,4294967295,1283,406,65535,4294967295,1284,407,65535,4294967295,1285,408,65535,4294967295,1286,129,65535,4294967295,1287,129,364,4294967295,1288,129,65535,4294967295,1289,241,65535,4294967295,1290,241,2707,4294967295,413,130,65535,4294967295,1291,409,65535,4294967295,413,130,65535,4294967295,1292,130,65535,4294967295,734,242,65535,4294967295,1293,242,65535,4294967295,1294,131,65535,4294967295,1295,243,65535,4294967295,1296,243,126,4294967295,1297,224,65535,4294967295,1298,244,65535,4294967295,1299,244,124,4294967295,98,18,65535,197,1300,131,65535,4294967295,1301,132,65535,4294967295,1302,245,65535,4294967295,1303,410,65535,4294967295,1304,8,65535,198,1305,226,65535,4294967295,1306,246,65535,4294967295,1307,411,65535,4294967295,401,82,65535,199,1308,132,65535,4294967295,1309,405,65535,4294967295,1310,247,887,4294967295,1311,39,65535,4294967295,744,248,65535,4294967295,1312,248,65535,4294967295,1313,248,887,4294967295,1314,412,65535,4294967295,1315,249,887,4294967295,1316,413,65535,4294967295,1317,414,65535,4294967295,1318,250,65535,4294967295,1319,250,887,4294967295,1320,415,65535,4294967295,1321,70,65535,4294967295,1322,416,65535,4294967295,413,130,65535,4294967295,751,251,65535,4294967295,1323,251,65535,4294967295,1324,71,65535,4294967295,428,137,65535,4294967295,1325,296,65535,4294967295,1326,135,65535,4294967295,1327,284,65535,4294967295,1328,136,65535,4294967295,1329,72,65535,4294967295,111,25,65535,4294967295,428,137,65535,4294967295,1330,417,65535,4294967295,428,137,65535,4294967295,1331,137,65535,4294967295,1332,138,65535,4294967295,1333,252,65535,4294967295,1334,252,507,4294967295,1335,418,65535,4294967295,1336,419,65535,4294967295,1337,420,65535,4294967295,1338,421,65535,4294967295,764,139,65535,4294967295,1339,139,65535,4294967295,1340,40,65535,4294967295,1341,140,65535,4294967295,1342,190,65535,4294967295,1343,422,65535,4294967295,769,253,65535,4294967295,1344,253,65535,4294967295,1345,253,364,4294967295,1346,423,65535,4294967295,770,140,65535,4294967295,1347,424,65535,4294967295,771,254,65535,4294967295,1348,254,65535,4294967295,1349,254,1429,4294967295,1350,425,65535,4294967295,675,219,65535,4294967295,1351,426,65535,4294967295,1352,140,65535,4294967295,1353,140,364,4294967295,1354,140,65535,4294967295,1355,427,65535,4294967295,774,255,65535,4294967295,1356,255,65535,4294967295,1357,255,1452,4294967295,1358,428,65535,4294967295,1359,429,65535,4294967295,775,256,65535,4294967295,1360,256,65535,4294967295,1361,256,464,4294967295,1362,430,65535,4294967295,1363,73,65535,4294967295,695,228,65535,4294967295,1364,431,65535,4294967295,435,141,65535,4294967295,1365,141,65535,4294967295,1366,142,65535,4294967295,578,183,65535,4294967295,1367,142,65535,4294967295,1368,41,65535,4294967295,713,234,65535,4294967295,1369,432,65535,4294967295,438,143,65535,4294967295,1370,143,65535,4294967295,1371,433,65535,4294967295,787,257,65535,4294967295,1372,257,65535,4294967295,1373,434,65535,4294967295,1374,74,65535,4294967295,1375,258,65535,4294967295,1376,258,65535,4294967295,1375,258,65535,4294967295,1377,258,65535,4294967295,1378,258,1539,4294967295,1379,435,65535,4294967295,1380,436,65535,4294967295,1381,437,65535,4294967295,790,259,65535,4294967295,1382,259,65535,4294967295,1383,259,490,4294967295,1384,438,65535,4294967295,1385,75,65535,4294967295,744,248,65535,4294967295,1386,439,65535,4294967295,445,144,65535,4294967295,1387,144,65535,4294967295,1388,145,65535,4294967295,591,187,65535,4294967295,1389,145,65535,4294967295,1390,42,65535,4294967295,1391,440,65535,4294967295,799,260,65535,4294967295,1392,260,65535,4294967295,1393,441,65535,4294967295,1394,261,65535,4294967295,1394,261,65535,4294967295,1395,261,65535,4294967295,1396,261,1604,4294967295,1397,76,65535,4294967295,802,262,65535,4294967295,1398,262,65535,4294967295,1399,262,333,4294967295,1400,442,65535,4294967295,1401,263,65535,4294967295,1401,263,65535,4294967295,1402,263,65535,4294967295,1403,263,1707,4294967295,1404,77,65535,4294967295,806,264,65535,4294967295,1405,264,65535,4294967295,1406,264,1719,4294967295,821,266,65535,4294967295,1407,443,65535,4294967295,805,146,65535,4294967295,1408,146,65535,4294967295,1409,43,65535,4294967295,454,147,65535,4294967295,1410,444,65535,4294967295,454,147,65535,4294967295,1411,147,65535,4294967295,1412,445,65535,4294967295,1413,446,65535,4294967295,1414,447,65535,4294967295,812,148,65535,4294967295,1415,148,65535,4294967295,1416,44,65535,4294967295,454,147,65535,4294967295,1417,265,65535,4294967295,1418,448,65535,4294967295,1417,265,65535,4294967295,1419,265,65535,4294967295,1420,78,65535,4294967295,1421,449,65535,4294967295,1422,450,65535,4294967295,1423,451,65535,4294967295,461,150,65535,4294967295,1424,150,65535,4294967295,1425,452,65535,4294967295,821,266,65535,4294967295,1426,266,65535,4294967295,1427,266,992,4294967295,1428,267,992,4294967295,1429,453,65535,4294967295,1430,268,65535,4294967295,1431,268,992,4294967295,1432,79,65535,4294967295,113,27,65535,4294967295,114,28,65535,4294967295,1433,80,65535,4294967295,1434,223,65535,4294967295,1435,269,65535,4294967295,1436,269,333,4294967295,1437,454,65535,4294967295,1438,455,65535,4294967295,1439,456,65535,4294967295,1440,270,65535,4294967295,1441,270,65535,4294967295,276,80,65535,200,1442,271,65535,4294967295,1443,271,333,4294967295,1444,457,65535,4294967295,1445,458,65535,4294967295,1446,459,65535,4294967295,1447,152,65535,4294967295,1448,460,65535,4294967295,1449,272,65535,4294967295,1450,272,65535,4294967295,1451,272,142,4294967295,551,175,65535,201,1452,273,65535,4294967295,1453,273,333,4294967295,1454,461,65535,4294967295,840,274,65535,4294967295,1455,274,65535,4294967295,1456,274,142,4294967295,1457,462,65535,4294967295,557,153,65535,4294967295,1458,153,65535,4294967295,1459,45,65535,4294967295,1460,463,65535,4294967295,1461,275,65535,4294967295,1462,275,65535,4294967295,1463,275,142,4294967295,1464,464,65535,4294967295,1465,465,65535,202,1466,276,65535,4294967295,1467,466,65535,4294967295,1468,467,65535,4294967295,1469,276,142,4294967295,1470,468,65535,4294967295,1466,276,65535,203,1471,81,65535,4294967295,116,29,65535,4294967295,1472,298,65535,4294967295,1473,277,1429,4294967295,1474,82,65535,4294967295,1475,83,65535,4294967295,1476,469,65535,4294967295,1477,285,65535,4294967295,1478,154,65535,4294967295,1479,377,65535,4294967295,1480,278,65535,4294967295,1481,278,65535,4294967295,1482,279,65535,4294967295,1483,279,65535,4294967295,371,114,65535,204,1484,280,65535,4294967295,1485,280,65535,4294967295,1486,470,65535,4294967295,1487,280,65535,4294967295,1488,471,464,4294967295,1489,472,65535,4294967295,1490,284,65535,205,1491,473,65535,4294967295,1492,281,65535,4294967295,1493,281,333,4294967295,1494,474,65535,4294967295,1495,475,65535,4294967295,860,282,65535,4294967295,1496,282,65535,4294967295,1497,476,333,4294967295,1498,477,65535,4294967295,1499,478,65535,4294967295,883,283,65535,4294967295,1500,283,65535,4294967295,883,283,65535,206,1501,284,65535,4294967295,1502,479,65535,4294967295,1501,284,65535,207,1503,84,65535,4294967295,1504,278,65535,4294967295,627,206,65535,208,1505,476,65535,4294967295,1506,480,65535,4294967295,1507,47,65535,4294967295,1508,85,65535,4294967295,1509,481,65535,4294967295,1510,364,65535,4294967295,1511,86,65535,4294967295,1512,482,65535,4294967295,1513,285,65535,4294967295,1514,483,65535,4294967295,1513,285,65535,4294967295,1515,285,65535,4294967295,1513,285,65535,209,1516,286,65535,4294967295,1517,484,507,4294967295,1518,485,65535,4294967295,876,287,65535,4294967295,1519,287,65535,4294967295,1520,486,507,4294967295,1521,487,65535,4294967295,493,159,65535,4294967295,1522,488,65535,4294967295,493,159,65535,4294967295,1523,159,65535,4294967295,1524,489,65535,4294967295,879,288,65535,4294967295,1525,288,65535,4294967295,1526,288,142,4294967295,1527,490,65535,4294967295,1528,289,65535,4294967295,1529,491,65535,4294967295,1530,289,65535,4294967295,1531,492,484,4294967295,1532,493,65535,4294967295,1533,87,65535,210,1534,87,65535,4294967295,1535,290,65535,4294967295,1536,494,65535,4294967295,1537,473,65535,4294967295,1538,290,124,4294967295,1539,495,65535,4294967295,1535,290,65535,211,1540,493,65535,4294967295,1541,160,65535,4294967295,495,160,65535,4294967295,1542,160,65535,4294967295,495,160,65535,212,1543,291,65535,4294967295,1544,291,142,4294967295,119,30,65535,4294967295,900,295,65535,4294967295,1545,293,65535,4294967295,1546,220,65535,4294967295,1547,496,65535,4294967295,1548,293,65535,4294967295,1549,163,249,4294967295,1550,497,65535,4294967295,1551,8,65535,213,1552,94,65535,4294967295,1553,90,65535,4294967295,1554,90,65535,4294967295,1555,90,65535,4294967295,1556,295,65535,4294967295,1557,498,65535,4294967295,1558,294,65535,4294967295,1559,294,364,4294967295,1560,499,65535,4294967295,138,31,65535,214,1561,31,65535,4294967295,138,31,65535,4294967295,1562,295,65535,4294967295,1563,500,65535,4294967295,1564,295,65535,4294967295,507,163,65535,4294967295,1565,501,65535,4294967295,1566,295,65535,4294967295,1567,295,3260,4294967295,1568,502,65535,4294967295,1569,503,65535,4294967295,1570,296,1539,4294967295,1571,325,65535,4294967295,1572,504,65535,4294967295,1573,163,65535,4294967295,1574,163,364,4294967295,1575,163,65535,4294967295,1576,297,3287,4294967295,1577,505,65535,4294967295,1578,298,65535,4294967295,1579,506,65535,4294967295,1580,507,65535,4294967295,1581,298,65535,4294967295,1582,508,610,4294967295,1583,9,65535,215,1584,164,65535,4294967295,1585,164,65535,4294967295,1586,164,364,4294967295,1587,164,65535,4294967295,1588,508,65535,4294967295,1589,49,65535,4294967295,1590,49,65535,4294967295,1591,509,65535,4294967295,1592,165,65535,4294967295,1593,510,65535,4294967295,1594,165,65535,4294967295,1595,165,364,4294967295,1596,165,65535,4294967295,1597,511,65535,4294967295,1598,512,65535,4294967295,927,299,65535,4294967295,1599,513,65535,4294967295,1600,299,364,4294967295,1601,514,65535,4294967295,916,300,65535,4294967295,1602,207,65535,4294967295,1603,515,65535,4294967295,1604,300,65535,4294967295,1605,300,364,4294967295,1606,516,65535,4294967295,513,52,65535,4294967295,1607,91,65535,4294967295,1608,91,65535,4294967295,1609,517,65535,4294967295,1610,166,65535,4294967295,1611,166,364,4294967295,1612,166,65535,4294967295,1613,518,65535,4294967295,1614,301,65535,4294967295,1615,301,65535,4294967295,1616,301,629,4294967295,1617,519,65535,4294967295,385,65,65535,4294967295,1618,520,65535,4294967295,1619,167,364,4294967295,1620,167,65535,4294967295,1621,302,65535,4294967295,1622,302,3398,4294967295,1623,299,65535,4294967295,1624,303,65535,4294967295,1625,303,65535,4294967295,1626,521,65535,4294967295,1624,303,65535,4294967295,1627,303,65535,4294967295,1628,140,249,4294967295,1629,522,65535,4294967295,1630,523,65535,4294967295,929,304,65535,4294967295,1631,304,65535,4294967295,1632,524,610,4294967295,1633,525,65535,4294967295,522,168,65535,4294967295,1634,303,65535,4294967295,1635,168,364,4294967295,1636,168,65535,4294967295,1637,526,65535,4294967295,932,305,65535,4294967295,1638,305,65535,4294967295,1639,527,629,4294967295,1640,92,65535,4294967295,1641,92,65535,4294967295,1642,306,65535,4294967295,1643,306,1987,4294967295,1644,528,65535,4294967295,1645,169,364,4294967295,1646,169,65535,4294967295,1647,32,65535,4294967295,198,32,65535,4294967295,1648,170,65535,4294967295,1649,170,364,4294967295,1650,170,65535,4294967295,1651,529,65535,4294967295,1652,307,65535,4294967295,1653,307,629,4294967295,1654,530,65535,4294967295,1655,50,65535,4294967295,1656,50,65535,4294967295,1657,171,65535,4294967295,1658,171,364,4294967295,1659,171,65535,4294967295,1660,277,65535,4294967295,1661,531,65535,4294967295,1662,308,65535,4294967295,1663,532,1987,4294967295,1664,533,65535,4294967295,1665,9,65535,216,1666,93,65535,4294967295,1667,93,65535,4294967295,1668,309,3398,4294967295,1669,534,65535,4294967295,1670,535,65535,4294967295,1671,536,65535,4294967295,1672,310,65535,4294967295,1673,310,65535,4294967295,1674,310,364,4294967295,1675,310,65535,4294967295,1676,50,629,4294967295,1677,537,65535,4294967295,1678,538,65535,4294967295,1679,539,65535,4294967295,1680,311,65535,4294967295,1681,311,65535,4294967295,1682,311,364,4294967295,1683,540,65535,4294967295,1684,312,65535,4294967295,1685,541,65535,4294967295,1686,542,65535,4294967295,1687,543,65535,4294967295,1688,312,364,4294967295,1689,544,65535,4294967295,536,51,65535,4294967295,1690,94,65535,4294967295,1691,94,65535,4294967295,1692,537,65535,4294967295,1693,51,65535,4294967295,1694,51,65535,4294967295,889,163,65535,4294967295,137,17,65535,4294967295,1695,51,65535,4294967295,1696,538,65535,4294967295,1697,545,65535,4294967295,1698,173,65535,4294967295,1699,546,65535,4294967295,1700,174,65535,4294967295,1701,547,65535,4294967295,1702,548,65535,4294967295,986,313,65535,4294967295,1703,549,65535,4294967295,1704,313,65535,4294967295,1465,465,65535,4294967295,972,314,65535,4294967295,1705,550,65535,4294967295,1706,551,65535,4294967295,1707,314,65535,4294967295,972,314,65535,4294967295,1446,459,65535,4294967295,972,314,65535,217,1708,458,65535,4294967295,1709,175,65535,4294967295,1710,541,65535,4294967295,1711,315,65535,4294967295,1712,315,464,4294967295,1713,552,65535,4294967295,1714,553,65535,4294967295,976,177,65535,4294967295,1715,177,65535,4294967295,1716,554,65535,4294967295,1717,316,65535,4294967295,1718,316,484,4294967295,1719,317,65535,4294967295,1720,317,65535,4294967295,1721,317,65535,4294967295,1722,317,490,4294967295,1723,555,65535,4294967295,1724,556,65535,4294967295,555,178,65535,4294967295,1725,178,65535,4294967295,1726,318,65535,4294967295,1727,318,507,4294967295,1728,313,65535,4294967295,1729,319,65535,4294967295,1730,319,65535,4294967295,1731,557,65535,4294967295,1729,319,65535,4294967295,1732,319,65535,4294967295,1733,319,124,4294967295,1734,558,65535,4294967295,1735,559,65535,4294967295,988,320,65535,4294967295,1736,320,65535,4294967295,1737,320,126,4294967295,1738,560,65535,4294967295,556,179,65535,4294967295,1739,561,65535,4294967295,990,321,65535,4294967295,1740,321,65535,4294967295,1741,321,142,4294967295,1742,91,65535,4294967295,1743,97,65535,4294967295,1744,97,65535,4294967295,1745,97,65535,4294967295,1746,509,65535,4294967295,1747,322,65535,4294967295,1748,180,65535,4294967295,1749,562,65535,4294967295,1750,322,65535,4294967295,1751,322,65535,4294967295,1752,325,249,4294967295,1753,563,65535,4294967295,1754,55,65535,218,1755,564,65535,4294967295,1756,323,65535,4294967295,1757,565,1429,4294967295,1758,566,65535,4294967295,1759,180,65535,4294967295,1760,180,364,4294967295,1761,180,65535,4294967295,1762,324,65535,4294967295,1763,567,1452,4294967295,1764,568,65535,4294967295,1765,509,65535,4294967295,1766,52,65535,4294967295,1767,52,65535,4294967295,1768,98,65535,4294967295,1769,569,65535,4294967295,1770,570,65535,4294967295,1771,333,65535,4294967295,1772,325,65535,4294967295,1773,325,65535,4294967295,1774,325,249,4294967295,1775,571,65535,4294967295,1046,190,65535,219,1776,344,65535,4294967295,1777,98,65535,4294967295,1778,572,65535,4294967295,1779,326,65535,4294967295,1780,326,65535,4294967295,1781,573,65535,4294967295,1782,327,65535,4294967295,1783,327,65535,4294967295,1784,574,992,4294967295,1785,575,65535,4294967295,1786,55,65535,220,1787,576,65535,4294967295,1788,577,65535,4294967295,1011,328,65535,4294967295,1789,328,65535,4294967295,1790,437,464,4294967295,1791,182,65535,4294967295,1792,329,65535,4294967295,1793,183,65535,4294967295,1794,335,65535,4294967295,1795,329,65535,4294967295,1796,329,65535,4294967295,1797,329,124,4294967295,1798,578,65535,4294967295,146,20,65535,221,1799,53,65535,4294967295,1800,183,65535,4294967295,1801,330,65535,4294967295,1802,330,484,4294967295,1803,53,65535,4294967295,1804,579,65535,4294967295,1021,331,65535,4294967295,1805,331,65535,4294967295,1806,580,484,4294967295,1807,580,65535,4294967295,1808,184,65535,4294967295,1809,332,65535,4294967295,1810,433,853,4294967295,1811,581,65535,4294967295,1141,370,65535,4294967295,1812,257,65535,4294967295,1813,582,65535,4294967295,1814,333,65535,4294967295,1815,583,1539,4294967295,1816,584,65535,4294967295,1817,585,65535,4294967295,1028,334,65535,4294967295,1818,334,65535,4294967295,1819,428,490,4294967295,1820,586,65535,4294967295,1821,428,65535,4294967295,1822,186,65535,4294967295,1823,323,65535,4294967295,1824,350,65535,4294967295,1825,102,65535,4294967295,1826,187,65535,4294967295,1827,335,65535,4294967295,1828,335,490,4294967295,1829,336,65535,4294967295,1830,336,65535,4294967295,1831,336,126,4294967295,103,21,65535,222,1832,54,65535,4294967295,1833,187,65535,4294967295,1834,54,65535,4294967295,1041,337,65535,4294967295,1835,337,65535,4294967295,1836,442,507,4294967295,1837,188,65535,4294967295,668,216,65535,4294967295,1838,260,65535,4294967295,1839,253,65535,4294967295,1840,338,65535,4294967295,1841,338,65535,4294967295,1842,338,364,4294967295,1843,190,65535,4294967295,1844,587,65535,4294967295,1845,190,65535,4294967295,1846,190,364,4294967295,1847,190,65535,4294967295,1848,191,65535,4294967295,569,98,65535,4294967295,1849,489,65535,4294967295,1850,191,65535,4294967295,1851,588,65535,4294967295,1852,55,65535,4294967295,1853,192,65535,4294967295,338,102,65535,4294967295,1854,589,65535,4294967295,1855,192,65535,4294967295,1856,56,65535,4294967295,1857,339,65535,4294967295,1858,590,65535,4294967295,1859,591,65535,4294967295,1860,339,65535
_(report truncated; full output in workflow logs)_ |
tinovyatkin
added a commit
that referenced
this pull request
Jul 26, 2026
"Create a PR" here carries repo-specific conventions that were tribal knowledge: branch from origin/main, use gh with --body-file to avoid a shell-escaping fight, never open a draft (the reviewers only engage on ready PRs), and keep the description current as scope moves. Merges are squash-only with squash_merge_commit_message=COMMIT_MESSAGES, so the commit message -- not the PR description -- is what lands in permanent history. Verified against #215's squashed commit. Document all three AI reviewers and how they differ, because "one of them is happy" is not the finish line: CodeRabbit's out-of-diff comments are the ones worth the most and the easiest to skip; Codex is per-push and signals via eyes/+1 reactions, so re-triggering while eyes are present just wastes a pass; and Claude Code Review posts no status at all, so a green workflow is not approval -- the verdict is in the comment text. State the exit condition explicitly: loop until Codex reacts +1 and Claude Code Review reports no blockers.
tinovyatkin
added a commit
that referenced
this pull request
Jul 26, 2026
* ci: gate Claude review on CI success and cancel on quota limits Run the review only after the CI workflow has gone green for the same commit, and stop a usage-limit hit from red-flagging a healthy PR. Trigger is now `workflow_run` on CI, gated to same-repo pull_request runs that concluded success and were not triggered by a bot. CI also runs on push to main, so the event filter keeps this PR-only. The prompt states that clippy, the test suite and fixture validation already passed, so the reviewer spends its budget on logic, ANTLR parity and coverage instead of re-verifying mechanics. This required moving from claude-code-action to claude-code-base-action. The wrapper derives PR context from the webhook payload and classifies `workflow_run` as an automation event with no PR entity: `track_progress` throws outright, `use_sticky_comment` is gated behind `isPullRequestEvent` and silently no-ops, and no tracking comment is wired up, which removes the `update_claude_comment` tool. Overriding the event shape is not possible either -- GITHUB_EVENT_NAME is a reserved default and the assignment is ignored. The base action has no event-shape validation, so the sticky progress comment is reimplemented with `gh api`: a hidden first-line marker keys lookup, matching the convention the other bots on this repo already use, and the review is delivered by PATCHing that same comment so re-runs update one comment instead of piling up new ones. The marker match requires first-line position and a Bot author. Matching it anywhere would let a human who merely quotes the marker have their comment overwritten. The empty-string default sits after the line index because an empty or null comment body splits to `[]`, making `[0]` null, which makes `startswith` raise and abort the step. On failure the execution log is parsed for the result message's `api_error_status`. A 429 cancels the run rather than failing it: the code under review is fine, and a cancelled run reads as "did not happen" instead of leaving a red check and a failed required-check to re-run. Any other error still fails loudly, including an unparseable log. Fork PRs are skipped. `workflow_run` grants the base repository's secrets, so a fork head would be untrusted code running with those secrets. Gating to same-repo branches means the author already has write access, which is what keeps this to a single ordinary checkout of the PR head. base-action is pinned to a commit rather than a tag: the newest tag predates the claude_args, plugins and plugin_marketplaces inputs, and GitHub ignores unknown inputs silently, so a tag pin would drop the model flags and the review plugin with no error. * docs: record the PR workflow and the three AI reviewers "Create a PR" here carries repo-specific conventions that were tribal knowledge: branch from origin/main, use gh with --body-file to avoid a shell-escaping fight, never open a draft (the reviewers only engage on ready PRs), and keep the description current as scope moves. Merges are squash-only with squash_merge_commit_message=COMMIT_MESSAGES, so the commit message -- not the PR description -- is what lands in permanent history. Verified against #215's squashed commit. Document all three AI reviewers and how they differ, because "one of them is happy" is not the finish line: CodeRabbit's out-of-diff comments are the ones worth the most and the easiest to skip; Codex is per-push and signals via eyes/+1 reactions, so re-triggering while eyes are present just wastes a pass; and Claude Code Review posts no status at all, so a green workflow is not approval -- the verdict is in the comment text. State the exit condition explicitly: loop until Codex reacts +1 and Claude Code Review reports no blockers. * ci: harden review gating against stale and overlapping CI runs Addresses four findings from the Codex review of #216. Restore PR-scoped concurrency, which this branch had dropped when it replaced the pull_request trigger. Without it two overlapping successful CI completions for the same PR both resolve the same PR, reuse the same sticky comment, and PATCH it -- so the run that finishes last wins, which is not necessarily the newest commit. Keyed on head_branch rather than PR number because workflow_run carries no PR object and the group has to resolve before any step runs; one branch maps to at most one open PR here. Skip runs whose SHA is no longer the PR head. workflow_run completions can arrive out of commit order, and the payload supplies a PR number with no freshness guarantee, so an older green run could review an obsolete commit and overwrite a newer verdict. Concurrency does not cover this on its own because the two runs need not overlap. Drop the `|| true` on the fallback PR lookup. The endpoint already exits 0 with an empty array when no PR matches (verified), so a nonzero exit means a real failure -- auth, rate limit, transient 5xx. Swallowing it reported found=false and let the job go green as though the PR did not exist. Add an always() finalizer so the comment never stays stuck on "Claude Code is reviewing...". An auth error, API 5xx, malformed result, crashed runner or cancelled run all previously left the placeholder in place, which misrepresents a finished run. Detection uses a second sentinel marker carried only by the placeholder rather than matching its prose; the sentinel sits on line 2 so the first-line sticky lookup is unaffected. The step is continue-on-error because a cosmetic comment update must not fail an otherwise good job. * ci: validate PR state at resolve and re-check the head at publish Addresses the second Codex round on #216. Fold state and draft checks into the PR lookup that already ran for head.sha, so one API call answers three questions. The payload's pull_requests[] fast path carries only a number and skips the fallback's state filter, and head.sha is unchanged by closing or merging -- so checking the SHA alone would review a PR that ended while CI ran. Drafts need checking here too: CI triggers on a bare `pull_request:`, which includes them, and the old trigger's `types: [..., ready_for_review]` filter has no workflow_run equivalent, so a work-in-progress PR would burn a full review and quota. Re-check the head after the review and retract the verdict when it moved. The resolve-time check leaves a time-of-check/time-of-use window across dependency setup and a 10-20 minute review. Concurrency does not close it: it can only cancel this run once the newer commit's *successful* CI starts another review, so if that CI fails or is cancelled the stale verdict stands as the latest word on an unreviewed commit. The retraction points at the job run rather than showing findings that may no longer apply; the full text stays in the uploaded execution log. It keeps the sticky marker on line 1 and drops the in-progress sentinel, so the finalizer leaves it alone and the next run still reuses the comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #212.
The bug
A
.g4file saved with a UTF-8 byte order mark did not compile:The cause is in our pinned meta-grammar, not the runtime. antlr-ng collapsed Java ANTLR's two trailing
NameStartCharranges —'ﷰ'..''and''..'�'— into a single'ﷰ'..'�', which covers U+FEFF. That made the byte order mark a legal identifier-start character, so the lexer emitted oneIDtoken spanning the mark and the following keyword.Worth noting for reviewers: a
UnicodeBOM -> skiprule on its own would not have fixed this.IDstill matches longer, and the lexer takes the longest match.The fix
In
third_party/antlr-v4-grammar/ANTLRv4Lexer.g4:NameStartCharto exclude U+FEFF, matching Java'sANTLRLexer.gexactly.to theWSchar set so the mark lands off the default channel.One deliberate divergence from Java
Java spends a dedicated
UnicodeBOM : '' -> skip;rule on this. I implemented that first, then backed it out: a new lexer rule allocates a token type, renumbering everything after it (END_ARGUMENT75→76, …). That broketests/codegen-direct/frontend-snapshots.tsvfor the one corpus grammar using Argument-mode tokens (vscode-antlr4TParser.g4) — token count matched, hash did not. Those oracles are recorded against antlr-ng, which has noUnicodeBOMrule, so they cannot be faithfully regenerated.Folding into
WSis observationally identical and keeps max token type at 78.The mark is skipped, not stripped
This distinction is the reason the fix does not touch the read path. Stripping the 3 BOM bytes from the buffer would shift every
SourceSpanby −3 and desynchronize our diagnostics from on-disk offsets. Leaving the mark in place preserves ANTLR's columns.Verified against the 4.13.2 jar:
1:18/1:171:18/1:172:22:2STRING_LITERAL/LEXER_CHAR_SET.tokensvocabularies: a second, separate bugWhile validating line-ending handling I found
.tokenssidecars were hard-failing on a BOM:They are generated sidecars parsed by
text.lines(), so they never reach the grammar lexer — and U+FEFF is notchar::is_whitespacein Rust, so.trim()left the mark glued to the first token name. Fixed with astrip_prefixinsemantics.rs.Note this is better than upstream, not parity with it: Java's regex accepts
"ID"as the token name and silently imports a wrong vocabulary behind a misleading "implicit definition" warning. I did not think that failure mode was worth reproducing, but it is a one-line revert if you would rather match it bug-for-bug.Windows line endings
Audited as part of this work, since it is the same class of problem. CRLF was already correct — diagnostics land on identical line:column to the LF spelling, lone-CR (classic Mac) parses, and a raw CR inside a string literal is rejected at the positions upstream reports. All four spellings of a grammar (plain / BOM / CRLF / BOM+CRLF) now produce byte-identical generated code. The only genuine gap was the
.tokenscase above.Regenerated frontend — please read before reviewing the diff
The bulk of the diff (~1000 lines in
antlr_v4_parser.rs) is pre-existing drift, not this fix. The checked-in artifacts carried av0.15.2header against crate0.19.0and predated #204'sadd_parse_listenerfacade, soupdate-stage0.sh --checkwas already red onmainbefore this change. Regenerating repairs that too.Confirmation the churn is unrelated:
grep -icE 'feff|bom|namestart'over the parser diff returns0. The lexer's serialized ATN shows exactly the intended split —65008,65278(FDF0–FEFE) and65280,65533(FF00–FFFD), with 65279 excluded.Two things reviewers should weigh: this modifies a self-hosting bootstrap artifact, and
update-stage0.shis not wired into any CI workflow, which is how the drift went unnoticed. Adding it to CI feels worth a follow-up issue rather than scope creep here.Verification
cargo test --locked --features codegen— 1084 passed, 0 failedcargo clippy --locked --all-targets --all-features -- -D warnings— cleancargo run --release --bin antlr4-runtime-testsuite— 357 passed, 0 failed, 0 skippedtools/grammar-frontend/update-stage0.sh --check— Stage 1 ≡ Stage 2, all 6 hashes OK (was red before)New regression tests: BOM spans/columns, BOM-in-literals, and CRLF/lone-CR line endings in
frontend.rs; plus two CLI tests covering the four grammar spellings and the four vocab-file spellings.CHANGELOG.mdleft alone since release-please owns it.Summary by CodeRabbit
New Features
Documentation