feat(codegen)!: centralize recognizer facades - #300
Merged
Conversation
Move grammar-independent generated lexer and parser facade methods and trait delegation behind doc-hidden runtime macros. Generated modules now declare their concrete recognizer types, storage fields, metadata/ATN providers, semantic token dispatch, and optional reset state while the runtime owns the stable forwarding implementation. Keep generated imports explicit, qualify macro-owned paths through $crate or the standard-library roots, and emit concrete implementations rather than blanket trait impls. Preserve constructors, typed hooks, listener management, stream/tree accessors, DFA controls, semantic dispatch, and parser configuration behavior. Bump the generated-code API to revision 5 because newly generated recognizers require the facade macros. Keep revisions 1 through 4 accepted while their runtime surfaces remain available, and regenerate checked-in recognizers, compatibility snapshots, and documentation. Representative generated source drops by 3,376 bytes for a lexer-only grammar, 8,237 bytes for a parser-only grammar, and 11,604 bytes for the combined Rust grammar. The release generator binary is 16,512 bytes smaller; warm package-check samples are reported separately without claiming a compile-time improvement from source compaction.
|
Warning Review limit reached
Next review available in: 49 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe runtime now owns shared lexer and parser facade implementations. Generated recognizers invoke runtime macros for trait delegation and lifecycle APIs. The generated-code API revision advances to 5, with updated compatibility checks, snapshots, documentation, and checked-in grammar hashes. ChangesRecognizer facade migration
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Codegen
participant GeneratedLexerParser
participant RuntimeFacades
participant RuntimeTraits
Codegen->>GeneratedLexerParser: render lexer and parser facade invocations
GeneratedLexerParser->>RuntimeFacades: invoke lexer/parser facade macros
RuntimeFacades->>RuntimeTraits: provide delegated recognizer behavior
RuntimeTraits-->>GeneratedLexerParser: expose lifecycle, accessors, and parsing/tokenization APIs
Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Copy/Paste DetectionFound 9 duplication(s) across 10 changed non-generated Rust file(s) (threshold: 100 tokens). Show duplicationsFound a 26 line (142 tokens) duplication in the following files:
atn.set_end_state(1, 4).expect("block end state");
atn.add_transition(0, ParserTransitionSpec::Epsilon { target: 1 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 2 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 3 })
.expect("transition");
atn.add_transition(
2,
ParserTransitionSpec::Atom {
target: 4,
label: 1,
},
)
.expect("transition");
atn.add_transition(
3,
ParserTransitionSpec::Atom {
target: 4,
label: 2,
},
)
.expect("transition");
atn.add_transition(4, ParserTransitionSpec::Epsilon { target: 5 })
.expect("transition");
atn.add_decision_state(1).expect("decision state");
```rust
---
Found a 26 line (125 tokens) duplication in the following files:
* Starting at line 340 of crates/antlr-rust-runtime/src/generated.rs
* Starting at line 511 of crates/antlr-rust-runtime/src/generated.rs
```rust
$input: $crate::char_stream::CharStream,
$hooks: $crate::parser::SemanticHooks,
{
pub fn metadata() -> &'static $crate::generated::GrammarMetadata {
$metadata()
}
/// Adds a listener for lexer diagnostics.
pub fn add_error_listener<T>(&mut self, listener: T)
where
T: for<'a> $crate::errors::ErrorListener<dyn $crate::recognizer::Recognizer + 'a>
+ ::core::marker::Send
+ 'static,
{
$crate::recognizer::Recognizer::add_error_listener(&mut self.$base, listener);
}
/// Removes every lexer error listener, including the default console listener.
pub fn remove_error_listeners(&mut self) {
$crate::recognizer::Recognizer::remove_error_listeners(&mut self.$base);
}
/// Routes every token through ATN interpretation instead of the compiled
/// lexer DFA, so the learned-DFA trace (`lexer_dfa_string`) observes each
/// match.
pub fn set_force_interpreted(&mut self, force_interpreted: bool) {Found a 25 line (115 tokens) duplication in the following files:
atn.add_state(AtnStateKind::BlockStart, Some(0))
.expect("state")
.index(),
1
);
assert_eq!(
atn.add_state(AtnStateKind::Basic, Some(0))
.expect("state")
.index(),
2
);
assert_eq!(
atn.add_state(AtnStateKind::Basic, Some(0))
.expect("state")
.index(),
3
);
assert_eq!(
atn.add_state(AtnStateKind::BlockEnd, Some(0))
.expect("state")
.index(),
4
);
assert_eq!(
atn.add_state(AtnStateKind::RuleStop, Some(0))
```rust
---
Found a 22 line (112 tokens) duplication in the following files:
* Starting at line 4147 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 4221 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
fn plus_loop_atn() -> ParserAtn {
let mut atn = ParserAtnBuilder::new(2);
assert_eq!(
atn.add_state(AtnStateKind::RuleStart, Some(0))
.expect("state")
.index(),
0
);
assert_eq!(
atn.add_state(AtnStateKind::PlusBlockStart, Some(0))
.expect("state")
.index(),
1
);
assert_eq!(
atn.add_state(AtnStateKind::Basic, Some(0))
.expect("state")
.index(),
2
);
assert_eq!(
atn.add_state(AtnStateKind::BlockEnd, Some(0))Found a 27 line (110 tokens) duplication in the following files:
decision: 0,
alts: (1, 2),
track_alt_number: false,
allow_semantic_context: true,
force_context: false,
plus_loop: false,
fast_path: None,
body: &body,
},
0,
GeneratedStepRenderContext {
current_rule_index: 0,
embedded: None,
portable_locals: None,
decision_routing: DecisionRoutingRender::default(),
inline_action_statements: &BTreeMap::new(),
track_alt_numbers: false,
track_context_alt_numbers: false,
direct_generated_rule_calls: &[],
atn_preferred_rule_calls: &[],
adaptive_atn_preferred_rule_slots: &[],
adaptive_atn_probe_rule_slots: &[],
},
);
// The whole rendered star-loop captures the leading-predicate-to-exit-alt filtering.
insta::assert_snapshot!(
```rust
---
Found a 17 line (105 tokens) duplication in the following files:
* Starting at line 168 of crates/antlr-rust-runtime/src/generated.rs
* Starting at line 265 of crates/antlr-rust-runtime/src/generated.rs
```rust
fn __from_node_with_invocation_states(
node: $crate::RuleNodeView<'a>,
invocation_states: Option<Vec<isize>>,
) -> Self {
$(
let __default = <$attrs>::default();
let __attrs = node.generated_attrs::<$attrs>().unwrap_or(&__default);
)?
Self {
__node: __GeneratedRuleContext::Stored(node),
__invocation_states: invocation_states,
__state: std::marker::PhantomData,
$(
$($field: __attrs.$field.clone(),)+
)?
}
}Found a 25 line (104 tokens) duplication in the following files:
state: 1,
decision: 0,
track_alt_number: false,
allow_semantic_context: false,
force_context: false,
fast_path: None,
alts: &alts,
},
0,
GeneratedStepRenderContext {
current_rule_index: 0,
embedded: None,
portable_locals: None,
decision_routing: DecisionRoutingRender::default(),
inline_action_statements: &BTreeMap::new(),
track_alt_numbers: false,
track_context_alt_numbers: false,
direct_generated_rule_calls: &[],
atn_preferred_rule_calls: &[],
adaptive_atn_preferred_rule_slots: &[],
adaptive_atn_probe_rule_slots: &[],
},
);
assert!(rendered.contains("ll1_decision_prediction(atn(), 1)"));
```rust
---
Found a 28 line (102 tokens) duplication in the following files:
* Starting at line 3167 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 3327 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
state: 1,
decision: 0,
track_alt_number: false,
allow_semantic_context: true,
force_context: false,
fast_path: None,
alts: &alts,
},
0,
GeneratedStepRenderContext {
current_rule_index: 0,
embedded: None,
portable_locals: None,
decision_routing: DecisionRoutingRender::default(),
inline_action_statements: &BTreeMap::new(),
track_alt_numbers: false,
track_context_alt_numbers: false,
direct_generated_rule_calls: &[],
atn_preferred_rule_calls: &[],
adaptive_atn_preferred_rule_slots: &[],
adaptive_atn_probe_rule_slots: &[],
},
);
// One decision renders into a fresh String; snapshot the whole emitted control flow (the
// semantic-context gate, both predicate probes, the alt rewrite, the no-viable fallback)
// instead of six positive probes plus one negative guard.
insta::assert_snapshot!(Found a 16 line (101 tokens) duplication in the following files:
atn.set_loop_back_state(3, 4).expect("loop back state");
atn.add_transition(0, ParserTransitionSpec::Epsilon { target: 1 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 2 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 3 })
.expect("transition");
atn.add_transition(
2,
ParserTransitionSpec::Atom {
target: 4,
label: 1,
},
)
.expect("transition");
atn.add_transition(4, ParserTransitionSpec::Epsilon { target: 1 })
```rust |
|
Claude Code is working… I'll analyze this and get back to you. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/antlr-rust-runtime/src/generated.rs`:
- Around line 908-967: Update the facade_hygiene fixture to shadow Rc alongside
the other prelude names, covering the lexer facade’s TokenSource::source_text
expansion. Add a comment explaining that successful compilation of both facade
macro invocations is the hygiene assertion; retain the test only if needed to
exercise the fixture, without implying its size assertion is the purpose of the
module.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6c25d7a6-0f7f-436e-a887-4da592fd978b
⛔ Files ignored due to path filters (13)
crates/antlr-rust-codegen/src/generator/snapshots/antlr_rust_codegen__generator__tests__generated_lexer_lifecycle_facade.snapis excluded by!**/*.snapcrates/antlr-rust-codegen/src/generator/snapshots/antlr_rust_codegen__generator__tests__generated_module_file_header.snapis excluded by!**/*.snapcrates/antlr-rust-codegen/src/generator/snapshots/antlr_rust_codegen__generator__tests__generated_parser_optional_state_facade.snapis excluded by!**/*.snapcrates/antlr-rust-codegen/src/generator/snapshots/antlr_rust_codegen__generator__tests__generated_plain_recognizer_facades.snapis excluded by!**/*.snapcrates/antlr-rust-codegen/src/generator/snapshots/antlr_rust_codegen__generator__tests__generated_recognizers_reuse_cached_static_metadata.snapis excluded by!**/*.snapcrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/snapshots/antlr4_rust_gen_cli__cli__generated_codegen_api_checks.snapis excluded by!**/*.snapcrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/snapshots/antlr4_rust_gen_cli__cli__generated_codegen_api_mismatch_diagnostic.snapis excluded by!**/*.snapcrates/antlr-rust-g4-parser/src/generated/antlr_v4_lexer.rsis excluded by!**/generated/**crates/antlr-rust-g4-parser/src/generated/antlr_v4_parser.rsis excluded by!**/generated/**crates/antlr-rust-rs-parser/src/generated/rust_lexer.rsis excluded by!**/generated/**crates/antlr-rust-rs-parser/src/generated/rust_parser.rsis excluded by!**/generated/**crates/antlr-rust-runtime/src/xpath/generated/x_path_lexer.rsis excluded by!**/generated/**docs/migration.mdis excluded by!**/docs/**
📒 Files selected for processing (12)
README.mdcrates/antlr-rust-codegen/src/generator/tests.rscrates/antlr-rust-codegen/src/lexer/render.rscrates/antlr-rust-codegen/src/parser/render/mod.rscrates/antlr-rust-codegen/src/parser/render_model.rscrates/antlr-rust-codegen/src/parser/routing.rscrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/cli.rscrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/support.rscrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/typed_tree.rscrates/antlr-rust-runtime/src/generated.rscrates/antlr-rust-runtime/src/lib.rsthird_party/antlr-v4-grammar/self-hosted.sha256
💤 Files with no reviewable changes (1)
- crates/antlr-rust-codegen/src/parser/render_model.rs
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
📊 Source Code Metrics (this PR vs
|
| File | Cyclomatic | Cognitive | Functions | LLOC | MI |
|---|---|---|---|---|---|
| crates/antlr-rust-codegen/src/generator/tests.rs | 284 (main: 280) 🔴 | 48 ⚪ | 199 (main: 197) 🔴 | 1344 (main: 1362) 🟢 | 0 ⚪ |
| crates/antlr-rust-codegen/src/parser/render_model.rs | 90 (main: 92) 🟢 | 61 ⚪ | 20 (main: 22) 🟢 | 130 (main: 132) 🟢 | 0 ⚪ |
| crates/antlr-rust-runtime/src/generated.rs | 20 (main: 18) 🔴 | 0 ⚪ | 14 (main: 12) 🔴 | 21 (main: 19) 🔴 | 5.44 (main: 13.31) 🔴 |
| crates/antlr-rust-codegen/src/parser/render/mod.rs | 43 ⚪ | 41 ⚪ | 3 ⚪ | 91 (main: 93) 🟢 | 7.02 (main: 4.56) 🟢 |
| crates/antlr-rust-runtime/src/lib.rs | 4 ⚪ | 3 ⚪ | 1 ⚪ | 7 ⚪ | 29.85 (main: 29.93) 🔴 |
Generated by mehen v1.8.0 — the code quality watcher.
Contributor
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #278.
Summary
doc-hidden runtime macros while keeping grammar-specific metadata, semantic
dispatch, typed hooks, and optional parser state in generated modules.
imports or blanket implementations; qualify macro-owned paths so invocation
site names cannot shadow support code.
through 4, and update checked-in recognizers, compatibility tests, snapshots,
hashes, and migration documentation.
Measured Impact
Warm
cargo check --locked --quiet -p antlr-rust-g4-parsersamples were0.11/0.21/0.17/0.15sbefore and0.10/0.08/0.07/0.08safter. These shortsamples are reported separately as requested; this PR does not claim a
compile-time improvement from source compaction.
Validation
cargo test --locked --workspace --all-featurescargo clippy --locked --workspace --all-targets --all-features -- -D warningscargo fmt --all -- --checkgit diff --checkSummary by CodeRabbit
New Features
Compatibility
Documentation
Tests