feat(codegen)!: centralize typed context mechanics - #299
Merged
Conversation
Move generated typed parser context storage, constructors, conversions, common accessors, and invocation-state display behind a doc-hidden runtime macro. Generated contexts now declare only their rule identity, optional attributes, context kind, and collision-safe method names. Do not emit empty per-rule attribute payloads or attribute lookups for attrless rules. Preserve the existing stored, active, recovered, validated, listener, visitor, and downcast surfaces through the shared implementation. Bump the generated-code API to revision 4 because new generated parsers require the runtime support macro. Keep revisions 1 through 3 accepted while their runtime surfaces remain available, and regenerate all checked-in recognizers, compatibility snapshots, and documentation. For the checked-in ANTLRv4 frontend, generated source falls from 26,446 to 14,118 lines and from 3,966,951 to 3,511,229 bytes. Seven warm package checks remain effectively unchanged at 1.561s baseline versus 1.571s current; the release generator binary is 14,592 bytes smaller.
Copy/Paste DetectionFound 14 duplication(s) across 12 changed non-generated Rust file(s) (threshold: 100 tokens). Show duplicationsFound a 26 line (142 tokens) duplication in the following files:
atn.set_end_state(1, 4).expect("block end state");
atn.add_transition(0, ParserTransitionSpec::Epsilon { target: 1 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 2 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 3 })
.expect("transition");
atn.add_transition(
2,
ParserTransitionSpec::Atom {
target: 4,
label: 1,
},
)
.expect("transition");
atn.add_transition(
3,
ParserTransitionSpec::Atom {
target: 4,
label: 2,
},
)
.expect("transition");
atn.add_transition(4, ParserTransitionSpec::Epsilon { target: 5 })
.expect("transition");
atn.add_decision_state(1).expect("decision state");
```rust
---
Found a 23 line (139 tokens) duplication in the following files:
* Starting at line 1021 of crates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/compatibility.rs
* Starting at line 1080 of crates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/compatibility.rs
```rust
ID: [a-z]+;\n",
)
.expect("delegate grammar should be writable");
let output = run_antlr4_rust_gen(&[
root.as_os_str(),
OsStr::new("-I"),
temp.path().as_os_str(),
OsStr::new("--actions"),
OsStr::new("embedded"),
OsStr::new("--sem-unknown"),
OsStr::new("error"),
OsStr::new("--require-full-semantics"),
OsStr::new("--out-dir"),
out.as_os_str(),
]);
assert!(
output.status.success(),
"stdout: {}\nstderr: {}",
utf8(&output.stdout),
utf8(&output.stderr)
);
let parser = fs::read_to_string(out.join("root_parser.rs")).expect("parser should be emitted");Found a 22 line (132 tokens) duplication in the following files:
"parser grammar Delegate;\ndelegated: {isTypeName()}? ID;\n",
)
.expect("delegate grammar should be writable");
fs::write(&tokens, "lexer grammar Tokens;\nID: [a-z]+;\n")
.expect("token grammar should be writable");
let output = run_antlr4_rust_gen(&[
root.as_os_str(),
tokens.as_os_str(),
OsStr::new("-I"),
temp.path().as_os_str(),
OsStr::new("--out-dir"),
out.as_os_str(),
]);
assert!(
output.status.success(),
"stdout: {}\nstderr: {}",
utf8(&output.stdout),
utf8(&output.stderr)
);
let parser = fs::read_to_string(out.join("root.rs")).expect("parser should be emitted");
assert!(parser.contains("pub trait RootHooks"), "{parser}");
```rust
---
Found a 25 line (115 tokens) duplication in the following files:
* Starting at line 3999 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 4209 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
atn.add_state(AtnStateKind::BlockStart, Some(0))
.expect("state")
.index(),
1
);
assert_eq!(
atn.add_state(AtnStateKind::Basic, Some(0))
.expect("state")
.index(),
2
);
assert_eq!(
atn.add_state(AtnStateKind::Basic, Some(0))
.expect("state")
.index(),
3
);
assert_eq!(
atn.add_state(AtnStateKind::BlockEnd, Some(0))
.expect("state")
.index(),
4
);
assert_eq!(
atn.add_state(AtnStateKind::RuleStop, Some(0))Found a 22 line (112 tokens) duplication in the following files:
fn plus_loop_atn() -> ParserAtn {
let mut atn = ParserAtnBuilder::new(2);
assert_eq!(
atn.add_state(AtnStateKind::RuleStart, Some(0))
.expect("state")
.index(),
0
);
assert_eq!(
atn.add_state(AtnStateKind::PlusBlockStart, Some(0))
.expect("state")
.index(),
1
);
assert_eq!(
atn.add_state(AtnStateKind::Basic, Some(0))
.expect("state")
.index(),
2
);
assert_eq!(
atn.add_state(AtnStateKind::BlockEnd, Some(0))
```rust
---
Found a 27 line (110 tokens) duplication in the following files:
* Starting at line 3353 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 3465 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
decision: 0,
alts: (1, 2),
track_alt_number: false,
allow_semantic_context: true,
force_context: false,
plus_loop: false,
fast_path: None,
body: &body,
},
0,
GeneratedStepRenderContext {
current_rule_index: 0,
embedded: None,
portable_locals: None,
decision_routing: DecisionRoutingRender::default(),
inline_action_statements: &BTreeMap::new(),
track_alt_numbers: false,
track_context_alt_numbers: false,
direct_generated_rule_calls: &[],
atn_preferred_rule_calls: &[],
adaptive_atn_preferred_rule_slots: &[],
adaptive_atn_probe_rule_slots: &[],
},
);
// The whole rendered star-loop captures the leading-predicate-to-exit-alt filtering.
insta::assert_snapshot!(Found a 20 line (107 tokens) duplication in the following files:
.expect("type-position grammar should be writable");
let output = run_antlr4_rust_gen(&[
grammar.as_os_str(),
OsStr::new("--actions"),
OsStr::new("embedded"),
OsStr::new("--sem-unknown"),
OsStr::new("error"),
OsStr::new("--require-full-semantics"),
OsStr::new("--out-dir"),
out.as_os_str(),
]);
assert!(
output.status.success(),
"stdout: {}\nstderr: {}",
utf8(&output.stdout),
utf8(&output.stderr)
);
let parser =
fs::read_to_string(out.join("type_position_parser.rs")).expect("parser should be emitted");
```rust
---
Found a 18 line (106 tokens) duplication in the following files:
* Starting at line 243 of crates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/compatibility.rs
* Starting at line 13 of crates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/semantics.rs
```rust
fixtures.join("AliasCollision.g4").as_os_str(),
OsStr::new("--actions"),
OsStr::new("embedded"),
OsStr::new("--sem-unknown"),
OsStr::new("error"),
OsStr::new("--require-full-semantics"),
OsStr::new("--out-dir"),
out.as_os_str(),
]);
assert!(
output.status.success(),
"stdout: {}\nstderr: {}",
utf8(&output.stdout),
utf8(&output.stderr)
);
let manifest =
fs::read_to_string(out.join("semantics.json")).expect("manifest should be emitted");Found a 17 line (105 tokens) duplication in the following files:
fn __from_node_with_invocation_states(
node: $crate::RuleNodeView<'a>,
invocation_states: Option<Vec<isize>>,
) -> Self {
$(
let __default = <$attrs>::default();
let __attrs = node.generated_attrs::<$attrs>().unwrap_or(&__default);
)?
Self {
__node: __GeneratedRuleContext::Stored(node),
__invocation_states: invocation_states,
__state: std::marker::PhantomData,
$(
$($field: __attrs.$field.clone(),)+
)?
}
}
```rust
---
Found a 25 line (104 tokens) duplication in the following files:
* Starting at line 3093 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 3260 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
state: 1,
decision: 0,
track_alt_number: false,
allow_semantic_context: false,
force_context: false,
fast_path: None,
alts: &alts,
},
0,
GeneratedStepRenderContext {
current_rule_index: 0,
embedded: None,
portable_locals: None,
decision_routing: DecisionRoutingRender::default(),
inline_action_statements: &BTreeMap::new(),
track_alt_numbers: false,
track_context_alt_numbers: false,
direct_generated_rule_calls: &[],
atn_preferred_rule_calls: &[],
adaptive_atn_preferred_rule_slots: &[],
adaptive_atn_probe_rule_slots: &[],
},
);
assert!(rendered.contains("ll1_decision_prediction(atn(), 1)"));Found a 28 line (102 tokens) duplication in the following files:
state: 1,
decision: 0,
track_alt_number: false,
allow_semantic_context: true,
force_context: false,
fast_path: None,
alts: &alts,
},
0,
GeneratedStepRenderContext {
current_rule_index: 0,
embedded: None,
portable_locals: None,
decision_routing: DecisionRoutingRender::default(),
inline_action_statements: &BTreeMap::new(),
track_alt_numbers: false,
track_context_alt_numbers: false,
direct_generated_rule_calls: &[],
atn_preferred_rule_calls: &[],
adaptive_atn_preferred_rule_slots: &[],
adaptive_atn_probe_rule_slots: &[],
},
);
// One decision renders into a fresh String; snapshot the whole emitted control flow (the
// semantic-context gate, both predicate probes, the alt rewrite, the no-viable fallback)
// instead of six positive probes plus one negative guard.
insta::assert_snapshot!(
```rust
---
Found a 16 line (101 tokens) duplication in the following files:
* Starting at line 4099 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 4251 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
atn.set_loop_back_state(3, 4).expect("loop back state");
atn.add_transition(0, ParserTransitionSpec::Epsilon { target: 1 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 2 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 3 })
.expect("transition");
atn.add_transition(
2,
ParserTransitionSpec::Atom {
target: 4,
label: 1,
},
)
.expect("transition");
atn.add_transition(4, ParserTransitionSpec::Epsilon { target: 1 })Found a 19 line (100 tokens) duplication in the following files:
);
render_repeated_accessor_validation(
&mut rendered.validation,
method,
view_name,
&label.source_name,
label.cardinality.min,
validation_error_name,
);
return;
}
let lookup = match label.selector {
ContextLabelSelector::Nth(occurrence) => format!(".nth({occurrence})"),
ContextLabelSelector::LastAfter(skip) => format!(".skip({skip}).last()"),
ContextLabelSelector::AllAfter(_) => unreachable!("handled above"),
};
if label.cardinality.is_required_single() {
let _ = writeln!(
rendered.recovered,
```rust
---
Found a 20 line (100 tokens) duplication in the following files:
* Starting at line 15 of crates/antlr-rust-codegen/src/parser/surface/names.rs
* Starting at line 346 of crates/antlr-rust-codegen/src/parser/surface/support_abi.rs
```rust
for (rule_index, rule) in model.rules.iter().enumerate() {
if !rule.has_attrs() {
continue;
}
let struct_name = embedded::attrs_struct_name(rule_index);
let mut fields = String::new();
for attr in &rule.attrs {
let _ = writeln!(
fields,
" pub {}: {},",
embedded::escape_keyword(&attr.name),
attr.ty
);
}
let _ = writeln!(
out.attrs_structs,
"#[derive(Clone, Debug, Default)]\n#[allow(non_snake_case, dead_code)]\npub struct {struct_name} {{\n{fields}}}\n"
);
}
out.module_items.push_str(EMBEDDED_INPUT_FACADE); |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (47)
📒 Files selected for processing (14)
💤 Files with no reviewable changes (1)
📝 WalkthroughWalkthroughGenerated parser contexts now use a new exported runtime macro, ChangesRuntime context macro consolidation
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Codegen as Codegen surface (contexts.rs)
participant Macro as __antlr4_rust_context! macro
participant Runtime as antlr-rust-runtime
participant Generated as Generated parser module
Codegen->>Macro: emit rule attributes and context-kind match
Macro->>Runtime: expand into stored/active/validated context types
Runtime-->>Generated: typed context struct with accessors, Display, conversions
Generated->>Runtime: request compatibility check via __antlr4_rust_require_codegen_api!
Runtime-->>Generated: confirm revision 4 support (accepts 1-4)
Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Claude Code is working… I'll analyze this and get back to you. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
📊 Source Code Metrics (this PR vs
|
| File | Cyclomatic | Cognitive | Functions | LLOC | MI |
|---|---|---|---|---|---|
| crates/antlr-rust-codegen/src/generator/tests.rs | 280 (main: 278) 🔴 | 48 ⚪ | 197 (main: 195) 🔴 | 1362 (main: 1347) 🔴 | 0 ⚪ |
| crates/antlr-rust-codegen/src/parser/surface/support_abi.rs | 164 (main: 163) 🔴 | 102 (main: 100) 🔴 | 31 ⚪ | 269 (main: 267) 🔴 | 0 ⚪ |
| crates/antlr-rust-codegen/src/parser/surface/accessors.rs | 236 (main: 237) 🟢 | 192 ⚪ | 28 (main: 29) 🟢 | 329 (main: 331) 🟢 | 0 ⚪ |
| crates/antlr-rust-codegen/src/parser/surface/names.rs | 65 (main: 64) 🔴 | 40 (main: 38) 🔴 | 16 ⚪ | 109 (main: 107) 🔴 | 5.04 (main: 5.26) 🔴 |
| crates/antlr-rust-runtime/src/generated.rs | 18 ⚪ | 0 ⚪ | 12 ⚪ | 19 ⚪ | 13.31 (main: 23.50) 🔴 |
| crates/antlr-rust-codegen/src/parser/surface/contexts.rs | 25 (main: 27) 🟢 | 22 (main: 26) 🟢 | 4 ⚪ | 72 (main: 90) 🟢 | 11.52 (main: 10.12) 🟢 |
| crates/antlr-rust-runtime/src/lib.rs | 4 ⚪ | 3 ⚪ | 1 ⚪ | 7 ⚪ | 29.93 (main: 30.02) 🔴 |
Generated by mehen v1.8.0 — the code quality watcher.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #276.
Summary
conversions, common accessors, and invocation-state display behind the
doc-hidden runtime
__antlr4_rust_context!support macro.optional grammar attributes, and collision-safe method names while preserving
the existing public stored, active, recovered, validated, listener, visitor,
and downcast behavior.
__RuleAttrsNpayloads and attribute lookups for ruleswithout generated attributes.
while their runtime surfaces remain available, and regenerate checked-in
recognizers, snapshots, hashes, and compatibility documentation.
Measured impact
Seven interleaved warm package-check rebuilds were effectively unchanged:
1.561 s baseline versus 1.571 s current. The release
antlr4-rust-genbinarydecreased from 11,632,512 to 11,617,920 bytes, a 14,592-byte (0.125%) reduction.
Validation
cargo fmt --all -- --checkcargo clippy --locked --workspace --all-targets --all-features -- -D warningscargo test --locked --workspace --all-featurestools/grammar-frontend/update-stage0.sh --checktools/rust-syntax/update-generated.sh --checkSummary by CodeRabbit