feat(codegen): share the validated-parse surface via runtime types - #331
Conversation
…mmar-agnostic types
Every generated parser re-declared an identical (modulo grammar-name prefix)
148-line validated-parse block: `<Grammar>ValidatedTree`, `ValidatedRuleNode`,
`FromValidatedRuleNode`, and `<Grammar>ValidationError` with its
Display/Error/From machinery, plus a ~12-line InvalidChildCount check per
required list child in `validate_tree_structure`. Measured across six
checked-in parsers (three here, three in mehen), the blocks hash identically
after stripping the prefix, so a multi-parser binary compiled the same
machinery once per grammar.
The runtime now owns the surface as grammar-agnostic types in the new
`validated` module: `ValidatedTree`, `ValidatedRuleNode`,
`FromValidatedRuleNode`, `ValidationError`, and a `require_min_count` helper
that collapses each child-count check to a single call. Variant data,
Display texts, `Error::source`, and both `From` conversions are preserved
exactly (thiserror-based; `MissingChild` uses `#[error("{0}")]` + `#[from]`
rather than `transparent` so `source()` still returns the inner error).
Grammar-specific strings stay in generated code as variant data, keeping the
runtime grammar-agnostic.
Generated modules now emit only the module-local `ValidatedTreeContext`
marker plus two plain type aliases (`pub type TomlValidatedTree =
antlr4_runtime::ValidatedTree;`), and re-export the unprefixed names so the
runtime context macros keep resolving unchanged. The marker must remain
module-local: the accessor macro's recovery-oriented and validated inherent
impls stay coherent only while rustc can prove the local marker never
implements the runtime's `__RecoveryContextState`. Aliasing deliberately
makes the validated types of different grammars interchangeable — one
compiled copy, uniform error handling across parsers.
This changes the emitted-source contract, so `__ANTLR4_RUST_CODEGEN_API` is
now 10. Revisions 1 through 9 remain accepted: the runtime only gained API,
and the macros keep unqualified name resolution, so older generated modules
that define the surface locally still compile (verified by building the
workspace with the previous checked-in parsers before regenerating). All
checked-in recognizers are regenerated; the compatibility test now sweeps
revisions 1-9 and pins the new diagnostic.
Generated-source savings: toml 2851->2658 lines, g4 7323->7050, rust
30703->30300 (869 lines, 24,211 bytes across the three parsers); a stripped
release binary linking the TOML and G4 parsers shrinks by 1,320 bytes with
byte-identical output.
Closes #319
Copy/Paste DetectionFound 10 duplication(s) across 11 changed non-generated Rust file(s) (threshold: 100 tokens). Show duplicationsFound a 20 line (161 tokens) duplication in the following files:
fn __walk<E, T: CodegenApiListener<E>>(
listener: &mut T,
tree: antlr4_runtime::Node<'_>,
mut invocation_states: Option<Vec<isize>>,
) -> Result<(), E> {
enum Event<'tree> {
Enter(antlr4_runtime::Node<'tree>),
Exit(RuleNodeView<'tree>),
}
let mut stack = vec![Event::Enter(tree)];
while let Some(event) = stack.pop() {
match event {
Event::Enter(node) => match node.kind() {
antlr4_runtime::NodeKind::Rule => {
let context = node.as_rule().expect("rule node kind checked");
if let Some(states) = &mut invocation_states {
states.insert(0, context.invoking_state());
}
listener.enter_every_rule(context)?;
```rust
---
Found a 26 line (142 tokens) duplication in the following files:
* Starting at line 4215 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 4438 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
atn.set_end_state(1, 4).expect("block end state");
atn.add_transition(0, ParserTransitionSpec::Epsilon { target: 1 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 2 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 3 })
.expect("transition");
atn.add_transition(
2,
ParserTransitionSpec::Atom {
target: 4,
label: 1,
},
)
.expect("transition");
atn.add_transition(
3,
ParserTransitionSpec::Atom {
target: 4,
label: 2,
},
)
.expect("transition");
atn.add_transition(4, ParserTransitionSpec::Epsilon { target: 5 })
.expect("transition");
atn.add_decision_state(1).expect("decision state");Found a 26 line (125 tokens) duplication in the following files:
$input: $crate::char_stream::CharStream,
$hooks: $crate::parser::SemanticHooks,
{
pub fn metadata() -> &'static $crate::generated::GrammarMetadata {
$metadata()
}
/// Adds a listener for lexer diagnostics.
pub fn add_error_listener<T>(&mut self, listener: T)
where
T: for<'a> $crate::errors::ErrorListener<dyn $crate::recognizer::Recognizer + 'a>
+ ::core::marker::Send
+ 'static,
{
$crate::recognizer::Recognizer::add_error_listener(&mut self.$base, listener);
}
/// Removes every lexer error listener, including the default console listener.
pub fn remove_error_listeners(&mut self) {
$crate::recognizer::Recognizer::remove_error_listeners(&mut self.$base);
}
/// Routes every token through ATN interpretation instead of the compiled
/// lexer DFA, so the learned-DFA trace (`lexer_dfa_string`) observes each
/// match.
pub fn set_force_interpreted(&mut self, force_interpreted: bool) {
```rust
---
Found a 25 line (115 tokens) duplication in the following files:
* Starting at line 4186 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 4396 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
atn.add_state(AtnStateKind::BlockStart, Some(0))
.expect("state")
.index(),
1
);
assert_eq!(
atn.add_state(AtnStateKind::Basic, Some(0))
.expect("state")
.index(),
2
);
assert_eq!(
atn.add_state(AtnStateKind::Basic, Some(0))
.expect("state")
.index(),
3
);
assert_eq!(
atn.add_state(AtnStateKind::BlockEnd, Some(0))
.expect("state")
.index(),
4
);
assert_eq!(
atn.add_state(AtnStateKind::RuleStop, Some(0))Found a 22 line (112 tokens) duplication in the following files:
fn plus_loop_atn() -> ParserAtn {
let mut atn = ParserAtnBuilder::new(2);
assert_eq!(
atn.add_state(AtnStateKind::RuleStart, Some(0))
.expect("state")
.index(),
0
);
assert_eq!(
atn.add_state(AtnStateKind::PlusBlockStart, Some(0))
.expect("state")
.index(),
1
);
assert_eq!(
atn.add_state(AtnStateKind::Basic, Some(0))
.expect("state")
.index(),
2
);
assert_eq!(
atn.add_state(AtnStateKind::BlockEnd, Some(0))
```rust
---
Found a 27 line (110 tokens) duplication in the following files:
* Starting at line 3540 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 3652 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
decision: 0,
alts: (1, 2),
track_alt_number: false,
allow_semantic_context: true,
force_context: false,
plus_loop: false,
fast_path: None,
body: &body,
},
0,
GeneratedStepRenderContext {
current_rule_index: 0,
embedded: None,
portable_locals: None,
decision_routing: DecisionRoutingRender::default(),
inline_action_statements: &BTreeMap::new(),
track_alt_numbers: false,
track_context_alt_numbers: false,
direct_generated_rule_calls: &[],
atn_preferred_rule_calls: &[],
adaptive_atn_preferred_rule_slots: &[],
adaptive_atn_probe_rule_slots: &[],
},
);
// The whole rendered star-loop captures the leading-predicate-to-exit-alt filtering.
insta::assert_snapshot!(Found a 17 line (105 tokens) duplication in the following files:
fn __from_node_with_invocation_states(
node: $crate::RuleNodeView<'a>,
invocation_states: Option<Vec<isize>>,
) -> Self {
$(
let __default = <$attrs>::default();
let __attrs = node.generated_attrs::<$attrs>().unwrap_or(&__default);
)?
Self {
__node: __GeneratedRuleContext::Stored(node),
__invocation_states: invocation_states,
__state: std::marker::PhantomData,
$(
$($field: __attrs.$field.clone(),)+
)?
}
}
```rust
---
Found a 25 line (104 tokens) duplication in the following files:
* Starting at line 3280 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 3447 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
state: 1,
decision: 0,
track_alt_number: false,
allow_semantic_context: false,
force_context: false,
fast_path: None,
alts: &alts,
},
0,
GeneratedStepRenderContext {
current_rule_index: 0,
embedded: None,
portable_locals: None,
decision_routing: DecisionRoutingRender::default(),
inline_action_statements: &BTreeMap::new(),
track_alt_numbers: false,
track_context_alt_numbers: false,
direct_generated_rule_calls: &[],
atn_preferred_rule_calls: &[],
adaptive_atn_preferred_rule_slots: &[],
adaptive_atn_probe_rule_slots: &[],
},
);
assert!(rendered.contains("ll1_decision_prediction(atn(), 1)"));Found a 28 line (102 tokens) duplication in the following files:
state: 1,
decision: 0,
track_alt_number: false,
allow_semantic_context: true,
force_context: false,
fast_path: None,
alts: &alts,
},
0,
GeneratedStepRenderContext {
current_rule_index: 0,
embedded: None,
portable_locals: None,
decision_routing: DecisionRoutingRender::default(),
inline_action_statements: &BTreeMap::new(),
track_alt_numbers: false,
track_context_alt_numbers: false,
direct_generated_rule_calls: &[],
atn_preferred_rule_calls: &[],
adaptive_atn_preferred_rule_slots: &[],
adaptive_atn_probe_rule_slots: &[],
},
);
// One decision renders into a fresh String; snapshot the whole emitted control flow (the
// semantic-context gate, both predicate probes, the alt rewrite, the no-viable fallback)
// instead of six positive probes plus one negative guard.
insta::assert_snapshot!(
```rust
---
Found a 16 line (101 tokens) duplication in the following files:
* Starting at line 4286 of crates/antlr-rust-codegen/src/generator/tests.rs
* Starting at line 4438 of crates/antlr-rust-codegen/src/generator/tests.rs
```rust
atn.set_loop_back_state(3, 4).expect("loop back state");
atn.add_transition(0, ParserTransitionSpec::Epsilon { target: 1 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 2 })
.expect("transition");
atn.add_transition(1, ParserTransitionSpec::Epsilon { target: 3 })
.expect("transition");
atn.add_transition(
2,
ParserTransitionSpec::Atom {
target: 4,
label: 1,
},
)
.expect("transition");
atn.add_transition(4, ParserTransitionSpec::Epsilon { target: 1 }) |
📝 WalkthroughWalkthroughThe runtime now provides shared validated-parse types and validation helpers. Generated parsers use grammar-branded aliases and revision-specific downcast handling. API revision 10 support, revision-9 compatibility tests, generated fixtures, and documentation were updated. ChangesValidated-parse runtime migration
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Claude finished @tinovyatkin's task in 18m 13s —— View job Review of
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1375b90265
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// Alias of the grammar-agnostic `antlr4_runtime::ValidatedTree`; the | ||
| /// validated-parse types of every generated parser are interchangeable. | ||
| pub type {validated_tree} = antlr4_runtime::ValidatedTree; |
There was a problem hiding this comment.
When a binary links two generated parsers, this alias erases grammar identity from their tree nodes: for example, TomlValidatedTree::tree() now produces the same ValidatedRuleNode accepted by GrammarSpecContext<ValidatedTreeContext>. The runtime macro's FromValidatedRuleNode implementation checks only the grammar-local numeric rule index/context kind (both TOML document and ANTLRv4 grammarSpec use index 0), so downcast_ref returns Some for the wrong grammar and subsequent infallible accessors can wrap unrelated children or panic. Keep a grammar-specific brand on ValidatedTree/ValidatedRuleNode even if the validation error type remains shared.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in baa6c4f — good catch, this was a real regression. ValidatedTree and ValidatedRuleNode are now branded with a Grammar type parameter that each generated module instantiates with its own module-local ValidatedTreeContext marker:
pub type TomlValidatedTree = antlr4_runtime::ValidatedTree<ValidatedTreeContext>;
pub type ValidatedRuleNode<'a> = antlr4_runtime::ValidatedRuleNode<'a, ValidatedTreeContext>;FromValidatedRuleNode carries the brand as an associated type (type Grammar), and downcast_ref bounds on Grammar = Grammar, so TomlValidatedTree::tree() no longer produces a node that GrammarSpecContext<ValidatedTreeContext> accepts — the cross-grammar downcast is a compile error (E0271) again. Pinned by a new test, validated_rule_nodes_of_different_grammars_do_not_cross_downcast, which generates two grammars into one crate, asserts the cross-grammar downcast fails to compile, and asserts the same-grammar downcast still resolves at runtime. ValidationError remains the one shared type, as you suggested.
| #[doc(hidden)] | ||
| #[must_use] | ||
| pub const fn __new(node: RuleNodeView<'a>) -> Self { | ||
| Self { node } |
There was a problem hiding this comment.
#[doc(hidden)] does not restrict access, so any downstream caller can take a RuleNodeView from an ordinary recovery-oriented ParsedFile and invoke this safe public constructor without running syntax or structural validation. The forged node can then be downcast into a generated validated context, whose required-child accessors use unreachable! when invariants are absent, turning recovered or malformed parses into panics; previously the generated node's private field prevented this. Seal this construction path, require a validation proof, or otherwise prevent ordinary safe code from minting ValidatedRuleNode (and likewise ValidatedTree) directly.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Partially addressed in baa6c4f; the remainder is documented as a known limitation rather than sealed, and here is why. Once the types live in the runtime, any constructor reachable by generated code in downstream crates is necessarily pub; a sealed-token scheme just moves the problem, because the token type must itself be public for generated code to name it (Claude's review reached the same conclusion). What the commit does instead:
- The accidental-misuse path you demonstrated is closed structurally: minting or downcasting now requires naming a specific grammar's
ValidatedTreeContextbrand, and a node forged from grammar A can no longer be downcast into grammar B's contexts (compile error, covered by a new negative-compile test). - Deliberate misuse — calling a
#[doc(hidden)]dunder constructor with an unvalidated tree of the same grammar — remains possible and panics, exactly as calling any other__-prefixed runtime item out of contract would misbehave. The__newdoc comments anddocs/migration.mdnow state this plainly (previously the docs overclaimed "only generated code may construct").
If you see a construction that seals this fully without making the token public to generated code in downstream crates, I'm happy to adopt it.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
📊 Source Code Metrics (this PR vs
|
| File | Cyclomatic | Cognitive | Functions | LLOC | MI |
|---|---|---|---|---|---|
| crates/antlr-rust-runtime/src/validated.rs | 30 🆕 | 2 🆕 | 20 🆕 | 31 🆕 | 14.54 🆕 |
| crates/antlr-rust-codegen/src/parser/surface/contexts.rs | 25 ⚪ | 22 ⚪ | 4 ⚪ | 69 ⚪ | 16.63 (main: 16.57) 🟢 |
| crates/antlr-rust-codegen/src/parser/surface/traversal.rs | 6 ⚪ | 2 ⚪ | 3 ⚪ | 36 ⚪ | 17.77 (main: 15.68) 🟢 |
| crates/antlr-rust-runtime/src/lib.rs | 4 ⚪ | 3 ⚪ | 1 ⚪ | 7 ⚪ | 28.97 (main: 29.51) 🔴 |
Generated by mehen v1.8.1 — the code quality watcher.
…den the revision contract Addresses PR #331 review findings. Codex P1: the initial hoist made all grammars' validated trees one type, so `downcast_ref` on a TOML validated node accepted another grammar's context whenever the grammar-local rule index happened to match, and the resulting "infallible" accessors could panic. `ValidatedTree<Grammar>` and `ValidatedRuleNode<'a, Grammar>` are now branded with the generated module's own `ValidatedTreeContext` marker (which already had to stay module-local for accessor-impl coherence), and `FromValidatedRuleNode` carries the brand as an associated type so the trait keeps its `FromValidatedRuleNode<'a>` arity for user-written bounds. Generated modules emit branded aliases; cross-grammar downcasts are compile errors again, pinned by a new negative-compile test. `ValidationError` deliberately stays one shared unbranded type. The `__antlr4_rust_context!` macro gains an optional `validated_downcast: branded,` field: revision-10 codegen selects the impl against the runtime-owned branded surface, while invocations without it keep expanding against the module-local validated types that revisions 9 and earlier declare, so all old revision arms remain accepted. That claim is now enforced by CI (Claude review #3): a frozen revision-9 generated module, produced by the pre-hoist generator, is checked in as a fixture and compiled against the current runtime in the compatibility test. Also from review: the error surface gets its own tests in the runtime (Display snapshot per variant, source() expectations, From conversions, require_min_count both ways); required-child validation sites shrink to `context.method()?;` via the runtime `From<MissingChildError>` conversion, retiring the dead validation_error_name plumbing in the accessor renderers; the doc-hidden `__new` constructors and the migration/README pages now state plainly that the constructors are a contract rather than a sealed boundary and that per-grammar trait impls on the shared error type collapse (E0119). Checked-in recognizers regenerated. Updated savings vs main: 530 lines / 19,334 bytes across the three parsers; the two-parser probe binary stays 1,216 bytes smaller with identical output.
|
@claude Thanks for the deep pass — every finding is addressed in baa6c4f:
Verification on the new commit: clippy |
|
Claude finished @tinovyatkin's task in 1m 53s —— View job Verification of baa6c4f fixesI reviewed
1. Branding (E0119 scope narrowed to 2. 3. Revision-9 compile guard. 4. Error-surface tests. 5. README wording. 6. 7. Required-accessor simplification. Note on local verification: this sandboxed environment requires interactive approval for No further concerns — all seven points from the prior deep-pass review are substantively addressed, not just documented around. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: baa6c4f40d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// Current generated-source/runtime contract revision emitted by the bundled generator. | ||
| #[doc(hidden)] | ||
| pub const __ANTLR4_RUST_CODEGEN_API: u32 = 9; | ||
| pub const __ANTLR4_RUST_CODEGEN_API: u32 = 10; |
There was a problem hiding this comment.
When bumping the current generated-code API to 10, every checked-in recognizer must be regenerated, but a repo-wide search still finds crates/antlr-rust-runtime/src/xpath/generated/x_path_lexer.rs declaring revision 8 from generator 0.31.0. Although the retained revision-8 arm keeps builds passing, this leaves the current runtime package dependent on stale generated output and prevents the revision inventory from accurately representing the checked-in recognizers; regenerate the XPath lexer alongside the other recognizers.
AGENTS.md reference: AGENTS.md:L17-L23
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 864ba82 — the XPath lexer is regenerated with the current generator and now declares __antlr4_rust_require_codegen_api!(10, "0.32.0"). The regenerated output is byte-identical apart from the header (generator version and revision handshake), and its semantics.json is unchanged, so this is purely the inventory correction you asked for. Runtime tests and clippy re-verified after the swap.
(For the record: it had been left at revision 8 following the precedent of the revision-9 bump, but you're right that AGENTS.md says all checked-in recognizers, and there's no reason for the runtime's own recognizer to lag.)
Codex review on PR #331 flagged that the revision-10 bump left crates/antlr-rust-runtime/src/xpath/generated/x_path_lexer.rs declaring revision 8 from generator 0.31.0, so the checked-in recognizer inventory no longer matched the current revision even though the retained arm kept it compiling. Regenerated with the current generator; the output is byte-identical apart from the header (generator version and revision handshake), and semantics.json is unchanged.
|
@codex review |
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Closes #319.
Summary
Moves the per-grammar validated-parse surface out of generated code into the runtime, and collapses each repeated-child count check in
validate_tree_structureto a single runtime call.New
antlr4_runtime::validatedmodule (re-exported at the crate root):ValidatedTree<Grammar>,ValidatedRuleNode<'a, Grammar>,FromValidatedRuleNode,ValidationError, andrequire_min_count(actual, minimum, context, child).Generated modules now emit only the module-local
ValidatedTreeContextmarker plus branded aliases and one trait re-export:Each min-count site becomes one
antlr4_runtime::require_min_count(...)?;call (site counts match the issue: toml 6, g4 14, rust 27), and — per review — each required-child site becomescontext.method()?;via the runtimeFrom<MissingChildError>conversion.Design decisions
Trees and nodes are branded per grammar; the error type is shared.
ValidatedTree/ValidatedRuleNodecarry aGrammartype parameter instantiated with the generated module'sValidatedTreeContextmarker (which already had to stay module-local so the recovery-oriented and validated accessor impls remain coherent — the #318 interplay). This keeps different grammars' validated trees nominally distinct and makes cross-grammardowncast_refa compile error, since rule indexes and context kinds are grammar-local numbers (review finding, Codex P1; pinned by a negative-compile test).FromValidatedRuleNodecarries the brand as an associated type, so the trait keeps its one-lifetime arity for user-written bounds.ValidationErroris deliberately one unbranded type: grammar identity was never encoded in its behavior — all grammar-specific detail (context/child names) is variant data — so one type means one compiled copy of theDisplay/Error/Frommachinery and uniform error handling in multi-parser binaries. Trade-off (documented indocs/migration.md): downstream code with per-grammar impls on the error name (e.g. two thiserror#[from]variants for two grammars) now hits E0119 and must collapse them into a single impl.Error behavior preserved exactly.
ValidationErroris thiserror-based but keeps the previous semantics: same variants andDisplaytexts,Error::sourcereturns the inner error forRecognition/MissingChild(#[error("{0}")]+#[from], deliberately not#[error(transparent)], which would forwardsource()one level too far), and bothFromconversions. The contract is now pinned by tests in the runtime (Display snapshot per variant,source()expectations,Fromconversions,require_min_countboth ways).Known limitation (from review, Codex P2 / Claude #2): the doc-hidden
__newconstructors used by generated code are technically callable from any crate, whereas revision-9 modules kept them module-private. Hand-constructing a validated value over an unvalidated parse makes the infallible accessors panic. This is an inherent cost of hoisting across the crate boundary and cannot be fully sealed with public-API mechanisms; it is stated plainly in the__newdocs anddocs/migration.md. The accidental-misuse path (cross-grammar confusion) is closed by the branding above.Compatibility contract
__ANTLR4_RUST_CODEGEN_APIis now 10. Revisions 1–9 remain accepted: the__antlr4_rust_context!macro gained an optionalvalidated_downcast: branded,field — revision-10 codegen selects the branded impl against the runtime-owned surface, while invocations without it keep expanding against the module-local validated types older modules declare themselves. Per review (Claude #3), this is no longer just a manual check: a frozen revision-9 generated module (produced by the pre-hoist v0.32.0 generator, declaring its ownValidatedTree/ValidatedRuleNode/FromValidatedRuleNode/ValidationError) is checked in undertests/antlr4_rust_gen_cli/fixtures/revision9/and compiled against the current runtime in the compatibility test. The revision-literal sweep now covers 1–9 and the diagnostic names revisions 1–10. All checked-in recognizers are regenerated, including the runtime-internal XPath lexer (revision 10; its regenerated output is byte-identical apart from the header).Existing user code compiles unchanged apart from the handshake:
<G>ValidationError::Variantpaths, exhaustive matches,validate(),parse_validated,downcast_ref::<Ctx<'_, ValidatedTreeContext>>(),T: FromValidatedRuleNode<'a>bounds, and the validated listener/visitor surfaces all work through the branded aliases.Measurements
Generated source (checked-in parsers, before → after; branding costs one
validated_downcast: branded,line per context vs the first revision of this PR):toml_parser.rsantlr_v4_parser.rsrust_parser.rsBinary size: a stripped release binary linking the TOML and G4 parsers and exercising
parse_validatedplus errorDisplaywent from 4,105,568 to 4,104,352 bytes (−1,216), with byte-identical program output. The delta is modest because the linker already deduplicated much of the identical machinery; the win is mainly source-level and grows per additional linked grammar.Testing
cargo clippy --locked --workspace --all-targets --all-features -- -D warnings— cleancargo test --workspace— 1,451 passed, 0 failed, including:frozen_revision9_generated_source_compiles_against_current_runtime(new)validated_rule_nodes_of_different_grammars_do_not_cross_downcast(new; asserts the E0271 compile failure and a successful same-grammar downcast at runtime)validated::testserror-contract suite (new)cargo fmt --check --all,rumdl fmt— cleantools/toml-syntax/update-generated.sh --check,tools/rust-syntax/update-generated.sh --check,tools/grammar-frontend/update-stage0.sh --check— all report currentNote for reviewers: the local
hkpre-commit harness (hk 1.54 vs the pinned 1.49 config) fails on any commit touching.rsfiles because its cargo-fmt builtin passesrust-toolchain.tomlas--manifest-path; all hook steps were run manually instead (results above).