Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1015,6 +1015,55 @@ rules. It removes generated rule methods, context types, and listener/visitor
callbacks, so consumers that invoke other rules directly must declare them with
`--entry-rule` or leave pruning disabled.

### Trivial-rule inlining

`--inline-trivial-rules` is an explicit, off-by-default optimization that
inlines two classes of pure parser rules into their call sites before ATN
construction, so the caller's decision sees the actual tokens instead of a
rule transition:

- **token-set rules** — a rule whose body is nothing but an alternation of
single terminals (keyword lists, operator names). Every reference is
replaced by the flattened token set, however many call sites exist;
expansion is bounded by construction at one element per site.
- **single-use pure sequences** — a single-alternative rule referenced
exactly once whose body carries no observable surface. Its body moves into
the call site as a parenthesized block.

Inlined rules are removed, so this pass is **recognition preserving**, not
tree/API preserving: the callee's rule method, context type, and
listener/visitor callbacks disappear, its parse-tree level vanishes, and its
recovery boundary moves to the caller. The accepted language and consumed
input for valid text are unchanged.

Candidates are inlined all-or-nothing and fail closed. A candidate is
declined — with a reason recorded in the manifest — when it is a configured or
inferred entry rule, recursive, nullable, referenced by grammar target code,
carries labels, attributes, actions, predicates, options, or exception
clauses, or when any call site binds a label, passes arguments, or pins
precedence. Discovery re-runs after every accepted rewrite, so alias chains
(`a : b ; b : X | Y ;`) collapse in one invocation while every application
removes exactly one rule.

Every applied run writes `optimizations.json` recording each candidate's
status, reason, removed rule, and rewritten call sites with original source
spans. Inspect the same deterministic report without generating or changing a
parser with:

```bash
antlr4-rust-gen Grammar.g4 \
--report-trivial-rules \
--out-dir target/grammar-report
```

Report mode writes only `optimizations.json`. Apply reviewed candidates with:

```bash
antlr4-rust-gen Grammar.g4 \
--inline-trivial-rules \
--out-dir src/generated
```

### Precedence-ladder optimization

`--optimize-precedence-ladders` is an explicit, off-by-default source
Expand Down
21 changes: 21 additions & 0 deletions crates/antlr-rust-codegen/src/builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ pub struct Builder {
fixed_lookahead: Option<usize>,
entry_rules: BTreeSet<String>,
prune_unreachable: bool,
inline_trivial_rules: bool,
report_trivial_rules: bool,
optimize_precedence_ladders: bool,
report_precedence_ladders: bool,
}
Expand All @@ -85,6 +87,8 @@ impl Default for Builder {
fixed_lookahead: None,
entry_rules: BTreeSet::new(),
prune_unreachable: false,
inline_trivial_rules: false,
report_trivial_rules: false,
optimize_precedence_ladders: false,
report_precedence_ladders: false,
}
Expand Down Expand Up @@ -177,6 +181,16 @@ impl Builder {
self
}

pub const fn inline_trivial_rules(mut self, enabled: bool) -> Self {
self.inline_trivial_rules = enabled;
self
}

pub const fn report_trivial_rules(mut self, enabled: bool) -> Self {
self.report_trivial_rules = enabled;
self
}

pub const fn optimize_precedence_ladders(mut self, enabled: bool) -> Self {
self.optimize_precedence_ladders = enabled;
self
Expand All @@ -201,6 +215,11 @@ impl Builder {
let output_directory = self
.output_directory
.ok_or_else(|| Error::configuration("an output directory is required"))?;
if self.inline_trivial_rules && self.report_trivial_rules {
return Err(Error::configuration(
"trivial-rule inlining and report-only mode are mutually exclusive",
));
}
if self.optimize_precedence_ladders && self.report_precedence_ladders {
return Err(Error::configuration(
"precedence-ladder optimization and report-only mode are mutually exclusive",
Expand Down Expand Up @@ -240,6 +259,8 @@ impl Builder {
fixed_lookahead: self.fixed_lookahead,
entry_rules: self.entry_rules,
prune_unreachable: self.prune_unreachable,
inline_trivial_rules: self.inline_trivial_rules,
report_trivial_rules: self.report_trivial_rules,
optimize_precedence_ladders: self.optimize_precedence_ladders,
report_precedence_ladders: self.report_precedence_ladders,
test_rig: None,
Expand Down
10 changes: 10 additions & 0 deletions crates/antlr-rust-codegen/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,14 @@ struct CliArgs {
#[arg(long)]
prune_unreachable: bool,

/// Inline trivial pure parser rules into their call sites (changes tree/API).
#[arg(long, conflicts_with = "report_trivial_rules")]
inline_trivial_rules: bool,

/// Dry-run trivial-rule inlining and emit only optimizations.json.
#[arg(long, conflicts_with = "inline_trivial_rules")]
report_trivial_rules: bool,

/// Collapse proven linear precedence ladders (changes tree/API).
#[arg(long, conflicts_with = "report_precedence_ladders")]
optimize_precedence_ladders: bool,
Expand Down Expand Up @@ -195,6 +203,8 @@ impl CliArgs {
fixed_lookahead: self.fixed_lookahead.map(usize::from),
entry_rules: self.entry_rules.into_iter().collect(),
prune_unreachable: self.prune_unreachable,
inline_trivial_rules: self.inline_trivial_rules,
report_trivial_rules: self.report_trivial_rules,
optimize_precedence_ladders: self.optimize_precedence_ladders,
report_precedence_ladders: self.report_precedence_ladders,
test_rig: None,
Expand Down
4 changes: 4 additions & 0 deletions crates/antlr-rust-codegen/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ pub(crate) struct CompilerConfig {
pub(crate) entry_rules: BTreeSet<String>,
/// Remove parser rules unreachable from every inferred/configured entry.
pub(crate) prune_unreachable: bool,
/// Inline trivial pure parser rules into their call sites (issue #130).
pub(crate) inline_trivial_rules: bool,
/// Analyze trivial-rule inlining on a shadow model and emit only its manifest.
pub(crate) report_trivial_rules: bool,
/// Recognition-preserving source rewrite from issue #225.
pub(crate) optimize_precedence_ladders: bool,
/// Analyze the same pass on a shadow model and emit only its manifest.
Expand Down
163 changes: 163 additions & 0 deletions crates/antlr-rust-codegen/src/grammar/transform/analysis.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ use std::collections::{BTreeMap, BTreeSet};
use petgraph::algo::tarjan_scc;
use petgraph::graph::DiGraph;

use crate::grammar::action::{ActionReferenceKind, ActionReferenceParser};
use crate::grammar::model::{Block, Element, ElementKind, GrammarUnit, Quantifier, RuleId};

#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
Expand Down Expand Up @@ -206,3 +207,165 @@ fn recursive_components(call_graph: &BTreeMap<RuleId, Vec<RuleId>>) -> Vec<Vec<R
components.sort();
components
}

pub(crate) fn visit_elements(block: &Block, visitor: &mut impl FnMut(&Element)) {
for alternative in &block.alternatives {
for element in &alternative.elements {
visitor(element);
if let ElementKind::Block(nested) = &element.kind {
visit_elements(nested, visitor);
}
}
}
}

/// Rules whose generated context or accessors are referenced from target
/// code anywhere in the unit.
///
/// Fails closed: any target-code body the action-reference parser cannot
/// fully resolve marks every rule as observed.
pub(crate) fn observed_rule_contexts(
unit: &GrammarUnit,
rules_by_name: &BTreeMap<String, RuleId>,
action_reference_parser: ActionReferenceParser,
) -> BTreeSet<RuleId> {
let mut observed = BTreeSet::new();
let mut has_opaque_target_code = false;
for action in &unit.actions {
collect_target_code_rule_references(
&action.body,
rules_by_name,
&mut observed,
&mut has_opaque_target_code,
action_reference_parser,
);
}
for rule in &unit.rules {
for clause in rule
.arguments
.iter()
.chain(rule.returns.iter())
.chain(rule.locals.iter())
{
collect_target_code_rule_references(
&clause.text,
rules_by_name,
&mut observed,
&mut has_opaque_target_code,
action_reference_parser,
);
}
for action in &rule.actions {
collect_target_code_rule_references(
&action.body,
rules_by_name,
&mut observed,
&mut has_opaque_target_code,
action_reference_parser,
);
}
for handler in &rule.catches {
collect_target_code_rule_references(
&handler.body,
rules_by_name,
&mut observed,
&mut has_opaque_target_code,
action_reference_parser,
);
}
if let Some(action) = &rule.finally_action {
collect_target_code_rule_references(
&action.body,
rules_by_name,
&mut observed,
&mut has_opaque_target_code,
action_reference_parser,
);
}
visit_elements(&rule.block, &mut |element| match &element.kind {
ElementKind::RuleCall(call) => {
if let Some(arguments) = &call.arguments {
collect_target_code_rule_references(
arguments,
rules_by_name,
&mut observed,
&mut has_opaque_target_code,
action_reference_parser,
);
}
}
ElementKind::Action { body, .. } => {
collect_target_code_rule_references(
body,
rules_by_name,
&mut observed,
&mut has_opaque_target_code,
action_reference_parser,
);
}
ElementKind::Predicate { body, fail, .. } => {
collect_target_code_rule_references(
body,
rules_by_name,
&mut observed,
&mut has_opaque_target_code,
action_reference_parser,
);
if let Some(fail) = fail {
collect_target_code_rule_references(
fail,
rules_by_name,
&mut observed,
&mut has_opaque_target_code,
action_reference_parser,
);
}
}
ElementKind::Terminal(_)
| ElementKind::Range(..)
| ElementKind::Set { .. }
| ElementKind::Block(_)
| ElementKind::Epsilon => {}
});
}
if has_opaque_target_code {
observed.extend(rules_by_name.values().copied());
}
observed
}

fn collect_target_code_rule_references(
body: &str,
rules_by_name: &BTreeMap<String, RuleId>,
observed: &mut BTreeSet<RuleId>,
has_opaque_target_code: &mut bool,
action_reference_parser: ActionReferenceParser,
) {
*has_opaque_target_code |= !body.trim().is_empty();
for reference in action_reference_parser(body) {
let name = match reference.kind {
ActionReferenceKind::Attribute { name, .. }
| ActionReferenceKind::Qualified { name, .. } => Some(name),
ActionReferenceKind::NonLocal { rule, .. } => Some(rule),
};
if let Some(rule) = name.and_then(|name| rules_by_name.get(name)) {
observed.insert(*rule);
}
}
}

/// Whether the rule carries any rule-level surface that generated consumers
/// or target code can observe: modifiers, attribute clauses, `throws`,
/// options, named actions, exception handlers, or case-insensitivity.
pub(crate) const fn rule_surface_is_observable(rule: &crate::grammar::model::Rule) -> bool {
!rule.modifiers.is_empty()
|| rule.arguments.is_some()
|| rule.returns.is_some()
|| rule.locals.is_some()
|| !rule.throws.is_empty()
|| !rule.options.is_empty()
|| !rule.actions.is_empty()
|| !rule.catches.is_empty()
|| rule.finally_action.is_some()
|| rule.case_insensitive.is_some()
}
Loading
Loading