Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,42 @@ cargo run --release --quiet --bin antlr4-runtime-testsuite -- --case ParserError

Per-case scratch crates land under `target/antlr-runtime-testsuite/<case>/`. Stale dirs from a killed run can fail a re-run with `Os { code: 66, ... DirectoryNotEmpty }` — `rm -rf target/antlr-runtime-testsuite/*` to recover.

## Snapshot tests (insta)

Prefer `insta` snapshots over hand-written assertions whenever a test pins a
*value* rather than a *property*: multi-field struct/enum equality, the contents
of a collection, a formatted diagnostic or error message, generated-code
strings, and token/tree/ATN/DFA dumps. The "assert `.len()` then spot-check a few
fields" pattern is the clearest win — snapshot the whole structure and the count
is implied. Snapshots are more observable regression targets and subsume negative
`!contains(...)` guards by showing the full output. Keep explicit `assert!` for
genuine properties a snapshot would weaken — boolean predicates
(`assert!(x.is_empty())`), bounds (`dur < LIMIT`), round-trip/algebraic
invariants (`decode(encode(x)) == x`), ordering checks — and layer a snapshot
alongside them when both the value and the invariant matter.

House style is named external snapshots stored under a sibling `snapshots/` dir —
`insta::assert_debug_snapshot!("descriptive_snake_name", value)`, or
`assert_snapshot!(...)` for a `String`/`Display` value; use inline (`@"..."`)
only for small, stable values. Project specifics:

- **Every test module (or bare `#[test]` fn) that calls an insta macro needs
`#[allow(clippy::disallowed_methods)] // insta assertion macros unwrap internal I/O.`**
— `.clippy.toml` bans `.unwrap()` and the macros unwrap internally, so CI
clippy fails without it (see `src/bin_support/grammar/semantics.rs`).
- **insta is `default-features = false`**: only `assert_snapshot!`,
`assert_debug_snapshot!`, and `assert_compact_debug_snapshot!` are available.
The YAML/JSON/redaction macros need serde, which the runtime does not use.
- **Determinism**: never snapshot `HashMap`/`HashSet` iteration order — the
runtime's `PredictionFxHasher` maps (`prediction.rs`, `dfa.rs`) are unordered;
collect into a `Vec` and sort by a stable key first. `BTreeMap`/`BTreeSet`
(used throughout the generator) are already ordered and safe. `TokenView`'s
`Debug` omits `byte_span`, so snapshot the explicit tuple when that field is
the point of the test.
- **Workflow**: `cargo insta test` records pending `.snap.new`/`.pending-snap`;
review each, then `cargo insta accept` (do not pass `--all-features` — it is
rejected). `cargo-insta` 1.48+ is available.

## CI parity

CI runs `cargo clippy --locked --all-targets --all-features -- -D warnings`, so reproduce locally with the same flags before pushing — `clippy::excessive-nesting`, `clippy::disallowed_types`, and similar nursery/pedantic lints all promote to errors there.
Expand Down
36 changes: 36 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,42 @@ fixtures in `src/atn/lexer_dfa.rs`, laid out one record-per-line to mirror the
ANTLR layout — carries `#[rustfmt::skip]`; leave those attributes in place rather
than letting fmt explode the block to one element per line.

## Snapshot tests (insta)

Prefer `insta` snapshots over hand-written assertions whenever a test pins a
*value* rather than a *property*: multi-field struct/enum equality, the contents
of a collection, a formatted diagnostic or error message, generated-code
strings, and token/tree/ATN/DFA dumps. The "assert `.len()` then spot-check a few
fields" pattern is the clearest win — snapshot the whole structure and the count
is implied. Snapshots are more observable regression targets and subsume negative
`!contains(...)` guards by showing the full output. Keep explicit `assert!` for
genuine properties a snapshot would weaken — boolean predicates
(`assert!(x.is_empty())`), bounds (`dur < LIMIT`), round-trip/algebraic
invariants (`decode(encode(x)) == x`), ordering checks — and layer a snapshot
alongside them when both the value and the invariant matter.

House style is named external snapshots stored under a sibling `snapshots/` dir —
`insta::assert_debug_snapshot!("descriptive_snake_name", value)`, or
`assert_snapshot!(...)` for a `String`/`Display` value; use inline (`@"..."`)
only for small, stable values. Project specifics:

- **Every test module (or bare `#[test]` fn) that calls an insta macro needs
`#[allow(clippy::disallowed_methods)] // insta assertion macros unwrap internal I/O.`**
— `.clippy.toml` bans `.unwrap()` and the macros unwrap internally, so CI
clippy fails without it (see `src/bin_support/grammar/semantics.rs`).
- **insta is `default-features = false`**: only `assert_snapshot!`,
`assert_debug_snapshot!`, and `assert_compact_debug_snapshot!` are available.
The YAML/JSON/redaction macros need serde, which the runtime does not use.
- **Determinism**: never snapshot `HashMap`/`HashSet` iteration order — the
runtime's `PredictionFxHasher` maps (`prediction.rs`, `dfa.rs`) are unordered;
collect into a `Vec` and sort by a stable key first. `BTreeMap`/`BTreeSet`
(used throughout the generator) are already ordered and safe. `TokenView`'s
`Debug` omits `byte_span`, so snapshot the explicit tuple when that field is
the point of the test.
- **Workflow**: `cargo insta test` records pending `.snap.new`/`.pending-snap`;
review each, then `cargo insta accept` (do not pass `--all-features` — it is
rejected). `cargo-insta` 1.48+ is available.

## Source layout

- `src/lib.rs` — public exports
Expand Down
55 changes: 10 additions & 45 deletions src/atn/parser.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1724,6 +1724,7 @@ fn dfa_state_display(state: ParserDfaStateView<'_>) -> String {
}

#[cfg(test)]
#[allow(clippy::disallowed_methods)] // `insta` assertion macros unwrap internal I/O.
mod tests {
use super::*;
use crate::atn::AtnStateKind;
Expand Down Expand Up @@ -1935,21 +1936,9 @@ mod tests {
let prediction = simulator
.adaptive_predict_info_with_precedence(0, 0, [1])
.expect("prediction");
assert_eq!(
prediction,
ParserAtnPrediction {
alt: 1,
requires_full_context: true,
has_semantic_context: false,
diagnostic: Some(ParserAtnPredictionDiagnostic {
kind: ParserAtnPredictionDiagnosticKind::Ambiguity,
start_index: 0,
sll_stop_index: 0,
ll_stop_index: 0,
conflicting_alts: vec![1, 2],
exact: false,
}),
}
insta::assert_debug_snapshot!(
"adaptive_predict_marks_sll_conflict_for_full_context",
prediction
);

let dfa = &simulator.decision_dfas()[0];
Expand Down Expand Up @@ -2029,21 +2018,9 @@ mod tests {
.adaptive_predict_stream_info_with_precedence(0, 0, &mut input)
.expect("prediction");

assert_eq!(
prediction,
ParserAtnPrediction {
alt: 1,
requires_full_context: true,
has_semantic_context: false,
diagnostic: Some(ParserAtnPredictionDiagnostic {
kind: ParserAtnPredictionDiagnosticKind::Ambiguity,
start_index: 0,
sll_stop_index: 0,
ll_stop_index: 0,
conflicting_alts: vec![1, 2],
exact: false,
}),
}
insta::assert_debug_snapshot!(
"adaptive_predict_stream_retries_full_context_conflict",
prediction
);
assert_eq!(input.index(), 0);
}
Expand Down Expand Up @@ -2087,21 +2064,9 @@ mod tests {
.adaptive_predict_stream_info_with_context(0, 0, &mut input, EMPTY_CONTEXT)
.expect("prediction");

assert_eq!(
prediction,
ParserAtnPrediction {
alt: 2,
requires_full_context: true,
has_semantic_context: false,
diagnostic: Some(ParserAtnPredictionDiagnostic {
kind: ParserAtnPredictionDiagnosticKind::ContextSensitivity,
start_index: 0,
sll_stop_index: 0,
ll_stop_index: 1,
conflicting_alts: vec![1, 2],
exact: false,
}),
}
insta::assert_debug_snapshot!(
"context_prediction_reports_context_sensitivity_for_dfa_conflict",
prediction
);
assert_eq!(input.index(), 0);
}
Expand Down
25 changes: 14 additions & 11 deletions src/atn/serialized.rs
Original file line number Diff line number Diff line change
Expand Up @@ -853,6 +853,7 @@ fn read_index(value: i32, label: &str) -> Result<usize, AntlrError> {
}

#[cfg(test)]
#[allow(clippy::disallowed_methods)] // `insta` assertion macros unwrap internal I/O.
mod tests {
use super::*;

Expand All @@ -877,12 +878,18 @@ mod tests {
let atn = AtnDeserializer::new(&serialized)
.deserialize_parser()
.expect("artificial parser ATN should deserialize");
assert_eq!(atn.max_token_type(), 9);
assert_eq!(atn.states().len(), 2);
assert_eq!(atn.rule_to_start_state().iter().collect::<Vec<_>>(), [0]);
assert_eq!(atn.rule_to_stop_state().iter().collect::<Vec<_>>(), [1]);
assert_eq!(atn.decision_to_state().iter().collect::<Vec<_>>(), [0]);
assert_eq!(atn.stats().transitions, 1);
// One summary snapshot of the deserialized shape supersedes the six per-accessor pokes and
// additionally locks the full ParserAtnStats layout (states count included).
insta::assert_debug_snapshot!(
"reads_small_parser_atn",
(
atn.max_token_type(),
atn.rule_to_start_state().iter().collect::<Vec<_>>(),
atn.rule_to_stop_state().iter().collect::<Vec<_>>(),
atn.decision_to_state().iter().collect::<Vec<_>>(),
atn.stats(),
)
);
}

#[test]
Expand All @@ -891,10 +898,6 @@ mod tests {
let error = AtnDeserializer::new(&serialized)
.deserialize()
.expect_err("parser input must not create a lexer graph");
assert!(
error
.to_string()
.contains("AtnDeserializer::deserialize_parser()")
);
insta::assert_snapshot!(error.to_string(), @"unsupported runtime feature: parser ATNs require AtnDeserializer::deserialize_parser()");
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
source: src/atn/parser.rs
expression: prediction
---
ParserAtnPrediction {
alt: 1,
requires_full_context: true,
has_semantic_context: false,
diagnostic: Some(
ParserAtnPredictionDiagnostic {
kind: Ambiguity,
start_index: 0,
sll_stop_index: 0,
ll_stop_index: 0,
conflicting_alts: [
1,
2,
],
exact: false,
},
),
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
source: src/atn/parser.rs
expression: prediction
---
ParserAtnPrediction {
alt: 1,
requires_full_context: true,
has_semantic_context: false,
diagnostic: Some(
ParserAtnPredictionDiagnostic {
kind: Ambiguity,
start_index: 0,
sll_stop_index: 0,
ll_stop_index: 0,
conflicting_alts: [
1,
2,
],
exact: false,
},
),
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
source: src/atn/parser.rs
expression: prediction
---
ParserAtnPrediction {
alt: 2,
requires_full_context: true,
has_semantic_context: false,
diagnostic: Some(
ParserAtnPredictionDiagnostic {
kind: ContextSensitivity,
start_index: 0,
sll_stop_index: 0,
ll_stop_index: 1,
conflicting_alts: [
1,
2,
],
exact: false,
},
),
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
source: src/atn/serialized.rs
expression: "(atn.max_token_type(), atn.rule_to_start_state().iter().collect::<Vec<_>>(),\natn.rule_to_stop_state().iter().collect::<Vec<_>>(),\natn.decision_to_state().iter().collect::<Vec<_>>(), atn.stats(),)"
---
(
9,
[
0,
],
[
1,
],
[
0,
],
ParserAtnStats {
states: 2,
transitions: 1,
interval_sets: 0,
interval_ranges: 0,
inline_token_sets: 0,
dense_token_sets: 0,
interval_token_sets: 0,
token_bitset_bytes: 0,
decisions: 1,
rules: 1,
packed_bytes: 204,
},
)
Loading
Loading