feat(codegen): run trusted Rust support bundles - #304
Merged
Conversation
Detect sibling Rust/transformGrammar.py bundles and require an explicit trust decision keyed by repository, revision, and content fingerprint before execution. Stage the complete grammar tree and run the approved transform in a bundled RustPython child so the source checkout remains untouched. Automatically select embedded strict generated-parser behavior, acknowledge transformed superclass metadata, relocate shipped Rust modules behind a generated aggregator, and emit transformed grammars plus rust-support.json for auditability. Keep Python compatibility tests scoped to the two current grammars-v4 C and Java transforms.
Copy/Paste DetectionFound 1 duplication(s) across 15 changed non-generated Rust file(s) (threshold: 100 tokens). Show duplicationsFound a 17 line (112 tokens) duplication in the following files:
let data = LexerCodegenData::from_compiled(compiled, &compilation.sources);
let support_enabled =
source_uses_rust_support(&data, &compilation.sources, &prepared_support);
let option_hooks = option_hooks(args, &data, support_enabled);
let options = collect_structural_grammar_options(&data, &option_hooks)?;
if support_enabled {
enforce_require_full_options(true, &options)?;
}
grammar_options.extend(options);
let embedded_actions = args.embedded_actions || support_enabled;
let sem_unknown = if support_enabled {
SemUnknownPolicy::Error
} else {
args.sem_unknown
};
let require_full_semantics = args.require_full_semantics || support_enabled;
let entries = collect_lexer_semantics(
```rust |
|
Claude Code is working… I'll analyze this and get back to you. |
|
Warning Review limit reached
Next review available in: 18 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe CLI discovers trusted grammars-v4 Rust support bundles, transforms grammar copies with bundled RustPython, integrates generated support modules, emits audit artifacts, and validates C and Java workflows through end-to-end tests. ChangesRust support transformation pipeline
Estimated code review effort: 5 (Critical) | ~90 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant CLI
participant Compiler
participant RustSupport
participant RustPython
User->>CLI: provide grammar path and trust fingerprint
CLI->>Compiler: create RustSupportOptions
Compiler->>RustSupport: prepare support bundle
RustSupport->>RustPython: transform staged grammar
RustPython-->>RustSupport: return transformed grammar and support files
RustSupport-->>Compiler: provide roots, modules, and manifest
Compiler-->>User: generate parser and diagnostics
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
📊 Source Code Metrics (this PR vs
|
| File | Cyclomatic | Cognitive | Functions | LLOC | MI |
|---|---|---|---|---|---|
| crates/antlr-rust-codegen/src/builder.rs | 46 ⚪ | 10 ⚪ | 24 ⚪ | 55 ⚪ | 13.59 (main: 13.71) 🔴 |
| crates/antlr-rust-codegen/src/rust_support/mod.rs | 136 🆕 | 94 🆕 | 23 🆕 | 160 🆕 | 0 🆕 |
| crates/antlr-rust-codegen/src/rust_support/identity.rs | 96 🆕 | 54 🆕 | 22 🆕 | 154 🆕 | 0 🆕 |
| crates/antlr-rust-codegen/src/testrig_cli.rs | 75 ⚪ | 34 ⚪ | 21 ⚪ | 88 ⚪ | 2.42 (main: 2.48) 🔴 |
| crates/antlr-rust-codegen/src/driver.rs | 111 (main: 79) 🔴 | 89 (main: 61) 🔴 | 9 (main: 7) 🔴 | 154 (main: 115) 🔴 | 0 (main: 5.20) 🔴 |
| crates/antlr-rust-codegen/src/rust_support/stage.rs | 51 🆕 | 39 🆕 | 9 🆕 | 61 🆕 | 17.93 🆕 |
| crates/antlr-rust-codegen/src/cli.rs | 23 (main: 14) 🔴 | 13 (main: 6) 🔴 | 5 (main: 4) 🔴 | 27 (main: 17) 🔴 | 17.20 (main: 21.34) 🔴 |
| crates/antlr-rust-codegen/src/rust_support/prompt.rs | 26 🆕 | 18 🆕 | 5 🆕 | 31 🆕 | 27.80 🆕 |
| crates/antlr-rust-codegen/src/lib.rs | 3 ⚪ | 0 ⚪ | 2 ⚪ | 4 ⚪ | 38.57 (main: 38.77) 🔴 |
| crates/antlr-rust-codegen/src/rust_support/python.rs | 11 🆕 | 7 🆕 | 1 🆕 | 27 🆕 | 36.52 🆕 |
| crates/antlr-rust-codegen/src/config.rs | 1 ⚪ | 0 ⚪ | 0 ⚪ | 0 ⚪ | 42.33 (main: 43.23) 🔴 |
Generated by mehen v1.8.1 — the code quality watcher.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 13
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/antlr-rust-codegen/src/cli.rs`:
- Around line 189-195: Update the CLI argument definition and the into_config
method so a --no-rust-support option is accepted and propagated into
RustSupportOptions.enabled. Default the option to enabled for backward
compatibility, but set enabled to false when the opt-out flag is supplied so
Rust support discovery skips trust validation.
In `@crates/antlr-rust-codegen/src/driver.rs`:
- Around line 318-324: Suppress structured positions when the diagnostic source
path is remapped, matching the existing byte_span behavior. Update the
computation around primary and structured_position in the current diagnostic
flow, and apply the same condition in compilation_diagnostics so positions from
staged grammar files are omitted unless they are mapped back to the original
source.
In `@crates/antlr-rust-codegen/src/rust_support/identity.rs`:
- Around line 264-301: Update default_trust_store_path so the Windows fallback
uses USERPROFILE when HOME is unavailable, allowing the AppData\Roaming path to
be constructed on standard Windows environments without HOME. Preserve the
existing APPDATA and non-Windows home-directory behavior.
- Around line 240-261: Update TrustStore::save to re-read the existing
trust-store file immediately before serialization, merge any records added by
other processes into the in-memory state, and then persist the merged contents.
Preserve the current directory creation, temporary-file, and error-handling
behavior while ensuring concurrent approvals are not discarded.
In `@crates/antlr-rust-codegen/src/rust_support/mod.rs`:
- Around line 458-495: Update write_file_name_array to stop unconditionally
expecting UTF-8 file names and instead return an io::Error when path.file_name()
cannot be converted with to_str, since transformed_grammars can reach this path
from the caller. Keep the existing JSON array writing logic in
write_file_name_array and json_string unchanged for valid names, but propagate
the conversion failure through the same error-handling flow used by
rewrite_support_paths rather than panicking.
- Around line 261-267: Update the candidate loop around identify,
stage::copy_grammar_directory, and stage::execute_transform so the grammar is
copied first, then identify the staged Rust directory and call ensure_trusted on
that staged identity. Remove the trust check against the original
support_directory, and execute the transform only after the staged bundle’s
digest has been approved.
- Around line 269-278: The support-file discovery currently reads the original
support directory before transformation, so generated artifacts use stale paths.
Update the flow around execute_transform and rewrite_support_paths to call
top_level_files_with_extension on the staged Rust directory after the transform
completes, then pass that resulting list to rewrite_support_paths and subsequent
artifact aggregation.
In `@crates/antlr-rust-codegen/src/rust_support/stage.rs`:
- Around line 7-15: Add a concise comment immediately before the
fs::create_dir_all call in copy_grammar_directory explaining that transform
scripts expect the staged grammar/src directory to exist. Keep the unconditional
directory creation and surrounding staging behavior unchanged.
- Around line 42-50: Update execute_transform to avoid unbounded Command::output
behavior: spawn the child, drain stdout and stderr concurrently into capped
buffers, and enforce a configurable execution deadline that terminates the child
when exceeded. Preserve the existing bounded diagnostic handling and return an
appropriate io::Error for timeout or child-process failures.
- Around line 78-89: Update top_level_files_with_extension in stage.rs to stop
discarding per-entry read_dir errors; replace the filter_map(Result::ok) path
with error propagation so any failed directory entry returns an io::Error
instead of silently skipping the file. Keep the existing extension filtering,
file collection, and sorting behavior unchanged, and make sure the read_dir
iterator in top_level_files_with_extension is the only place that changes.
In `@crates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/rust_support.rs`:
- Around line 41-60: Add a test alongside
noninteractive_bundle_requires_its_exact_fingerprint that invokes the command
with a valid-looking but different sha256: trust value, then assert the command
fails. Reuse the existing temporary grammar, generated-output, trust-store, and
command helpers, and verify the mismatched fingerprint is rejected rather than
accepted.
- Around line 62-129: Update both untrusted invocations in
current_java_transform_runs_from_the_staged_tree and the corresponding test
block to assert !untrusted.status.success() immediately after each run, before
extracting its fingerprint. Use normalize_cli_snapshot on stderr before every
untrusted_fingerprint call, including the existing parsing site near line 56, so
all fingerprint extraction uses normalized CLI output.
In
`@crates/antlr-rust-codegen/tests/fixtures/antlr4-rust-gen/rust-support/c/Rust/transformGrammar.py`:
- Around line 15-20: In the _lt function, remove the unnecessary f-string
prefixes from the chained string fragments that contain no interpolation, while
retaining the f prefix on the fragment that interpolates k.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: dc376cdb-5daa-47b6-b503-c17a92581ad0
⛔ Files ignored due to path filters (3)
Cargo.lockis excluded by!**/*.lockcrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/snapshots/antlr4_rust_gen_cli__cli__antlr4_rust_gen_help.snapis excluded by!**/*.snapcrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/snapshots/antlr4_rust_gen_cli__rust_support__untrusted_rust_support_diagnostic.snapis excluded by!**/*.snap
📒 Files selected for processing (23)
README.mdcrates/antlr-rust-codegen/Cargo.tomlcrates/antlr-rust-codegen/README.mdcrates/antlr-rust-codegen/src/builder.rscrates/antlr-rust-codegen/src/cli.rscrates/antlr-rust-codegen/src/config.rscrates/antlr-rust-codegen/src/driver.rscrates/antlr-rust-codegen/src/lib.rscrates/antlr-rust-codegen/src/rust_support/identity.rscrates/antlr-rust-codegen/src/rust_support/mod.rscrates/antlr-rust-codegen/src/rust_support/prompt.rscrates/antlr-rust-codegen/src/rust_support/python.rscrates/antlr-rust-codegen/src/rust_support/stage.rscrates/antlr-rust-codegen/src/testrig_cli.rscrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli.rscrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/rust_support.rscrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/support.rscrates/antlr-rust-codegen/tests/fixtures/antlr4-rust-gen/rust-support/c/CParser.g4crates/antlr-rust-codegen/tests/fixtures/antlr4-rust-gen/rust-support/c/Rust/c_parser_base.rscrates/antlr-rust-codegen/tests/fixtures/antlr4-rust-gen/rust-support/c/Rust/transformGrammar.pycrates/antlr-rust-codegen/tests/fixtures/antlr4-rust-gen/rust-support/java/JavaLexer.g4crates/antlr-rust-codegen/tests/fixtures/antlr4-rust-gen/rust-support/java/JavaParser.g4crates/antlr-rust-codegen/tests/fixtures/antlr4-rust-gen/rust-support/java/Rust/transformGrammar.py
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ac0a06bb1d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Fingerprint the staged bundle bytes that will execute and harvest support modules from post-transform staged state. Bound child runtime and output, propagate filesystem and path errors, and suppress transformed coordinates when diagnostics point back to untouched sources. Serialize concurrent trust-store updates under a file lock, add the Windows home fallback, and strengthen exact-fingerprint and trust-flow coverage.
Enforce strict grammar options only for staged support sources and retain the caller's semantic policy for mixed-root manifests. Re-export shipped modules through a stable generated rust_support surface so bundle-level reset and preprocessing helpers remain reachable without consumer module declarations.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e44822dd2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Derive artifact identities from repository, revision, and bundle path so identical support contents from different grammar directories cannot collide. Relocate crate-root references inside shipped support modules and prepare the conventional src output directory only when bundle metadata includes Rust modules.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6a0dc192df
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Encode support path and file lengths as u64 so trusted fingerprints remain target-independent. Short-circuit exact CLI fingerprints before loading persisted trust state, allowing reproducible CI even when an unrelated user trust store is unreadable or corrupt.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/rust_support.rs`:
- Around line 254-257: Update the parser output assertions in the relevant
rust_support test to build the expected #[path] prefix using the
platform-specific separator and compare it against the generated path via
to_string_lossy(). Remove the hard-coded `antlr-rust-support/` prefix while
preserving the existing assertions for the support module and alias.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 1019d862-c223-455e-9ceb-eaff76d3272e
⛔ Files ignored due to path filters (1)
crates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/snapshots/antlr4_rust_gen_cli__cli__antlr4_rust_gen_help.snapis excluded by!**/*.snap
📒 Files selected for processing (8)
crates/antlr-rust-codegen/README.mdcrates/antlr-rust-codegen/src/cli.rscrates/antlr-rust-codegen/src/driver.rscrates/antlr-rust-codegen/src/rust_support/identity.rscrates/antlr-rust-codegen/src/rust_support/mod.rscrates/antlr-rust-codegen/src/rust_support/stage.rscrates/antlr-rust-codegen/tests/antlr4_rust_gen_cli/rust_support.rscrates/antlr-rust-codegen/tests/fixtures/antlr4-rust-gen/rust-support/c/Rust/c_parser_base.rs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5a1b87fb27
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 464149a894
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rust/transformGrammar.pybundles and identify them by repository, revision, path, and content fingerprint--trust-rust-supportfingerprintsuperClassacknowledgementrust_supportre-export, and emit transformed grammars plusrust-support.jsonWhy
Issue #265 identified two grammars-v4 Rust support folders whose transforms target the older antlr4rust generated API. The compatibility surfaces from #267 and committed action timing from #266 are now available, so codegen can consume those bundles without modifying the source checkout or requiring users to run Python, copy files, add flags, or declare support modules manually.
The execution contract is deliberately trusted-source-first. Staging protects the checkout and makes outputs auditable, but it is not presented as a host security sandbox.
Scope
antlr4-rust-genCLI feature;Builderand TestRig do not execute sibling scriptsc/Rustandjava/java/RusttransformsValidation
cargo test --locked --workspace --all-targets --all-featurescargo clippy --locked --workspace --all-targets --all-features -- -D warningscargo package --locked -p antlr-rust-codegen --allow-dirtyANTLR_RUST_RELEASE_ALLOW_DIRTY=1 tools/release/verify-package-contents.sh antlr-rust-codegen42b1ea521705c126d84331186b221c9819db1058Summary by CodeRabbit