Skip to content

fix(parser): don't pop from lexer errors to preserve error after rewind#13460

Closed
ulrichstark wants to merge 2 commits intooxc-project:mainfrom
ulrichstark:dont-pop-from-lexer-errors
Closed

fix(parser): don't pop from lexer errors to preserve error after rewind#13460
ulrichstark wants to merge 2 commits intooxc-project:mainfrom
ulrichstark:dont-pop-from-lexer-errors

Conversation

@ulrichstark
Copy link
Contributor

@ulrichstark ulrichstark commented Aug 31, 2025

Closes #12816 and leads to more and better error messages as you can see in the updated coverage snapshots.
I'm not sure why the size of antd.js suddenly changed and I had to update the allocation snaphots as a result.

Copilot AI review requested due to automatic review settings August 31, 2025 08:54
@github-actions github-actions bot added A-parser Area - Parser C-bug Category - Bug labels Aug 31, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes an issue where lexer errors were being lost after parser rewind operations by changing from popping errors to referencing the last error. This ensures error messages are preserved and available even after the parser backtracks, leading to better error reporting.

  • Changes error handling from destructive pop() to non-destructive last() access
  • Adds error cloning to maintain ownership semantics
  • Preserves lexer errors for better diagnostic reporting after parser rewind operations

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@codspeed-hq
Copy link

codspeed-hq bot commented Aug 31, 2025

CodSpeed Instrumentation Performance Report

Merging #13460 will not alter performance

Comparing ulrichstark:dont-pop-from-lexer-errors (09bc1b6) with main (b92fa4f)

Summary

✅ 37 untouched benchmarks

Copy link
Member

@Boshen Boshen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is producing two identical errors:

e.g.:

  × Unterminated string
    ╭─[test262/test/language/types/string/S8.4_A14_T1.js:16:11]
 15 │ 
 16 │ var str = ";
    ·           ───
    ╰────

  × Unterminated string
    ╭─[test262/test/language/types/string/S8.4_A14_T1.js:16:11]
 15 │ 
 16 │ var str = ";
    ·           ───
    ╰────

@ulrichstark
Copy link
Contributor Author

This is producing two identical errors:

e.g.:

  × Unterminated string
    ╭─[test262/test/language/types/string/S8.4_A14_T1.js:16:11]
 15 │ 
 16 │ var str = ";
    ·           ───
    ╰────

  × Unterminated string
    ╭─[test262/test/language/types/string/S8.4_A14_T1.js:16:11]
 15 │ 
 16 │ var str = ";
    ·           ───
    ╰────

Ok thanks! You are right. I missed that.

My first approach to fix that issue was to clone all errors when creating a checkpoint and then re-assign them when rewinding instead of storing the count of errors in the snapshot. This way the expected behaviour stays the same and the error is preserved after rewinding even if it was popped from the Vec. The only disadvantage is that it has a slightly worse performance and memory footprint.

Are you okay with cloning the lexer errors into the checkpoint or does it make more sense to figure out how to handle the duplicate errors?

@overlookmotel
Copy link
Member

I'm not sure why the size of antd.js suddenly changed and I had to update the allocation snaphots as a result.

This is fixed by #13465.

@Boshen
Copy link
Member

Boshen commented Aug 31, 2025

@ulrichstark I'm not sure, gut feeling is Kind::Eof should have special treatment ... any token reaching EOF is a fatal error that cannot be recovered?

@ulrichstark
Copy link
Contributor Author

Closing this PR in favour of #13494, because the other PR is a cleaner fix and the performance impact is unexpectedly neutral instead of negative and even slightly positive on some benchmarks.

Boshen pushed a commit that referenced this pull request Sep 1, 2025
Closes #12816.
Alternative to #13460.

This PR is the slightly cleaner way to fix the issue compared to the
other PR, but comes at a potential performance cost because now instead
of remembering how many errors there were, we are storing a clone of all
errors and re-assign them when rewinding.

This makes it possible to fix the issue because now even errors before a
checkpoint, that were popped/removed after the checkpoint, are properly
rewinded.
@ulrichstark ulrichstark deleted the dont-pop-from-lexer-errors branch September 8, 2025 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-parser Area - Parser C-bug Category - Bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Panic called Option::unwrap() on a None value in rules/eslint/new_cap.rs

4 participants