fix(linter/plugins): decode escapes in identifier tokens

[overlookmotel]

Fix tokens in linter. Use options for tokens which match ESLint's tokens style, instead of matching TS-ESLint's.

[overlookmotel]

• feat(estree/tokens): add function to update tokens in place #19856 : 2 dependent PRs (#19858 , #19865 )
• refactor(estree/tokens): reduce repeated code #19857
• ci(benchmarks): add benchmark for updating tokens for raw transfer #19854
• feat(parser): add mutate_tokens Cargo feature #19853
• refactor(parser): Token::set_* method visible only within lexer #19852
• perf(estree/tokens): preallocate sufficient space for tokens JSON #19851
• refactor(estree/tokens): break up into multiple files #19850
• perf(estree/tokens): introduce ESTreeTokenConfig trait #19842
• refactor(estree/tokens): move logic for JSX identifiers into JSXState struct #19839
• fix(linter/plugins): decode escapes in identifier tokens #19838 👈 (View in Graphite)
• refactor(estree/tokens): re-order code #19837
• docs(estree/tokens): improve comments #19836
• refactor(estree/tokens): make ESTreeUnsafeToken type private #19835
• refactor(estree/tokens): rename types #19834
• fix(estree/tokens): convert this tokens in TSTypeName #19815
• perf(estree/tokens): do not JSON-encode keyword, punctuator, etc tokens #19814
• perf(estree/tokens): do not JSON-encode this identifiers #19813
• perf(estree/tokens): do not JSON-encode JSX identifiers #19812
• perf(estree/tokens): handle regex tokens separately #19796
• perf(estree/tokens): use strings from AST for identifier tokens #19744 : 1 other dependent PR (#19735 )
• refactor(estree/tokens): add TokenType type #19789
• feat(estree): add IS_COMPACT const to Formatter trait #19787
• feat(estree): expose buffer and formatter of serializers #19773
• perf(estree): use CodeBuffer::print_strs_array to reduce bounds checks #19766
• feat(data_structures): add CodeBuffer::print_strs_array #19760
• perf(estree): use CodeBuffer::print_indent #19727
• perf(estree/tokens): serialize tokens while visiting AST #19726
• perf(estree/tokens): serialize with ESTree not serde #19725
• feat(estree): oxc_estree crate export config and formatter types #19724
• refactor(estree/tokens): convert tokens spans before AST spans #19723
• refactor(estree/tokens): simplify code #19722
• perf(estree/tokens): do not branch on presence of override twice #19721
• refactor(estree/tokens): move logic for excluding legacy identifiers into visitor #19720
• perf(estree/tokens): replace hash map with Vec #19718
• perf(estree/tokens): replace multiple hash sets into a single hash map #19716
• refactor(estree/tokens): replace stored token when re-lexing #19698
• perf(parser): remove branches from finish_next_inner #19695
• perf(parser): remove const generic param from finish_next_inner #19684
• feat(parser): add LexerConfig::TOKENS_METHOD_IS_STATIC const #19683
• docs(parser): improve doc comments for ParserConfig and LexerConfig #19682
• refactor(estree/tokens): use wildcard import for AST types #19681
• test(estree): ESTree conformance check files with hashbangs #19680
• fix(estree/tokens): reverse field order of regex object in tokens #19679
• perf(estree/tokens): serialize tokens to compact JSON #19572
• ci(benchmarks): correct comment #19650
• perf(parser/tokens): pre-allocate capacity for tokens #19543
• fix(estree/tokens): generate tokens for files with BOM #19535
• refactor(estree/tokens): combine collect_token_context into to_estree_tokens_json #19534
• refactor(estree): reorder tokens code #19533
• refactor(coverage): standardize ESTree tokens testing functions #19532
• perf(linter/plugins): remove typescript from bundle #19531
• docs(linter/plugins): correct doc comments for initTokens #19530
• refactor(linter/plugins): combine functions for getting tokens #19529
• ci(benchmarks): benchmark parser with tokens and ESTree tokens conversion #19649
• perf(linter/plugins): use Oxc tokens in plugins #19498
• test(estree/tokens): do not skip tokens conformance tests for files with hashbangs #19643
• fix(estree): fix tokens for JSX #19524
• perf(parser): introduce ParserConfig #19637
• feat(parser,estree,coverage): collect tokens in parser and convert to ESTree format #19497
• feat(coverage): add estree token conformance scaffolding #19496
• fix(tasks/coverage): bring back acorn-jsx estree testing #19516 : 1 other dependent PR (#19515 )
• chore(submodules): update submodule SHAs to latest commits #19518
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• 0-merge - adds this PR to the back of the merge queue
• hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Copilot]

Pull request overview

This PR fixes token serialization in the linter and JS plugin layer to use ESLint-compatible (Espree) token values for identifiers containing Unicode escape sequences (e.g., \u0061). Previously, the linter() constructor had decode_identifier_escapes: false (matching TS-ESLint's behavior), while ESLint's Espree parser decodes escapes in the token value. The fix replaces ESTreeTokenOptions::linter() with the existing ESTreeTokenOptions::test262() (Espree-compatible) and removes the now-redundant linter() constructor.

Changes:

• Remove ESTreeTokenOptions::linter() and replace both call sites with ESTreeTokenOptions::test262()
• Add escaped_idents.js fixture file covering Unicode-escaped identifiers and legacy keyword identifiers
• Update the test snapshot to reflect correctly decoded token values

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
crates/oxc_estree_tokens/src/lib.rs	Removes the linter() constructor (which differed from test262() only in decode_identifier_escapes: false)
crates/oxc_linter/src/lib.rs	Switches token options from removed linter() to test262()
apps/oxlint/src/js_plugins/parse.rs	Switches token options from removed linter() to test262()
apps/oxlint/test/fixtures/tokens/files/escaped_idents.js	New test fixture with Unicode escape sequences in identifiers and legacy keywords
apps/oxlint/test/fixtures/tokens/output.snap.md	Updated snapshot showing decoded identifier values matching ESLint behavior

[codspeed-hq]

Merging this PR will not alter performance

✅ 52 untouched benchmarks
⏩ 3 skipped benchmarks¹

---

_{Comparing om/02-22-fix_linter_plugins_decode_escapes_in_identifier_tokens (a175b31) with main (0681ca3)²}

Footnotes

1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on om/02-28-refactor_estree_tokens_re-order_code (0f666d6) during the generation of this report, so main (0681ca3) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[overlookmotel]

Merge activity

• Feb 28, 12:58 PM UTC: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
• Feb 28, 12:58 PM UTC: overlookmotel added this pull request to the Graphite merge queue.
• Feb 28, 1:11 PM UTC: Merged by the Graphite merge queue.