perf(napi/parser, linter/plugins): faster deserialization of raw fields#20923
Conversation
How to use the Graphite Merge QueueAdd either label to this PR to merge it via the merge queue:
You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. This stack of pull requests is managed by Graphite. Learn more about stacking. |
Merging this PR will not alter performance
Comparing Footnotes
|
There was a problem hiding this comment.
Pull request overview
This PR speeds up JS-side raw-transfer deserialization for raw fields by avoiding string decoding and instead deriving raw directly from the already-available sourceText using span offsets.
Changes:
- Add
#[estree(from_span)]attribute support inast_toolsschema + derive parsing. - Update raw-transfer JS deserializer generation to emit
sourceText.slice(start, end)for eligible fields and to inlineOption-None checks. - Apply
#[estree(from_span)]to relevant AST noderawfields and regenerate parser/oxlint deserializers accordingly.
Reviewed changes
Copilot reviewed 5 out of 14 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tasks/ast_tools/src/schema/extensions/estree.rs | Adds from_span flag to ESTree field config (docs currently need alignment with implementation). |
| tasks/ast_tools/src/derives/estree.rs | Parses #[estree(from_span)] on struct fields into schema metadata. |
| tasks/ast_tools/src/generators/raw_transfer.rs | Emits sourceText.slice(start, end) and refactors Option-None detection helper (needs an additional validation guard). |
| crates/oxc_ast/src/ast/literal.rs | Marks raw on several literals as from_span so JS deserializer can slice directly from source. |
| crates/oxc_ast/src/ast/jsx.rs | Marks JSXText.raw as from_span. |
| napi/parser/src-js/generated/deserialize/*.js | Regenerated parser deserializers now compute raw via sourceText.slice(start, end). |
| apps/oxlint/src-js/generated/deserialize.js | Regenerated oxlint deserializer now computes raw via sourceText.slice(start, end). |
1ba5af3 to
3170f01
Compare
Merge activity
|
…elds (#20923) `raw` field of `NumericLiteral`, `StringLiteral`, `BigIntLiteral`, `RegExpLiteral`, and `JSXText` are slices of source text. So in raw transfer deserializer, skip going through `deserializeStr`, which can be slow when source contains any non-ASCII characters. Instead just slice the string from the source text directly with `sourceText.slice(start, end)`. String decoding is the slowest part of raw transfer, so this should be a significant speed gain.
3170f01 to
0503a78
Compare
Use local copy of `oxc-parser` after oxc-project/oxc#20923. This reduces the number of strings per file.
### 🐛 Bug Fixes - fc7f60c allocator: Revert changes to `get_current_chunk_footer_field_offset` (#20964) (overlookmotel) - 31316c8 semantic: Rebind class expressions before identifier checks (#20916) (camc314) ### ⚡ Performance - fb52383 napi/parser, linter/plugins: Clear buffers and source texts earlier (#21025) (overlookmotel) - 3b7dec4 napi/parser, linter/plugins: Use `utf8Slice` for decoding UTF-8 strings (#21022) (overlookmotel) - 012c924 napi/parser, linter/plugins: Speed up decoding strings in raw transfer (#21021) (overlookmotel) - 55e1e9b napi/parser, linter/plugins: Initialize vars as 0 (#21020) (overlookmotel) - c25ef02 napi/parser, linter/plugins: Simplify branch condition in `deserializeStr` (#21019) (overlookmotel) - 9f494c3 napi/parser, linter/plugins: Raw transfer use `String.fromCharCode` in string decoding (#21018) (overlookmotel) - 91cf105 allocator: Increase initial chunk size from 512B to 16KB (#20968) (overlookmotel) - cbc0c21 allocator: Add `#[cold]` to to error handling functions (#20967) (overlookmotel) - 0503a78 napi/parser, linter/plugins: Faster deserialization of `raw` fields (#20923) (overlookmotel) - a24f75e napi/parser: Optimize string deserialization for non-ASCII sources (#20834) (Joshua Tuddenham) ### 📚 Documentation - c78a57a syntax: Fix typo (#21044) (camc314) - f5e228f allocator: Fix typo in comment (#20972) (overlookmotel) - 7159d51 allocator: Improve doc comment examples for `vec2::Vec` (#20969) (overlookmotel) - b1da750 allocator, data_structures: Correct comments (#20966) (overlookmotel) Co-authored-by: Boshen <1430279+Boshen@users.noreply.github.com>
# Oxlint ### 💥 BREAKING CHANGES - 22ce6af oxlint/lsp: [**BREAKING**] Show/fix safe suggestions by default (#19816) (Sysix) ### 🚀 Features - 7a7b7b8 oxlint/lsp: Add source.fixAllDangerous.oxc code action kind (#20526) (bab) - 9cfe57e linter/unicorn: Implement prefer-import-meta-properties rule (#20662) (Irfan - ئىرفان) - 1edb391 linter/eslint: Implement `no-restricted-exports` rule (#20592) (Nicolas Le Cam) - 0f12bcd linter/react: Implement `hook-use-state` rule (#20986) (Khaled Labeb) - 1513a9f oxlint/lsp: Show note field for lsp diagnostic (#20983) (Sysix) - 7fdf722 linter/unicorn: Implement `no-useless-iterator-to-array` rule (#20945) (Mikhail Baev) - 39c8f2c linter/jest: Implement padding-around-after-all-blocks (#21034) (Sapphire) - ac39e51 linter/eslint-vitest-plugin: Prefer importing vitest globals (#20960) (Said Atrahouch) - 0b84de1 oxlint: Support allow option for prefer-promise-reject-errors (#20934) (camc314) - 23db851 linter/consistent-return: Move rule from nursery to suspicious (#20920) (camc314) - 9a27e32 linter/no-unnecessary-type-conversion: Move rule from nursery to suspicious (#20919) (camc314) - 1ca7b58 linter/dot-notation: Move rule from nursery to style (#20918) (camc314) - 73ba81a linter/consistent-type-exports: Move rule from nursery to style (#20917) (camc314) - b9199b1 linter/unicorn: Implement switch-case-break-position (#20872) (Mikhail Baev) - 3435ff8 linter: Implements `prefer-snapshot-hint` rule in Jest and Vitest (#20870) (Said Atrahouch) - 98510d2 linter: Implement react/prefer-function-component (#19652) (Connor Shea) - 871f9d9 linter: Implement no-useless-assignment (#15466) (Zhaoting Zhou) - 0f01fbd linter: Implement eslint/object-shorthand (#17688) (yue) ### 🐛 Bug Fixes - dd2df87 npm: Export package.json for oxlint and oxfmt (#20784) (kazuya kawaguchi) - 9bc77dd linter/no-unused-private-class-members: False positive with await expr (#21067) (camc314) - 60a57cd linter/const-comparisons: Detect equality contradictions (#21065) (camc314) - 2bb2be2 linter/no-array-index-key: False positive when index is passed as function argument (#21012) (bab) - 6492953 linter/no-this-in-sfc: Only flag `this` used as member expression object (#20961) (bab) - 9446dcc oxlint/lsp: Skip `node_modules` in oxlint config walker (#21004) (copilot-swe-agent) - af89923 linter/no-namespace: Support glob pattern matching against basename (#21031) (bab) - 64a1a7e oxlint: Don't search for nested config outside base config (#21051) (Sysix) - 3b953bc linter/button-has-type: Ignore `document.createElement` calls (#21008) (Said Atrahouch) - 8c36070 linter/unicorn: Add support for `Array.from()` for `prefer-set-size` rule (#21016) (Mikhail Baev) - c1a48f0 linter: Detect vitest import from vite-plus/test (#20976) (Said Atrahouch) - 5c32fd1 lsp: Prevent corrupted autofix output from overlapping text edits (#19793) (Peter Wagenet) - ca79960 linter/no-array-index-key: Move span to `key` property (#20947) (camc314) - 2098274 linter: Add suggestion for `jest/prefer-equality-matcher` (#20925) (eryue0220) - 6eb77ec linter: Allow default-import barrels in import/named (#20757) (Bazyli Brzóska) - 9c218ef linter/eslint-vitest-plugin: Remove pending fix status for require-local-test-context-for-concurrent-snapshot (#20890) (Said Atrahouch) ### ⚡ Performance - fb52383 napi/parser, linter/plugins: Clear buffers and source texts earlier (#21025) (overlookmotel) - 3b7dec4 napi/parser, linter/plugins: Use `utf8Slice` for decoding UTF-8 strings (#21022) (overlookmotel) - 012c924 napi/parser, linter/plugins: Speed up decoding strings in raw transfer (#21021) (overlookmotel) - 55e1e9b napi/parser, linter/plugins: Initialize vars as 0 (#21020) (overlookmotel) - c25ef02 napi/parser, linter/plugins: Simplify branch condition in `deserializeStr` (#21019) (overlookmotel) - 9f494c3 napi/parser, linter/plugins: Raw transfer use `String.fromCharCode` in string decoding (#21018) (overlookmotel) - 0503a78 napi/parser, linter/plugins: Faster deserialization of `raw` fields (#20923) (overlookmotel) - a24f75e napi/parser: Optimize string deserialization for non-ASCII sources (#20834) (Joshua Tuddenham) ### 📚 Documentation - af72b80 oxlint: Fix typo for --tsconfig (#20889) (leaysgur) - 70c53b1 linter: Highlight that tsconfig is not respected in type aware linting (#20884) (camc314) # Oxfmt ### 🚀 Features - 35cf6e8 oxfmt: Add node version hint for ts config import failures (#21046) (camc314) ### 🐛 Bug Fixes - dd2df87 npm: Export package.json for oxlint and oxfmt (#20784) (kazuya kawaguchi) - 9d45511 oxfmt: Propagate file write errors instead of panicking (#20997) (leaysgur) - 139ddd9 formatter: Handle leading comment after array elision (#20987) (leaysgur) - 4216380 oxfmt: Support `.editorconfig` `tab_width` fallback (#20988) (leaysgur) - d10df39 formatter: Resolve pending space in fits measurer before expanded-mode early exit (#20954) (Dunqing) - f9ef1bd formatter: Avoid breaking after `=>` when arrow body has JSDoc type cast (#20857) (bab) Co-authored-by: Boshen <1430279+Boshen@users.noreply.github.com>

rawfield ofNumericLiteral,StringLiteral,BigIntLiteral,RegExpLiteral, andJSXTextare slices of source text.So in raw transfer deserializer, skip going through
deserializeStr, which can be slow when source contains any non-ASCII characters. Instead just slice the string from the source text directly withsourceText.slice(start, end).String decoding is the slowest part of raw transfer, so this should be a significant speed gain.