perf(parser): make pushing tokens faster#19960
Merged
graphite-app[bot] merged 1 commit intomainfrom Mar 3, 2026
Merged
Conversation
Member
Author
How to use the Graphite Merge QueueAdd either label to this PR to merge it via the merge queue:
You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. This stack of pull requests is managed by Graphite. Learn more about stacking. |
github-actions
bot
added
A-parser
overlookmotel
force-pushed
the
om/03-03-feat_allocator_add_vec_push_fast_method
branch
from
e31e409 to
5a987e9
Compare
overlookmotel
force-pushed
the
om/03-03-perf_parser_make_pushing_tokens_faster
branch
from
f0bd5c4 to
68eecce
Compare
Merging this PR will not alter performance
Comparing 
Footnotes
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the lexer’s token collection hot path to use Vec::push_fast when appending tokens, aiming to reduce code size in Lexer::finish_next and improve branch prediction behavior during parsing.
Changes:
- Switch token collection from
self.tokens.push(token)toself.tokens.push_fast(token)infinish_next_inner. - Add an inline comment explaining why
push_fastis expected to be beneficial here.
Contributor
Merge activity
|
Use `Vec::push_fast` method (introduced in #19959) for pushing to the tokens `Vec`. We allocate sufficient capacity upfront in the `Vec`, so it should never need to grow. `Vec::push_fast` makes the "need to grow" path cold and `#[inline(never)]`, which should reduce the size of `Lexer::finish_next`, and help the branch predictor make accurate predictions. +1% on benchmark for parsing with tokens in CodSpeed. *Possible* it'll have more impact in real world as CodSpeed doesn't take branch predictor into account in its simulations.
graphite-app
bot
force-pushed
the
om/03-03-feat_allocator_add_vec_push_fast_method
branch
from
5a987e9 to
f83be30
Compare
graphite-app
bot
force-pushed
the
om/03-03-perf_parser_make_pushing_tokens_faster
branch
from
68eecce to
8ba61dd
Compare
Base automatically changed from
om/03-03-feat_allocator_add_vec_push_fast_method
to
main
March 3, 2026 14:02
graphite-app
bot
deleted the
om/03-03-perf_parser_make_pushing_tokens_faster
branch
camc314
pushed a commit
that referenced
this pull request
Mar 9, 2026
### 🚀 Features - e8547cc parser: Report error for using declarations in ambient contexts (#19934) (camc314) - 8345318 allocator: Add methods for boxed slices `ArenaBox<[T]>` (#19968) (overlookmotel) - f83be30 allocator: Add `Vec::push_fast` method (#19959) (overlookmotel) ### 🐛 Bug Fixes - 291d867 transformer_plugins: Unwrap ChainExpression after define replacement removes optional markers (#20058) (IWANABETHATGUY) - 36b2e56 codegen: Print type for TSImportEqualsDeclaration (#20128) (camc314) - 5a246ec codegen: Print type arguments for JSXOpeningElement (#20127) (camc314) - a40870e codegen: Preserve parens for TSNonNullExpression (#20125) (camc314) - ae830b2 codegen: Print `declare` for `TSInterfaceDeclaration` (#20124) (camc314) - 92cfb14 linter/plugins: Fix types for `walkProgram` and `walkProgramWithCfg` (#20081) (overlookmotel) - ee0491e apps,napi: Explicitly specify libs in tsconfigs (#20071) (camc314) - 588009e codegen: Print `static` keyword for TSIndexSignature (#19755) (Dunqing) - 5a8799c codegen: Print `with_clause` for `ExportNamedDeclaration` (#20002) (Dunqing) - 7502afe parser: Correct capacity for tokens `Vec` (#19967) (overlookmotel) ### ⚡ Performance - 4ea8f9a napi: Remove `napi_build::setup()` from `oxc_napi` to avoid redundant rebuilds (#20094) (Boshen) - 2baa5fb napi: Unify build-test profile to coverage for cache sharing (#20090) (Boshen) - 8ba61dd parser: Make pushing tokens faster (#19960) (overlookmotel) Co-authored-by: Boshen <1430279+Boshen@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Use
Vec::push_fastmethod (introduced in #19959) for pushing to the tokensVec. We allocate sufficient capacity upfront in theVec, so it should never need to grow.Vec::push_fastmakes the "need to grow" path cold and#[inline(never)], which should reduce the size ofLexer::finish_next, and help the branch predictor make accurate predictions.+1% on benchmark for parsing with tokens in CodSpeed. Possible it'll have more impact in real world as CodSpeed doesn't take branch predictor into account in its simulations.