perf(linter/plugins): store tokens as a Box<[Token]>

[overlookmotel]

Small perf optimization.

Once tokens are collected, we have no reason to add / remove any. So store tokens as a Box<'a, [Token]> (16 bytes) instead of a Vec<'a, Token> (24 bytes).

Unlike with std's Vec<T>, converting arena Vec<'a, T> to Box<'a, [T]> is zero cost.

Also, now that it's only 16 bytes, remove the Option wrapper, as it serves little purpose except to force you to unwrap it. "No tokens" is represented as an empty slice instead.

[overlookmotel]

• perf(linter/plugins): collect tokens even when not needed #19970
• perf(linter/plugins): store tokens as a Box<[Token]> #19969 👈 (View in Graphite)
• feat(allocator): add methods for boxed slices ArenaBox<[T]> #19968
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• 0-merge - adds this PR to the back of the merge queue
• hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

Merging this PR will not alter performance

✅ 4 untouched benchmarks
⏩ 52 skipped benchmarks¹

---

_{Comparing om/03-03-perf_linter_plugins_store_tokens_as_a_box_token_ (c13b437) with om/03-03-feat_allocator_add_methods_for_boxed_slices_arenabox_t_ (d8c7299)²}

Footnotes

1. 52 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on om/03-03-feat_allocator_add_methods_for_boxed_slices_arenabox_t_ (64eee51) during the generation of this report, so 58a7847 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[Copilot]

Pull request overview

This PR optimizes token storage in the linter runtime by switching from arena Vec (and an Option wrapper) to an arena Box<[Token]>, representing “no tokens” as an empty slice to reduce per-section/context memory overhead.

Changes:

• Replace Option<ArenaVec<'a, Token>> with ArenaBox<'a, [Token]> for parser tokens across runtime/context storage.
• Adjust parsing pipeline to convert collected tokens to a boxed slice (into_boxed_slice) and propagate it through section/module processing.
• Update external-linter/raw-transfer preparation code paths to work with always-present (possibly empty) token slices.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
crates/oxc_linter/src/service/runtime.rs	Store section tokens as ArenaBox<[Token]> and return boxed slices from parsing.
crates/oxc_linter/src/context/host.rs	Persist parser tokens as ArenaBox<[Token]> and expose &[Token] / &mut ArenaBox<[Token]> APIs.
crates/oxc_linter/src/lib.rs	Remove Option handling around tokens when preparing JS-plugin raw transfer.

[overlookmotel]

Merge activity

• Mar 3, 4:23 PM UTC: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
• Mar 3, 4:23 PM UTC: overlookmotel added this pull request to the Graphite merge queue.
• Mar 3, 4:31 PM UTC: Merged by the Graphite merge queue.