Skip to content

Comments

fix(linter): OOM problems with custom plugins#17082

Merged
overlookmotel merged 6 commits intomainfrom
c/12-17-draft
Dec 18, 2025
Merged

fix(linter): OOM problems with custom plugins#17082
overlookmotel merged 6 commits intomainfrom
c/12-17-draft

Conversation

@overlookmotel
Copy link
Member

This PR fixes OOM (Out of Memory) problems when using JS plugins together with the import plugin for cross-module analysis.

Previously, a stopgap solution simply refused to run when >10,000 files were linted with both features enabled.

This PR implements a real fix using a deferred cloning strategy: standard allocators are used for parsing/linting (efficient, dynamic sizing), and the AST is only copied into a fixed-size allocator briefly when calling JS plugins.

This dramatically reduces memory usage from n_files × 6 GiB to n_files × dynamic + thread_count × 6 GiB.

Copilot AI review requested due to automatic review settings December 18, 2025 22:49
@github-actions github-actions bot added A-linter Area - Linter A-cli Area - CLI C-bug Category - Bug labels Dec 18, 2025
@overlookmotel overlookmotel self-assigned this Dec 18, 2025
@overlookmotel overlookmotel added the A-linter-plugins Area - Linter JS plugins label Dec 18, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses Out of Memory (OOM) issues that occur when using JavaScript plugins together with the import plugin for cross-module analysis. The implementation introduces a deferred cloning strategy that dramatically reduces memory usage.

Key Changes:

  • Implements a dual-allocator strategy: standard allocators for parsing/linting, with AST cloning to fixed-size allocators only when calling JS plugins
  • Removes the previous stopgap limitation that prevented linting >10,000 files with both features enabled
  • Adds conditional compilation support to restrict JS plugin functionality to 64-bit little-endian platforms

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
crates/oxc_linter/src/service/runtime.rs Adds separate js_allocator_pool field and logic to choose between standard and fixed-size allocators based on feature combination
crates/oxc_linter/src/lib.rs Implements new clone_into_fixed_size_allocator_and_run_external_rules function for deferred cloning strategy and refactors external rule execution
crates/oxc_allocator/src/pool/fixed_size.rs Removes ID wraparound assertion that is no longer needed
apps/oxlint/src/result.rs Removes TooManyFilesWithImportAndJsPlugins error variant that is now obsolete
apps/oxlint/src/lint.rs Removes the 10,000 file limit check and updates file system configuration logic for the new approach

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@codspeed-hq
Copy link

codspeed-hq bot commented Dec 18, 2025

CodSpeed Performance Report

Merging #17082 will not alter performance

Comparing c/12-17-draft (feabf6e) with main (d209c21)

Summary

✅ 42 untouched
⏩ 3 skipped1

Footnotes

  1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@overlookmotel overlookmotel merged commit fb9e193 into main Dec 18, 2025
28 checks passed
@overlookmotel overlookmotel deleted the c/12-17-draft branch December 18, 2025 23:01
overlookmotel added a commit that referenced this pull request Dec 19, 2025
### 🚀 Features

- d209c21 allocator: Add cap to FixedSizeAllocatorPool and block when
exhausted (#17023) (Cameron)
- fb2af91 allocator: Add bitset utils (#17042) (zhaoting zhou)
- c16082c tasks/compat_data: Integrate `node-compat-table` (#16831)
(Boshen)
- 5586823 span: Extract TS declaration file check to its own function
(#17037) (camchenry)
- 3d2b492 minifier: Fold iife arrow functions in call expressions
(#16477) (Armano)
- 67e9f9e codegen: Keep comments on the export specifiers (#16943)
(夕舞八弦)
- cb515fa parser: Improve error message for `yield` as identifier usage
(#16950) (sapphi-red)
- dcc856b parser: Add help for `new_dynamic_import` error (#16949)
(sapphi-red)
- c3c79f8 parser: Improve import attribute value error message (#16948)
(sapphi-red)
- 291b57b ast_tools: Generate TS declaration files for deserializer and
walk files (#16912) (camc314)
- 74eae13 minifier: Remove unused import specifiers (#16797) (camc314)

### 🐛 Bug Fixes

- fb9e193 linter: OOM problems with custom plugins (#17082)
(overlookmotel)
- e59132b parser/napi: Fix lazy deser (#17069) (overlookmotel)
- a92faf0 ast_tools: Support `u128` in `assert_layouts` generator
(#17050) (overlookmotel)
- 47b4c2f minifier/docs: Correct hyperlink path in OPTIMIZATIONS.md
(#16986) (GRK)
- 3002649 transformer/typescript: Remove unused import equals
declaration (#16776) (Dunqing)
- 5a2af88 regular_expression: Correct named capture group reference
error (#16952) (sapphi-red)

### ⚡ Performance

- b657bb6 allocator: Reduce time `Mutex` lock is held in
`FixedSizeAllocatorPool::get` (#17079) (overlookmotel)
- 1f3b19b ast: `#[ast]` macro use `#[repr(transparent)]` for
single-field structs (#17052) (overlookmotel)
- 225f229 parser: Use SmallVec for duplicate default export detection
(#16801) (camc314)

### 📚 Documentation

- a9c419f traverse: Update safety comments (#16944) (overlookmotel)

Co-authored-by: overlookmotel <557937+overlookmotel@users.noreply.github.com>
overlookmotel added a commit that referenced this pull request Dec 19, 2025
# Oxlint
### 🚀 Features

- 6cc3fdf linter/no-inferrable-types: Implement fixer (#17090) (camc314)
- 2067997 linter/no-negation-in-equality-check: Implement suggestion
(#17084) (camc314)
- 552f9ef vscode: Auto-generate VSCode README configuration from
package.json (#16970) (Copilot)
- 9190c4b linter/no-unnecessary-array-flat-depth: Implement fixer
(#17057) (camc314)
- ed789de linter/misrefactored-assign-op: Implement fixer (#17056)
(camc314)
- a0f74a0 linter/config: Allow aliasing plugin names to allow names the
same as builtin plugins (#15569) (Cameron)
- a43d251 linter/plugins: `RuleTester` support `languageOptions.globals`
(#17009) (overlookmotel)
- 35070d9 linter/bad-bitwise-operator: Implement fixer (#17006)
(camc314)
- 322d995 linter/prefer-enum-initializers: Implement fixer (#17004)
(camc314)
- ae1e5bc vscode: Add support for tsgolint binary configuration (#16921)
(ColemanDunn)
- 3bfe31e linter/eslint-plugin-vitest: Add prefer-called-with as vitest
compatible jest rule (#16993) (Said Atrahouch)
- 0cd075f linter/eslint-plugin-jest: Add fix capabilities to
prefer-called-with rule (#16987) (Said Atrahouch)
- 357564b linter: Add options for
`typescript/require-array-sort-compare` rule. (#16980) (connorshea)
- 2b0ffba linter: Add options for
`typescript/no-meaningless-void-operator` rule. (#16981) (connorshea)
- 8bc4287 linter/plugins: Validate options against schema (#16974)
(overlookmotel)
- fdc7d08 linter: Implement eslint/capitalized-comments (#16896) (Tu
Shaokun)
- f8b6561 linter: Add support for `test.for` in vitest (#16925)
(camchenry)
- 7ee0379 linter/eslint-plugin-vitest: Implement prefer-spy-on (#16426)
(Said Atrahouch)
- fc96ee0 linter: Implement jest/prefer-to-have-been-called-times
(#16938) (秦宇航)
- 291b57b ast_tools: Generate TS declaration files for deserializer and
walk files (#16912) (camc314)
- e31da2a linter: Implement jest/perfer-to-have-been-called (#16899)
(秦宇航)
- 1a31306 linter/eslint-plugin-vitest: Add require-hook as vitest
compatible jest rule (#16880) (Said Atrahouch)
- cd3db21 linter: Add ignoredTypeNames option to no-base-to-string rule
(#16898) (camc314)
- 763b25a linter: Implement eslint/no-inline-comments (#16885) (Tu
Shaokun)

### 🐛 Bug Fixes

- fb9e193 linter: OOM problems with custom plugins (#17082)
(overlookmotel)
- 005ec25 linter: Permit `$schema` `.oxlintrc.json` struct (#17060)
(Copilot)
- fd03131 linter/plugins: Handle plugin names containing slashes
(#17073) (overlookmotel)
- b2a4fac linter/plugins: Error if plugin name alias is not normalized
(#17071) (overlookmotel)
- e046c4e linter/no-misused-spread: Add rule options support (#17054)
(camc314)
- 5c1a9e0 linter/no-deprecated: Add rule options support (#17053)
(camc314)
- 8c9cafe linter: `import/consistent-type-specifier-style`: add support
for declaration files (#16979) (camchenry)
- dab4780 linter/no-empty-pattern: Misleading help message for arrays
(#17039) (Copilot)
- 67f8c5d linter/plugins: Get correct plugin name in all cases (#17033)
(overlookmotel)
- 674dab9 linter/plugins: Fix indentation in error message (#17018)
(overlookmotel)
- 6524f72 linter/plugins: Add `@types/node` dev dependency to `oxlint`
package (#17016) (overlookmotel)
- 7a35513 linter/plugins: Better error for `null` in `globals` in
`RuleTester` (#17011) (overlookmotel)
- 42603ba linter/plugins: Always define `languageOptions.globals`
(#17008) (overlookmotel)
- 4cdc2f8 linter: Fix VITEST override rule list and add test for
alphabetizing the two lists (#16975) (Connor Shea)
- e466562 linter/consistent-type-definitions: Handle parenthesized types
in rule (#16998) (camc314)
- fce267c linter: Correct vitest plugin source to be
`@vitest/eslint-plugin` (#16976) (connorshea)
- 477bb57 linter: Fix `vitest/no-restricted-vi-methods` and add tests
for it. (#16971) (connorshea)
- 7d6974d linter: Ignore oxlint directive comments in
capitalized-comments (#16989) (Tu Shaokun)
- 23ac6b1 linter/plugins: Apply defaults from `meta.schema` to options
(#16930) (overlookmotel)
- 2f946cf linter/plugins: Error if `defaultOptions` is not
JSON-serializable (#16959) (overlookmotel)
- d8b8a57 linter/plugins: Freeze whole of merged options (#16958)
(overlookmotel)
- d446c43 linter: Prevent extra fields from being present on oxlint
config file (#16874) (connorshea)
- b845871 linter/plugins: Correctly handle object with `__proto__` keys
in options merging (#16928) (overlookmotel)
- c897794 linter: Fix eslint/sort-imports allowSeparatedGroups not
working with single empty line (#16012) (Duc Nghiem Xuan)
- 0c347a1 linter/array-type: Handle satisfies expression (#16903)
(camc314)

### ⚡ Performance

- 70d853c linter: Avoid cloning source text when cloning AST into
fixed-size allocator (#17088) (overlookmotel)
- 4d389f7 linter: Less bounds checks in `normalize_plugin_name` (#17030)
(overlookmotel)
- fd8e9c6 linter/plugins: Speed up cloning JSON objects (#16997)
(overlookmotel)
- d77e22d linter/plugins: Use `DEFAULT_OPTIONS` for rules with empty
array as default options (#16913) (overlookmotel)

### 📚 Documentation

- 6d053b4 linter: Fix typo in doc comment (#17091) (overlookmotel)
- b5f3c91 linter: Document intentional exclusion of ignoreCase option in
jsx-no-duplicate-props (#17046) (Copilot)
- a0bf5d8 linter: Fix the config option docs for no-inline-comments
rule. (#16983) (connorshea)
- ca26a11 linter/plugins: Fix typo in doc comment (#16966)
(overlookmotel)
- 3183bf8 linter/plugins: Fix typo in JSDoc comment (#16900)
(overlookmotel)
# Oxfmt
### 🚀 Features

- 15dfb55 oxfmt: Respect single nearest `.editorconfig` (#17043)
(leaysgur)
- 8c33ff4 oxfmt: Expose Node.js API: `format(fileName, sourceText,
options?)` (#16939) (leaysgur)

### 🐛 Bug Fixes

- d340c87 oxfmt: Update api `FormatOptions` type with `& Record<string,
unknown>` (#17036) (leaysgur)
- 827a256 oxfmt: Place ignorePatterns at bottom of JSON in --migrate
prettier (#16926) (Boshen)

Co-authored-by: overlookmotel <557937+overlookmotel@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-cli Area - CLI A-linter Area - Linter A-linter-plugins Area - Linter JS plugins C-bug Category - Bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants