fix(linter): OOM problems with custom plugins#17082
Merged
overlookmotel merged 6 commits intomainfrom Dec 18, 2025
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses Out of Memory (OOM) issues that occur when using JavaScript plugins together with the import plugin for cross-module analysis. The implementation introduces a deferred cloning strategy that dramatically reduces memory usage.
Key Changes:
- Implements a dual-allocator strategy: standard allocators for parsing/linting, with AST cloning to fixed-size allocators only when calling JS plugins
- Removes the previous stopgap limitation that prevented linting >10,000 files with both features enabled
- Adds conditional compilation support to restrict JS plugin functionality to 64-bit little-endian platforms
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
crates/oxc_linter/src/service/runtime.rs |
Adds separate js_allocator_pool field and logic to choose between standard and fixed-size allocators based on feature combination |
crates/oxc_linter/src/lib.rs |
Implements new clone_into_fixed_size_allocator_and_run_external_rules function for deferred cloning strategy and refactors external rule execution |
crates/oxc_allocator/src/pool/fixed_size.rs |
Removes ID wraparound assertion that is no longer needed |
apps/oxlint/src/result.rs |
Removes TooManyFilesWithImportAndJsPlugins error variant that is now obsolete |
apps/oxlint/src/lint.rs |
Removes the 10,000 file limit check and updates file system configuration logic for the new approach |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
51396f3 to
feabf6e
Compare
CodSpeed Performance ReportMerging #17082 will not alter performanceComparing Summary
Footnotes
|
This was referenced Dec 19, 2025
overlookmotel
added a commit
that referenced
this pull request
Dec 19, 2025
### 🚀 Features - d209c21 allocator: Add cap to FixedSizeAllocatorPool and block when exhausted (#17023) (Cameron) - fb2af91 allocator: Add bitset utils (#17042) (zhaoting zhou) - c16082c tasks/compat_data: Integrate `node-compat-table` (#16831) (Boshen) - 5586823 span: Extract TS declaration file check to its own function (#17037) (camchenry) - 3d2b492 minifier: Fold iife arrow functions in call expressions (#16477) (Armano) - 67e9f9e codegen: Keep comments on the export specifiers (#16943) (夕舞八弦) - cb515fa parser: Improve error message for `yield` as identifier usage (#16950) (sapphi-red) - dcc856b parser: Add help for `new_dynamic_import` error (#16949) (sapphi-red) - c3c79f8 parser: Improve import attribute value error message (#16948) (sapphi-red) - 291b57b ast_tools: Generate TS declaration files for deserializer and walk files (#16912) (camc314) - 74eae13 minifier: Remove unused import specifiers (#16797) (camc314) ### 🐛 Bug Fixes - fb9e193 linter: OOM problems with custom plugins (#17082) (overlookmotel) - e59132b parser/napi: Fix lazy deser (#17069) (overlookmotel) - a92faf0 ast_tools: Support `u128` in `assert_layouts` generator (#17050) (overlookmotel) - 47b4c2f minifier/docs: Correct hyperlink path in OPTIMIZATIONS.md (#16986) (GRK) - 3002649 transformer/typescript: Remove unused import equals declaration (#16776) (Dunqing) - 5a2af88 regular_expression: Correct named capture group reference error (#16952) (sapphi-red) ### ⚡ Performance - b657bb6 allocator: Reduce time `Mutex` lock is held in `FixedSizeAllocatorPool::get` (#17079) (overlookmotel) - 1f3b19b ast: `#[ast]` macro use `#[repr(transparent)]` for single-field structs (#17052) (overlookmotel) - 225f229 parser: Use SmallVec for duplicate default export detection (#16801) (camc314) ### 📚 Documentation - a9c419f traverse: Update safety comments (#16944) (overlookmotel) Co-authored-by: overlookmotel <557937+overlookmotel@users.noreply.github.com>
overlookmotel
added a commit
that referenced
this pull request
Dec 19, 2025
# Oxlint ### 🚀 Features - 6cc3fdf linter/no-inferrable-types: Implement fixer (#17090) (camc314) - 2067997 linter/no-negation-in-equality-check: Implement suggestion (#17084) (camc314) - 552f9ef vscode: Auto-generate VSCode README configuration from package.json (#16970) (Copilot) - 9190c4b linter/no-unnecessary-array-flat-depth: Implement fixer (#17057) (camc314) - ed789de linter/misrefactored-assign-op: Implement fixer (#17056) (camc314) - a0f74a0 linter/config: Allow aliasing plugin names to allow names the same as builtin plugins (#15569) (Cameron) - a43d251 linter/plugins: `RuleTester` support `languageOptions.globals` (#17009) (overlookmotel) - 35070d9 linter/bad-bitwise-operator: Implement fixer (#17006) (camc314) - 322d995 linter/prefer-enum-initializers: Implement fixer (#17004) (camc314) - ae1e5bc vscode: Add support for tsgolint binary configuration (#16921) (ColemanDunn) - 3bfe31e linter/eslint-plugin-vitest: Add prefer-called-with as vitest compatible jest rule (#16993) (Said Atrahouch) - 0cd075f linter/eslint-plugin-jest: Add fix capabilities to prefer-called-with rule (#16987) (Said Atrahouch) - 357564b linter: Add options for `typescript/require-array-sort-compare` rule. (#16980) (connorshea) - 2b0ffba linter: Add options for `typescript/no-meaningless-void-operator` rule. (#16981) (connorshea) - 8bc4287 linter/plugins: Validate options against schema (#16974) (overlookmotel) - fdc7d08 linter: Implement eslint/capitalized-comments (#16896) (Tu Shaokun) - f8b6561 linter: Add support for `test.for` in vitest (#16925) (camchenry) - 7ee0379 linter/eslint-plugin-vitest: Implement prefer-spy-on (#16426) (Said Atrahouch) - fc96ee0 linter: Implement jest/prefer-to-have-been-called-times (#16938) (秦宇航) - 291b57b ast_tools: Generate TS declaration files for deserializer and walk files (#16912) (camc314) - e31da2a linter: Implement jest/perfer-to-have-been-called (#16899) (秦宇航) - 1a31306 linter/eslint-plugin-vitest: Add require-hook as vitest compatible jest rule (#16880) (Said Atrahouch) - cd3db21 linter: Add ignoredTypeNames option to no-base-to-string rule (#16898) (camc314) - 763b25a linter: Implement eslint/no-inline-comments (#16885) (Tu Shaokun) ### 🐛 Bug Fixes - fb9e193 linter: OOM problems with custom plugins (#17082) (overlookmotel) - 005ec25 linter: Permit `$schema` `.oxlintrc.json` struct (#17060) (Copilot) - fd03131 linter/plugins: Handle plugin names containing slashes (#17073) (overlookmotel) - b2a4fac linter/plugins: Error if plugin name alias is not normalized (#17071) (overlookmotel) - e046c4e linter/no-misused-spread: Add rule options support (#17054) (camc314) - 5c1a9e0 linter/no-deprecated: Add rule options support (#17053) (camc314) - 8c9cafe linter: `import/consistent-type-specifier-style`: add support for declaration files (#16979) (camchenry) - dab4780 linter/no-empty-pattern: Misleading help message for arrays (#17039) (Copilot) - 67f8c5d linter/plugins: Get correct plugin name in all cases (#17033) (overlookmotel) - 674dab9 linter/plugins: Fix indentation in error message (#17018) (overlookmotel) - 6524f72 linter/plugins: Add `@types/node` dev dependency to `oxlint` package (#17016) (overlookmotel) - 7a35513 linter/plugins: Better error for `null` in `globals` in `RuleTester` (#17011) (overlookmotel) - 42603ba linter/plugins: Always define `languageOptions.globals` (#17008) (overlookmotel) - 4cdc2f8 linter: Fix VITEST override rule list and add test for alphabetizing the two lists (#16975) (Connor Shea) - e466562 linter/consistent-type-definitions: Handle parenthesized types in rule (#16998) (camc314) - fce267c linter: Correct vitest plugin source to be `@vitest/eslint-plugin` (#16976) (connorshea) - 477bb57 linter: Fix `vitest/no-restricted-vi-methods` and add tests for it. (#16971) (connorshea) - 7d6974d linter: Ignore oxlint directive comments in capitalized-comments (#16989) (Tu Shaokun) - 23ac6b1 linter/plugins: Apply defaults from `meta.schema` to options (#16930) (overlookmotel) - 2f946cf linter/plugins: Error if `defaultOptions` is not JSON-serializable (#16959) (overlookmotel) - d8b8a57 linter/plugins: Freeze whole of merged options (#16958) (overlookmotel) - d446c43 linter: Prevent extra fields from being present on oxlint config file (#16874) (connorshea) - b845871 linter/plugins: Correctly handle object with `__proto__` keys in options merging (#16928) (overlookmotel) - c897794 linter: Fix eslint/sort-imports allowSeparatedGroups not working with single empty line (#16012) (Duc Nghiem Xuan) - 0c347a1 linter/array-type: Handle satisfies expression (#16903) (camc314) ### ⚡ Performance - 70d853c linter: Avoid cloning source text when cloning AST into fixed-size allocator (#17088) (overlookmotel) - 4d389f7 linter: Less bounds checks in `normalize_plugin_name` (#17030) (overlookmotel) - fd8e9c6 linter/plugins: Speed up cloning JSON objects (#16997) (overlookmotel) - d77e22d linter/plugins: Use `DEFAULT_OPTIONS` for rules with empty array as default options (#16913) (overlookmotel) ### 📚 Documentation - 6d053b4 linter: Fix typo in doc comment (#17091) (overlookmotel) - b5f3c91 linter: Document intentional exclusion of ignoreCase option in jsx-no-duplicate-props (#17046) (Copilot) - a0bf5d8 linter: Fix the config option docs for no-inline-comments rule. (#16983) (connorshea) - ca26a11 linter/plugins: Fix typo in doc comment (#16966) (overlookmotel) - 3183bf8 linter/plugins: Fix typo in JSDoc comment (#16900) (overlookmotel) # Oxfmt ### 🚀 Features - 15dfb55 oxfmt: Respect single nearest `.editorconfig` (#17043) (leaysgur) - 8c33ff4 oxfmt: Expose Node.js API: `format(fileName, sourceText, options?)` (#16939) (leaysgur) ### 🐛 Bug Fixes - d340c87 oxfmt: Update api `FormatOptions` type with `& Record<string, unknown>` (#17036) (leaysgur) - 827a256 oxfmt: Place ignorePatterns at bottom of JSON in --migrate prettier (#16926) (Boshen) Co-authored-by: overlookmotel <557937+overlookmotel@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes OOM (Out of Memory) problems when using JS plugins together with the import plugin for cross-module analysis.
Previously, a stopgap solution simply refused to run when >10,000 files were linted with both features enabled.
This PR implements a real fix using a deferred cloning strategy: standard allocators are used for parsing/linting (efficient, dynamic sizing), and the AST is only copied into a fixed-size allocator briefly when calling JS plugins.
This dramatically reduces memory usage from n_files × 6 GiB to n_files × dynamic + thread_count × 6 GiB.