feat!: rewrite pools without tinypool

[AriPerkkio]

Description

Removes Tinypool and adds new custom internal pool for Vitest only. Some previous thoughts I wrote ~2 months ago about this below.

• Fixes Regression: vitest 3.2 hangs when using workspaces, bail and single threaded #8484
• Fixes Error: Terminating worker thread #8564
• Fixes Channel closed error as of recent version of Vitest #8201 (that one has no reproduction so can't say for sure 🤷)
• Fixes 'Closing rpc while "fetch" was pending' on v4 beta #8649 (need to recheck)
• Fixes 4.0.0-beta post 10 partially breaks for Bun runtime #8650
• Enables Per-file opt-in isolation #8010 if we want it
• Enables Detect hanging async operations #3119, let's implement it later 🥳

Vitest uses tinypool for orchestrating how test files are run. Originally Tinypool was a fork of piscina which implemented node:worker_threads pooling only. Codebase of Piscina is very complex. It has features Vitest users do not need or cannot even use. When ever we add new features to Tinypool, we need to make sure we don't break those unused features that some other dependent package might be using.

Tinypool has some flaws that are making it hard to use in Vitest. When user is running Vitest workspaces with multiple pools, we easily end up in a situation where we allocate too many resources at the same time. For example in a 4 core CPU we may end up spinning 4 threads AND 4 processes at the same time. This is twice the amount of resources we intended to use. Tinypool does not support splitting resourceses across runtime. Also mixing isolate: false with isolate: true is very complex from Vitest's point of view - and not supported properly at the moment.

Originally Tinypool did not support node:child_process runtime. That was added for Vitest in tinylibs/tinypool#65. While this feature works fine, there's still some overhead that the original node:worker_threads runtime required. I think there's room for some optimization and performance gains. Note that adding this feature to Piscina's codebase was unnecessarily difficult.

For the past 1-2 years, whenever we've seen Tinypool related weird errors in Vitest I've thought about rewriting Tinypool completely. However Tinypool is nowadays used by large packages like facebook/docusaurus, jest-light-runner and @rstest/core. Rewriting Tinypool just for Vitest's needs might break other's use cases. It would also be better if we could make changes to Vitest's pool implementation quickly, without any major releases.

We should replace Tinypool usage in Vitest with a custom internal pool implementation that lives in Vitest monorepo.

Requirements

• Basic functionality of Tinypool that Vitest currently uses
• node:worker_threads and node:child_process runtimes
• 🆕 Interface for easily adding new runtimes in future, e.g. browser or even a async function pool
• 🆕 Resource limits (maxWorkers/maxThreads/maxForks) that are shared across whole pool
• 🆕 The runtime should be configurable on Task level, not on Pool level
  • Queue that is not runtime (forks/threads/custom) specific
  • A single task should describe what runtime it uses
  • Instead of new Pool({ runtime: 'worker_threads' }), we do pool = new Pool(); pool.run({ runtime: 'worker_threads' }).
  • Allows mixing forks and threads specific tests parallel
• 🆕 isolate should be configurable on Task level, not on Pool level
  • Pool/Queue sorts tasks based on isolate option, so that workers can be reused between tasks
• Do not use minThreads option at all, no idle workers.

TODO:

• Add Browser pool back
• Check TODOs in code
• Documentation
• Migration guide
• Fix tests
• Add new tests
• Custom pool support - it might work already but needs types and tests
• Verify manually that vmMemory works. Worker should be recycled (e.g. pid changes) after limit is reached

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

• It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
• Ideally, include a test that fails without this PR but passes with it.
• Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.
• Please check Allow edits by maintainers to make review process faster. Note that this option is not available for repositories that are owned by Github organizations.

Tests

• Run the tests with pnpm test:ci.

Documentation

• If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

• Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

[netlify]

✅ Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	d798468
🔍 Latest deploy log	https://app.netlify.com/projects/vitest-dev/deploys/68f712dbf1e32f00082c869a
😎 Deploy Preview	https://deploy-preview-8705--vitest-dev.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[sheremet-va]

/ecosystem-ci run

[vitest-ecosystem-ci]

📝 Ran ecosystem CI: Open

suite	result
effect	⏹️ cancelled
elk	❌ failure
vitest-browser-examples	✅ success
lerna-lite	✅ success
vitest-coverage-large	⏹️ cancelled
vitest-sonar-reporter	✅ success
sveltejs-cli	✅ success
vue	❌ failure
nuxt	✅ success
zustand	✅ success
aria-live-capture	❌ failure
nuxt-test-utils	❌ failure
vitest-in-webcontainer	❌ failure
vitest-benchmark-large	⏹️ cancelled
vitest-vscode	❌ failure
vite	❌ failure
webcontainer-test	❌ failure
vitest-reporters-large	✅ success