feat: Bun Runtime Support. Migrate from Piscina to Tinypool for worker thread management#716
feat: Bun Runtime Support. Migrate from Piscina to Tinypool for worker thread management#716
Conversation
Replace Piscina with Tinypool to significantly reduce bundle size (800KB → 38KB) while maintaining full API compatibility and performance. This migration affects all worker thread pools used in file processing, security checks, and metrics calculations. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Updated all references throughout the codebase: - Import statements in 5 core modules - Function calls in file processing, metrics, and security modules - Test mocks and descriptions - Maintained backward compatibility and functionality 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Add Bun test and build jobs to CI pipeline to ensure compatibility across different JavaScript runtimes. This validates the Tinypool migration works correctly with both Node.js and Bun environments. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThe changes replace the Piscina worker pool library with Tinypool throughout the codebase. All concurrency initialization functions, imports, and related tests are updated accordingly. Additionally, the CI workflow is extended to include jobs for testing and building with the Bun JavaScript runtime on multiple operating systems and Bun versions. Changes
Sequence Diagram(s)sequenceDiagram
participant CI as GitHub Actions CI
participant Bun as Bun Runtime
participant Node as Node.js Runtime
participant Repo as Code Repository
CI->>Repo: Checkout code
alt For each OS and Bun version
CI->>Bun: Setup Bun runtime
CI->>Bun: bun install
CI->>Bun: bun run test
CI->>Bun: bun run build
CI->>Bun: bun install --production
CI->>Bun: node bin/repomix.cjs (run CLI)
CI->>CI: Upload build artifact
end
alt Node.js jobs (existing)
CI->>Node: Run tests and build as before
end
sequenceDiagram
participant Main as Main Thread
participant Tinypool as Tinypool Worker Pool
participant Worker as Worker Thread
Main->>Tinypool: initWorker(numOfTasks, workerPath)
loop For each task
Main->>Tinypool: run(task)
Tinypool->>Worker: Execute workerPath with task
Worker-->>Tinypool: Return result
Tinypool-->>Main: Return result
end
✨ Finishing Touches🧪 Generate Unit Tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
|
Claude finished @yamadashy's task —— View job PR Review: Piscina to Tinypool MigrationReview Todo List:
Overall Assessment: ✅ APPROVEDThis is an excellent migration that demonstrates strong architectural principles and attention to detail. The changes are well-executed with proper abstraction, comprehensive testing, and significant bundle size benefits. Key Strengths🎯 Excellent Abstraction LayerThe migration is brilliantly abstracted through 📦 Significant Bundle Size Reduction
🔧 Consistent ImplementationAll modules ( const initTaskRunner = (numOfTasks: number) => {
const pool = initWorker(numOfTasks, workerPath);
return (task: Task) => pool.run(task);
};🧪 Comprehensive Testing Strategy
Technical ReviewConfiguration CompatibilityThe Tinypool configuration in
Error HandlingProper error handling maintained across all modules with consistent patterns:
API DesignThe function rename from
CI/CD EnhancementsBun IntegrationThe CI improvements are well-architected:
Job StructureThe parallel job structure is efficient and comprehensive:
Minor SuggestionsDocumentationConsider adding a brief comment in Type SafetyThe current implementation is already well-typed, but consider adding a generic type parameter to export const initWorker = <T, R>(
numOfTasks: number,
workerPath: string
): Tinypool<T, R> => { ... }Performance MonitoringConsider adding a simple performance comparison comment in the PR description showing before/after metrics for typical workloads. Security Review✅ No security concerns identified
Final RecommendationThis PR represents a high-quality migration that:
Recommendation: ✅ APPROVE and MERGE The migration is production-ready and will provide immediate benefits to users while maintaining the same functionality and performance characteristics. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @yamadashy, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request significantly optimizes the application's dependency footprint by replacing the Piscina worker thread management library with Tinypool. This strategic swap not only slashes the bundle size by over 95% but also maintains the existing API and parallel processing performance, ensuring a lightweight yet powerful solution. Furthermore, the change improves the codebase's abstraction for worker pool management and expands CI coverage to include Bun runtime compatibility.
Highlights
- Core Worker Thread Migration: The primary change in this pull request is the migration of the worker thread pool implementation from the
Piscinalibrary toTinypool. - Significant Bundle Size Reduction: This migration has resulted in a dramatic reduction of the application's bundle size, shrinking it from 800KB to just 38KB, representing a 95% decrease.
- API Compatibility and Performance Preservation: The new
Tinypoolimplementation maintains full API compatibility with existing worker thread functionality and ensures no degradation in parallel processing performance. - Improved Abstraction: The worker pool initialization function has been renamed from
initPiscinatoinitWorker, providing a more abstract and future-proof API for managing worker threads. - Enhanced Cross-Runtime Compatibility: New CI jobs have been added to specifically test compatibility across Node.js and Bun runtimes, ensuring broader support and stability.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
Major refactoring to replace Piscina with Tinypool for worker thread management, achieving 95% bundle size reduction (800KB → 38KB) while maintaining API compatibility.
- Renamed
initPiscinatoinitWorkerinsrc/shared/processConcurrency.tsfor better abstraction - Added comprehensive Bun runtime testing across 3 OS platforms in
.github/workflows/ci.yml - Updated worker thread implementation in 5 core modules (fileCollect, fileProcess, securityCheck, calculateAllFileMetrics, calculateOutputMetrics)
- Maintained all 685 unit tests passing and existing performance benchmarks
- Successfully validated cross-runtime compatibility between Node.js and Bun
11 files reviewed, 1 comment
Edit PR Review Bot Settings | Greptile
Deploying repomix with
|
| Latest commit: |
4e6dfc2
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://354969ab.repomix.pages.dev |
| Branch Preview URL: | https://feat-tinypool.repomix.pages.dev |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #716 +/- ##
=======================================
Coverage 90.36% 90.36%
=======================================
Files 99 99
Lines 5561 5561
Branches 1186 1186
=======================================
Hits 5025 5025
Misses 536 536 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Response to Review Comments@greptile-apps[bot] - Bun CLI execution suggestion ✅Great suggestion! You're absolutely right that running Node.js CLI commands after Bun build may miss Bun-specific runtime issues. I've updated the workflow to use Changes made:
@copilot - Test expectations completeness ✅The test is already asserting all the necessary configuration parameters:
The test coverage is complete and properly validates the entire configuration object passed to Tinypool constructor. @gemini-code-assist[bot] - Node.js version requirement ✅The Node.js version bump from 18.0.0 to 20.0.0 was actually from a previous unrelated change. However, I verified that Tinypool officially requires Node.js 18+ as of 2024, so this version requirement is appropriate and not caused by the Tinypool migration. The version requirement aligns with Tinypool's ESM-only design and modern Node.js features. @gemini-code-assist[bot] - WorkerPool abstraction 🤔I added a This provides documentation value and sets up for future abstraction while keeping the migration focused and safe. All tests pass (685/685) and TypeScript compilation is clean. Thanks for the thorough review! 🙏 |
There was a problem hiding this comment.
I see you want me to help update the build-and-run-bun job in the CI workflow to use bun instead of node for running the CLI commands. This change would align with testing the actual Bun runtime execution. Here's how we can modify those lines with the exact same indentation:
- run: bun bin/repomix.cjs
- run: bun bin/repomix.cjs --version
- run: bun bin/repomix.cjs --help
This maintains the exact same indentation as the original (6 spaces before each line) while switching from node to bun for executing the CLI commands. The change ensures we're testing the actual Bun runtime execution rather than Node.js execution of the bundled CJS file.
The changes should verify both basic startup and commonly used CLI flags (--version and --help) under the Bun runtime.
Note that we're only modifying the three lines that execute repomix.cjs - all other parts of the job configuration remain unchanged to maintain the integrity of the build and artifact upload steps.
|
Thanks for the detailed response and for addressing the points raised! Your clarifications are very helpful. Regarding the Node.js version requirement, it's good to know that the bump was unrelated to the Tinypool migration and that the current version aligns with Tinypool's requirements. This makes perfect sense. For the |
Address PR review feedback by switching to native Bun CLI execution in CI and adding WorkerPool interface for better abstraction while maintaining type safety. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Update setup-bun action hash for security and latest features. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Consolidate build process by removing separate clean script and add build-bun alias for consistency with Bun CI workflows. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Reduce CI complexity by testing only the latest Bun version while maintaining comprehensive OS coverage for optimal testing efficiency. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Ensure consistency by applying the same Bun version matrix simplification to the build-and-run-bun job, reducing from 6 to 3 CI combinations. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Pull Request Overview
This PR replaces the existing Piscina-based worker pool with Tinypool, renames the initializer to initWorker, and updates all related imports, tests, docs, and CI to use the new implementation.
- Swap
piscinafortinypooland renameinitPiscina→initWorker - Update all core modules and tests to import and call
initWorker - Adjust documentation and CI to reflect the new dependency and add Bun support
Reviewed Changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/shared/processConcurrency.ts | Replaced Piscina import and initializer with Tinypool |
| tests/shared/processConcurrency.test.ts | Updated test to mock and assert Tinypool initialization |
| src/core/security/securityCheck.ts | Switched pool initializer import to initWorker |
| tests/core/security/securityCheck.test.ts | Updated mock from initPiscina to initWorker |
| src/core/metrics/*.ts | Replaced Piscina with initWorker in metrics modules |
| src/core/file/*.ts | Switched file-processing modules to use initWorker |
| package.json | Removed piscina dependency and added tinypool |
| repomix-instruction.md | Updated documentation to reference Tinypool |
| .github/workflows/ci.yml | Added Bun test and build matrix |
yamadashy
changed the title
This PR migrates the worker thread pool implementation from Piscina to Tinypool, achieving significant bundle size reduction while maintaining full API compatibility and performance.
Related:
bunx#591Summary
Key Changes
Core Migration
piscinadependency withtinypool(^1.1.1)initTinypooltoinitWorkerfor better abstractionAffected Components
CI Enhancements
Benefits
Testing
Checklist
npm run testnpm run lint🤖 Generated with Claude Code