perf(core): Pre-initialize metrics worker pool to overlap tiktoken WASM loading#1302
perf(core): Pre-initialize metrics worker pool to overlap tiktoken WASM loading#1302
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR introduces a metrics task runner warm-up system that initializes token-counting tasks before metrics calculation, refactors dependency injection in metrics handling, optimizes file output writes to run concurrently, moves binary-extension checking earlier in file reading, and converts top-level Changes
Possibly Related PRs
Estimated Code Review Effort🎯 4 (Complex) | ⏱️ ~60 minutes 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
⚡ Performance Benchmark
Details
History
|
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request focuses on significant performance enhancements across several core components of the application. By strategically pre-initializing the metrics worker pool, deferring non-essential module loading, optimizing file system interactions, and parallelizing I/O operations, the changes aim to reduce overall execution time and improve efficiency, particularly in scenarios involving large codebases or complex configurations. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1302 +/- ##
==========================================
- Coverage 87.09% 87.09% -0.01%
==========================================
Files 115 115
Lines 4341 4354 +13
Branches 1009 1010 +1
==========================================
+ Hits 3781 3792 +11
- Misses 560 562 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Code Review
This pull request introduces several performance optimizations. Key changes include lazy-loading the jiti module, optimizing binary file detection by checking extensions before I/O operations, and parallelizing the writing of split output files using Promise.all. The most significant change is the introduction of a pre-initialized metrics task runner with a warm-up mechanism to overlap tiktoken WASM loading with other pipeline stages, aiming to improve overall performance. A review comment highlights an inefficiency where the metrics task runner is unnecessarily initialized and cleaned up in the skillGenerate path, suggesting conditional initialization to avoid this overhead.
src/core/packager.ts
Outdated
| await warmupPromise; | ||
| await metricsTaskRunner.cleanup(); |
There was a problem hiding this comment.
In the skillGenerate path, the metricsTaskRunner is warmed up and then immediately cleaned up here without being used. This introduces unnecessary overhead from initializing and tearing down the worker pool.
To optimize this, consider making the creation of metricsTaskRunner on line 97 conditional, so it's only initialized when config.skillGenerate is not set. This would avoid the wasted work in this code path while preserving the pre-warming optimization for the main path.
Deploying repomix with
|
| Latest commit: |
67f6bcb
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://172bbdee.repomix.pages.dev |
| Branch Preview URL: | https://perf-pre-init-metrics-worker.repomix.pages.dev |
Code ReviewNice set of focused performance optimizations! The benchmark numbers look solid. A few items worth addressing before merge: 1. Worker pool leak on pipeline errors (Recommended)
Suggested fix: Wrap the section from pool creation through cleanup in a Exampleconst metricsTaskRunner = deps.createMetricsTaskRunner(allFilePaths.length);
const warmupPromise = metricsTaskRunner.run({ content: '', encoding: config.tokenCount.encoding });
try {
// ... existing pipeline stages ...
} finally {
await metricsTaskRunner.cleanup();
}2. Warmup is wasted in the
|
…SM loading Pipeline-level optimizations that produce measurable end-to-end improvement: - Pre-initialize metrics worker pool during file collection phase so tiktoken WASM loading overlaps with security checks and file processing. First token count task dropped from 381ms to 22ms (worker already warmed). - Lazy-load Jiti via dynamic import — only loaded when TS/JS config files are detected, saving startup time for the common JSON/default config path. - Fix O(n²) file path re-grouping in packager by using Map + Set for O(1) membership checks instead of .find() + .includes(). - Move binary extension check before fs.stat in fileRead to skip unnecessary stat syscalls for binary files. - Parallelize split output file writes with Promise.all instead of sequential for-loop. Benchmark (15 runs each, median ± IQR, packing repomix repo ~1000 files): main branch: 3515ms (P25: 3443, P75: 3581) perf branch: 3318ms (P25: 3215, P75: 3383) Improvement: -197ms (-5.6%) Pipeline stage breakdown (instrumented): - Metrics first-file init: 381ms → 22ms (worker pre-warmed) - Total metrics stage: 793ms → ~450ms All 1096 tests pass. Lint clean. https://claude.ai/code/session_01JoNjFe7S2roMfHfNcw6bso
43c74d0 to
4d2bbcf
Compare
PR Review —
|
Wrap the metrics worker pool lifecycle in try/finally to prevent worker thread leaks when any pipeline stage throws. Also suppress unhandled promise rejection on the warmup task with .catch() since it floats between creation and await. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Code ReviewNice set of performance optimizations — the pre-warm strategy, O(n) regrouping, lazy jiti, and binary check reorder are all solid improvements. A few findings worth addressing: 1. Skill path does not use the pre-warmed worker poolIn This means the entire warmup optimization is wasted on the skill generation path, and two worker pools are allocated instead of one. Suggestion: Pass 2. Misleading comment on
|
Incremental Code ReviewPrevious reviews covered the main items well. The 1.
|
| Item | Status |
|---|---|
| Worker pool leak on error paths | ✅ Fixed (try/finally added) |
.catch(() => 0) swallows errors permanently (comment says "errors surface when awaited" but they don't) |
⏳ Still present — consider at minimum fixing the comment |
| Warmup wasted on skill path (two pools created) | ⏳ Still present — acknowledged as acceptable trade-off |
| Test coverage gaps (cleanup-on-error, warmup failure, skill path) | ⏳ Still present |
overrideDeps: Partial<> DI pattern deviation |
⏳ Noted, not blocking |
Overall the PR is in good shape. The remaining items are low severity — the .catch(() => 0) comment fix would be the most impactful small change.
🤖 Generated with Claude Code
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
59f425e to
67f6bcb
Compare
Code Review —
|
Summary
Cherry-picked from #1295. Pre-initializes the metrics worker pool during the file collection phase so that tiktoken WASM loading overlaps with security checks and file processing instead of blocking the metrics stage.
Changes
calculateMetrics.ts,packager.ts): ExtractcreateMetricsTaskRunnerfactory that starts the worker pool and fires a warm-up task to trigger tiktoken WASM initialization in the background. By the time metrics calculation starts, the worker is already warm.configLoad.ts): DefercreateJitiimport to when TypeScript config loading is actually needed.fileRead.ts): Check binary extension (no I/O) before callingfs.stat(I/O), avoiding unnecessary stat calls for binary files.packager.ts): Replace O(n²) array spread in file path re-grouping with Map-based O(n) approach.produceOutput.ts): Write multiple split output files concurrently withPromise.allinstead of sequentially.Benchmark (from automated perf suite)
Checklist
npm run testnpm run lint