fix(tasks): restore parallel starts with poetry via list_paths cache and stable exec-env cache#6237
fix(tasks): restore parallel starts with poetry via list_paths cache and stable exec-env cache#6237
Conversation
Hyperfine Performance
|
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2025.9.6 x -- echo |
18.3 ± 0.3 | 17.9 | 20.9 | 1.00 |
mise x -- echo |
18.9 ± 0.2 | 18.5 | 19.7 | 1.03 ± 0.02 |
mise env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2025.9.6 env |
17.8 ± 0.3 | 17.3 | 20.0 | 1.00 |
mise env |
18.4 ± 0.4 | 17.8 | 21.8 | 1.03 ± 0.03 |
mise hook-env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2025.9.6 hook-env |
17.4 ± 0.2 | 17.0 | 18.3 | 1.00 |
mise hook-env |
18.0 ± 0.3 | 17.6 | 20.4 | 1.03 ± 0.02 |
mise ls
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2025.9.6 ls |
15.9 ± 0.2 | 15.5 | 16.9 | 1.00 |
mise ls |
16.0 ± 0.2 | 15.5 | 17.5 | 1.01 ± 0.02 |
xtasks/test/perf
| Command | mise-2025.9.6 | mise | Variance |
|---|---|---|---|
| install (cached) | 163ms | ✅ 102ms | +59% |
| ls (cached) | 62ms | 62ms | +0% |
| bin-paths (cached) | 65ms | 68ms | -4% |
| task-ls (cached) | 465ms | 463ms | +0% |
✅ Performance improvement: install cached is 59%
There was a problem hiding this comment.
Pull Request Overview
This PR fixes parallel task execution with poetry by implementing a comprehensive caching strategy to prevent serialization of concurrent tasks. The issue was that tasks were recomputing environment and path information repeatedly, causing them to run sequentially instead of in parallel.
- Introduces a global
LIST_PATHS_CACHEto cache toolset path computations across identical configurations - Removes spurious cache invalidations from poetry exec-env cache by eliminating the
dirs::DATAfreshness check - Adds detailed tracing to identify performance bottlenecks in task execution
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/toolset/mod.rs | Implements global LIST_PATHS_CACHE with stable cache keys and adds tracing for list_bin_paths operations |
| src/cli/run.rs | Adds comprehensive timing traces for task execution phases including semaphore acquisition and environment rendering |
| src/backend/external_plugin_cache.rs | Refactors cache managers to use Arc for thread safety and adds timing traces for cache operations |
| src/backend/asdf.rs | Removes dirs::DATA freshness check and adds exec_env timing traces |
| e2e/tasks/test_task_parallel_execution | Enhances test with poetry tooling and multiple runs for reliability verification |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
|
||
| // Cache Toolset::list_paths results across identical toolsets within a process. | ||
| // Keyed by project_root plus sorted list of backend@version pairs currently installed. | ||
| static LIST_PATHS_CACHE: Lazy<DashMap<String, Vec<PathBuf>>> = Lazy::new(DashMap::new); |
There was a problem hiding this comment.
The cache key construction could lead to collisions if project paths contain the delimiter '|'. Consider using a more robust serialization approach like JSON or a structured hash.
src/backend/external_plugin_cache.rs
Outdated
| list_bin_paths: RwLock<HashMap<ToolRequest, Arc<CacheManager<Vec<String>>>>>, | ||
| exec_env: RwLock<HashMap<ToolRequest, Arc<CacheManager<EnvMap>>>>, |
There was a problem hiding this comment.
The nested Arc<CacheManager> pattern creates unnecessary indirection. Consider using DashMap directly for better performance and simpler code structure.
- Add LIST_PATHS_CACHE to cache toolset list_paths results - Use Arc<CacheManager> in ExternalPluginCache to release locks earlier - Remove dirs::DATA fresh check that caused spurious cache invalidations - Add trace logs for debugging task startup timing - Update test to demonstrate the bug with poetry installed
### 🐛 Bug Fixes - **(env)** allow mixed map for env._.file by @risu729 in [#6148](#6148) - **(tasks)** restore parallel starts with poetry via list_paths cache and stable exec-env cache by @jdx in [#6237](#6237) - add 'unknown' to the list of OS patterns by @efussi in [#6235](#6235) - propagate errors from backend installs by @jdx in [#6236](#6236) ### 📦️ Dependency Updates - update taiki-e/install-action digest to 0c5db7f by @renovate[bot] in [#6244](#6244) - update golang docker tag to v1.25.1 by @renovate[bot] in [#6247](#6247) - update dependency vitepress to v1.6.4 by @renovate[bot] in [#6246](#6246) ### New Contributors - @efussi made their first contribution in [#6235](#6235) Co-authored-by: mise-en-dev <release@mise.jdx.dev>
## [2025.9.12](https://github.com/jdx/mise/compare/v2025.9.11..v2025.9.12) - 2025-09-16 ### 🐛 Bug Fixes - **(ci)** properly exclude aqua-registry files from hk linting by @jdx in [42d7758](jdx/mise@42d7758) ### Chore - **(release)** embed aqua-registry under crate and publish like vfox by @jdx in [#6306](jdx/mise#6306) - ignore aqua-registry assets from prettier by @jdx in [047d77b](jdx/mise@047d77b) - disable "useless cat" shellcheck by @jdx in [a6def59](jdx/mise@a6def59) ## [2025.9.11](https://github.com/jdx/mise/compare/v2025.9.10..v2025.9.11) - 2025-09-16 ### 📦 Registry - indicate aws-cli is v2 by @jayvdb in [#6300](jdx/mise#6300) ### 🚀 Features - **(ci)** run all registry tools when workflow_dispatch is triggered by @jdx in [#6295](jdx/mise#6295) - **(cli)** handle non-existent working directories gracefully by @jdx in [#6304](jdx/mise#6304) - **(set)** add -E/--env flag to mise set command by @jdx in [#6291](jdx/mise#6291) - **(tasks)** make auto outputs default by @risu729 in [#6137](jdx/mise#6137) ### 🐛 Bug Fixes - align crate versions to fix release-plz script by @jdx in [5a464e9](jdx/mise@5a464e9) ### 🚜 Refactor - **(aqua)** extract aqua registry into internal subcrate by @jdx in [#6293](jdx/mise#6293) ### 📚 Documentation - fix mkdir paths by @risu729 in [#6298](jdx/mise#6298) - fix rust profile default by @risu729 in [#6305](jdx/mise#6305) ### Chore - **(aqua-registry)** remove unused aqua-registry files by @jdx in [#6294](jdx/mise#6294) - **(release)** make release-plz idempotent for existing crate versions by @jdx in [dbdfd6a](jdx/mise@dbdfd6a) - **(release)** skip publishing mise when aqua-registry is a path dep by @jdx in [47efffd](jdx/mise@47efffd) - keep aqua-registry LICENSE file by @risu729 in [#6297](jdx/mise#6297) ### New Contributors - @jayvdb made their first contribution in [#6300](jdx/mise#6300) ## [2025.9.10](https://github.com/jdx/mise/compare/v2025.9.9..v2025.9.10) - 2025-09-13 ### 📦 Registry - use asdf to install semver-tool by @jylenhof in [#6233](jdx/mise#6233) ### 🐛 Bug Fixes - **(tool-stub)** detect binary names from single-file downloads by @jdx in [#6281](jdx/mise#6281) ### 🚜 Refactor - allow any collection types in deserialize_arr by @risu729 in [#6282](jdx/mise#6282) - use deserialize_arr for task runs by @risu729 in [#6253](jdx/mise#6253) ### 📚 Documentation - **(getting-started)** add backends step with diagram, github example, env vars and simple tasks by @jdx in [#6288](jdx/mise#6288) - simplify OS detection in tool plugin development by @jdx in [#6287](jdx/mise#6287) - update backend plugin template references by @jdx in [942f5eb](jdx/mise@942f5eb) ### 📦️ Dependency Updates - update rust crate chrono to v0.4.42 by @renovate[bot] in [#6274](jdx/mise#6274) - update taiki-e/install-action digest to 0154864 by @renovate[bot] in [#6273](jdx/mise#6273) ### Chore - **(schema)** fix schema for subtasks by @risu729 in [#6289](jdx/mise#6289) - update render:schema task by @risu729 in [#6275](jdx/mise#6275) ## [2025.9.9](https://github.com/jdx/mise/compare/v2025.9.8..v2025.9.9) - 2025-09-11 ### 🐛 Bug Fixes - **(backend)** make HTTP installs atomic and serialize with cache lock by @jdx in [#6259](jdx/mise#6259) - **(env)** allow nested env._.path directives by @risu729 in [#6160](jdx/mise#6160) - **(env)** disallow nested env objects by @risu729 in [#6268](jdx/mise#6268) - **(schema)** allow nested arrays in task.depends by @risu729 in [#6265](jdx/mise#6265) - **(task)** resolve jobs=1 hang and keep-order panic; improve Ctrl-C handling by @jdx in [#6264](jdx/mise#6264) - **(tasks)** stop CLI group after first failure without --continue-on-error by @jdx in [#6270](jdx/mise#6270) ### 📚 Documentation - fixed toml issues in URL replacements settings documentation by @ThomasSteinbach in [#6269](jdx/mise#6269) ### Chore - **(schema)** strict oneOf branches and DRY env_directive in schemas by @jdx in [#6271](jdx/mise#6271) - add schema linter by @risu729 in [#6267](jdx/mise#6267) ## [2025.9.8](https://github.com/jdx/mise/compare/v2025.9.7..v2025.9.8) - 2025-09-10 ### 🐛 Bug Fixes - **(tasks)** prevent hang when task fails in sequence by @jdx in [#6260](jdx/mise#6260) - **(version)** fetch mise version if cached version is older than the current by @risu729 in [#6252](jdx/mise#6252) ### 📦️ Dependency Updates - update rhysd/action-setup-vim action to v1.4.3 by @renovate[bot] in [#6249](jdx/mise#6249) ## [2025.9.7](https://github.com/jdx/mise/compare/v2025.9.6..v2025.9.7) - 2025-09-09 ### 🐛 Bug Fixes - **(env)** allow mixed map for env._.file by @risu729 in [#6148](jdx/mise#6148) - **(tasks)** restore parallel starts with poetry via list_paths cache and stable exec-env cache by @jdx in [#6237](jdx/mise#6237) - add 'unknown' to the list of OS patterns by @efussi in [#6235](jdx/mise#6235) - propagate errors from backend installs by @jdx in [#6236](jdx/mise#6236) ### 📦️ Dependency Updates - update taiki-e/install-action digest to 0c5db7f by @renovate[bot] in [#6244](jdx/mise#6244) - update golang docker tag to v1.25.1 by @renovate[bot] in [#6247](jdx/mise#6247) - update dependency vitepress to v1.6.4 by @renovate[bot] in [#6246](jdx/mise#6246) ### New Contributors - @efussi made their first contribution in [#6235](jdx/mise#6235)
Summary
Minimal fix for parallel task execution with poetry. Tasks now start concurrently again (~4ms spread vs ~6s).
Problem
With poetry installed, parallel tasks were serializing due to:
Solution
Testing
Updated test_task_parallel_execution to demonstrate the bug with poetry installed. Test now passes reliably with all tasks starting within milliseconds.
Verification