Skip to content

set uv cache dir to .platformio/.cache folder for easy clean up#386

Merged
Jason2866 merged 4 commits into
developfrom
uv_cache
Feb 10, 2026
Merged

set uv cache dir to .platformio/.cache folder for easy clean up#386
Jason2866 merged 4 commits into
developfrom
uv_cache

Conversation

@Jason2866
Copy link
Copy Markdown

@Jason2866 Jason2866 commented Feb 10, 2026

Description:

to try to recover better from broken install attempts when Github is not reachable and broken? leftovers are stored in uv cache.

Checklist:

  • The pull request is done against the latest develop branch
  • Only relevant files were touched
  • Only one feature/fix was added per PR, more changes are allowed when changing boards.json
  • I accept the CLA

Summary by CodeRabbit

  • New Features
    • Added support for optional UV cache directory configuration, enabling customizable caching for Python dependency and esptool installations to improve build performance.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Feb 10, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds optional UV_CACHE_DIR support to Python dependency and esptool installation flows. The uv_cache_dir parameter is propagated through install_python_deps, install_esptool, and _install_esptool_from_tl_install functions, with UV_CACHE_DIR environment variable set in all uv subprocess calls and cache directory defaulting to build_dir/.uv-cache.

Changes

Cohort / File(s) Summary
UV Cache Directory Support
builder/penv_setup.py
Added uv_cache_dir optional parameter to install_python_deps, install_esptool, and _install_esptool_from_tl_install. Propagates UV_CACHE_DIR environment variable through uv subprocess calls. Computes default cache location at build_dir/.uv-cache in _setup_python_environment_core.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

Poem

🐰 Down the build-dir rabbit hole,
UV cache is now my goal,
Esptool and deps align,
In .uv-cache, all things shine!

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Title check ⚠️ Warning The title mentions .platformio/.cache folder but the implementation actually sets cache to build_dir/.uv-cache per the summary. Update the PR title to accurately reflect the implementation: change to something like 'set uv cache dir to build folder for easy cleanup' to match the actual code changes.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch uv_cache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Jason2866
Copy link
Copy Markdown
Author

@coderabbitai full review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Feb 10, 2026

✅ Actions performed

Full review triggered.

@Jason2866
Copy link
Copy Markdown
Author

@coderabbitai Are there negative side effects to expect by moving the uv cache folder in the pio build directory?

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Feb 10, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🌐 Web query:

uv cache directory behavior default location shared cache

💡 Result:

For Astral’s uv (Python package manager), the cache is global (per-user) by default and is therefore shared across projects/environments unless you override it. [1]

Default cache location

uv chooses the cache directory in this order: [1]

  1. Temporary cache if you pass --no-cache (still used within that single invocation). [1]
  2. The directory set via --cache-dir, UV_CACHE_DIR, or tool.uv.cache-dir. [1][2][3]
  3. OS default:
    • macOS/Linux: $XDG_CACHE_HOME/uv or $HOME/.cache/uv
    • Windows: %LOCALAPPDATA%\uv\cache [3]

To print the active path, run: uv cache dir. [3]

“Shared cache” behavior (what that means in practice)

  • The cache is designed to be safe for concurrent uv commands (thread-safe/append-only), though you shouldn’t modify/delete cache contents by hand. [1]
  • For speed, uv may link files from the cache into environments; if the cache is on a different filesystem than the target environment, it may fall back to slower copies. [1]
  • Cache data is bucketed and versioned, so multiple uv versions can share a cache directory, but if a bucket version changes you may see duplicate entries for that bucket. [1]

Quick controls

  • Override location: UV_CACHE_DIR=/path/to/cache (or --cache-dir). [2][3]
  • Clear/prune:
    • uv cache clean (wipe all) [1]
    • uv cache prune (remove unused) [1]
  • Prefer --refresh over --no-cache in most cases (updates cache but doesn’t read from it). [1]

Sources: [1] [2] [3]


@Jason2866: Yes, there are some trade-offs to consider when moving the uv cache to the PIO build directory:

Negative side effects:

  1. Loss of cache reuse across projects: By default, uv uses a global shared cache (e.g., ~/.cache/uv on Linux/macOS). This means packages are downloaded once and reused across all your ESP32 projects. With per-project caches, each project will maintain its own cache, duplicating packages.

  2. Increased disk space usage: If you have multiple ESP32 projects, each will store its own copy of the same packages (esptool, dependencies, etc.) in their respective build directories.

  3. Slower initial builds: When starting a new project or after cleaning the build directory, the cache must be rebuilt from scratch instead of reusing existing cached packages.

  4. Cache lost on clean: PlatformIO users often clean build directories to resolve issues. With this change, cleaning the build directory also wipes the uv cache, requiring a full re-download on the next build.

  5. Build directory bloat: The .uv-cache folder adds to the build directory size, which some users track in version control or sync across machines.

Positive trade-offs (your intended benefits):

  1. Isolated cache corruption: A broken cache in one project won't affect others
  2. Easy recovery: Simply cleaning the build directory resolves cache issues
  3. Self-contained builds: Each project is independent and doesn't rely on global state

Recommendation:

The trade-off makes sense for your use case (recovering from GitHub connectivity issues), but users with multiple projects will see increased disk usage and slower clean builds. You might consider:

  • Documenting this behavior change in release notes
  • Potentially making it configurable via an environment variable if users want to opt back to shared cache

Does this align with your expectations, or would you like to reconsider the approach?


🧠 Learnings used
Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 268
File: builder/penv_setup.py:87-96
Timestamp: 2025-08-26T14:52:29.044Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 prefers to trust system executables when falling back to PATH-based command resolution (like "uv"), rather than implementing additional validation checks for security concerns.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 281
File: builder/frameworks/component_manager.py:1047-1054
Timestamp: 2025-09-04T15:27:18.112Z
Learning: In the pioarduino/platform-espressif32 project, Jason2866 prefers not to add guard checks for missing arduino_libs_mcu in backup operations, preferring to let the operation proceed rather than silently skip when the directory might be missing.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 0
File: :0-0
Timestamp: 2025-09-23T12:35:35.508Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 has explicitly instructed to NEVER create issues or PRs outside of his repositories (pioarduino/*). All issue and PR creation must be strictly limited to his own repositories only, never in upstream or other repositories.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 0
File: :0-0
Timestamp: 2025-09-23T12:35:35.508Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 has explicitly instructed to NEVER create issues or PRs outside of his repositories (pioarduino/*). All issue and PR creation must be strictly limited to his own repositories only, never in upstream or other repositories.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 371
File: builder/spiffsgen.py:620-629
Timestamp: 2025-12-27T16:33:38.710Z
Learning: In the pioarduino/platform-espressif32 project, when extracting SPIFFS files in builder/spiffsgen.py, Jason2866 prefers to check if rel_path is empty after lstrip('/') and skip such files with a warning message, rather than attempting to handle empty parent directories in os.makedirs calls. This treats empty paths as invalid/corrupted data.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 263
File: platform.py:80-84
Timestamp: 2025-08-20T14:13:13.842Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 prefers to always override IDF_TOOLS_PATH environment variable with PlatformIO's core directory, rather than respecting any existing user-defined IDF_TOOLS_PATH values. This ensures consistent ESP32 tool management within PlatformIO.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 296
File: builder/main.py:49-50
Timestamp: 2025-09-22T14:47:08.855Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 prefers that critical setup operations like platform.setup_python_env should crash immediately if they fail rather than having fallback mechanisms, as this ensures problems are detected early and clearly rather than being masked.

@Jason2866
Copy link
Copy Markdown
Author

@coderabbitai Do analyze the effects now. uv cache folder is moved into folder .platformio/.cache

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Feb 10, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check the actual implementation of uv_cache_dir in the PR branch
git fetch origin pull/386/head:pr-386
git checkout pr-386

# Find where uv_cache_dir is set
rg "uv_cache_dir\s*=" -A 2 -B 2 builder/penv_setup.py

# Also check how UV_CACHE_DIR is used
rg "UV_CACHE_DIR" -A 1 -B 1 builder/penv_setup.py

Length of output: 1375


@Jason2866: Ah, that changes everything! Moving to .platformio/.cache/uv is much better than per-project caching. Here's the updated analysis:

Effects of .platformio/.cache/uv location:

Positive effects:

  1. Cache shared across all PlatformIO projects: All your ESP32 projects (and other PlatformIO projects) still share the same uv cache - no disk duplication
  2. Survives project clean operations: Unlike build directory, this cache persists when users clean individual projects
  3. Isolated recovery: A corrupt cache from GitHub issues only affects PlatformIO builds, not system-wide uv usage
  4. Easy manual cleanup: Users can remove just .platformio/.cache/uv to fix cache issues without affecting their system uv cache
  5. No disk space bloat: Single cache for all PlatformIO work, similar to the system default

⚠️ Minor trade-offs:

  1. Separation from system uv cache: If users run uv outside PlatformIO (e.g., for personal Python projects), those won't share cache with PlatformIO builds - slight disk duplication between the two
  2. Not auto-cleared on PlatformIO updates: The cache persists across PlatformIO version changes (though this is usually desirable)

Comparison summary:

Location Shared across projects? Survives clean? Isolated recovery?
System default (~/.cache/uv) ✅ All projects ✅ Yes ❌ Affects all uv usage
.platformio/.cache/uv (your PR) ✅ All PlatformIO projects ✅ Yes ✅ PlatformIO only
Per-project build dir ❌ No ❌ No ✅ Per-project

Verdict:

This is an excellent middle ground! You get:

  • All the benefits of shared caching (speed, disk efficiency)
  • Isolation for recovery scenarios (your original goal)
  • Persistence across project operations

The only users who might see slight disk increase are those who actively use uv for non-PlatformIO projects, but that's a reasonable trade-off for the isolation benefits you're targeting.

This approach resolves my earlier concerns - the implementation is well-designed! 🎯


🧠 Learnings used
Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 268
File: builder/penv_setup.py:87-96
Timestamp: 2025-08-26T14:52:29.044Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 prefers to trust system executables when falling back to PATH-based command resolution (like "uv"), rather than implementing additional validation checks for security concerns.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 281
File: builder/frameworks/component_manager.py:1047-1054
Timestamp: 2025-09-04T15:27:18.112Z
Learning: In the pioarduino/platform-espressif32 project, Jason2866 prefers not to add guard checks for missing arduino_libs_mcu in backup operations, preferring to let the operation proceed rather than silently skip when the directory might be missing.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 263
File: platform.py:80-84
Timestamp: 2025-08-20T14:13:13.842Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 prefers to always override IDF_TOOLS_PATH environment variable with PlatformIO's core directory, rather than respecting any existing user-defined IDF_TOOLS_PATH values. This ensures consistent ESP32 tool management within PlatformIO.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 371
File: builder/spiffsgen.py:620-629
Timestamp: 2025-12-27T16:33:38.710Z
Learning: In the pioarduino/platform-espressif32 project, when extracting SPIFFS files in builder/spiffsgen.py, Jason2866 prefers to check if rel_path is empty after lstrip('/') and skip such files with a warning message, rather than attempting to handle empty parent directories in os.makedirs calls. This treats empty paths as invalid/corrupted data.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 296
File: builder/penv_setup.py:612-647
Timestamp: 2025-09-23T16:55:39.788Z
Learning: In the pioarduino/platform-espressif32 project, Jason2866 prefers no fallback mechanisms in penv setup functions. The penv virtual environment setup must work properly and should crash immediately rather than falling back to host Python environments, to avoid using "unknown env" configurations.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 0
File: :0-0
Timestamp: 2025-09-23T12:35:35.508Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 has explicitly instructed to NEVER create issues or PRs outside of his repositories (pioarduino/*). All issue and PR creation must be strictly limited to his own repositories only, never in upstream or other repositories.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 0
File: :0-0
Timestamp: 2025-09-23T12:35:35.508Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 has explicitly instructed to NEVER create issues or PRs outside of his repositories (pioarduino/*). All issue and PR creation must be strictly limited to his own repositories only, never in upstream or other repositories.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 296
File: builder/main.py:49-50
Timestamp: 2025-09-22T14:47:08.855Z
Learning: For the pioarduino/platform-espressif32 project, Jason2866 prefers that critical setup operations like platform.setup_python_env should crash immediately if they fail rather than having fallback mechanisms, as this ensures problems are detected early and clearly rather than being masked.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 0
File: :0-0
Timestamp: 2025-09-22T10:59:14.425Z
Learning: In the pioarduino/platform-espressif32 project, the penv setup ensures that uv gets installed in the venv (penv), so additional safeguards for uv availability are not needed.

Learnt from: Jason2866
Repo: pioarduino/platform-espressif32 PR: 255
File: builder/penv_setup.py:265-275
Timestamp: 2025-08-10T16:59:15.533Z
Learning: In the ESP32 platform builder's penv_setup.py, the esptool_repo_path used in subprocess calls comes from PlatformIO's internal package management system (platform.get_package_dir()) which is a trusted source. Command injection concerns are not applicable here since this is build infrastructure code with controlled inputs, not user-facing code accepting arbitrary input.

@Jason2866
Copy link
Copy Markdown
Author

@swoboda1337 your 2 cents please. Thx! If you dont see any road blocks i will merge.

@Jason2866 Jason2866 changed the title set uv cache dir to build folder to clean up set uv cache dir to .platformio/.cache folder to clean up Feb 10, 2026
@Jason2866 Jason2866 changed the title set uv cache dir to .platformio/.cache folder to clean up set uv cache dir to .platformio/.cache folder for easy clean up Feb 10, 2026
@swoboda1337
Copy link
Copy Markdown

@swoboda1337 your 2 cents please. Thx! If you dont see any road blocks i will merge.

Yeah I don't think it can hurt. Makes it easier for users to clean everything.

@Jason2866 Jason2866 merged commit e63eda3 into develop Feb 10, 2026
1 check passed
@Jason2866 Jason2866 deleted the uv_cache branch February 10, 2026 21:08
swoboda1337 added a commit to swoboda1337/esphome that referenced this pull request Feb 10, 2026
Upgrade uv from 0.6.14 to 0.10.1 to pick up the fix for HTTP/2
connection reset retry handling (astral-sh/uv#15675, released in
0.8.16). Also set UV_HTTP_RETRIES=10 (default 3) to better handle
transient network errors during PlatformIO penv bootstrap.

Remove the UV_CACHE_DIR override since pioarduino now handles this
upstream (pioarduino/platform-espressif32#386).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants