Skip to content

Share downloaded URL wheels across hash fragments - #21288

Draft
charliermarsh wants to merge 7 commits into
mainfrom
charlie/normalize-wheel-cache-hashes
Draft

charliermarsh wants to merge 7 commits into
mainfrom
charlie/normalize-wheel-cache-hashes

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Aug 24, 2026 •

Copy link
Copy Markdown
Member

Summary

We omit known hash fields from CanonicalUrl, preserving source selectors and unknown fragments, and index downloaded URL wheels by their canonical URL and computed hashes. This lets uv sync --offline reuse a wheel downloaded from a fragment-hashed URL when the lockfile stores its URL and hashes separately.

Keep HTTP metadata, source builds, and in-flight downloads scoped to their original hash declarations. Sharing checks the complete hash policy and reads metadata from the selected archive, so a newer revision cannot hide an older cached wheel or pair new wheel bytes with old metadata. Existing cache keys remain valid, and references to older revisions survive cache pruning.

Apply the same filename, hash, size, and archive-existence checks to every cached URL wheel candidate. When online, refresh cached contents that no longer match the requested hashes or size before resolving dependencies. Fresh integrity failures are rejected without an extra retry, including in the full-download fallback.

Follows #21279, which added URL hash reuse during resolution.

@konstin

konstin commented Aug 24, 2026

Copy link
Copy Markdown
Member

That's the same as #21210, though I don't mind which one we merge.

@codspeed

codspeed Bot commented Aug 24, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 8.55%

⚡ 3 improved benchmarks
✅ 22 untouched benchmarks
⏩ 12 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Simulation resolve_warm_jupyter 72.9 ms 66 ms +10.47%
⚡ Simulation resolve_warm_jupyter_universal 314.8 ms 289.6 ms +8.71%
⚡ Simulation resolve_warm_airflow 800.2 ms 751.4 ms +6.51%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing charlie/normalize-wheel-cache-hashes (558b601) with main (a22198b)

Open in CodSpeed

Footnotes

  1. 12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@charliermarsh
charliermarsh marked this pull request as draft August 24, 2026 21:43
@charliermarsh
charliermarsh removed this pull request from stack #21289 September 9, 2026 00:39
@charliermarsh
charliermarsh added this pull request to stack #21550 September 9, 2026 00:40
Base automatically changed from charlie/reuse-url-hashes to main September 10, 2026 14:29
@charliermarsh
charliermarsh force-pushed the charlie/normalize-wheel-cache-hashes branch from 107ee98 to 3311391 Compare September 10, 2026 14:58
@astral-sh-bot

astral-sh-bot Bot commented Sep 10, 2026

Copy link
Copy Markdown

uv test inventory changes

This PR changes the tests when compared with the main base revision.

  • Added tests: 8
  • Removed tests: 0
  • Changed suites: 5
uv-cache: +1 / -0

Added:

  • uv-cache::wheel::tests::archive_and_metadata_url_identity

Removed: none

uv-cache-key: +2 / -0

Added:

  • uv-cache-key::canonical_url::tests::canonical_url_ignores_known_hash_fragments
  • uv-cache-key::canonical_url::tests::canonical_url_preserves_other_fragments

Removed: none

uv::lock: +2 / -0

Added:

  • uv::lock::lock::lock_wheel_url_fragment_download
  • uv::lock::lock::lock_wheel_url_hash_cache

Removed: none

uv::pip: +2 / -0

Added:

  • uv::pip::pip_sync::direct_url_invalid_cache
  • uv::pip::pip_sync::direct_url_wheel_cache_refresh

Removed: none

uv::pip_install: +1 / -0

Added:

  • uv::pip_install::pip_install::direct_url_hash_cache_metadata

Removed: none

This branch was previously deployed

1 inactive deployment
automations — 558b601e Deployed Sep 10, 2026 by charliermarsh via review / security review #46316
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants