Skip to content

Avoid repeated resolver prefetches - #20465

Closed
charliermarsh wants to merge 1 commit into
mainfrom
charlie/codex-prefetch-once
Closed

Avoid repeated resolver prefetches#20465
charliermarsh wants to merge 1 commit into
mainfrom
charlie/codex-prefetch-once

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Jul 16, 2026

Copy link
Copy Markdown
Member

Summary

Unit propagation repeatedly tightens ranges for packages that were already speculatively prefetched. Selecting a candidate for every new range is relatively expensive even when the likely distribution metadata is already in memory. This prefetches each package once per fork and leaves eventual candidate selection/fetching unchanged.

This is deliberately experimental: avoiding later speculative requests can trade away some cold-cache overlap on backtracking-heavy resolutions.

Performance

Measured with profiling binaries, a warm local cache, --offline, pinned CPUs (8-15), fixed UV_EXCLUDE_NEWER, alternating base/head order, 10 warmups, and 60 paired runs. This is incremental on current main (including #20461) plus #20450, marker fastpaths, and cache-payload offload. Every pair produced identical lockfile hashes.

Workload Wall before → after Wall Δ CPU before → after CPU Δ
Transformers 296.0 → 296.4 ms +0.1% 353.7 → 345.8 ms -2.2%
Home Assistant 119.9 → 116.2 ms -3.1% 84.8 → 85.1 ms +0.3%
Warehouse 157.0 → 158.2 ms +0.8% 184.6 → 184.3 ms -0.2%
JupyterLab 111.2 → 111.2 ms 0.0% 60.1 → 60.1 ms +0.1%
Semantic Kernel 117.8 → 115.8 ms -1.7% 72.9 → 72.4 ms -0.7%

Transformers is the meaningful CPU win: block-aware paired analysis was -2.51% CPU (95% CI -3.51 to -1.47), with neutral wall time. A resolver trace confirms candidate selections drop from 4,647 to 3,572 (-23.1%) for the same resulting lock.

The focused resolver suite passes (63 tests); formatting and diff checks are clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants