Select single-index prefetch candidates in one pass

[charliermarsh]

Summary

Batch prefetch currently selects up to 50 candidates by repeatedly invoking the candidate selector and intersecting a singleton complement into the remaining range. On backtracking-heavy packages, those repeated traversals and increasingly fragmented ranges add meaningful resolver CPU before metadata requests are issued.

This adds a lazy, single-index candidate iterator that visits the version map once, preserves highest/lowest ordering, prerelease and cutoff handling, compatible/incompatible candidates, and the compatible-to-in-order transition, then enqueues each eligible wheel immediately. Multi-index resolution and an existing in-order phase keep the current selector path. Unlike the earlier eager prototype in astral-sh#20487, this does not collect a batch before the first request, preserving cold-cache overlap and early-abandon behavior.

Performance

Measured against current main (1535a6767) with --profile profiling binaries and a prewarmed PyPI cache. All runs were offline uv pip compile, Python 3.12, fixed UV_EXCLUDE_NEWER=2024-08-08T00:00:00Z, UV_CONCURRENT_CACHE_READS=4, CPUs 8-15, 20 warmups, and 80 alternating pairs. Every generated requirements file matched byte-for-byte. Intervals are paired-bootstrap 95% confidence intervals.

workload	wall before → after	wall change (95% CI)	CPU before → after	CPU change (95% CI)
jupyter (97 packages)	102.7 → 99.4 ms	-3.2% [-4.9, -0.4]	50.0 → 49.9 ms	-0.1% [-1.5, +1.2]
boto3 (7 packages, backtracking)	167.3 → 160.6 ms	-4.0% [-6.9, -2.1]	239.4 → 227.8 ms	-4.8% [-8.9, -1.6]
airflow (559 packages)	230.7 → 232.6 ms	+0.9% [-1.2, +3.0]	329.3 → 328.4 ms	-0.3% [-1.5, +1.1]
large-index (26 large Simple pages, no deps)	103.3 → 103.0 ms	-0.2% [-3.0, +0.7]	49.7 → 50.3 ms	+1.2% [-0.1, +2.3]

The meaningful win is the backtracking-heavy boto3 case; the other CPU intervals include no change. A focused selector-equivalence matrix covers highest/lowest, prerelease modes including an all-prerelease IfNecessary map, segmented ranges, incompatible and cutoff entries, and the compatible-to-in-order transition. All 66 resolver unit tests, 11 focused Packse/backtracking integration tests, strict clippy, formatting, and diff checks pass; the Python-3.8-only prefetch integration case was unavailable in this environment.