BUG: pyarrow-nightly CI fails during Pixi install due to stale locked nightly wheel

[kjmin622]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

bash
pixi install -e pyarrow-nightly --locked
Error: The process '/home/runner/.pixi/bin/pixi' failed with exit code 1

Issue Description

The pyarrow-nightly environment uses the scientific-python nightly wheel index. However, pixi.lock pins a direct URL for the pyarrow-25.0.0.dev48 wheel, and that nightly artifact has been removed from the index, causing a 404 during installation.

Expected Behavior

The pyarrow-nightly CI job should not fail simply because an older nightly wheel artifact has been pruned.

Installed Versions

Details

[kjmin622]

I have confirmed that this issue is being resolved in PR#65668.

[jorisvandenbossche]

This is actually something that will keep happening in the future as well, with the current setup, so re-opening so we can discuss the best way to deal with this.

The reason the URL disappears is because the nightly index only keeps the last 250 files, and with the number of wheels pyarrow has, and the number of uploads, that will mean that the URL is no longer available after around 5 days.

We don't have this issue with numpy, because they have a "fixed" nightly version (like 2.5.0.dev0), so the URL always stays the same until there is an actual version bump.

[jorisvandenbossche]

cc @mroeschke (and might be worth checking with the pixi folks if they have ideas about this situation)

[kjmin622]

@jorisvandenbossche I closed PR #65689 because I understood that the issue was already being addressed. However, could we also consider updating pyarrow nightly at CI test time?

[mroeschke]

Started prefix-dev/pixi#6186 to see if the Pixi community has some ideas.

However, could we also consider updating pyarrow nightly at CI test time?

This works but defeats the purpose of the lock file where the entire environment is solved beforehand.

[rhshadrach]

We now refresh the lock file each week, so this should no longer be an ongoing issue. Closing.