feat: add a --clean-cache command to clean up locations specified at CIBW_CACHE_PATH

[agriyakhetarpal]

As described in #2397, this PR adds a --clean-cache command to clean CIBW_CACHE_PATH. It does not implement granular cache cleans at the moment (it does not clean per supported platform), and shutil.rmtree()s the entire folder. I added three tests that account for possible code paths: if cache files exist, if they don't, or if the cache files are somehow in a non-readable/writable location (though I think that aspect should also be validated at the creation time of the cache paths).

This is especially useful for Pyodide builds, as the Emscripten SDK can occupy up to 1.5 GiB of space; however, it is also beneficial for builds on all other platforms.

Closes #2397

[joerick]

Thanks for putting this together!

[henryiii]

The one thought I had was could we check for something specific to make sure this is in fact a cibuildwheel cache dir? Just to protect for a mistake like setting the cache dir to something like $HOME. :)

[agriyakhetarpal]

The one thought I had was could we check for something specific to make sure this is in fact a cibuildwheel cache dir? Just to protect for a mistake like setting the cache dir to something like $HOME. :)

Do you mean we should check if CIBW_CACHE_PATH contains cibuildwheel or if it matches platformdirs.user_cache_path()?

Realistically, I think we'd have no defense against this unless we make the cache path non-configurable and drop the ability to set it via the DEFAULT_CIBW_CACHE_PATH environment variable. This is unless I've misunderstood you here :)

[henryiii]

I was thinking looking for a specific file or folder that would exist if it was cibuildwheel's cache.

[agriyakhetarpal]

Right, we could include a sentinel .cibuildwheel_cached file inside the cache folder; we do similar things with Pyodide's xbuildenv cache.

[joerick]

Right, we could include a sentinel .cibuildwheel_cache file inside the cache folder; we do similar things with Pyodide's xbuildenv cache.

That's a nice idea. Might save somebody's day.

[agriyakhetarpal]

CACHEDIR.TAG makes a lot of sense. Linking the specification here for future reference: https://bford.info/cachedir/

[agriyakhetarpal]

Should we also comply with https://arc.net/l/quote/bosrzvuf?

Edit: quoting here, it says the following:

For the benefit of anyone who happens to find and look at a cache directory tag directly, it is recommended that applications include in the file a comment referring back to this specification. For example:

Signature: 8a477f597d28d172789f06886806bc55
# This file is a cache directory tag created by (application name).
# For information about cache directory tags, see:
# http://www.brynosaurus.com/cachedir/

The official "home" URL of this specification may change from the above at some point if/when this proposal becomes > more of a formal standard, but in that case an appropriate forwarding link will be left in the old location.

[henryiii]

I'd be fine with that - I was tempted to add it but was a little hard in comments.

FWIW, uv does not add this.

[henryiii]

@freakboy3742 I'm guessing that since Azure hasn't migrated "macOS-latest" to 15 yet, there's nothing we can do about the Azure Pipelines failures, since those are on 14?

[joerick]

really nice job on this!

[freakboy3742]

@freakboy3742 I'm guessing that since Azure hasn't migrated "macOS-latest" to 15 yet, there's nothing we can do about the Azure Pipelines failures, since those are on 14?

Azure started moving macos-latest to macos-15 on August 4; the failed run was running on macos-15.

GitHub's response to the problem appears to be "yeah, looks like macOS-15 is broken; try using macOS-14 for now, and maybe it will be fixed in the next release".

I've opened #2563 to switch the Azure image, and add some documentation about the problem. That documentation is hopefully temporary, but since I can't say for certain, it doesn't hurt to throw a paragraph in.