[BREAKING INCOMPATIBLE CHANGES] in the official GitHub Actions CI/CD artifact upload/download actions

[webknjaz]

GitHub Actions Advisory for @achimnol @bdraco @Dreamsorcerer @jettify @Nothing4You @mjpieters @aio-libs/admins and anybody setting up CI across the org

TL;DR My current recommendation is to actively prevent these actions from being upgraded to v4.

1. Two days ago, GitHub released v4 of actions/upload-artifact and actions/download-artifact: https://github.blog/changelog/2023-12-14-github-actions-artifacts-v4-is-now-generally-available/
2. Yesterday, Dependabot started making PRs with the upgrade
3. v3 and v4 are incompatible and only pairs of upload+download of the same version work; it's not possible to upload via v3, but download with v4, and vice versa
4. When we have a composite action (or a reusable workflow) integrated, that has actions/upload-artifact@v3 used under the hood, and we update to actions/download-artifact@v4 in our calling workflow, things break, meaning that now it's necessary to know the implementation details of all the actions
5. actions/upload-artifact@v4 no longer supports uploading more files into the same GHA artifact, which effectively render our matrixes with the dists fully unusable; when said upload matrixes are invoked behind the release gate, the breakage will only become evident at the last moment, at release time, which is too late and unpleasant (and will waste hours to notice and deal with)
6. There's a limit of 10 artifacts per job now. Though, without testing, it's unclear if matrix jobs count as separate ones of as one.

Context:

• Provide guidance for workflows broken by @4? actions/upload-artifact#472
• https://twitter.com/hynek/status/1735568463093997752
• https://hynek.me/articles/ditch-codecov-python/
• https://github.com/hynek/build-and-inspect-python-package/releases/tag/v2.0.0
• Note about actions/upload-artifact@v4 pypa/cibuildwheel#1699
• (unclear if related but may be related to GitHub's motivation) [BUG] Race condition when an existing artifact is being fetched right about when new files are added there actions/toolkit#1235

[webknjaz]

So they just added pattern-based downloads through the new pattern: and merge-multiple: inputs: actions/download-artifact#259.

What does this mean for us? Mainly, careful manual updates, starting with looking into the actions that might make use of the older APIs under the hood.
Using the download action needs changing from name: thing to pattern: things-*, and an additional merge-multiple: true. But it's still incompatible with the v3 artifacts. So it'll require a lot of coordination to update actions and workflows simultaneously (and maybe, some composite actions before that!)
Additionally, the upload action use will have to change to generate unique artifact names (things-{something-unique}), while maintaining a common prefix so that the its download counterpart could retrieve all those files and fuse them into the common directory structure. This may require making use of python3 -c 'import uuid; print(uuid.uuid4())' or python3 -c 'import secrets; print(secrets.token_hex(16))'.

[webknjaz]

Somebody in the upstream discussion suggested that combining the ${{ strategy }} and ${{ github }} contexts might be an adequate replacement for generating random strings. If that works, the migration should be easier. But it appears that it's unclear how to make things globally unique across reruns.

[webknjaz]

UPD: Enough people report more flakiness with the v4 (the download failing before the upload catches up with the CDN). So the advice is still to stick with v3 for the time being.