Always upload on branch/nightly builds

[raydouglass]

Since the build.yaml workflow only runs on branch pushes, tag pushes, or nightly calls, it should always upload the wheel to PyPI like it does for conda packages.

This will fix the missing release uploads like this: https://github.com/rapidsai/dask-cuda/actions/runs/4678841210/jobs/8288889977

[ajschmidt8]

this change will cause releases to occur when PRs are merged now.

because of that, we should probably also use the skip-existing parameter below:

• https://github.com/pypa/gh-action-pypi-publish#tolerating-release-package-file-duplicates

[vyasr]

I'm fine with this change, just want to make sure that the consequences are clear:

• If there are no PRs on a given day, then we get a true nightly at the end of the day.
• If there is exactly one PR on a given day, the "nightly" for that day will be the package uploaded when that PR is merged.
• If there are multiple PRs, then the pushed package will be from the first merged PR. All subsequent builds will simply skip uploading.

It's an acceptable tradeoff, it just means that sometimes the latest nightly won't actually contain the latest changes.

[ajschmidt8]

approving but also CC'ing @pentschev for some feedback based on Vyas's comment

[pentschev]

If there are multiple PRs, then the pushed package will be from the first merged PR. All subsequent builds will simply skip uploading.
It's an acceptable tradeoff, it just means that sometimes the latest nightly won't actually contain the latest changes.

I must say this is a bit weird, essentially the newest nightly will not contain the latest changes. I don't know whether anyone actually uses Dask-CUDA PyPI nightly builds but our own CI. If you think this is an acceptable tradeoff I'm ok with that, but what this is saying is that we may need to wait up to ~48h for changes to show up in CI on those days (assuming two PRs are merged shortly after the latest nightly was generated), which could mean it will be blocked for up to 48h. Dask-CUDA has low traffic so this is probably not going to be an occurrence more than a few days a year, but just wanted to point it out.

[ajschmidt8]

Is there an alternative versioning scheme we can adopt for nightlies to fix this weirdness?

e.g. one of the following, where GIT_DESCRIBE_NUMBER is the number of commits since the most recent tag:

23.6.0a230524<GIT_DESCRIBE_NUMBER>
23.6.0a230524.dev<GIT_DESCRIBE_NUMBER>

both of these are PEP 440 compliant.

and using GIT_DESCRIBE_NUMBER would ensure that each commit gets released.

[pentschev]

Is there an alternative versioning scheme we can adopt for nightlies to fix this weirdness?

I have no preference here, anything would be ok. I'm also ok with leaving it like this for now and revisit should this become important. I'll leave it up to @vyasr who is more knowledgeable what would be better here or if he has any preference, otherwise I'm ok merging this as is for now.

[ajschmidt8]

Ok. I assume we can merge this now then and revisit this conversation in the future if necessary.

I'll defer to Ray to merge since it's his PR.

[raydouglass]

/merge

[jakirkham]

Thanks all! 🙏

[vyasr]

Sorry was out of town when the discussion was being concluded. Yes, there are alternative version schemes that we can adopt to fix the problem. However, we would probably want to apply this consistently across all of RAPIDS. Furthermore, it would affect our dependency specs because of the way pip installs nightlies (which is also why the rest of RAPIDS doesn't currently use proper nightly versions). Happy to revisit this discussion in the future though.