chore(ci): more tweaks to the release process - #361
Conversation
There was a problem hiding this comment.
Pull request overview
Updates the release workflow to run automatically on version tag pushes and to always perform the full release sequence (package publishing + GitHub release creation).
Changes:
- Switch trigger from manual
workflow_dispatchinputs topushon tags matchingv* - Always publish the built wheel to both Test PyPI and PyPI (removes dry-run gating)
- Always create a GitHub release and switch release notes to
--generate-notes
Comments suppressed due to low confidence (2)
.github/workflows/release.yml:94
- The PyPI upload step now always runs, but it doesn’t use
--skip-existing. Any workflow rerun (or re-push of the same tag) will fail once the wheel is already on PyPI, which makes recovery from later-job failures difficult. Consider adding--skip-existing(or an explicit pre-check) so reruns are idempotent.
- name: Publish to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_PERSONAL_TOKEN }}
run: |
uvx twine upload \
--non-interactive \
--verbose \
dist/*.whl
.github/workflows/release.yml:90
- This workflow now automatically publishes to production PyPI on any
v*tag push. Without an environment/approval gate, an accidental or premature tag push will immediately ship a release. Consider adding a protected GitHub Environment for the PyPI publish step/job (or another explicit guard) to reduce the blast radius.
- name: Publish to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_PERSONAL_TOKEN }}
run: |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| gh release create "v${VERSION}" \ | ||
| dist/*.whl \ | ||
| --title "v${VERSION}" \ | ||
| --notes-file CHANGELOG.md \ | ||
| --generate-notes \ | ||
| $PRERELEASE_FLAG |
There was a problem hiding this comment.
gh release create "v${VERSION}" derives the tag name from the built wheel version, but the workflow is triggered by a pushed tag. If the pushed tag and computed VERSION diverge, this can create a release (and possibly a new tag) that doesn’t match the triggering ref. Consider using the triggering tag (github.ref_name) as the release tag and failing fast if it doesn’t match the wheel-derived version.
b46f3d9 to
6187fc7
Compare
| 3. Build wheel - Builds the production wheel | ||
| 2. Build wheel - Builds the production wheel | ||
| 3. Push to test PyPI | ||
| 4. Publish to PyPI - Uploads to PyPI (or test PyPI for dry runs) |
There was a problem hiding this comment.
question: dry run is no longer a concept for the workflow, right?
6187fc7 to
aa7e5f2
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -98,7 +84,6 @@ jobs: | |||
| dist/*.whl | |||
|
|
|||
| - name: Publish to PyPI | |||
There was a problem hiding this comment.
This change makes the production PyPI publish unconditional for every v* tag. That increases the blast radius of an accidental/incorrect tag push (including RC tags if you intend those to be test-only). If RC tags should not publish to production PyPI, add an explicit condition (e.g., based on tag naming) and/or gate the PyPI publish behind a protected environment approval.
| - name: Publish to PyPI | |
| - name: Publish to PyPI | |
| if: ${{ !contains(github.ref_name, 'rc') && !contains(github.ref_name, 'alpha') && !contains(github.ref_name, 'beta') && !contains(github.ref_name, '.dev') }} |
* Always run on push to `v*` * Always create a gh release * No longer runnable as a workflow action Signed-off-by: Matt Kornfield <mkornfield@nvidia.com>
Signed-off-by: Matt Kornfield <mkornfield@nvidia.com>
aa7e5f2 to
6b9c0b6
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | [conventional-commit.yml](conventional-commit.yml) | PRs | Validates PR titles follow conventional commit format | | ||
| | [docs.yml](docs.yml) | Push to `main` (docs paths) | Builds and deploys documentation to GitHub Pages | | ||
| | [release.yml](release.yml) | Manual dispatch | Builds and publishes package to Test PyPI or PyPI (production) | | ||
| | [release.yml](release.yml) | Push tags to `v*` | Builds and publishes package to Test PyPI or PyPI (production) | |
There was a problem hiding this comment.
The release workflow now publishes to both Test PyPI and PyPI on every v* tag push, but this table entry still says “Test PyPI or PyPI”. Update the wording to match the current behavior (e.g., “Test PyPI then PyPI”).
| | [release.yml](release.yml) | Push tags to `v*` | Builds and publishes package to Test PyPI or PyPI (production) | | |
| | [release.yml](release.yml) | Push tags to `v*` | Builds and publishes package to Test PyPI then PyPI | |
| ## Release Workflow (Production) | ||
|
|
||
| The production release workflow uses the [FW-CI-templates `_release_library.yml`](https://github.com/NVIDIA-NeMo/FW-CI-templates) reusable workflow to publish to PyPI. | ||
| The production release workflow publishes to test PyPI and regular PyPI. It also creates release notes |
There was a problem hiding this comment.
This sentence is missing punctuation and is a bit unclear. Consider ending it with a period and clarifying that GitHub Release notes are auto-generated (via gh release create --generate-notes).
| The production release workflow publishes to test PyPI and regular PyPI. It also creates release notes | |
| The production release workflow publishes to Test PyPI and PyPI. It also creates GitHub Release notes automatically via `gh release create --generate-notes`. |
| conventional -.->|reuses| FW-CI-templates | ||
| secrets -.->|reuses| FW-CI-templates | ||
| release -.->|reuses| FW-CI-templates | ||
| ``` |
There was a problem hiding this comment.
The diagram removes the release reuse link, but the later “Reusable Workflows” section still states that “All compliance and release workflows reuse templates …” and lists _release_library.yml. Since release.yml is now implemented inline, update that section (or the diagram) to avoid misleading readers.
| pr --> ci & conventional & secrets | ||
| manual --> release | ||
| tag[Tag push v[0-9]*] --> internalRelease | ||
| tag[Tag push v[0-9]*] --> release |
There was a problem hiding this comment.
The workflow diagram still includes an “Internal Release” subgraph, but there is no internal-release.yml workflow in this repository (and the Internal Release section was removed below). Either remove the Internal Release nodes from the diagram or add back the corresponding workflow/docs so the diagram matches reality.
| - name: Publish to PyPI | ||
| if: ${{ !inputs.dry-run }} | ||
| env: | ||
| TWINE_USERNAME: __token__ | ||
| TWINE_PASSWORD: ${{ secrets.PYPI_PERSONAL_TOKEN }} |
There was a problem hiding this comment.
Now that this workflow always publishes to PyPI on tag pushes, it’s hard to recover from partial failures by re-running the job: PyPI rejects re-uploading the same file, and the upload command here doesn’t use --skip-existing (unlike the Test PyPI step). Consider adding --skip-existing or handling the “already exists” error so reruns can proceed to GitHub release creation.
v*