run nightly-vs-upstream-main on 2.7.1 and multi-gpu also#2929
Conversation
WalkthroughThe nightly test workflow was updated to add a new test configuration for PyTorch 2.7.1 with CUDA 12.6.3 and Python 3.11. Additionally, a new job was introduced to run multi-GPU end-to-end tests using Modal, triggered only for a specific repository owner. Changes
Sequence Diagram(s)sequenceDiagram
participant GitHub Actions
participant Pre-commit
participant Pytest
participant Docker E2E Tests
participant Docker E2E MultiGPU
GitHub Actions->>Pre-commit: Run pre-commit checks
Pre-commit->>Pytest: On success, run pytest
Pytest->>Docker E2E Tests: On success, run docker-e2e-tests (matrix includes new config)
Docker E2E Tests->>Docker E2E MultiGPU: On success, run docker-e2e-multigpu-tests (if owner matches)
Docker E2E MultiGPU->>Modal: Launch cicd.multigpu test job
Suggested reviewers
Poem
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
.github/workflows/tests-nightly.yml (2)
142-144: Custom runner labelmodaltrips action-lint
actionlintwarns thatmodalis an unknown label.
If this is an internal self-hosted runner group, add it to.github/actionlint.yaml’srunnerLabels:allow-list to silence false positives and keep CI linting green.
158-181: High duplication between single- and multi-GPU jobs
docker-e2e-testsanddocker-e2e-multigpu-testsshare identical setup steps (checkout, Python, Modal install, env export). Consider extracting them into:
- A reusable workflow (
.github/workflows/reusable-e2e.yml) invoked viauses:- Or a composite action stored in
.github/actions/setup-e2e.This keeps the workflow DRY and reduces the risk of the two diverging.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/tests-nightly.yml(2 hunks)
🧰 Additional context used
🪛 actionlint (1.7.7)
.github/workflows/tests-nightly.yml
143-143: label "modal" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-2019", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "ubuntu-20.04", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
170-170: property "axolotl_args" is not defined in object type {axolotl_extras: string; cuda: number; cuda_version: string; nightly_build: bool; num_gpus: number; python_version: number; pytorch: string}
(expression)
🔇 Additional comments (1)
.github/workflows/tests-nightly.yml (1)
109-115: 2.7.1 added only to E2E matrix – unit/CLI tests still pinned to ≤ 2.7.0
You extended thedocker-e2e-testsmatrix with PyTorch 2.7.1, but the upstreampytestjob (line 29) still enumerates["2.6.0", "2.7.0"].
If the intent is full coverage on 2.7.1, remember to bump that list as well; otherwise regressions might slip through unit tests while being caught (and diagnosed much later) in the slower E2E phase.
| run: | | ||
| echo "BASE_TAG=main-base-py${{ matrix.python_version }}-cu${{ matrix.cuda }}-${{ matrix.pytorch }}" >> $GITHUB_ENV | ||
| echo "PYTORCH_VERSION=${{ matrix.pytorch}}" >> $GITHUB_ENV | ||
| echo "AXOLOTL_ARGS=${{ matrix.axolotl_args}}" >> $GITHUB_ENV | ||
| echo "AXOLOTL_EXTRAS=${{ matrix.axolotl_extras}}" >> $GITHUB_ENV | ||
| echo "CUDA=${{ matrix.cuda }}" >> $GITHUB_ENV | ||
| echo "N_GPUS=${{ matrix.num_gpus }}" >> $GITHUB_ENV | ||
| echo "NIGHTLY_BUILD=${{ matrix.nightly_build }}" >> $GITHUB_ENV |
There was a problem hiding this comment.
matrix.axolotl_args is never defined – workflow will fail at run-time
Both E2E jobs export:
echo "AXOLOTL_ARGS=${{ matrix.axolotl_args }}" >> $GITHUB_ENVYet none of the matrix.include dictionaries declare an axolotl_args key.
GitHub Actions treats an undefined matrix property as a hard error (Context does not contain property 'axolotl_args'), so the job will exit before tests start.
Minimal fix – add an empty value to every matrix entry:
num_gpus: 1
axolotl_extras:
+ axolotl_args: ""
nightly_build: "true"…and similarly for the multi-GPU row.
🧰 Tools
🪛 actionlint (1.7.7)
170-170: property "axolotl_args" is not defined in object type {axolotl_extras: string; cuda: number; cuda_version: string; nightly_build: bool; num_gpus: number; python_version: number; pytorch: string}
(expression)
🤖 Prompt for AI Agents
In .github/workflows/tests-nightly.yml around lines 170 to 177, the matrix
property axolotl_args is used but not defined in any matrix.include entries,
causing a runtime error. To fix this, add an axolotl_args key with an empty
string value to every matrix.include dictionary where it is missing, ensuring
the property exists for all matrix configurations and prevents the workflow from
failing.
Codecov ReportAll modified and coverable lines are covered by tests ✅ 📢 Thoughts on this report? Let us know! |
1 participant
manually triggered workflow:
Summary by CodeRabbit