Skip to content

build: fix build-container flow (pass --torch-versions in Dockerfile and set explicit EP meson overrides) - #1866

Merged
ovidiusm merged 2 commits into
ai-dynamo:mainfrom
lishapira:fix-build-container-ep-explicit-overrides-and-torch-versions
Jul 1, 2026
Merged

ovidiusm merged 2 commits into
ai-dynamo:mainfrom
lishapira:fix-build-container-ep-explicit-overrides-and-torch-versions

Conversation

@lishapira

@lishapira lishapira commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Fixes for nixl-ci-build-container flow

What?

  1. contrib/Dockerfile: add a new WHL_TORCH_VERSIONS build arg (default 2.11,2.12,2.13, mirroring Dockerfile.manylinux) and forward it to contrib/build-wheel.sh via --torch-versions when BUILD_NIXL_EP=true.
  2. examples/device/ep/meson.build: extend the EP target's override_options from ['buildtype=release'] to ['buildtype=release', 'optimization=3', 'debug=false'].

Why?

After #1802 + #1775 landed, nixl-ci-build-container (debug, BUILD_NIXL_EP=true) has been failing in two distinct places:

  1. Wheel step failure: NIXL EP wheels: Add support for multiple PyTorch versions #1775 added --torch-versions support to contrib/build-wheel.sh and updated Dockerfile.manylinux, but contrib/Dockerfile was still calling --build-nixl-ep alone. Every EP container build errored at the wheel step with --build-nixl-ep requires --torch-versions.
  2. EP nvcc compile failure: ci: Add nixl ep to new dl ci pipeline #1802 sets nixl_ep target with override_options = ['buildtype=release'] so a global --buildtype=debug doesn't drop -G onto the EP nvcc line. In practice --device-debug still reached the EP .cu compiles on nixl-ci-build-containe compilation path.

Summary by CodeRabbit

  • New Features

    • Added support for configuring which Torch versions are included when building the wheel, with a sensible default set.
  • Bug Fixes

    • Improved EP builds in debug environments by forcing release-style compilation settings, helping avoid debug flags affecting CUDA compilation.

@copy-pr-bot

copy-pr-bot Bot commented Jul 1, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

👋 Hi lishapira! Thank you for contributing to ai-dynamo/nixl.

Your PR reviewers will review your contribution then trigger the CI to test your changes.

🚀

lishapira added 2 commits July 1, 2026 03:34
`override_options` in meson treats buildtype, optimization, and debug
as independent knobs. Setting only `buildtype=release` on the EP target
is not always enough to keep --device-debug (-G) off the EP nvcc line
under a global --buildtype=debug. Be explicit about all three so the
EP target is built without -G regardless of the surrounding context.

EP kernels need more registers than the 128-register limit -G imposes.
Dockerfile.manylinux, but left contrib/Dockerfile calling --build-nixl-ep alone.
So every EP container build has since errored at the wheel step with
"--build-nixl-ep requires --torch-versions".

Mirror the manylinux default (WHL_TORCH_VERSIONS=2.11,2.12,2.13) so the EP
wheel step completes.
@lishapira
lishapira force-pushed the fix-build-container-ep-explicit-overrides-and-torch-versions branch from dc3f316 to 9e501e4 Compare July 1, 2026 10:45
@lishapira

Copy link
Copy Markdown
Contributor Author

/ok to test 9e501e4

@lishapira

Copy link
Copy Markdown
Contributor Author

/build

@lishapira

Copy link
Copy Markdown
Contributor Author

@lishapira

Copy link
Copy Markdown
Contributor Author

@lishapira
lishapira marked this pull request as ready for review July 1, 2026 12:29
@lishapira
lishapira requested review from a team, ebarilanM, itayalroy and rakhmets as code owners July 1, 2026 12:29
@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds a WHL_TORCH_VERSIONS build argument to the Docker wheel build, passing --torch-versions to build-wheel.sh when building the NIXL EP, and renames a flag variable. Separately, it extends the Meson EP debug buildtype override to also disable debug and force optimization level 3.

Changes

Build Configuration Updates

Layer / File(s) Summary
Dockerfile wheel build torch versions
contrib/Dockerfile
Adds WHL_TORCH_VERSIONS build arg (default 2.11,2.12,2.13); wheel build step now passes --torch-versions $WHL_TORCH_VERSIONS alongside --build-nixl-ep, renaming EP_BUILD_FLAG to EP_BUILD_FLAGS and reformatting the command.
Meson EP debug override
examples/device/ep/meson.build
When buildtype is debug, the EP target override now also sets optimization=3 and debug=false in addition to forcing buildtype=release.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

  • ai-dynamo/nixl#1775: Introduces the same WHL_TORCH_VERSIONS build arg and EP_BUILD_FLAGEP_BUILD_FLAGS rename for the NIXL EP wheel build.

Suggested labels: size/M

Suggested reviewers: brminich, Alexey-Rivkin, aranadive, ovidiusm

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is specific and matches the two main changes in the PR.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The PR description follows the template with clear What and Why sections and covers both code changes; the optional How section is omitted.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@ovidiusm
ovidiusm merged commit 78985be into ai-dynamo:main Jul 1, 2026
17 checks passed
nv-nmailhot pushed a commit that referenced this pull request Jul 1, 2026
…and set explicit EP meson overrides) (#1866) (#1872)

Cherry pick of #1866

Fixes for nixl-ci-build-container flow

1. contrib/Dockerfile: add a new WHL_TORCH_VERSIONS build arg (default
2.11,2.12,2.13, mirroring Dockerfile.manylinux) and forward it to
contrib/build-wheel.sh via --torch-versions when BUILD_NIXL_EP=true.
2. examples/device/ep/meson.build: extend the EP target's
override_options from ['buildtype=release'] to ['buildtype=release',
'optimization=3', 'debug=false'].

After #1802 + #1775 landed, nixl-ci-build-container (debug,
BUILD_NIXL_EP=true) has been failing in two distinct places:
1. Wheel step failure: #1775 added --torch-versions support to
contrib/build-wheel.sh and updated Dockerfile.manylinux, but
contrib/Dockerfile was still calling --build-nixl-ep alone. Every EP
container build errored at the wheel step with --build-nixl-ep requires
--torch-versions.
2. EP nvcc compile failure: #1802 sets nixl_ep target with
override_options = ['buildtype=release'] so a global --buildtype=debug
doesn't drop -G onto the EP nvcc line. In practice --device-debug still
reached the EP .cu compiles on nixl-ci-build-containe compilation path.

Co-authored-by: lishapira <lishapira@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants