Repository navigation
[CI/Build][NVIDIA] Build NIXL with NIXL EP from source in Rubin image - #60090
Conversation
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
Documentation preview: https://vllm--60090.org.readthedocs.build/en/60090/ |
mergify
Bot
added
documentation
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
@wangshangsam, hold on a bit, I need to confirm first that everything is correct. Give me some time |
|
/ci run |
|
✅ Triggered Buildkite CI #93077 for commit |
CI selector (shadow): 183 test steps (259 jobs) instead of 194 (270 jobs)Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today's rules. How it works. Feedback welcome: reply here if it would skip a step this change needs, or runs something unrelated.
Today's rules run everything for this diff. Selector would run (183)
Would skip (today's rules run them) (11)
Would add (today's rules do not run them) (0)none AMD mirrors: would skip (162)
AMD mirrors: would add (0)none 6 changed files · base |
|
The main issue with building from source is that if you are deviating from our procedure in small ways, we will have surprises later when we try to do integration I'm working on packaging a wheel that matches the vLLM PyTorch pin https://github.com/ovidiusm/nixl/tree/release-1.5.0-torch215-cu134 I need to do some testing and then we can find a way to publish this as a preview |
ovidiusm
left a comment
ovidiusm
left a comment
There was a problem hiding this comment.
I don't think building from source is a good idea. We have nightly wheel build pipeline, we only need to publish it
Hey Ovidiu, I thought that, per our discussion in https://nvidia.enterprise.slack.com/archives/C08MF9LBA4E/p1790925450746089 and https://nvidia.enterprise.slack.com/archives/C08MF9LBA4E/p1789183979475839 , the earliest time when nixl releases a torch 2.15 build would be the 1.6.0 release, which is still several weeks away? We just want the EP extension to be usable in the meanwhile. |
|
@ovidiusm, we decided to build nixl from source because it is really urgent thing that we need right now -- to have nixl support in Rubin docker image. Of course, this approach won't be there forever and we fall back on installing nixl from wheel package once it has build against torch 2.15. Until then I believe we will have to use approach from this PR to have that support in our containers.
Could you tell us please what is time estimation for nixl build against torch 2.15 to be released?
I do believe this is not the last time we have to build nixl from source. In this PR I tried to update our own script |
|
@wangshangsam, just in case, I confirm, I built Rubin image manually and successfully run manual disaggregated test on top of it -- I see no issues currently, even though this is my first attempt to modify public vLLM docker image. So, I assume we can move forward with review of this PR, but maybe a couple of more eyes would be awesome to have on it. |
| set -eu; \ | ||
| if [ "${BUILD_NIXL}" = "true" ]; then \ | ||
| ls /tmp/nixl-dist/*.whl; \ | ||
| uv pip uninstall --system nixl nixl-cu12 nixl-cu13; \ |
There was a problem hiding this comment.
I would emphasise your attention on this particular step. Here I delete already installed nixl packages to use instead our own build of nixl. One important thing is that I remove nixl-cu12 package and then do NOT reinstall it again. For nixl and nixl-cu13 packages everything is fine -- I reinstall them from the source.
Here I just have no idea why vLLM by default installs both packages nixl-cu12 and nixl-cu13 if it is already known that we use cuda 13.4. If I understand right in that case we do not need nixl-cu12 and it's better to not have it installed to prevent from dependency conflicts and simply for smaller size.
If you have any explanation why we still DO need to install nixl-cu12 even though we are using cuda13.4, then please explain it to me in the comments here. Thank you!
There was a problem hiding this comment.
Ooops ... I just saw this message.
So, docker/Dockerfile is used to build nightly images with 3 different CUDA versions: 12.9, 13.0 and 13.4, and we should make sure all 3 works.
I can guess that installing both nixl-cu12 and nixl-cu13 was for this Dockerfile to build both 12.9 and 13.0 images back then, but why installing both instead of installing only the correct one, that was probably a mistake that haven't been caught so far.
…e script for our issue with Rubin Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
/ci run |
|
✅ Triggered Buildkite CI #93139 for commit |
|
/ci retry |
|
✅ Triggered Buildkite CI #93371 for commit |
…vllm-project#60090) Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Purpose
Official vLLM Rubin docker image currently lacks of proper nixl support since nixl does not have a wheel built against torch 2.15 that is core requirement for Rubin image. See issue report to nixl official github repo.
To resolve this problem I created a new temporary script
tools/build_nixl_with_torch215_for_rubin.pythat I will delete from our codebase once nixl will publish official wheel built against torch 2.15.Testing Done
Manually built docker image for Rubin using buildkite pipeline.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.