[https://nvbugs/5433581][infra] Update install docs and CI script for SBSA deep_gemm workaround #6607
Conversation
📝 WalkthroughWalkthroughThe changes update Linux installation documentation to clarify CUDA and PyTorch requirements, add a workaround for SBSA platform installation issues, and adjust the Jenkins test script to apply this workaround programmatically for SBSA by explicitly installing the Changes
Sequence Diagram(s)sequenceDiagram
participant Jenkins
participant FileSystem
participant Pip
Jenkins->>Jenkins: Determine cpu_arch
alt cpu_arch == SBSA
Jenkins->>FileSystem: Read requirements.txt
FileSystem-->>Jenkins: Return content
Jenkins->>Jenkins: Search for deep_gemm line
alt deep_gemm found
Jenkins->>Pip: pip install deep_gemm_package --extra-index-url
else deep_gemm not found
Jenkins->>Jenkins: Log absence of deep_gemm
end
end
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Suggested reviewers
Note ⚡️ Unit Test Generation is now available in beta!Learn more here, or try it out under "Finishing Touches" below. ✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
coderabbitai
bot
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
docs/source/installation/linux.md (3)
12-14: Clarify CUDA Toolkit environment setupConsider adding a brief example (
export CUDA_HOME=/usr/local/cuda) or mention that some distro packages set it automatically. This helps first-time users verify the variable is actually present.
16-23: Pin companion package versions for PyTorch
torch==2.7.1is pinned buttorchvision/torchaudioare not. Mixing mismatched versions can break the wheel resolver. Recommend:pip3 install torch==2.7.1 torchvision==0.18.1 torchaudio==2.7.1 …
27-27: Minor wording / plural
platform→platforms(“…on prior GPUs or Linux x86_64 platforms”).jenkins/L0_Test.groovy (1)
2094-2099: Local variable name can be confusing
def platform = …shadows the earlierplatformparameter used throughout the method and risks accidental misuse later.
Rename to something explicit such ascudaRepoArchto avoid ambiguity.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
docs/source/installation/linux.md(1 hunks)jenkins/L0_Test.groovy(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: in tensorrt-llm testing, it's common to have both cli flow tests (test_cli_flow.py) and pytorch api ...
Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.
Applied to files:
jenkins/L0_Test.groovydocs/source/installation/linux.md
📚 Learning: in tensorrt-llm, examples directory can have different dependency versions than the root requirement...
Learnt from: yibinl-nvidia
PR: NVIDIA/TensorRT-LLM#6506
File: examples/models/core/mixtral/requirements.txt:3-3
Timestamp: 2025-08-01T15:14:45.673Z
Learning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.
Applied to files:
jenkins/L0_Test.groovydocs/source/installation/linux.md
📚 Learning: applies to **/*.py : the code developed for tensorrt-llm should conform to python 3.8+....
Learnt from: CR
PR: NVIDIA/TensorRT-LLM#0
File: CODING_GUIDELINES.md:0-0
Timestamp: 2025-08-04T02:12:17.582Z
Learning: Applies to **/*.py : The code developed for TensorRT-LLM should conform to Python 3.8+.
Applied to files:
docs/source/installation/linux.md
🔇 Additional comments (1)
jenkins/L0_Test.groovy (1)
2108-2117: Verify deep_gemm install path resolution
readFile("${LLM_ROOT}/requirements.txt")runs on the agent; inside a pod the workspace path can differ, causing a “file not found” at runtime.
Please run a quick check that the file exists in this context or fall back tosh "grep '^deep_gemm' ${LLM_ROOT}/requirements.txt"to avoid pipeline failures.
chzblych
changed the title
|
Tested internally with https://nv/trt-llm-cicd/job/main/job/L0_MergeRequest/29121. |
… failure on SBSA platform Signed-off-by: Yanchao Lu <[email protected]>
|
/bot skip --comment "Tested internally with https://nv/trt-llm-cicd/job/main/job/L0_MergeRequest/29121". |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
docs/source/installation/linux.md (3)
12-14: Provide a concreteCUDA_HOMEexample to avoid mis-configurationThe new sentence instructs users to “make sure
CUDA_HOMEenvironment variable is properly set”, but gives no illustration. A quick one-liner prevents common mistakes when multiple CUDA versions are present.Install CUDA Toolkit following the [CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/) and -make sure `CUDA_HOME` environment variable is properly set. +export the correct `CUDA_HOME`, e.g.: + +```bash +export CUDA_HOME=/usr/local/cuda-12.4 +``` + +Verify with `nvcc --version`.
19-23: Make DeepGEMM pinning maintainableHard-coding a personal GitHub fork and commit hash in user-facing docs will break silently when the fork disappears or the API changes.
- Move the exact requirement to
requirements/deep_gemm_sbsa.txt(or reuse the existing global requirements.txt) and reference that file in the doc.- Add a brief rationale (“temporary SBSA workaround for nvbugs/5433581”) so future readers know when it can be removed.
27-27: Fix typo and clarify sentence-PyTorch CUDA 12.8 package is required for supporting NVIDIA Blackwell GPUs and SBSA platform. On prior GPUs or Linux x86_64 platform, this extra installation is not required. +The CUDA 12.8 PyTorch wheel is required only on NVIDIA Blackwell GPUs **or** the SBSA (AArch64) platform. On earlier GPUs or standard x86_64 Linux hosts, skip this step.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
docs/source/installation/linux.md(1 hunks)jenkins/L0_Test.groovy(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- jenkins/L0_Test.groovy
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: in tensorrt-llm, examples directory can have different dependency versions than the root requirement...
Learnt from: yibinl-nvidia
PR: NVIDIA/TensorRT-LLM#6506
File: examples/models/core/mixtral/requirements.txt:3-3
Timestamp: 2025-08-01T15:14:45.673Z
Learning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.
Applied to files:
docs/source/installation/linux.md
📚 Learning: in tensorrt-llm testing, it's common to have both cli flow tests (test_cli_flow.py) and pytorch api ...
Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.
Applied to files:
docs/source/installation/linux.md
📚 Learning: applies to **/*.py : the code developed for tensorrt-llm should conform to python 3.8+....
Learnt from: CR
PR: NVIDIA/TensorRT-LLM#0
File: CODING_GUIDELINES.md:0-0
Timestamp: 2025-08-04T02:12:17.582Z
Learning: Applies to **/*.py : The code developed for TensorRT-LLM should conform to Python 3.8+.
Applied to files:
docs/source/installation/linux.md
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Pre-commit Check
🔇 Additional comments (1)
docs/source/installation/linux.md (1)
16-18: Double-check availability of torch==2.7.1 wheel on the cu128 indexOur automated checks against https://download.pytorch.org/whl/cu128/ did not surface any
torch-2.7.1wheels. If this release isn’t publicly published, please choose one of the following:
- Downgrade the example to the latest public build (e.g.
pip3 install torch==2.4.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128)- Or add a callout that
torch==2.7.1for cu128 is an internal preview and requires access to your organization’s mirrorLocation to update:
• docs/source/installation/linux.md (lines 16–18)
|
PR_Github #14075 [ skip ] triggered by Bot |
|
PR_Github #14075 [ skip ] completed with state |
… SBSA deep_gemm workaround (NVIDIA#6607) Signed-off-by: Yanchao Lu <[email protected]> Signed-off-by: Lanyu Liao <[email protected]>
… SBSA deep_gemm workaround (NVIDIA#6607) Signed-off-by: Yanchao Lu <[email protected]>
3 participants
Summary by CodeRabbit
Documentation
Bug Fixes