[Bugfix] Auto-configure TRITON_PTXAS_PATH for new GPU architectures

[danielostrow]

Summary

Triton bundles a ptxas binary from CUDA 12.8 that does not support GPU architectures sm_110a (Jetson Thor) or sm_121a (DGX Spark GB10). This causes Triton kernel compilation to fail with:

ptxas fatal: Value 'sm_121a' is not defined for option 'gpu-name'

This PR adds automatic detection of new GPU architectures and configures TRITON_PTXAS_PATH to use the system CUDA toolkit's ptxas when needed.

Changes

• Add _configure_triton_ptxas_for_new_gpus() function to vllm/triton_utils/importing.py
• Uses Triton's native backend detection (triton.backends.backends) to get GPU architecture
• Sets TRITON_PTXAS_PATH for GPUs with arch >= 110 (compute capability 11.0+)
• Respects user-configured TRITON_PTXAS_PATH if already set
• Fails gracefully if detection is unavailable
• Add unit tests for the new functionality

Testing

Tested on NVIDIA GB10 (DGX Spark) with:

• CUDA 13.0 (V13.0.88)
• Triton 3.5.1
• PyTorch 2.9.1+cu130

Verified that:

• Triton kernels compile and execute successfully with the fix
• Triton kernels fail without the fix (expected ptxas error)

Related Issues

Fixes #31269
Fixes #32093

Related to #29469

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[mergify]

Hi @danielostrow, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[gemini-code-assist]

Code Review

This pull request introduces a helpful auto-configuration mechanism for TRITON_PTXAS_PATH to support new GPU architectures. The implementation is sound and the accompanying tests are thorough. My review focuses on improving the debuggability of the new logic by adding logging to error-handling paths that currently fail silently. These changes will make it easier to diagnose issues if the auto-configuration does not behave as expected.

[danielostrow]

How are we looking on this? is it relevant?

[changqingla]

I'm experiencing the same problem and hope this PR can be merged as soon as possible.

[danielostrow]

Please note-- this script acts as a handoff when Triton updates are behind CUDA updates. in the future this handoff may be deemed irrelevant on Triton update

[mgoin]

I think it is likely that this will be resolved by torch==2.10.0 update here #30525 since that pins to triton==3.6.0

[Kaweees]

I think it is likely that this will be resolved by torch==2.10.0 update here #30525 since that pins to triton==3.6.0

@mgoin I could be completely wrong, but my configuration in #34470 uses torch>=2.10.0 and triton>=3.6.0 but still has issues

[johnnynunez]

I think it is likely that this will be resolved by torch==2.10.0 update here #30525 since that pins to triton==3.6.0

@mgoin I could be completely wrong, but my configuration in #34470 uses torch>=2.10.0 and triton>=3.6.0 but still has issues

because that is only solved in future triton 3.7.0
you have to build from main due not cut branch yet
and use https://dev-discuss.pytorch.org/t/pytorch-2-11-rc1-produced-for-pytorch-torchvision/3316

then build vllm from upstream...

I had running yesterday nemotron nvfp4 in jetson agx thor and dgx spark