[JIT Kernel] Fix NVFP4 multi-arch compilation failure - #20874
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
| # JIT compilation targets only the current device, unlike AOT fat-binaries; | ||
| # adding extra architectures here would clash with the single SGL_CUDA_ARCH | ||
| # value injected by load_jit(). | ||
| return f"{major}.{minor}a" |
There was a problem hiding this comment.
Can we remove this context? JIT kernel already use current arch major.minor. We just need to check that whether major >= 10 here.
There was a problem hiding this comment.
Don't we need the a suffix though? _get_cuda_arch_list() in utils.py returns major.minor without a, but NVFP4 requires sm_*a targets
There was a problem hiding this comment.
ic. let's just keep it for now.
There was a problem hiding this comment.
@DarkSharpness shall we land this now or do you still see any issues?
|
LGTM for mitigating the issue. It seems that the JIT multi-arch build logic was copied from AOT build logic, but JIT only targets the current device, so simply removing it is right. |
|
/rerun-stage stage-c-test-4-gpu-b200 |
|
✅ Triggered |
|
/rerun-stage stage-b-test-small-1-gpu |
|
✅ Triggered |
|
/rerun-failed-ci |
4 participants
Summary
Remove unnecessary extra
10.3aarch append in_get_nvfp4_cuda_arch_list(). Before #20103 this was harmless (just wasted compilation for an unused arch on CUDA 13+), but now thatload_jit()injects a singleSGL_CUDA_ARCH, compiling for multiple archs triggers astatic_assertmismatch.cc @DarkSharpness