[pengbowang-nv]

@coderabbitai summary

Description

Cherry-pick of #19591 (main commit f08592ada02b8b7b94db68ee6972f3ebcecd857d) onto release/1.3 as a0dd7bcd7cc5738b0c262e8ea481d8ca5767f21c.

Update trtllm-gen to main@f8a7d6e2004bb9b7c864b33246f18c33b20cdd47, as in the original PR. This cherry-pick has no changes beyond the original FMHA artifact update.

Test Coverage

• No build or runtime tests were rerun for this cherry-pick.
• git diff --check passed, and the FMHA Git tree matches the original merged main commit.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why.

• PR follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions).

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities.

• CODEOWNERS updated if ownership changes.

• Documentation updated as needed.

• Update tava architecture diagram if there is a significant design change in PR.

• Please check this after reviewing the above items as appropriate for this PR.

[pengbowang-nv]

/bot run

[tensorrt-cicd]

PR_Github #76676 [ run ] triggered by Bot. Commit: a0dd7bc Link to invocation

[github-actions]

Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.

[tensorrt-cicd]

PR_Github #76676 [ run ] completed with state SUCCESS. Commit: a0dd7bc
/LLM/release-1.3/L0_MergeRequest_PR pipeline #90 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

CI Report

Link to invocation

[farazkh80]

mDtypeKv is set from the K dtype only, so for the new FP8-K/NVFP4-V kernels the SwapsAb generation check would pick NVRTC instead of the cubin.

Should we derive it from both K and V, or reject the K=E4m3/V=E2m1 pair on release/1.3 until that is done?

[pengbowang-nv]

@farazkh80 The new kernels are not enabled by trtllm so far and I would say it's not an enabled feature. It should be fixed and enabled when the kernel is needed. So I prefer to keep the PR as it is, a pure update to trtllm library without enabling new features.

[github-actions]

✅ LFS objects already in storage (7534 files) — no sync needed.

These LFS-tracked files are already present in this repository's LFS storage:

• cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100aKernel_QE4m3KvE2m1OBfloat16H128PagedKvCausalP32VarSeqQ256Kv128PersistentContext.cubin.tar.zst
• cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100aKernel_QE4m3KvE2m1OBfloat16H128PagedKvCausalP32VarSeqQ256Kv128StaticContext.cubin.tar.zst
• cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100aKernel_QE4m3KvE2m1OBfloat16H128PagedKvCustomP32MultiCtasKvCgaVarSeqQ16Kv128StaticSwapsAbForGen.cubin.tar.zst
• cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100aKernel_QE4m3KvE2m1OBfloat16H128PagedKvCustomP32MultiCtasKvCgaVarSeqQ32Kv128StaticSwapsAbForGen.cubin.tar.zst
• cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100aKernel_QE4m3KvE2m1OBfloat16H128PagedKvCustomP32MultiCtasKvCgaVarSeqQ8Kv128StaticSwapsAbForGen.cubin.tar.zst
• ...and 7529 more