[TRTLLM-11538][feat] Blackwell custom mask fmha support by sunnyqgg · Pull Request #12958 · NVIDIA/TensorRT-LLM

sunnyqgg · 2026-04-12T02:29:20Z

Summary

Update 81 existing bf16/fp16 SM100f custom mask FMHA cubins for KeepsAbForGen variants
Add 16 new FP8 E4M3/E2M1 custom mask FMHA cubin kernels for SM100f Blackwell (H64/H128)
Register the new E4M3/E2M1 kernels in kernelMetaInfo.h (extern declarations + data entries)

Changes

cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/*.cpp — 81 updated + 16 new cubin files
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h — 48 new lines (16 extern char + 16 extern int + 16 data entries)

Test plan

CI build passes with new cubins
EAGLE3 dynamic tree spec decoding e2e test on B200 with FP8 E4M3/E2M1

Summary by CodeRabbit

Chores
- Updated precompiled kernel binaries for NVIDIA SM100f GPU architecture with new optimized implementations supporting E4M3 and E2M1 low-precision data types.
- Added new kernel variants with improved performance characteristics across multiple configuration options for attention operations.

sunnyqgg · 2026-04-12T02:29:26Z

/bot run

tensorrt-cicd · 2026-04-12T02:36:13Z

PR_Github #42831 [ run ] triggered by Bot. Commit: ecf5b67 Link to invocation

coderabbitai · 2026-04-12T02:36:16Z

📝 Walkthrough

Walkthrough

This pull request updates and extends the FMHA kernel library for SM100 GPUs by refreshing compiled CUDA kernel binaries (cubins) through Git LFS pointer updates and introducing new E4M3/E2M1 precision format variants. The kernel metadata registry is updated to expose these new kernels for runtime selection.

Changes

Cohort / File(s)	Summary
Existing cubin LFS updates `cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_Qkv*_cubin.cpp` (68 files)	Updated Git LFS object identifiers (`oid`) and file sizes for existing FMHA kernel variants (Bfloat16, E4m3, Fp16, E4m3/E2m1), reflecting recompiled binary artifacts.
New E4M3/E2M1 cubin pointers `cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1*.._cubin.cpp` (16 files)	Added new Git LFS pointer files for E4M3/E2M1 precision format FMHA kernel variants covering H128 and H64 configurations with multiple paging and sequence handling modes.
Kernel metadata registry `cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h`	Added 16 new `extern unsigned char` and `extern unsigned int` declarations for E4M3/E2M1 cubin artifacts and their lengths, plus corresponding `TllmGenFmhaKernelMetaInfo` registry entries to enable kernel discovery and selection at runtime.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

[None][feat] add trtllm-gen kernels for glm4.7 and support groupsTokensHeadsQ + e2m1 output #11643: Updates trtllmGen FMHA cubin artifacts with focus on E4M3/E2M1 kernel variants and kernel registry extensions.
[TRTLLM-11119][feat] Blackwell SageAttention, Integrate into AttentionOp API #11718: Adds SM100 FMHA cubin pointers and extends kernel metadata declarations with similar scope and pattern.
[None][feat] Trtllm-gen FMHA JIT support #12612: Introduces and manages FMHA/JIT cubin artifacts and kernel registry that directly relate to these metadata and artifact updates.

Suggested labels

VisualGen

Suggested reviewers

niukuo
PerkzZheng
dongxuy04

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check	✅ Passed	The pull request description is clear, concise, and covers the essential changes (cubin updates, new FP8 kernels, kernel registration) with a test plan.
Title check	✅ Passed	The PR title '[TRTLLM-11538][feat] Blackwell custom mask fmha support' is related to the main objective of adding custom mask FMHA support for Blackwell (SM100f) GPUs. However, it is overly broad and does not clearly convey the specific scope of changes: updating 81 existing cubins for KeepsAbForGen variants and adding 16 new FP8 E4M3/E2M1 kernels. The title refers to the general feature domain but lacks specificity about the key changes.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 2

🧹 Nitpick comments (6)

cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp (1)
1-3: Reduce CI noise by excluding Git LFS pointer stubs from C/C++ static analysis.

The reported syntax errors are expected when analyzers parse unresolved LFS pointer text as .cpp. Consider filtering these cubin pointer files (or hydrating LFS before analysis) to avoid false-positive failures.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`
around lines 1 - 3, The CI static analyzer is parsing Git LFS pointer stub files
(files that start with the line "version https://git-lfs.github.com/spec/v1" and
contain "oid sha256:...") as .cpp and raising false syntax errors; update the
analysis pipeline/config to either (a) skip files that match the LFS pointer
pattern (detect the "version https://git-lfs.github.com/spec/v1" header or an
"oid sha256:" line) or (b) hydrate Git LFS objects before running analysis so
real .cpp content is present; ensure the rule/ignore list references these
marker patterns (or the specific file name
FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp)
so the analyzer won’t treat pointer stubs as C/C++ source.
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp (1)
1-3: Exclude LFS cubin pointer files from C/C++ static-analysis passes.

These lines are valid Git LFS pointer metadata, so the reported syntax errors are tooling noise rather than code defects. Please filter cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/*_cubin.cpp out of Cppcheck/Clang parsing to keep CI signal clean.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`
around lines 1 - 3, The CI static-analysis is parsing Git LFS pointer files (LFS
metadata) named with the suffix _cubin.cpp and reporting spurious syntax errors;
update the static-analyzer configuration or wrapper scripts (e.g.,
cppcheck/clang/clang-tidy invocation or the analyzer pre-scan) to skip files
that match the pattern *_cubin.cpp (or any file whose contents start with the
Git LFS pointer header like "version https://git-lfs.github.com/spec/v1") so
these pointer files are excluded from C/C++ parsing; implement the exclusion in
the analyzer config or by adding a pre-filter that detects LFS pointer headers
and omits those files from analysis.
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp (1)
2-3: LFS pointer refresh looks correct.

This change is a standard Git LFS pointer update (oid + size) for a cubin artifact; no source-level issue is evident here. Static-analysis parse errors on this file are expected noise from scanning LFS pointer text as C++.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`
around lines 2 - 3, This is an LFS pointer update (oid
4752608e09c346cd92237d6cfa556daf1c0fbadd7afe3a855d0e6087e0ad026f, size 832858)
rather than source code; do not modify C++ sources—keep the file as the Git LFS
pointer blob, ensure the .cubin extension is tracked in .gitattributes and that
the repository stores the actual binary in LFS (verify with git lfs ls-files),
then approve/merge the change.
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp (1)
1-3: Exclude Git LFS pointer stubs from C++ parsing/tooling.

Line 1–3 are valid Git LFS pointer metadata, but they are not valid C++. The Clang/Cppcheck errors here are tooling noise; please ensure static-analysis/compilation inputs skip unresolved *_cubin.cpp LFS pointer stubs (or enforce LFS fetch before analysis) to avoid false failures.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`
around lines 1 - 3, The CI/static-analysis is trying to parse Git LFS pointer
files (example:
FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp)
which contain LFS metadata (version/oid/size) and are not valid C++; update the
tooling to either skip files matching the *_cubin.cpp pattern (or any file whose
contents match an LFS pointer stub) during C++ parsing, or ensure Git LFS is
fetched before analysis so the real binary cubin files replace the pointer
stubs—implement the change in the static-analysis/compilation input discovery
logic to detect LFS pointer headers (e.g., lines starting with "version
https://git-lfs.github.com/spec/v1" or "oid sha256:") and exclude those files or
trigger an LFS fetch.
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp (1)
1-3: Optional: exclude Git LFS pointer files from C/C++ static analysis.

The reported syntax/type errors are tool noise from parsing LFS pointer text as C++. Consider filtering this cubin pointer path pattern from C++ analyzers.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`
around lines 1 - 3, Static analysis is picking up Git LFS pointer files (e.g.,
files that start with the literal line "version
https://git-lfs.github.com/spec/v1" and contain "oid sha256:...") and reporting
spurious C/C++ errors; update the analyzer configuration or pre-scan to exclude
files matching that LFS pointer pattern (match files whose content begins with
"version https://git-lfs.github.com/spec/v1" and/or contain an "oid sha256:"
line and/or the cubin pointer naming convention) so they are not fed to the
C/C++ parser, or add an exclude glob for these cubin pointer artifacts in your
analyzer rules or CI step that runs the static analysis.
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp (1)
1-3: Add an explicit CI guard for Git LFS hydration on cubin pointer files.

This file is a valid LFS pointer update; the reported Clang/Cppcheck syntax errors are expected when pointer text is parsed as C++. To prevent flaky failures outside LFS-enabled jobs, add a preflight step that verifies/pulls LFS artifacts before compilation.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`
around lines 1 - 3, Add a preflight CI step that detects and hydrates Git LFS
pointer files before compilation: check for the LFS pointer header string
("version https://git-lfs.github.com/spec/v1") or run `git lfs status`/`git lfs
ls-files` and if pointers are present run `git lfs pull` (or fail fast with a
clear message) prior to invoking the build/compile task (e.g., in your build
script or CI job that calls the compile step for the cubin files), so the C/C++
compiler never sees raw pointer text like the cubin pointer in
FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp`:
- Line 1: This file is missing the required NVIDIA copyright header; locate the
LFS pointer file named
FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
and either (A) add the standard NVIDIA copyright header with the appropriate
year of latest meaningful modification at the top of the file, or (B) if this
and similar Git LFS pointer files should be excluded from the rule, update
repository rules (e.g., .gitattributes or the CI/lint config that enforces the
header) to exclude *_cubin.cpp or LFS pointer files so the check no longer flags
them.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp`:
- Around line 1-3: The pre-commit workflow runs clang-format over all C/C++/CUDA
files (the clang-format hook in .pre-commit-config.yaml uses types_or: [c++, c,
cuda]) and fails on git-lfs pointer files like *cubin.cpp because
precommit-check.yml does not hydrate LFS; fix by either enabling LFS on the
checkout step (add lfs: true to the actions/checkout@v6 step in
precommit-check.yml) or exclude cubin files from the clang-format hook (add an
exclude pattern like *cubin.cpp to the clang-format hook entry in
.pre-commit-config.yaml) so clang-format won’t try to parse non-hydrated LFS
pointers.

---

Nitpick comments:
In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`:
- Around line 1-3: The CI/static-analysis is trying to parse Git LFS pointer
files (example:
FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp)
which contain LFS metadata (version/oid/size) and are not valid C++; update the
tooling to either skip files matching the *_cubin.cpp pattern (or any file whose
contents match an LFS pointer stub) during C++ parsing, or ensure Git LFS is
fetched before analysis so the real binary cubin files replace the pointer
stubs—implement the change in the static-analysis/compilation input discovery
logic to detect LFS pointer headers (e.g., lines starting with "version
https://git-lfs.github.com/spec/v1" or "oid sha256:") and exclude those files or
trigger an LFS fetch.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`:
- Around line 2-3: This is an LFS pointer update (oid
4752608e09c346cd92237d6cfa556daf1c0fbadd7afe3a855d0e6087e0ad026f, size 832858)
rather than source code; do not modify C++ sources—keep the file as the Git LFS
pointer blob, ensure the .cubin extension is tracked in .gitattributes and that
the repository stores the actual binary in LFS (verify with git lfs ls-files),
then approve/merge the change.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`:
- Around line 1-3: Add a preflight CI step that detects and hydrates Git LFS
pointer files before compilation: check for the LFS pointer header string
("version https://git-lfs.github.com/spec/v1") or run `git lfs status`/`git lfs
ls-files` and if pointers are present run `git lfs pull` (or fail fast with a
clear message) prior to invoking the build/compile task (e.g., in your build
script or CI job that calls the compile step for the cubin files), so the C/C++
compiler never sees raw pointer text like the cubin pointer in
FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`:
- Around line 1-3: Static analysis is picking up Git LFS pointer files (e.g.,
files that start with the literal line "version
https://git-lfs.github.com/spec/v1" and contain "oid sha256:...") and reporting
spurious C/C++ errors; update the analyzer configuration or pre-scan to exclude
files matching that LFS pointer pattern (match files whose content begins with
"version https://git-lfs.github.com/spec/v1" and/or contain an "oid sha256:"
line and/or the cubin pointer naming convention) so they are not fed to the
C/C++ parser, or add an exclude glob for these cubin pointer artifacts in your
analyzer rules or CI step that runs the static analysis.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`:
- Around line 1-3: The CI static-analysis is parsing Git LFS pointer files (LFS
metadata) named with the suffix _cubin.cpp and reporting spurious syntax errors;
update the static-analyzer configuration or wrapper scripts (e.g.,
cppcheck/clang/clang-tidy invocation or the analyzer pre-scan) to skip files
that match the pattern *_cubin.cpp (or any file whose contents start with the
Git LFS pointer header like "version https://git-lfs.github.com/spec/v1") so
these pointer files are excluded from C/C++ parsing; implement the exclusion in
the analyzer config or by adding a pre-filter that detects LFS pointer headers
and omits those files from analysis.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp`:
- Around line 1-3: The CI static analyzer is parsing Git LFS pointer stub files
(files that start with the line "version https://git-lfs.github.com/spec/v1" and
contain "oid sha256:...") as .cpp and raising false syntax errors; update the
analysis pipeline/config to either (a) skip files that match the LFS pointer
pattern (detect the "version https://git-lfs.github.com/spec/v1" header or an
"oid sha256:" line) or (b) hydrate Git LFS objects before running analysis so
real .cpp content is present; ensure the rule/ignore list references these
marker patterns (or the specific file name
FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp)
so the analyzer won’t treat pointer stubs as C/C++ source.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f20f5cd9-cf93-4467-8128-bddbc4cc2b5d

📥 Commits

Reviewing files that changed from the base of the PR and between 4ece13c and ecf5b67.

📒 Files selected for processing (97)

cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H128PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H128PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H128PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H128PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H64PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H64PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H64PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H64PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H64PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvBfloat16OBfloat16H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OBfloat16H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H64PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H64PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H64PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H64PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H64PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE2m1H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H64PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H64PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H64PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H64PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H64PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OE4m3H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H128PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H128PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H128PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H128PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvE4m3OFp16H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H128PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H64PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H64PagedKvCustomP32MultiCtasKvCgaVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H64PagedKvCustomP32MultiCtasKvVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H64PagedKvCustomP32MultiCtasKvVarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H64PagedKvCustomP32VarSeqQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H64PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128PersistentKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/FmhaSm100fKernel_QkvFp16OFp16H64PagedKvCustomP32VarSeqSkipsSoftmaxQ128Kv128StaticKeepsAbForGen_cubin.cpp
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h

tensorrt-cicd · 2026-04-12T04:37:37Z

PR_Github #42831 [ run ] completed with state SUCCESS. Commit: ecf5b67
/LLM/main/L0_MergeRequest_PR pipeline #33498 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

sunnyqgg · 2026-04-12T06:13:43Z

/bot run

tensorrt-cicd · 2026-04-12T06:20:44Z

PR_Github #42851 [ run ] triggered by Bot. Commit: ecf5b67 Link to invocation

tensorrt-cicd · 2026-04-12T07:58:00Z

PR_Github #42851 [ run ] completed with state SUCCESS. Commit: ecf5b67
/LLM/main/L0_MergeRequest_PR pipeline #33517 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

sunnyqgg · 2026-04-12T12:43:56Z

/bot run

tensorrt-cicd · 2026-04-12T12:49:56Z

PR_Github #42871 [ run ] triggered by Bot. Commit: ecf5b67 Link to invocation

tensorrt-cicd · 2026-04-12T14:31:16Z

PR_Github #42871 [ run ] completed with state SUCCESS. Commit: ecf5b67
/LLM/main/L0_MergeRequest_PR pipeline #33535 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

sunnyqgg · 2026-04-13T08:56:55Z

/bot run

tensorrt-cicd · 2026-04-13T09:02:26Z

PR_Github #43001 [ run ] triggered by Bot. Commit: ecf5b67 Link to invocation

tensorrt-cicd · 2026-04-13T11:10:33Z

PR_Github #43001 [ run ] completed with state SUCCESS. Commit: ecf5b67
/LLM/main/L0_MergeRequest_PR pipeline #33653 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

sunnyqgg · 2026-05-14T14:13:55Z

/bot run

sunnyqgg · 2026-05-15T07:28:03Z

/bot run

sunnyqgg · 2026-05-15T07:29:20Z

Update pushed in 7165e4fec0.

Refreshed FMHA artifacts from clean trtllm-gen f805ce8f37e992f1e92d664d39fb80ca781e8a3a.
Export used the final full path with internal cubin generation plus public x86 and public ARM libs; no --no-arm.
TLLM_GEN_VERSION is now f805ce8f.
Cubins are in the final compressed TRT-LLM form: 3388 *.cubin.tar.zst archives; intermediate *_cubin.cpp files are removed by the export script.
Refreshed both x86_64-linux-gnu and aarch64-linux-gnu FMHA static libraries.

sunnyqgg · 2026-05-18T06:59:48Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-18T07:05:22Z

PR_Github #48852 [ run ] triggered by Bot. Commit: 7c8657c Link to invocation

tensorrt-cicd · 2026-05-18T19:53:41Z

PR_Github #48852 [ run ] completed with state SUCCESS. Commit: 7c8657c
/LLM/main/L0_MergeRequest_PR pipeline #38606 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

mikeiovine · 2026-05-21T20:05:34Z

Would make things a bit easier to review if we could split the cubins into a separate commit

sunnyqgg · 2026-05-25T10:18:57Z

Would make things a bit easier to review if we could split the cubins into a separate commit

#14291 This PR has updated the custom cubins and trtllm-gen jit so I'll only update the runtime staff in this PR and will let you know when it's ready.Thanks

Signed-off-by: qgai <qgai@nvidia.com>

sunnyqgg · 2026-06-08T14:40:37Z

/bot run --disable-fail-fast

sunnyqgg · 2026-06-09T00:55:01Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-09T01:01:07Z

PR_Github #52895 [ run ] triggered by Bot. Commit: f96ceca Link to invocation

yihwang-nv

Thanks, LGTM!

sunnyqgg · 2026-06-09T07:28:54Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-09T07:35:12Z

PR_Github #53001 [ run ] triggered by Bot. Commit: f96ceca Link to invocation

tensorrt-cicd · 2026-06-09T07:40:23Z

PR_Github #52895 [ run ] completed with state ABORTED. Commit: f96ceca

Link to invocation

tensorrt-cicd · 2026-06-09T13:01:47Z

PR_Github #53001 [ run ] completed with state SUCCESS. Commit: f96ceca
/LLM/main/L0_MergeRequest_PR pipeline #42228 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

sunnyqgg · 2026-06-10T02:35:18Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-10T02:41:15Z

PR_Github #53202 [ run ] triggered by Bot. Commit: f96ceca Link to invocation

tensorrt-cicd · 2026-06-10T06:06:46Z

PR_Github #53202 [ run ] completed with state FAILURE. Commit: f96ceca
/LLM/main/L0_MergeRequest_PR pipeline #42399 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: qgai <qgai@nvidia.com>

sunnyqgg · 2026-06-10T07:36:35Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-10T07:42:23Z

PR_Github #53265 [ run ] triggered by Bot. Commit: a40d071 Link to invocation

tensorrt-cicd · 2026-06-10T14:41:07Z

PR_Github #53265 [ run ] completed with state SUCCESS. Commit: a40d071
/LLM/main/L0_MergeRequest_PR pipeline #42457 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

sunnyqgg · 2026-06-11T01:47:20Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-06-11T01:53:35Z

PR_Github #53425 [ run ] triggered by Bot. Commit: a40d071 Link to invocation

tensorrt-cicd · 2026-06-11T03:11:03Z

PR_Github #53425 [ run ] completed with state SUCCESS. Commit: a40d071
/LLM/main/L0_MergeRequest_PR pipeline #42594 completed with status: 'SUCCESS'

CI Report

Link to invocation

Signed-off-by: qgai <qgai@nvidia.com>

sunnyqgg · 2026-06-11T05:18:03Z

/bot run --help

tensorrt-cicd · 2026-06-11T05:26:20Z

PR_Github #53485 Bot args parsing error: usage: /bot [-h]
{run,kill,skip,submit,reviewers,reuse-pipeline,reuse-review} ...
/bot: error: unrecognized arguments: --help

Link to invocation

sunnyqgg · 2026-06-11T05:36:25Z

/bot reuse-pipeline

tensorrt-cicd · 2026-06-11T05:42:12Z

PR_Github #53488 [ reuse-pipeline ] triggered by Bot. Commit: fb7c9fb Link to invocation

tensorrt-cicd · 2026-06-11T05:48:11Z

PR_Github #53488 [ reuse-pipeline ] completed with state SUCCESS. Commit: fb7c9fb
Reusing PR_Github #53425 for commit fb7c9fb

Link to invocation

github-actions Bot assigned sunnyqgg Apr 12, 2026

coderabbitai Bot reviewed Apr 12, 2026

View reviewed changes

Comment thread ...m100fKernel_QkvE4m3OBfloat16H128PagedKvCustomP32VarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp Outdated

Comment thread ...kvE4m3OBfloat16H64PagedKvCustomP32MultiCtasKvCgaVarSeqQ128Kv128StaticKeepsAbForGen_cubin.cpp Outdated

sunnyqgg changed the title ~~[None][feat] Add SM100f custom mask FMHA cubins for KeepsAbForGen (bf16/fp16 update + FP8 E4M3/E2M1)~~ [TRTLLM-11538][feat] Add SM100f custom mask FMHA cubins for KeepsAbForGen (bf16/fp16 update + FP8 E4M3/E2M1) Apr 13, 2026

This was referenced Apr 17, 2026

[TRTLLM-11540][feat] Add EAGLE3 dynamic tree speculative decoding support #12062

Merged

[TRTLLM-11540][feat] Revert revert of EAGLE3 dynamic tree speculative decoding support #13081

Merged

sunnyqgg changed the title ~~[TRTLLM-11538][feat] Add SM100f custom mask FMHA cubins for KeepsAbForGen (bf16/fp16 update + FP8 E4M3/E2M1)~~ [TRTLLM-11538][feat] Blackwell custom mask fmha support Apr 24, 2026

sunnyqgg force-pushed the dyanmic_tree_support_b100_cubin branch from ecf5b67 to ae69f46 Compare May 14, 2026 14:13

sunnyqgg force-pushed the dyanmic_tree_support_b100_cubin branch from 7165e4f to 7c8657c Compare May 18, 2026 03:48

sunnyqgg force-pushed the dyanmic_tree_support_b100_cubin branch from 7c8657c to 4a289aa Compare May 29, 2026 07:32

sunnyqgg added 4 commits June 8, 2026 07:36

[None][fix] Make custom mask offsets deterministic

5b63849

Signed-off-by: qgai <qgai@nvidia.com>

[None][fix] Restore NVRTC block-size guard

478c662

Signed-off-by: qgai <qgai@nvidia.com>

[None][refactor] Remove unused spec-dec max draft field

6b37be1

Signed-off-by: qgai <qgai@nvidia.com>

[None][fix] Fix spec-dec mask layout and scope block-size guard

f96ceca

Signed-off-by: qgai <qgai@nvidia.com>

sunnyqgg force-pushed the dyanmic_tree_support_b100_cubin branch from 1b211e1 to f96ceca Compare June 8, 2026 14:40

yihwang-nv approved these changes Jun 9, 2026

View reviewed changes

[None][fix] Keep spec-dec KV length scoped

a40d071

Signed-off-by: qgai <qgai@nvidia.com>

[None][refactor] Remove Eagle3 speculative change

fb7c9fb

Signed-off-by: qgai <qgai@nvidia.com>

sunnyqgg merged commit a622e30 into NVIDIA:main Jun 11, 2026
7 checks passed

Uh oh!

Conversation

sunnyqgg commented Apr 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Test plan

Summary by CodeRabbit

Uh oh!

sunnyqgg commented Apr 12, 2026

Uh oh!

tensorrt-cicd commented Apr 12, 2026

Uh oh!

coderabbitai Bot commented Apr 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tensorrt-cicd commented Apr 12, 2026

Uh oh!

sunnyqgg commented Apr 12, 2026

Uh oh!

tensorrt-cicd commented Apr 12, 2026

Uh oh!

tensorrt-cicd commented Apr 12, 2026

Uh oh!

sunnyqgg commented Apr 12, 2026

Uh oh!

tensorrt-cicd commented Apr 12, 2026

Uh oh!

tensorrt-cicd commented Apr 12, 2026

Uh oh!

sunnyqgg commented Apr 13, 2026

Uh oh!

tensorrt-cicd commented Apr 13, 2026

Uh oh!

tensorrt-cicd commented Apr 13, 2026

Uh oh!

sunnyqgg commented May 14, 2026

Uh oh!

sunnyqgg commented May 15, 2026

Uh oh!

sunnyqgg commented May 15, 2026

Uh oh!

sunnyqgg commented May 18, 2026

Uh oh!

tensorrt-cicd commented May 18, 2026

Uh oh!

tensorrt-cicd commented May 18, 2026

Uh oh!

mikeiovine commented May 21, 2026

Uh oh!

sunnyqgg commented May 25, 2026

Uh oh!

sunnyqgg commented Jun 8, 2026

Uh oh!

sunnyqgg commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

yihwang-nv left a comment

Choose a reason for hiding this comment

Uh oh!

sunnyqgg commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

tensorrt-cicd commented Jun 9, 2026

Uh oh!

sunnyqgg commented Apr 12, 2026 •

edited

Loading

coderabbitai Bot commented Apr 12, 2026 •

edited

Loading