Repository navigation
[None][chore] Update trtllm-gen FMHA artifacts - #19591
Conversation
|
/bot run --disable-fail-fast |
|
Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements. |
|
PR_Github #75231 [ run ] triggered by Bot. Commit: |
|
PR_Github #75231 [ run ] completed with state
|
|
Evaluate semantic conflicts between the fixed head and target revisions below. Read the repository and verify all three full commit IDs and their merge-base. Inspect affected callers, implementations, imports, tests and test doubles, Use read-only source and Git inspection. Do not modify repository files, execute Report concrete incompatibilities with their trigger, observable failure, End with a standalone heading Put the fixed head and target source citations after the standalone heading too. Analyze only these fixed revisions in NVIDIA/TensorRT-LLM, not the hosting PR: |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -u
H=abd476ed8790ea493fa261c65a3ae75f8b93b58f
T=f00e9627fa9879a56544d5382f4e5193cb0cc2b2
B=c3785d305f4455a6f63f737747c360b4a2e55ef1
git remote -v
for id in "$H" "$T" "$B"; do echo "COMMIT $id"; git cat-file -t "$id"; git rev-parse "$id^{commit}"; done
echo MERGE_BASE
git merge-base "$H" "$T"
echo HEAD_DIFF_STAT
git diff --stat "$B" "$H"
echo TARGET_DIFF_STAT
git diff --stat "$B" "$T"
echo HEAD_NAMES
git diff --name-status "$B" "$H"
echo TARGET_NAMES
git diff --name-status "$B" "$T"Length of output: 201162 🏁 Script executed: #!/bin/bash
set -euo pipefail
H=abd476ed8790ea493fa261c65a3ae75f8b93b58f
T=f00e9627fa9879a56544d5382f4e5193cb0cc2b2
B=c3785d305f4455a6f63f737747c360b4a2e55ef1
echo HEAD_NON_CUBIN
git diff --name-status "$B" "$H" -- cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha | grep -v 'cubin.tar.zst' | head -120
echo HEAD_OTHER
git diff --name-status "$B" "$H" -- . ':!cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha' | head -120
echo TARGET_FMHA_RELATED
git diff --name-status "$B" "$T" -- cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention cpp/tensorrt_llm/kernels/trtllmGenKernels tensorrt_llm/_torch/attention tests/unittest/_torch/attention
echo HEAD_NON_ARTIFACT_PATCH
git diff "$B" "$H" -- cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha ':!*.cubin.tar.zst' ':!*.a' | head -n 1000
echo TARGET_FMHA_PATCH
git diff "$B" "$T" -- cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention cpp/tensorrt_llm/kernels/trtllmGenKernels tensorrt_llm/_torch/attention/backends/fmha tests/unittest/_torch/attention/test_context_fmha_kernel_presence.py | head -n 1000Length of output: 201879 🏁 Script executed: #!/bin/bash
set -u
H=abd476ed8790ea493fa261c65a3ae75f8b93b58f
T=f00e9627fa9879a56544d5382f4e5193cb0cc2b2
B=c3785d305f4455a6f63f737747c360b4a2e55ef1
echo HEAD_HEADER_DIFFS
git diff -U2 "$B" "$H" -- cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export | sed -n '1,750p'
echo TARGET_CPP_FMHA_DIFFS
git diff -U3 "$B" "$T" -- cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention | sed -n '1,500p'
echo TARGET_PY_FMHA_DIFFS
git diff -U2 "$B" "$T" -- tensorrt_llm/_torch/attention/backends/fmha | sed -n '1,450p'
echo TARGET_PRESENCE_TEST
git show "$T":tests/unittest/_torch/attention/test_context_fmha_kernel_presence.py | nl -ba | sed -n '1,250p'Length of output: 62681 🏁 Script executed: #!/bin/bash
set -u
H=abd476ed8790ea493fa261c65a3ae75f8b93b58f
T=f00e9627fa9879a56544d5382f4e5193cb0cc2b2
B=c3785d305f4455a6f63f737747c360b4a2e55ef1
echo CALLS
rg -n -C2 'getMmaOpsPerClk\s*\(|mStoreTransformedKvInTmem|mUsesMmaForSoftmaxSum' cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention cpp/tensorrt_llm/kernels/trtllmGenKernels -g '*.cpp' -g '*.h' | grep -v 'kernelMetaInfo.h' | sed -n '1,180p'
echo HEAD_TABLE_ANALYSIS
python - <<'PY'
import subprocess,re,collections
p='cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h'
for rev in ('c3785d305f4455a6f63f737747c360b4a2e55ef1','abd476ed8790ea493fa261c65a3ae75f8b93b58f'):
s=subprocess.check_output(['git','show',rev+':'+p],text=True)
print(rev,'lines',len(s.splitlines()))
for pattern in ('QkvBfloat16OBfloat16H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext','QkvFp16OFp16H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext','QE4m3KvE2m1OBfloat16H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext','QE4m3KvE2m1OE4m3H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext'):
print(pattern, len(re.findall(r'^\{[^\n]*'+pattern,s,re.M)))
for label in ('mStoreTransformedKvInTmem','mUsesMmaForSoftmaxSum'):
print(label,[i for i,line in enumerate(s.splitlines(),1) if label in line][:6])
PY
echo TARGET_LOOKUP
git diff -U2 "$B" "$T" -- cpp/tensorrt_llm/thop/attentionOp.cpp cpp/tensorrt_llm/common/attentionOp.h | sed -n '1,300p'
echo HEAD_VS_TARGET_CMAKE
git show "$T":cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention/fmhaRunner.cpp | nl -ba | sed -n '180,220p'Length of output: 26098 🏁 Script executed: #!/bin/bash
set -u
H=abd476ed8790ea493fa261c65a3ae75f8b93b58f
T=f00e9627fa9879a56544d5382f4e5193cb0cc2b2
echo TARGET_LOOKUP_SYMBOLS
git show "$T":cpp/tensorrt_llm/thop/attentionOp.cpp | nl -ba | grep -C12 'fused_context_fmha_kernel_exists'
echo TARGET_BINDING_SYMBOLS
git show "$T":cpp/tensorrt_llm/nanobind/thop/bindings.cpp | nl -ba | grep -C8 'fused_context_fmha_kernel_exists'
echo HEAD_META_H96
git show "$H":cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h | grep -n -m 4 'Bfloat16OBfloat16H96PagedKv'
echo HEAD_META_STRUCT
git show "$H":cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h | nl -ba | sed -n '15100,15152p'
echo TARGET_GEN_RUNTIME_SYMBOLS
git grep -n -E 'getMmaOpsPerClk|TllmGenFmhaKernelMetaInfo|mStoreTransformedKvInTmem|mUsesMmaForSoftmaxSum' "$T" -- cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention cpp/tensorrt_llm/thop cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha 2>/dev/null | grep -v 'kernelMetaInfo.h' | sed -n '1,95p'Length of output: 11139 🏁 Script executed: #!/bin/bash
set -u
H=abd476ed8790ea493fa261c65a3ae75f8b93b58f
T=f00e9627fa9879a56544d5382f4e5193cb0cc2b2
echo BINDING_OUTLINE
ast-grep outline cpp/tensorrt_llm/nanobind/thop/bindings.cpp --match fusedContextFmhaKernelExists --view expanded | sed -n '1,95p'
echo TARGET_BINDING
git show "$T":cpp/tensorrt_llm/nanobind/thop/bindings.cpp | nl -ba | sed -n '80,210p'
echo HEAD_FMHA_FACTORY
git show "$H":cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/fmhaKernels.h | nl -ba | sed -n '130,225p;1560,1605p'
echo HEAD_META_MATCHES
git show "$H":cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h | nl -ba | grep -E -m 9 '^\s*[0-9]+\s+\{ DATA_TYPE_(BF16|FP16|E4M3), DATA_TYPE_(BF16|FP16|E2M1), .*H64PagedKvCausalP32VarSeqQ256Kv128(Persistent|Static)Context' | cut -c1-380
echo HEAD_HEADER_DETAILS
git show "$H":cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export/FmhaAutoTuner.h | nl -ba | sed -n '39,57p'Length of output: 21424
SEMANTIC_REVIEWNo concrete semantic conflict found. Git verified all three full commit IDs and confirmed that I checked the head’s generated FMHA metadata and exported header changes against the target’s FMHA dispatcher, diagnostic binding, caller changes, and kernel-presence tests. In both directions, the inspected contracts remain compatible: the target’s H64 paged-context probes have matching head metadata entries. The target’s diagnostic queries FMHA support using the Q, KV, and output types represented by the head table. The head’s changed This is a best-effort source review, not proof of runtime safety. I did not execute tests or validate the generated cubins on a GPU. SEMANTIC_RESULT request_id=b2a8b294-2064-4a2f-bcd1-38e61499e75a head=abd476ed8790ea493fa261c65a3ae75f8b93b58f target=f00e9627fa9879a56544d5382f4e5193cb0cc2b2 merge_base=c3785d305f4455a6f63f737747c360b4a2e55ef1 verdict=PASS You are interacting with an AI system. |
Signed-off-by: Pengbo Wang <221450789+pengbowang-nv@users.noreply.github.com>
Signed-off-by: Pengbo Wang <221450789+pengbowang-nv@users.noreply.github.com>
abd476e to
a98c40d
Compare
|
/bot run --disable-fail-fast |
|
Important Review skippedWe couldn't safely recover the incremental review. No full review was started, and the last reviewed checkpoint was preserved. Retry later, or explicitly request a full review by commenting You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
PR_Github #75513 [ run ] triggered by Bot. Commit: |
|
Evaluate semantic conflicts between the fixed head and target revisions below. Read the repository and verify all three full commit IDs and their merge-base. Inspect affected callers, implementations, imports, tests and test doubles, Use read-only source and Git inspection. Do not modify repository files, execute Report concrete incompatibilities with their trigger, observable failure, End with a standalone heading Put the fixed head and target source citations after the standalone heading too. Analyze only these fixed revisions in NVIDIA/TensorRT-LLM, not the hosting PR: |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
B=0c58480ca680f2946bb94315af3517c9c7000bca
git remote -v | head -5
for id in "$H" "$T" "$B"; do printf 'commit %s type=' "$id"; git cat-file -t "$id"; git rev-parse "$id^{commit}"; done
printf 'actual merge-base: '; git merge-base "$H" "$T"
echo 'STAT:'
git diff --stat "$T" "$H" | tail -40
echo 'CHANGED PATHS (non-artifact):'
git diff --name-status "$T" "$H" | grep -Ev '\.(cubin|a|o|zst|so|ptx|fatbin|tar|bin)([[:space:]]|$)' | head -220
echo 'TOTAL COUNTS:'
git diff --name-status "$T" "$H" | awk '{a[$1]++} END {for (k in a) print k,a[k]}'Length of output: 4765 🏁 Script executed: #!/bin/bash
set -euo pipefail
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo 'HEADER PATCH:'
git diff -U2 "$T" "$H" -- "$P/trtllmGen_fmha_export" | head -950
echo 'META PATCH SHORT:'
git diff -U1 "$T" "$H" -- "$P/cubin/kernelMetaInfo.h" | head -160
echo 'TARGET RUNTIME REFERENCES:'
git grep -n -E 'getMmaOpsPerClk|mStoreTransformedKvInTmem|mUsesMmaForSoftmaxSum|FmhaAutoTuner|FmhaOptions|KernelTraits|KernelParams|TllmGenFmhaKernelMetaInfo' "$T" -- 'cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention' 'cpp/tensorrt_llm/kernels/trtllmGenKernels' 'cpp/tensorrt_llm/nanobind' 'cpp/tensorrt_llm/thop' 2>/dev/null | grep -v 'kernelMetaInfo.h' | head -105
echo 'TEST REFERENCES:'
git grep -n -E 'fused_context_fmha_kernel_exists|trtllm_gen_fmha|trtllmGen.*fmha' "$T" -- tests 2>/dev/null | head -70Length of output: 60884 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo 'TARGET EXTERNAL USERS OF CHANGED SYMBOLS:'
git grep -n -E 'getMmaOpsPerClk[[:space:]]*\(|mUsesMmaForSoftmaxSum|mStoreTransformedKvInTmem|mDtypeK|mDtypeV|mKvSf' "$T" -- 'cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention' 'cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/fmhaKernels.h' | sed -n '1,115p'
echo 'TARGET HEADER REFS:'
git grep -n -E 'getMmaOpsPerClk[[:space:]]*\(|makeTmaShapeStrideKvSf[[:space:]]*\(|setKernelParams[[:space:]]*\(' "$T" -- cpp | grep -v 'kernelMetaInfo.h' | sed -n '1,105p'
echo 'TESTS:'
git grep -n -E 'fused_context_fmha_kernel_exists|trtllm_gen_fmha' "$T" -- tests | sed -n '1,70p'
echo 'HEAD META PATTERNS & COLUMNS:'
git show "$H":"$P/cubin/kernelMetaInfo.h" | grep -n -m 3 'QkE4m3VE2m1OBfloat16H128PagedKvCausalP32VarSeqQ256Kv128PersistentContext' | cut -c1-440
echo 'TARGET RUNNER IDENTIFIERS:'
git show "$T":cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention/fmhaRunner.cpp | grep -n -E 'DtypeK|DtypeV|DtypeKv|sf|Sf|stride|Stride' | sed -n '1,100p'Length of output: 14277 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
ast-grep outline "$P/fmhaKernels.h" --match 'Fmha|Kernel' --view compact 2>/dev/null | sed -n '1,45p'
ast-grep outline "$P/trtllmGen_fmha_export/KernelParams.h" --match 'setKernelParams|makeTmaShapeStrideKvSf' --view compact 2>/dev/null | sed -n '1,40p'
echo 'TARGET LOOKUP + FILTER'
git show "$T":"$P/fmhaKernels.h" | nl -ba | sed -n '145,195p;265,290p;610,654p'
echo 'TARGET RUN CONFIG'
git show "$T":"$P/fmhaKernels.h" | nl -ba | sed -n '1035,1075p;1140,1195p;1230,1297p'
echo 'HEAD DESCRIPTOR'
git show "$H":"$P/trtllmGen_fmha_export/KernelParams.h" | nl -ba | sed -n '470,545p;665,797p'
echo 'TARGET TEST'
git show "$T":tests/unittest/_torch/attention/test_context_fmha_kernel_presence.py | nl -ba | sed -n '80,155p'Length of output: 34002 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo 'META ROWS MIXED:'
git show "$H":"$P/cubin/kernelMetaInfo.h" | nl -ba | grep -m 4 -E '\{ DATA_TYPE_E4M3, DATA_TYPE_E4M3, DATA_TYPE_E2M1|^\s*[0-9]+\s+\{ DATA_TYPE_E4M3, DATA_TYPE_E4M3, DATA_TYPE_E2M1' | cut -c1-510
echo 'META ROWS OLD:'
git show "$T":"$P/cubin/kernelMetaInfo.h" | nl -ba | grep -m 2 'DATA_TYPE_E2M1' | cut -c1-510
echo 'HEAD META FIELDS:'
git show "$H":"$P/cubin/kernelMetaInfo.h" | nl -ba | grep -n 'struct TllmGenFmhaKernelMetaInfo' | sed -n '1,3p'
echo 'TARGET META HASH CALLS:'
git show "$T":"$P/fmhaKernels.h" | nl -ba | sed -n '285,390p;935,1030p;1075,1140p'
echo 'TARGET HASH DEF SEARCH:'
git grep -n 'hashFromFmhaOptions' "$T" -- "$P" | sed -n '1,30p'
echo 'TARGET SCALE POINTER PATHS:'
git show "$T":"$P/fmhaKernels.h" | nl -ba | grep -C4 -E 'kSfBasePtr|vSfBasePtr|kvSfPtr|sfPtr' | sed -n '1,135p'Length of output: 19691 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo 'TARGET FACTORY API LOCATIONS:'
git grep -n -E 'TllmGenFmhaKernelFactory|TllmGenFmhaKernel<|get.*FmhaKernel|create.*FmhaKernel|dtypeV[[:space:]]*=' "$T" -- "$P" cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention cpp/tensorrt_llm/thop | sed -n '1,125p'
echo 'TARGET HASH LOADER:'
git show "$T":"$P/fmhaKernels.h" | nl -ba | sed -n '195,235p;775,850p'
echo 'HEAD METADATA STRUCT AND MIXED ROW:'
git show "$H":"$P/cubin/kernelMetaInfo.h" | nl -ba | sed -n '15845,15865p;15100,15165p' | cut -c1-650
echo 'HEAD MIXED META ROW EXACT:'
git show "$H":"$P/cubin/kernelMetaInfo.h" | nl -ba | grep -m 3 -E '^\s*[0-9]+\s*\{\s*DATA_TYPE_E4M3,\s*DATA_TYPE_E4M3,\s*DATA_TYPE_E2M1' | cut -c1-760
echo 'TARGET RUNNER PARAM DECL REFS:'
git grep -n 'struct TllmGenFmhaRunnerParams' "$T" -- "$P" cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttentionLength of output: 36719 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
python - <<'PY'
import subprocess,re
p='cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h'
for rev in ('0c58480ca680f2946bb94315af3517c9c7000bca','a98c40d35cd49948d0974df3f75d362b4b68f4f3'):
s=subprocess.check_output(['git','show',f'{rev}:{p}'],text=True)
matches=[(i,x) for i,x in enumerate(s.splitlines(),1) if re.match(r'\{\s*DATA_TYPE_E4M3,\s*DATA_TYPE_E4M3,\s*DATA_TYPE_E4M3,\s*DATA_TYPE_E2M1,',x)]
print(rev,'mixed rows',len(matches))
for i,x in matches[:3]: print(i,x[:750])
PY
echo 'FACTORY CALLS:'
git grep -n -E 'getTllmFmhaKernels[[:space:]]*\(|TllmGenFmhaRunner[[:space:]]*\(' "$T" -- cpp | sed -n '1,100p'
echo 'PARAMS DATA FIELDS'
git show "$T":"$P/fmhaRunnerParams.h" | nl -ba | sed -n '225,370p'
echo 'RUNNER INITIALIZATION'
git show "$T":"$P/fmhaRunner.cpp" | nl -ba | sed -n '35,110p'Length of output: 13644 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
python - <<'PY'
import subprocess
p='cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h'
for rev in ('0c58480ca680f2946bb94315af3517c9c7000bca','a98c40d35cd49948d0974df3f75d362b4b68f4f3'):
s=subprocess.check_output(['git','show',f'{rev}:{p}'],text=True)
matches=[(i,x) for i,x in enumerate(s.splitlines(),1) if x.startswith('{') and 'QkE4m3VE2m1OBfloat16H128PagedKvCausalP32VarSeqQ256Kv128PersistentContext' in x]
print(rev, 'mixed entries',len(matches))
for i,x in matches[:2]: print(i,x[:950])
PY
echo 'TARGET DISPATCHER SOURCES'
git show "$T":cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp | nl -ba | sed -n '32,77p'
git show "$T":cpp/tensorrt_llm/common/attentionOp.cpp | nl -ba | sed -n '3171,3202p'
git show "$T":cpp/tensorrt_llm/kernels/xqaDispatcher.cpp | nl -ba | sed -n '125,155p'
echo 'HEAD MIXED DESCRIPTOR AND TARGET PAGE REF'
git show "$H":"$P/trtllmGen_fmha_export/KernelParams.h" | nl -ba | sed -n '310,353p;383,455p'
git show "$T":"$P/fmhaKernels.h" | nl -ba | sed -n '985,1009p;1026,1033p'Length of output: 16265 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo 'UNPACK DECL/USE:'
git grep -n -E 'unpack_kv_data_type[[:space:]]*\(|DATA_TYPE_UNKNOWN|E4M3.*E2M1|E2M1.*E4M3' "$T" -- cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp cpp/tensorrt_llm/common cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/fmhaRunnerParams.h | sed -n '1,105p'
echo 'HEAD TRTLLM STRIDES:'
git show "$H":"$P/trtllmGen_fmha_export/KernelParams.h" | nl -ba | sed -n '349,385p'
echo 'TARGET layout enablement:'
git show "$T":"$P/fmhaKernels.h" | nl -ba | grep -n -C5 -E 'mIsTrtllmLayout|mQkvLayout' | sed -n '1,130p'
echo 'TARGET CMAKE ARTIFACT REFS:'
git grep -n -E 'libTrtLlmGenFmhaLib|kernelMetaInfo.h' "$T" -- "$P" | sed -n '1,60p'Length of output: 12586 🏁 Script executed: #!/bin/bash
set -u
T=0c58480ca680f2946bb94315af3517c9c7000bca
echo 'TARGET dtype decode definitions:'
git grep -n -E 'unpack_kv_data_type[[:space:]]*\(|PACKED.*KV|E4M3_E2M1|E4M3.*E2M1' "$T" -- cpp | sed -n '1,105p'
echo 'TARGET mixed dtype tests and shapes:'
git grep -n -E 'dtypeK.*dtypeV|dataTypeK.*dataTypeV|e4m3.*e2m1' "$T" -- tests cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha | grep -v -E 'kernelMetaInfo|cubin|\.zst' | sed -n '1,65p'Length of output: 57595 🏁 Script executed: #!/bin/bash
set -u
T=0c58480ca680f2946bb94315af3517c9c7000bca
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
git show "$T":cpp/tensorrt_llm/kernels/multiHeadAttentionCommon.h | nl -ba | sed -n '50,93p'
git show "$T":cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/fmhaKernels.h | nl -ba | sed -n '1574,1603p;1495,1532p'
git show "$H":cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h | nl -ba | sed -n '15370,15377p' | cut -c1-300Length of output: 8167
SEMANTIC_REVIEWFinding — the main FMHA dispatcher cannot select the new mixed K/V cubins. Confidence: high. The head adds a paged-context cubin for E4M3 K and E2M1 V in Line 15374. The target dispatcher obtains both K and V types from one Trigger and observable failure: When a caller uses I also inspected the target runner’s metadata filter, parameter setup, and kernel-presence tests against the head’s changed headers and metadata. I did not run tests or validate cubins on a GPU. SEMANTIC_RESULT request_id=2904bba8-b2c9-4d2c-a623-aea6ba714122 head=a98c40d35cd49948d0974df3f75d362b4b68f4f3 target=0c58480ca680f2946bb94315af3517c9c7000bca merge_base=0c58480ca680f2946bb94315af3517c9c7000bca verdict=FAIL 🧠 Learnings usedYou are interacting with an AI system. |
|
@CodeRabbit This only means this feature is not enabled. It shouldn't affect any pre-existing test or usage. |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
for id in "$H" "$T"; do git cat-file -t "$id"; git rev-parse "$id^{commit}"; done
git merge-base "$H" "$T"
echo 'changed nonbinary paths'
git diff --name-status "$T" "$H" | grep -Ev '\.(cubin|a|o|zst|so|ptx|fatbin|tar|bin)([[:space:]]|$)' | head -90
echo 'target dispatcher'
git show "$T":cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp | nl -ba | sed -n '35,86p'
echo 'target type decode'
git show "$T":cpp/tensorrt_llm/kernels/multiHeadAttentionCommon.h | nl -ba | sed -n '57,85p'
echo 'target kernel metadata loading'
git show "$T":"$P/fmhaKernels.h" | nl -ba | sed -n '145,230p;260,310p'
echo 'target type filtering'
git show "$T":"$P/fmhaKernels.h" | nl -ba | sed -n '780,845p'
echo 'head mixed metadata'
git show "$H":"$P/cubin/kernelMetaInfo.h" | nl -ba | sed -n '15371,15376p' | cut -c1-550
echo 'target test'
git show "$T":tests/unittest/_torch/attention/test_context_fmha_kernel_presence.py | nl -ba | sed -n '75,145p'Length of output: 25853
I withdraw my semantic-conflict finding. I did not run tests, so I cannot independently confirm that every existing path is unaffected. You are interacting with an AI system. |
|
@coderabbitai repost the sematic analysis result |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=0c58480ca680f2946bb94315af3517c9c7000bca
B=0c58480ca680f2946bb94315af3517c9c7000bca
for id in "$H" "$T" "$B"; do printf 'verified %s: ' "$id"; git rev-parse --verify "$id^{commit}"; done
printf 'merge-base: '; git merge-base "$H" "$T"
printf 'changed paths (first 45):\n'; git diff --name-status "$T" "$H" | head -45 || true
echo 'head mixed entry:'
git show "$H":cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h | nl -ba | sed -n '15372,15375p' | cut -c1-480
echo 'target dispatcher:'
git show "$T":cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp | nl -ba | sed -n '54,68p'
echo 'target unpack:'
git show "$T":cpp/tensorrt_llm/kernels/multiHeadAttentionCommon.h | nl -ba | sed -n '68,82p'
echo 'target metadata lookup:'
git show "$T":cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/fmhaKernels.h | nl -ba | sed -n '156,181p;274,292p'
echo 'target presence test:'
git show "$T":tests/unittest/_torch/attention/test_context_fmha_kernel_presence.py | nl -ba | sed -n '81,104p'Length of output: 16163
SEMANTIC_REVIEWNo concrete semantic conflict found. The head contains mixed E4M3-K/E2M1-V metadata. The target dispatcher uses a single KV type value, and the target type mapping cannot produce that mixed pair. The new configuration is therefore unavailable through this dispatcher. This is not evidence that an existing configuration fails. The source review compared SEMANTIC_RESULT request_id=2904bba8-b2c9-4d2c-a623-aea6ba714122 head=a98c40d35cd49948d0974df3f75d362b4b68f4f3 target=0c58480ca680f2946bb94315af3517c9c7000bca merge_base=0c58480ca680f2946bb94315af3517c9c7000bca verdict=PASS You are interacting with an AI system. |
|
/bot help |
GitHub Bot Help
Provide a user friendly way for developers to interact with a Jenkins server. Run See details below for each supported subcommand. Details
Launch build/test pipelines. All previously running jobs will be killed.
kill
Kill all running builds associated with pull request. skip
Skip testing for latest commit on pull request. reuse-pipeline
Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break. |
|
PR_Github #75513 [ run ] completed with state
|
|
/bot run --disable-fail-fast |
|
PR_Github #75532 [ run ] triggered by Bot. Commit: |
|
Evaluate behavioral incompatibilities involving the supplied PR changes and the Verify the three full commit IDs and their merge-base. For divergent branches, When head contains target ( First identify changed or removed interfaces and symbols, then search repository Classify findings in prose as cross-branch interactions or PR-local compatibility Use read-only source and Git inspection. Do not modify checked-out files, execute Assess independent findings separately. Uncertainty about one path does not
Write exactly one standalone heading
Then give the verdict, findings or inspected paths, and limitations. For PASS and Copy the supplied identity and commit IDs verbatim. Always include the result Analyze only these fixed revisions in NVIDIA/TensorRT-LLM, not the hosting PR: |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=336c4337a0a4d2f1479697c1f03e473b92fb3ee7
B=0c58480ca680f2946bb94315af3517c9c7000bca
git remote -v | head -4
for id in "$H" "$T" "$B"; do echo "OBJECT $id"; git cat-file -t "$id"; git rev-parse "$id^{commit}"; done
echo "MERGE_BASE"; git merge-base "$H" "$T"
echo "HEAD_CHANGES"; git diff --name-status "$B" "$H" | head -110
echo "TARGET_CHANGES"; git diff --name-status "$B" "$T" | head -220
echo "TARGET_STAT"; git diff --stat "$B" "$T" | tail -35
echo "HEAD_STAT"; git diff --stat "$B" "$H" | tail -15
echo "MERGE_PREVIEW"; git merge-tree --write-tree "$H" "$T" 2>&1 | head -70 || trueLength of output: 20276 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=336c4337a0a4d2f1479697c1f03e473b92fb3ee7
B=0c58480ca680f2946bb94315af3517c9c7000bca
echo HEAD_SOURCE_PATHS
git diff --name-status "$B" "$H" | grep -Ev '\.(cubin\.tar\.zst|a|o|so|ptx|fatbin|bin)$'
echo TARGET_PATHS
git diff --name-status "$B" "$T" | grep -Ev '\.(cubin\.tar\.zst|a|o|so|ptx|fatbin|bin)$' | sed -n '1,300p'
echo TARGET_STAT
git diff --stat "$B" "$T" | tail -30
echo MERGE_PREVIEW
git merge-tree --write-tree "$H" "$T" 2>&1 | sed -n '1,95p'
echo HEAD_SOURCE_DIFFSTAT
git diff --stat "$B" "$H" -- ':(exclude)*.cubin.tar.zst' | tail -35Length of output: 6597 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=336c4337a0a4d2f1479697c1f03e473b92fb3ee7
B=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo HEAD_CHANGED_HEADERS
git diff -U2 "$B" "$H" -- "$P/trtllmGen_fmha_export/FmhaAutoTuner.h" "$P/trtllmGen_fmha_export/FmhaOptions.h" "$P/trtllmGen_fmha_export/KernelConfigBase.h" "$P/trtllmGen_fmha_export/KernelParams.h" "$P/trtllmGen_fmha_export/KernelTraits.h" "$P/trtllmGen_fmha_export/trtllm/dev/CutlassUtils.h" "$P/trtllmGen_fmha_export/trtllm/dev/Fp4Utils.h" "$P/trtllmGen_fmha_export/trtllm/dev/Utils.h" "$P/trtllmGen_fmha_export/trtllm/gen/DtypeDecl.h" | sed -n '1,930p'
echo TARGET_CPP_REFS
git grep -n -E 'getMmaOpsPerClk[[:space:]]*\(|makeTmaShapeStrideKvSf[[:space:]]*\(|mStoreTransformedKvInTmem|mUsesMmaForSoftmaxSum|mDtypeK|mDtypeV|mKvSf|mQkvLayout' "$T" -- cpp ':!cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h' | sed -n '1,155p'
echo TARGET_FMHA_INDIRECT
git diff -U1 "$B" "$T" -- tensorrt_llm/_torch/pyexecutor tensorrt_llm/_torch/visual_gen tensorrt_llm/_torch/modules/fla | grep -n -E -C2 'fmha|FMHA|attention|dtype|kv_cache|block_size' | sed -n '1,150p'Length of output: 1520 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
B=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export
for f in FmhaAutoTuner.h FmhaOptions.h KernelConfigBase.h KernelParams.h KernelTraits.h trtllm/dev/CutlassUtils.h trtllm/dev/Fp4Utils.h trtllm/dev/Utils.h trtllm/gen/DtypeDecl.h; do
echo "DIFF $f"
git diff -U1 "$B" "$H" -- "$P/$f" | sed -n '1,115p'
doneLength of output: 26116 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=336c4337a0a4d2f1479697c1f03e473b92fb3ee7
B=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo PARAM_REMAINING
git diff -U2 "$B" "$H" -- "$P/trtllmGen_fmha_export/KernelParams.h" | sed -n '110,310p'
echo TRAITS_REMAINING
git diff -U1 "$B" "$H" -- "$P/trtllmGen_fmha_export/KernelTraits.h" | sed -n '110,290p'
echo TARGET_FMHA_CALLS
git grep -n -E 'getMmaOpsPerClk[[:space:]]*\(|dtypeNeedsPadding[[:space:]]*\(|setKernelParams[[:space:]]*\(|mStoreTransformedKvInTmem|mUsesMmaForSoftmaxSum' "$T" -- "$P/fmhaKernels.h" "$P/fmhaRunner.cpp" "$P/fmhaRunnerParams.h" cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp | sed -n '1,115p'
echo TARGET_PY_CHANGED_FMHA_REFS
git diff -U0 "$B" "$T" -- tensorrt_llm/_torch/pyexecutor tensorrt_llm/_torch/visual_gen tensorrt_llm/_torch/modules/fla | grep -iE '^\+[^+].*(fmha|trtllmgen|dtype|flashinfer)' | sed -n '1,70p'Length of output: 12484 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=336c4337a0a4d2f1479697c1f03e473b92fb3ee7
B=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo HEAD_TX
git show "$H":"$P/trtllmGen_fmha_export/KernelTraits.h" | nl -ba | sed -n '1135,1210p'
echo BASE_TX
git show "$B":"$P/trtllmGen_fmha_export/KernelTraits.h" | nl -ba | sed -n '1045,1120p'
echo TARGET_RUNNER_CALL
git show "$T":"$P/fmhaKernels.h" | nl -ba | sed -n '605,657p;175,195p;780,825p'
echo HEAD_PARAMS
git show "$H":"$P/trtllmGen_fmha_export/KernelParams.h" | nl -ba | sed -n '685,729p;760,810p'
echo TARGET_DISPATCH
git show "$T":cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp | nl -ba | sed -n '45,80p'
echo TARGET_TEST
git show "$T":tests/unittest/_torch/attention/test_context_fmha_kernel_presence.py | nl -ba | sed -n '80,125p'Length of output: 28561 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=336c4337a0a4d2f1479697c1f03e473b92fb3ee7
B=0c58480ca680f2946bb94315af3517c9c7000bca
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo TARGET_HEADER_SYMBOL_REFERENCES
git grep -n -E 'getMmaOpsPerClk[[:space:]]*\(|dtypeNeedsPadding[[:space:]]*\(|mUsesMmaForSoftmaxSum' "$T" -- "$P" cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention cpp/tensorrt_llm/kernels/fmhaDispatcher.cpp ':!cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h' | sed -n '1,100p'
echo HEAD_META_TOP_BOTTOM
git show "$H":"$P/cubin/kernelMetaInfo.h" | nl -ba | sed -n '1,18p;15843,15870p' | cut -c1-360
echo TARGET_META_TOP_BOTTOM
git show "$T":"$P/cubin/kernelMetaInfo.h" | nl -ba | sed -n '1,18p;15085,15125p' | cut -c1-360
echo HEAD_CONFIG_COUNTS
python - <<'PY'
import subprocess
p='cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h'
for rev in ('0c58480ca680f2946bb94315af3517c9c7000bca','336c4337a0a4d2f1479697c1f03e473b92fb3ee7','a98c40d35cd49948d0974df3f75d362b4b68f4f3'):
s=subprocess.check_output(['git','show',f'{rev}:{p}'],text=True)
print(rev, 'lines',len(s.splitlines()))
for pat in ('QkvBfloat16OBfloat16H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext', 'QkvFp16OFp16H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext','QE4m3KvE2m1OE4m3H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext','QkE4m3VE2m1OBfloat16H128PagedKvCausalP32VarSeqQ256Kv128PersistentContext'):
matches=[i for i,line in enumerate(s.splitlines(),1) if line.startswith('{') and pat in line]
print(pat, len(matches), matches[:3])
PY
echo HEAD_CONTEXT_USAGE
git show "$H":"$P/trtllmGen_fmha_export/FmhaOptions.h" | nl -ba | sed -n '612,636p'Length of output: 2036 🏁 Script executed: #!/bin/bash
set -u
H=a98c40d35cd49948d0974df3f75d362b4b68f4f3
T=336c4337a0a4d2f1479697c1f03e473b92fb3ee7
P=cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha
echo HEAD_META
git show "$H":"$P/cubin/kernelMetaInfo.h" | nl -ba | sed -n '15370,15378p;15845,15864p' | cut -c1-340
echo TARGET_RUNNER_FACTORY
git show "$T":"$P/fmhaKernels.h" | nl -ba | sed -n '150,172p;265,289p'
echo HEAD_AUTOTUNER
git show "$H":"$P/trtllmGen_fmha_export/FmhaAutoTuner.h" | nl -ba | sed -n '44,55p'
echo HEAD_OPTIONS
git show "$H":"$P/trtllmGen_fmha_export/FmhaOptions.h" | nl -ba | sed -n '614,634p'
echo TARGET_MIXED_DTYPE_DECODING
git show "$T":cpp/tensorrt_llm/kernels/multiHeadAttentionCommon.h | nl -ba | sed -n '65,82p'Length of output: 16049 🏁 Script executed: #!/bin/bash
set -u
python - <<'PY'
import subprocess
p='cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/cubin/kernelMetaInfo.h'
patterns=('QkvBfloat16OBfloat16H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext','QkvFp16OFp16H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext','QE4m3KvE2m1OE4m3H64PagedKvCausalP32VarSeqQ256Kv128PersistentContext')
for rev in ('0c58480ca680f2946bb94315af3517c9c7000bca','336c4337a0a4d2f1479697c1f03e473b92fb3ee7','a98c40d35cd49948d0974df3f75d362b4b68f4f3'):
s=subprocess.check_output(['git','show',f'{rev}:{p}'],text=True)
lines=s.splitlines()
print(rev,len(lines))
for pat in patterns:
print(pat, [i for i,x in enumerate(lines,1) if x.startswith('{') and pat in x][:5])
PY
echo TARGET_PY_REFERENCE_LINES
git show 336c4337a0a4d2f1479697c1f03e473b92fb3ee7:tensorrt_llm/_torch/pyexecutor/connectors/kv_cache_connector.py | nl -ba | sed -n '1,35p'Length of output: 3678
SEMANTIC_REVIEW
Verdict: PASS — no concrete behavioral incompatibility found in the inspected paths. Git verified all three full commit IDs. It confirmed the supplied merge-base. I compared both branch diffs. A three-way merge preview reported no textual conflict. That result does not establish runtime compatibility. The head changes generated FMHA metadata, headers, and artifacts. The target changes KV-cache connector and executor code, among other paths, but does not edit those FMHA files. I checked the head’s K/V descriptor and scale-factor changes against the target runner’s argument setup. The runner passes separate K and V scale-factor pointers to the head descriptor setup. I also checked metadata used by the target’s BF16, FP16, and NVFP4 kernel-presence tests. Corresponding entries remain in the head table, including the NVFP4 entry. The new mixed K/V entry remains unavailable through the target dispatcher’s single-KV-type mapping. That limitation does not establish a failure in an existing configuration. This was a source review. I did not execute tests, inspect GPU behavior, or validate the binary artifacts. SEMANTIC_RESULT request_id=4bf877cf-91cd-4949-a31f-5d16dbbe547f head=a98c40d35cd49948d0974df3f75d362b4b68f4f3 target=336c4337a0a4d2f1479697c1f03e473b92fb3ee7 merge_base=0c58480ca680f2946bb94315af3517c9c7000bca verdict=PASS You are interacting with an AI system. |
|
PR_Github #75532 [ run ] completed with state |
|
/bot run --disable-fail-fast --only-multi-gpu-test |
|
PR_Github #75566 [ run ] triggered by Bot. Commit: |
|
PR_Github #75566 [ run ] completed with state |
|
/bot run --disable-fail-fast |
|
PR_Github #75597 [ run ] triggered by Bot. Commit: |
|
PR_Github #75597 [ run ] completed with state |
|
✅ LFS objects already in storage (7534 files) — no sync needed. These LFS-tracked files are already present in this repository's LFS storage:
|
….3 (#19956) Signed-off-by: Pengbo Wang <221450789+pengbowang-nv@users.noreply.github.com>
@coderabbitai summary
Description
Update trtllm-gen to main@f8a7d6e2004bb9b7c864b33246f18c33b20cdd47.
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.