Fix Phi-3.5 ONNX Attention (static-cache) + GQA fp16 export bugs - #328

Closed

titaiwangms wants to merge 40 commits into

mainfrom

fix/phi35-onnx-attention-gqa-export

Commits on Jun 1, 2026

docs(skills): add mobius ONNX export gotchas
Copilot
committed
747579b
fix(static-cache): use is_causal=0 + explicit causal mask for opset-24 Attention

titaiwangms
and
Copilot
committed
140afae
fix(static-cache): phase-split static Attention to keep decode on Flash/XQA

titaiwangms
and
Copilot
committed
ce41067
docs(static-cache): clarify Flash->MEA decode-latency rationale

titaiwangms
and
Copilot
committed
ed57be8
Fix fp16 GQA export emitting fp32 packed weights in fold passes

titaiwangms
and
Copilot
committed
df203cc
test(static-cache): add fp16 runtime maskless-decode (Flash-eligibility) guard

titaiwangms
and
Copilot
committed
dd6bc15
test(static-cache): make phase-split mask guard fail-closed per-If

titaiwangms
and
Copilot
committed
0859a35
docs: update obsolete fp16 GQA export bug docs after df203cc fix

titaiwangms
and
Copilot
committed
a28b4e1
Strip dead pre-pack weights in FoldConcatInitializersPass

titaiwangms
and
Copilot
committed
71e84b3
test(static-cache): assert decode selects Flash, prefill MEA (direct kernel proof)

titaiwangms
and
Copilot
committed
80f0902
docs(static-cache): document load-bearing rename/pin/append ordering

titaiwangms
and
Copilot
committed
b39d99a
test(static-cache): assert decode honors nonpad bound (cache-poison guard)

titaiwangms
and
Copilot
committed
b094af3
test(dtype): assert warning fires on declared/const_value dtype disagreement

titaiwangms
and
Copilot
committed
f74d812
test(static-cache): make decode-Flash proof dispositive at head_dim=96

titaiwangms
and
Copilot
committed
215df63
docs(static-cache): note _capture_attention_kernel_log parallel-safety

titaiwangms
and
Copilot
committed
e0fd0c6
test(static-cache): version-gate the verbose-log Flash kernel proof

titaiwangms
and
Copilot
committed
03e827b
docs(skill): add VALUE-based fp16 GQA weight-integrity gotcha (section 5)

titaiwangms
and
Copilot
committed
26d74d2
fix(tasks): stamp explicit present KV-cache output shapes for GQA

titaiwangms
and
Copilot
committed
cf6c5c4
docs(skill): add GQA present-output head_dim gotcha (section 6)

titaiwangms
and
Copilot
committed
0911749
docs(skill): add initializer_dtype() convention to prevent fp32-default footgun

titaiwangms
and
Copilot
committed
5286dda
docs(skill): correct fp16 GQA mean|abs| to 0.015 and add signed-mean caveat

titaiwangms
and
Copilot
committed
cd2ea14
docs(skill): generalize fp16 GQA count-trap — init count unstable across fixes

titaiwangms
and
Copilot
committed
86a166c
test(fold-concat): name the live-packed-result invariant in DCE assert

titaiwangms
and
Copilot
committed
35d08f6
test(fold-concat): assert survived packed-QKV values are exact after DCE

titaiwangms
and
Copilot
committed
4e9bb5a

Commits on Jun 2, 2026

test(fold-concat): gate packed-QKV values through serialize→reload

titaiwangms
and
Copilot
committed
b3b08cc
test(fold-concat): add mean|abs| degeneracy assert + poison negative control

titaiwangms
and
Copilot
committed
ec5afb9
docs(skill): fix fp16 GQA count-trap direction to 293->197

titaiwangms
and
Copilot
committed
ef58730
Warn on partial present-shape param set in _register_kv_cache_outputs

titaiwangms
and
Copilot
committed
24fec65
Merge branch 'main' into fix/phi35-onnx-attention-gqa-export
titaiwangms
authored
d091474
style: apply lintrunner format + D205 fix for CI

titaiwangms
and
Copilot
committed
23564ff
Merge branch 'main' into fix/phi35-onnx-attention-gqa-export
titaiwangms
authored
5de4345

Commits on Jun 3, 2026

test: use IR-native ir.save instead of onnx.save in _fold_concat_test

titaiwangms
and
Copilot
committed
96ef1b1
test: use IR-native graph.all_nodes() and .as_graph() in build_graph_test

titaiwangms
and
Copilot
committed
6fc2644
ci: run static-cache decode runtime tests in integration-fast GPU job

titaiwangms
and
Copilot
committed
a7cbf92
ci(benchmark): waive intended static-cache phase-split node delta + changelog

titaiwangms
and
Copilot
committed
945e5f5
test(fp16): add e2e regression test guarding df203cc fold-pass dtype retention

titaiwangms
and
Copilot
committed
110f26b

Commits on Jun 4, 2026