[MLAS][KleidiAI] Apply fp16 split independent fixes#28720
Conversation
Squash the fp16 split independent fixes into a single IP review patch while preserving the original branch history separately. Signed-off-by: Cathal Lawlor <cathal.lawlor@arm.com>
There was a problem hiding this comment.
Pull request overview
Splits a small set of independent fixes from the larger Arm64 fp16 MLAS/KleidiAI enablement effort: a refactored KleidiAI Conv2D regression test, a WebGPU exclusion for an unsupported fp16 Attention case, and a parameter rename for naming consistency.
Changes:
- Extracted the KleidiAI LHS-pointer-cache-by-pad-growth regression into a named helper, gated on
USE_KLEIDIAI, and invoked it from bothExecuteShortand the existing long-test location. - Disabled WebGPU for
AttentionBatch1_Float16since WebGPU Attention does not supportmask_index. - Renamed
BatchSizeparameter toBatchNinArmKleidiAI::MlasDynamicQGemmBatchto match the publicMlasDynamicQGemmBatchdeclaration inmlas.h.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| onnxruntime/test/mlas/unittest/test_conv2d.h | Factor KleidiAI pad-growth regression into a helper and call it from short/long Conv2D tests; expand mlasi.h include guard to USE_KLEIDIAI. |
| onnxruntime/test/contrib_ops/attention_op_test.cc | Disable WebGPU EP for the fp16 mask_index Attention test. |
| onnxruntime/core/mlas/lib/kleidiai/qgemm_kleidiai.cpp | Rename BatchSize → BatchN to match declaration in mlas.h/mlasi_kleidiai.h. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Review — PR #28720 (MLAS/KleidiAI fp16-split independent fixes)Verdict: approve. This is a clean split of three unrelated, low-risk changes from the larger fp16 enablement work in #28487. Each is small, locally justified, and easy to verify. What's here
Things that are right
Minor comments1. void TestKleidiAiLhsPointerCacheByPadGrowth() {
#if defined(USE_KLEIDIAI)
...
#else
return; // ← unreachable; function ends immediately below
#endif
}Drop the 2. The regression-test comment migrated from the long-test body to the helper definition, but the new helper body has a much shorter version. The original comment explained the smaller-CI → larger-CI → smaller-CI sequence, the pad-buffer growth, the cache-invalidation hazard, and the issue link. The new helper comment is two sentences and drops the issue link. Please preserve the link to issue #26669 in the helper's comment — it's exactly the kind of context that gets lost otherwise. 3. 4. The RunAttentionTest(input_data, weight_data, bias_data, mask_index_data, output_data,
batch_size, sequence_length, hidden_size, number_of_heads, true /*use_float16*/,
false, false, 0, nullptr, nullptr,
AttentionMaskType::MASK_1D_KEY_SEQ_LEN, 0, 0,
false /*disable_cpu*/, false /*disable_cuda*/, false /*disable_dml*/, disable_webgpu);Three positional 5. The rename touches 7 lines in Bottom lineMerge after the trivial dead- |
Remove the dead non-KleidiAI return path from the pad-growth helper and restore the issue link plus reallocation rationale for the repeated shape sequence. Signed-off-by: Cathal Lawlor <cathal.lawlor@arm.com>
|
There are no other uses for BatchSize in the KleidiAI subtree for the MlasDynamicQGemmBatch function. |
Re-review — PR #28720 (commit
|
Description
This PR splits out a small set of independent fixes and test-scope updates from the larger CPU fp16 enablement work.
Changes:
mask_index, which WebGPU Attention does not support.BatchSizetoBatchNfor consistency with MLAS batch naming.Motivation and Context
These changes are independent from the wider fp16 MLAS/API and CPU EP enablement work in #28487.
The Conv2D regression exercises the small-CI -> large-CI -> small-CI sequence that can expose stale cached LHS pointer tables after the KleidiAI padding buffer grows. The Attention test exclusion keeps the test targeted to providers that support the tested
mask_indexinput combination.Note: This PR is the first offshoot from #28487.