kleidiai: Rework KleidiAI Build System/Integration

[JonathanC-ARM]

Overview

This PR reworks how kleidiai is included into llama.cpp via cmake.
The intent here is to remove the requirement for manually adding kernel source files, -march flags and other things that are specific to the kleidiai library and instead relying on the libraries own CMakeLists.

Various ifdef guards are also removed from ggml/src/ggml-cpu/kleidiai/kernels.cpp which is safe to do as kernel selection will be runtime feature safe from the changes included in PR 1 - #26076

Additional information

This pr is intended to be 2/3 in a series of pr's to improve the KleidiAI with its cpu feature detection, build system and documentation.

PR 1 - #26076
PR 3 - #26078

Requirements

• I have read and agree with the contributing guidelines - YES
• AI usage disclosure: YES - Ai was used to help understand llama.cpp

[ggml-gh-bot]

Hi @JonathanC-ARM, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 2 open PRs.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[JonathanC-ARM]

Hi @JonathanC-ARM, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

* **Multiple open PRs from a new contributor**: We limit new contributors (those without a previously merged PR) to **1 open PR at a time**. You currently have 2 open PRs.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

This is correct, however the approach here from my part was to give context to a series of work. This pr is draft and is not required to be acted upon and will only be updated and pursued if and when PR 1 in this series is accepted.

[JonathanC-ARM]

@chaxu01 @taronaeo would it be possible to get a review on this?

[taronaeo]

Lemme look into it the day after tomorrow :)

[chaxu01]

The SME2 Q4 GEMM appears to use FP16 instructions, but its required_cpu only includes CPU_FEATURE_SME2. KleidiAI itself guards this kernel with both SME2 and FP16 support.

Could we also require FP16 in the runtime dispatch? Otherwise an SME2-capable implementation without the required FP16 feature could select this kernel and hit an illegal instruction.

[JonathanC-ARM]

Agreed, I added in CPU_FEATURE_FP16 to the enum and added in the assignment in kleidiai.cpp also, same as other types.

[taronaeo]

I think you forgot to delete this line too?

[JonathanC-ARM]

I think in my haste I actually removed this which I didn't actually mean to, may have happened as part of a me rebasing perhaps. but i'll restore it.

[taronaeo]

Suggested change

	(runtime_feat.has_fp16 ? CPU_FEATURE_FP16 : CPU_FEATURE_NONE) |
	(runtime_feat.has_fp16 ? CPU_FEATURE_FP16 : CPU_FEATURE_NONE) |