ggml-zendnn : add Q8_0 quantization support

[z-sachin]

Overview

This PR adds Q8_0 quantization support in the ggml-zendnn backend.

The implementation enables ZenDNN execution paths for Q8_0 models and integrates the required handling for quantized weights and matmul operations.

Key changes:

• Added Q8_0 support in ggml-zendnn backend
• Enabled ZenDNN execution path for Q8_0 quantized matmul operations
• Added handling for Q8_0 tensor layouts and conversions
• Integrated backend execution support for Q8_0 models
• Also pointing to the latest ZenDNN

Benchmark Results

Benchmark configuration:

• threads = 96
• type_k = bf16
• type_v = bf16

Llama-3.1-8B-Instruct Q8_0

Prompt Size	GGML_CPU_Q8_0 t/s	ZenDNN_Q8_0 t/s	Gain
256	472.28	730.87	54.75%
512	450.86	832.48	84.64%
768	446.81	864.52	93.49%
1024	439.58	800.15	82.03%
2048	405.07	778.34	92.15%
tg128	33.08	33.14	0.18%

Mixtral-8x7B Q8_0

Prompt Size	GGML_CPU_Q8_0 t/s	ZenDNN_Q8_0 t/s	Gain
256	156.09	297.67	90.70%
512	156.63	389.44	148.64%
768	156.76	417.38	166.25%
1024	154.70	438.73	183.60%
2048	150.11	470.41	213.38%
tg128	20.95	20.92	-0.14%

gemma4 31B Q8_0

Prompt Size	GGML_CPU_Q8_0 t/s	ZenDNN_Q8_0 t/s	Gain
256	116.05	195.02	68.05%
512	112.53	229.12	103.61%
768	111.96	239.02	113.49%
1024	110.93	238.03	114.58%
2048	106.37	222.32	109.01%
tg128	8.50	8.47	-0.35%

gemma-4-26B-A4B-it Q8_0

Prompt Size	GGML_CPU_Q8_0 t/s	ZenDNN_Q8_0 t/s	Gain
256	570.87	597.84	4.72%
512	581.80	666.18	14.50%
768	588.67	683.91	16.18%
1024	574.79	684.13	19.02%
2048	562.26	642.08	14.20%
tg128	33.96	33.83	-0.38%

Observations

• Significant prompt-processing gains are observed for larger prompt sizes
• Decoding (tg128) performance remains comparable to ggml-cpu

Additional information

Validated on:

• Llama-3.1-8B-Instruct Q8_0
• Mixtral-8x7B Q8_0
• gemma4 31B Q8_0
• gemma-4-26B-A4B-it Q8_0

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes

[CISC]

cc/ @avinashcpandey @Jiten1parmar @z-vishal

[z-vishal]

8-bit quantization support was much awaited in ZenDNN backend, and the benchmark numbers look solid! thanks @z-sachin
big thanks to the ZenDNN team for making this happen
cc: @amukho @avinashcpandey @Jiten1parmar

[z-vishal]

@CISC the PR looks good
from my side I approved the changes, now we can merge :)

[z-vishal]

@ggml-org/maintainers Another approval required

[taronaeo]

Are we waiting for CI? Looks pretty jammed up.

[CISC]

I'm preemptively clearing the queue for Release fix.

[z-vishal]

@CISC could you trigger the cancelled checks again if the release is done?

[CISC]

@CISC could you trigger the cancelled checks again if the release is done?

No need, the ones that finished are good enough.