openvino: OV 2026.2, context-shift, Q5_1 support, gemma4 dense/embedding, and -fa off - #24503

Merged

ggerganov merged 129 commits into

ggml-org:masterfrom

ravi9:dev_backend_openvino

Jun 17, 2026

Commits on Jun 9, 2026

Add interface is_model_splitted() to check the c-graph is splited or not

zhaixuejun1993
authored and
ravi9
committed
4bc1eb1
Infer and propagate dynamic-dimension indices for all tensors in the GGML graph in api compute_model_outputs()

zhaixuejun1993
authored and
ravi9
committed
c49ec28
Only do this for fallback sub graph

zhaixuejun1993
authored and
ravi9
committed
76eb69e
Move dynamic dims compute in graph missmatch

zhaixuejun1993
authored and
ravi9
committed
7e6caef
ggml-openvino: fix tensor data handling for PERMUTE/VIEW ops in split models

zhaixuejun1993
authored and
ravi9
committed
d306b0b
ggml-openvino:add comments

zhaixuejun1993
authored and
ravi9
committed
01088c2
ggml-openvino: override VIEW op_case to 0 for split model inputs

zhaixuejun1993
authored and
ravi9
committed
126d758
openvino backend: Handle unsupported VIEW shape-mismatch in OpenVINO backend

zhaixuejun1993
authored and
ravi9
committed
32f9cb7
Enable additional mul_mat tests and add tensor data saving function (#81)

zhaixuejun1993
authored and
ravi9
committed
f812f78
ggml-openvino: fix CONT/TRANSPOSE mapping and improve dynamic-dimension handling

zhaixuejun1993
authored and
ravi9
committed
865f121
OpenVINO: add NORM/TANH support and rework SOFT_MAX translation

zhaixuejun1993
authored and
ravi9
committed
ca3a176
ggml-openvino: extend VIEW handling

zhaixuejun1993
authored and
ravi9
committed
a73a6dc
Enable -fa off (#118)

wine99
authored and
ravi9
committed
bfa4c53
Enable --context-shift

wine99
authored and
ravi9
committed
9c922b1
Fix llm param compute error for normal softmax not the softmax in attention

zhaixuejun1993
authored and
ravi9
committed
59f0e3c
OpenVINO backend: fix error for attention size compute in llm param

zhaixuejun1993
authored and
ravi9
committed
c8e9ce4
use tensor->extra in infer_request i/o

wine99
authored and
ravi9
committed
9f355ed
OpenVINO backend: refacter the compute_llm_params() func add get_attention_pattern_case to easy extand

zhaixuejun1993
authored and
ravi9
committed
dc5ed75
OpenVINO backend: clean unused code

zhaixuejun1993
authored and
ravi9
committed
e2ce59c
1to1 match op update (#146)

authored and
ravi9
committed
130ef39
initial gemma4 support

Mustafa Cavus
authored and
ravi9
committed
13ddbf3
removed hardcoded names for kv cache slicing

cavusmustafa
authored and
ravi9
committed
7597773
OpenVINO backend: Add new attention pattern for llm parameters compute

zhaixuejun1993
authored and
ravi9
committed
a1baa1a
flash attn Q shape static conversion

cavusmustafa
authored and
ravi9
committed
dad8acd
Remove slice in permute translation when n_seq is 1

cavusmustafa
authored and
ravi9
committed
760e86d
return optional in extract_layer_from_name

wine99
authored and
ravi9
committed
5a39967
OpenVINO backend: refactor VIEW related operation (#148)

zhaixuejun1993
authored and
ravi9
committed
d289bbd
OpenVINO backend: Add ops l2_norm & pad

zhaixuejun1993
authored and
ravi9
committed
e0caf43
OpenVINO backend does not support CPY with non-contiguous data or mismatched types

zhaixuejun1993
authored and
ravi9
committed
8c83092
add op SSM_CONV GATED_DELTA_NET

wine99
authored and
ravi9
committed
a08546f
OpenVINO backend: fix error for bf16 in OV gpu plugin

zhaixuejun1993
authored and
ravi9
committed
0d0fb42
reverted static Q input shape for attention layer

cavusmustafa
authored and
ravi9
committed
c064e87
OpenVINO backend: remove hardcode name inp_tokens, which ignore some leaf case

zhaixuejun1993
authored and
ravi9
committed
d44fa9c
Disable remote tensor due to bug in ov gpu

wine99
authored and
ravi9
committed
6afe652
Disable n_token > 1 GATED_DELTA_NET on gpu

wine99
authored and
ravi9
committed
d2279ae
OpenVINO backend: fix the view op dynamic handling issue in gemma4 & enable view + get_row

zhaixuejun1993
authored and
ravi9
committed
e6a7a9e
OpenVINO backend: clean code

zhaixuejun1993
authored and
ravi9
committed
418c5e5
OpenVINO backend: enable view + norm/rms_norm

zhaixuejun1993
authored and
ravi9
committed
c6efcb6
OpenVINO backend: concat op

zhaixuejun1993
authored and
ravi9
committed
eafd08e
OpenVINO backend: argsort op

zhaixuejun1993
authored and
ravi9
committed
89858ec
OpenVINO backend: enable unary + view & GGML_UNARY_OP_SOFTPLUS

zhaixuejun1993
authored and
ravi9
committed
e25ed8f
Fix issue for test-backend-ops in TOPK_MOE, which compare VIEW ops result, VIEW node in OpenVINO no need compare, the whole graph result is correct

zhaixuejun1993
authored and
ravi9
committed
996f0c7
OpenVINO backend: enable sum_rows

zhaixuejun1993
authored and
ravi9
committed
12863b8
OpenVINO backend: enable clamp

zhaixuejun1993
authored and
ravi9
committed
404d6b3
OpenVINO backend: enable DIV

zhaixuejun1993
authored and
ravi9
committed
41c35a3
OpenVINO backend: enable GGML_OP_MUL_MAT_ID

zhaixuejun1993
authored and
ravi9
committed
03e835c
OpenVINO backend: disable MUL_MAT_ID_FUSION case with large mem needed

zhaixuejun1993
authored and
ravi9
committed
08438be
OpenVINO backend: Disable GGML_OP_ARGSORT, cause test_backend-ops failed

zhaixuejun1993
authored and
ravi9
committed
904c608
OpenVINO backend: fix issue in mul_mat_id

zhaixuejun1993
authored and
ravi9
committed
d2ca0f8
OpenVINO backend: Disable DIV with broadcast on GPU

zhaixuejun1993
authored and
ravi9
committed
2aa3b2d
OpenVINO backend: update DIV

zhaixuejun1993
authored and
ravi9
committed
59e3d64
use ov internal op GatedDeltaNet

wine99
authored and
ravi9
committed
4472ce0
OpenVINO backend: enable llama erch test qwen3next

zhaixuejun1993
authored and
ravi9
committed
4bbb85f
OpenVINO backend: enable RMS_NORM + VIEW & remove op_case 2 for rope

zhaixuejun1993
authored and
ravi9
committed
3032423
OpenVINO backend: fix error

zhaixuejun1993
authored and
ravi9
committed
c4bd20f
suggested changes, need review

wine99
authored and
ravi9
committed
2c2541c
suggested changes, need review

wine99
authored and
ravi9
committed
d11e198
OpenVINO backend: clean unused code & fix build warning

zhaixuejun1993
authored and
ravi9
committed
c4f2ec7
OpenVINO backend: enable minicpm3 for arch test

zhaixuejun1993
authored and
ravi9
committed
46bddb1
Disable GDN op (#177)

wine99
authored and
ravi9
committed
bb38483
disable gated_delta_net

wine99
authored and
ravi9
committed
645df27
update stateful_kv_size correctly in mismatch case

wine99
authored and
ravi9
committed
08b4fd6
OpenVINO backend: enable arch test for qwen3vl

Xuejun
authored and
ravi9
committed
d2c7549
OpenVINO backend: enable cohere2 for arch test

zhaixuejun1993
authored and
ravi9
committed
e05da27
OpenVINO backend: enable t5 for arch test

zhaixuejun1993
authored and
ravi9
committed
c3c4dba
OpenVINO backend: enable jamba for arch test

zhaixuejun1993
authored and
ravi9
committed
a32aeb5
OpenVINO backend: remove warning for tmp

zhaixuejun1993
authored and
ravi9
committed
a0155c4
OpenVINO backend: enable kimi-linear for arch test

zhaixuejun1993
authored and
ravi9
committed
b1f6fb4
Remove unused

zhaixuejun1993
authored and
ravi9
committed
603c7dc
Fix gpt-oss accuracy issue

yangwang201911
authored and
ravi9
committed
21bab71
OpenVINO backend: enable arctic for arch test

zhaixuejun1993
authored and
ravi9
committed
f49b026
OpenVINO backend: enable grok for arch test

zhaixuejun1993
authored and
ravi9
committed
65ec35a
Gemma4 initial npu support (#179)

cavusmustafa
authored and
ravi9
committed
292b156
ggml-openvino: add GGML_OPENVINO_ENABLE_CACHE env var to control decoder cache. Add environment variable GGML_OPENVINO_ENABLE_CACHE (default: YES). When set to NO, the decoder_cache is bypassed and…

zhaixuejun1993
authored and
ravi9
committed
c832153
Revert "Gemma4 initial npu support (#179)"

wine99
authored and
ravi9
committed
af2a8e1
OpenVINO backend: disable debug log print

zhaixuejun1993
authored and
ravi9
committed
a16cfb4
Update TBB discovery. Delegated to OpenVINOs own config.
ravi9
committed
36c5cd5
OpenVINO backend: GGML_OPENVINO_ENABLE_CACHE YES -> 1

zhaixuejun1993
authored and
ravi9
committed
6df01a7
OpenVINO backend: fallback FLASH_ATTN_EXT in gemma3n to CPU backend

zhaixuejun1993
authored and
ravi9
committed
2ab4121
Add raw ov infer profiling metric

virajwad
authored and
ravi9
committed
6b1c5aa
Add OV raw infer time metric to static compute path

authored and
ravi9
committed
d194391
Modify precision of static profiling

virajwad
authored and
ravi9
committed
f1a5340
update to OV 2026.2, add OV windows CI
ravi9
committed
88f22fd
fix editorconfig-checks
ravi9
committed
ccb1b23
Initiall gemma4 npu support

cavusmustafa
authored and
ravi9
committed
df50c52
temp. fix for gemma4 accuracy bug on npu

cavusmustafa
authored and
ravi9
committed
b397e94
Remove hardcoded names for npu-fold handling

cavusmustafa
authored and
ravi9
committed
41ce1c7
revert static n tokens for cont translation as it is not needed

cavusmustafa
authored and
ravi9
committed
7baa213
removed unused variable

cavusmustafa
authored and
ravi9
committed
5fa8e5e
test-llama-archs fix

cavusmustafa
authored and
ravi9
committed
b9cba9d
Fix gemma4 flash_attn fallback

cavusmustafa
authored and
ravi9
committed
e8324ac
support im2col

mostafafaheem
authored and
ravi9
committed
10a2cfd
fix code style

mostafafaheem
authored and
ravi9
committed
38e9d59
disable add_rope_sin_cos optimization

wine99
authored and
ravi9
committed
9c0ca74
stateless boradcast and rope optimizations

cavusmustafa
authored and
ravi9
committed
bbc4319
Enable manual gqa attn by default for stateless gpu

cavusmustafa
authored and
ravi9
committed
e3bdd6b
manual gqa: fixed static batch

Mustafa Cavus
authored and
ravi9
committed
bc11c32
gemma4 llama-bench ctx update fix

cavusmustafa
authored and
ravi9
committed
d551f5b
Update OV win CI
ravi9
committed
2e33f25
stateful rope fusion temp. fix

cavusmustafa
authored and
ravi9
committed
699fd7d
OpenVINO backend: Conslolidate supported ops

mostafafaheem
authored and
ravi9
committed
d05ce54
Exclude unsupported GGML_OP_SUB cases

mostafafaheem
authored and
ravi9
committed
b32c04e
Exclude unsupported TOPK_MOE cases

mostafafaheem
authored and
ravi9
committed
1d9fa46
OpenVINO Backend: MUL_MAT enhancements

mostafafaheem
authored and
ravi9
committed
71ba113
Update OV CI
ravi9
committed
1c64362
support f16 mask input for npu

wine99
authored and
ravi9
committed
f7bbe7c
Make GGML_OPENVINO_* env vars usage uniform
ravi9
committed
efbc565
OpenVINO backend: Enhance envvar handling
mostafafaheem
committed
34b2bee
more cleanup
mostafafaheem
committed
b3f21ea
move ggml_openvino_env_flag to appropriate place
mostafafaheem
committed
e68a103
OpenVINO backend: add REPEAT translator, Q5_1 weights, and GLU view-input fix
cavusmustafa
committed
65e2ecc
Merge pull request #208 from mostafafaheem/envvar_cleanup
ravi9
authored
8ea91dd
ggml-openvino: fix -Werror=cast-qual in extract_q5_1_data
cavusmustafa
committed
341b615
Merge pull request #209 from cavusmustafa/op_translations_q51
ravi9
authored
4c878bd

Uh oh!

Commits

Commits on Jun 9, 2026

Commits on Jun 10, 2026

Commits on Jun 12, 2026

Commits on Jun 13, 2026

Commits on Jun 15, 2026

Commits on Jun 16, 2026