Skip to content

UPSTREAM PR #16796: vulkan: Fix crash when FP16 mul_mat accumulation is not supported #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

UPSTREAM PR #16796: vulkan: Fix crash when FP16 mul_mat accumulation is not supported #10

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5abfd16
Experimenting crash fix
rillomas Oct 28, 2025
c5474a6
added assert for aborting and fixed comment
rillomas Oct 29, 2025
7661f3b
Merge branch 'ggml-org:master' into fix-accf16-capability-crash
rillomas Oct 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions ggml/src/ggml-vulkan/ggml-vulkan.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5105,15 +5105,22 @@ static vk_matmul_pipeline ggml_vk_get_mul_mat_mat_id_pipeline(ggml_backend_vk_co

// XXX TODO 'prec' is not actually allowed in mul_mat_id.
bool prefer_fp16acc = ctx->device->fp16 /*&& prec == GGML_PREC_DEFAULT*/;
bool support_fp16acc = ctx->device->pipeline_dequant_mul_mat_mat_id[src0_type].f16acc != nullptr;
bool support_fp32acc = ctx->device->pipeline_dequant_mul_mat_mat_id[src0_type].f32acc != nullptr;

if (support_fp16acc && (prefer_fp16acc || !support_fp32acc)) {
return ctx->device->pipeline_dequant_mul_mat_mat_id[src0_type].f16acc;
} else {
GGML_ASSERT(support_fp32acc);
return ctx->device->pipeline_dequant_mul_mat_mat_id[src0_type].f32acc;
auto& mmp = ctx->device->pipeline_dequant_mul_mat_mat_id[src0_type];
if (ctx->device->coopmat_support) {
// coopmat may or may not have f16acc and f32acc. It may also lack some child pipelines
if (ctx->device->coopmat_acc_f16_support && prefer_fp16acc) {
return mmp.f16acc;
} else if (ctx->device->coopmat_acc_f32_support) {
return mmp.f32acc;
} else {
// f16acc nor f32acc is supported so we just abort
GGML_ASSERT(false);
}
}
// coopmat2 or no coopmat comes here.
// coopmat2 should support both f16acc and f32acc by default with all child pipelines.
// No coopmat should support both f16acc and f32acc when FP16 is preferred, but only f32acc when FP16 is not preferred.
return prefer_fp16acc ? mmp.f16acc : mmp.f32acc;
}

static vk_pipeline ggml_vk_get_dequantize_mul_mat_vec_id(ggml_backend_vk_context * ctx, ggml_type a_type, ggml_type b_type) {
Expand Down