Skip to content

[SYCL] Level Zero detection in ggml_sycl_init #23097

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ggerganov merged 3 commits into from
May 22, 2026
Merged

[SYCL] Level Zero detection in ggml_sycl_init #23097

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5c1aa5f
[SYCL] Centralize Level Zero detection in ggml_sycl_init
sanmai May 15, 2026
3c5b939
use the same wording
sanmai May 15, 2026
84e1b56
get back the warning
sanmai May 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ggml/src/ggml-sycl/common.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,8 @@ struct ggml_sycl_device_info {
std::array<float, GGML_SYCL_MAX_DEVICES> default_tensor_split = {};

int max_work_group_sizes[GGML_SYCL_MAX_DEVICES] = {0};

bool ext_oneapi_level_zero = true; // sycl::backend::ext_oneapi_level_zero used by all enumerated GPU devices
};

const ggml_sycl_device_info & ggml_sycl_info();
Expand Down
26 changes: 8 additions & 18 deletions ggml/src/ggml-sycl/ggml-sycl.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ static ggml_sycl_device_info ggml_sycl_init() {
for (int i = 0; i < info.device_count; ++i) {
info.devices[i].vmm = 0;
dpct::device_info prop;
sycl::device device = dpct::dev_mgr::instance().get_device(i);
auto & device = dpct::dev_mgr::instance().get_device(i);

SYCL_CHECK(CHECK_TRY_ERROR(dpct::get_device_info(
prop, device)));
Expand All @@ -116,6 +116,12 @@ static ggml_sycl_device_info ggml_sycl_init() {
info.devices[i].max_wg_per_cu = info.max_work_group_sizes[i] / prop.get_max_compute_units();
info.devices[i].hw_info = get_device_hw_info(&device);

// Only check GPU devices; CPU devices use OpenCL and would otherwise
// disable Level Zero for the GPUs on systems without ONEAPI_DEVICE_SELECTOR set.
if (device.is_gpu() && device.default_queue().get_backend() != sycl::backend::ext_oneapi_level_zero) {
GGML_LOG_WARN("SYCL GPU device %d does not use Level Zero backend, disabling Level Zero memory API\n", i);
info.ext_oneapi_level_zero = false;
}
Copy link
Copy Markdown
Contributor Author

@sanmai sanmai May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds like we can check for GGML_SYCL_ENABLE_LEVEL_ZERO here to skip #23381

Copy link
Copy Markdown
Contributor

@NeoZhangJianyu NeoZhangJianyu May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

}

for (int id = 0; id < info.device_count; ++id) {
Expand Down Expand Up @@ -229,26 +235,10 @@ static void ggml_check_sycl() try {
g_ggml_sycl_disable_dnn = get_sycl_env("GGML_SYCL_DISABLE_DNN", 0);
g_ggml_sycl_prioritize_dmmv = get_sycl_env("GGML_SYCL_PRIORITIZE_DMMV", 0);
#ifdef GGML_SYCL_SUPPORT_LEVEL_ZERO
g_ggml_sycl_enable_level_zero = get_sycl_env("GGML_SYCL_ENABLE_LEVEL_ZERO", 1);
g_ggml_sycl_enable_level_zero = get_sycl_env("GGML_SYCL_ENABLE_LEVEL_ZERO", ggml_sycl_info().ext_oneapi_level_zero);
#else
g_ggml_sycl_enable_level_zero = 0;
#endif
if (g_ggml_sycl_enable_level_zero) {
// Verify all GPU devices use the Level Zero backend before enabling L0 APIs.
// Only check GPU devices; CPU devices use OpenCL and would otherwise
// disable Level Zero for the GPUs on systems without ONEAPI_DEVICE_SELECTOR set.
for (unsigned int i = 0; i < dpct::dev_mgr::instance().device_count(); i++) {
auto & q = dpct::dev_mgr::instance().get_device(i).default_queue();
if (!q.get_device().is_gpu()) {
continue;
}
if (q.get_backend() != sycl::backend::ext_oneapi_level_zero) {
GGML_LOG_WARN("SYCL GPU device %d does not use Level Zero backend, disabling Level Zero memory API\n", i);
g_ggml_sycl_enable_level_zero = 0;
break;
}
}
}

#ifdef SYCL_FLASH_ATTN
g_ggml_sycl_enable_flash_attention = get_sycl_env("GGML_SYCL_ENABLE_FLASH_ATTN", 1);
Expand Down