sycl: set fattn_vec_nthreads to 256 for Battlemage

[Titaniumtown]

Overview

Adds a GGML_SYCL_FATTN_VEC_NTHREADS option which sets the VEC_NTHREADS option. This is useful when the hardware supports values different than 128.

Additional information

For 1st gen Intel ARC Graphics, 128 is the native option. For 2nd gen Arc Graphics, 256 is what the hardware supports.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[arthw]

The idea to set different value for hardware is good.
But the method of hard code is not good.
In base code, there is framework to check the hardware type and set the parameter value in initial stage.
Is it possible to refactor the code by referring them?

code to check the hardware type:

In ggml-sycl.cpp

if (!(ggml_sycl_info().devices[ctx.device].hw_info.arch ==
                gpu_arch::intel_gpu_acm_g10 &&
            src0->type == GGML_TYPE_Q4_0)) {
        use_dequantize_mul_mat_vec =
            use_dequantize_mul_mat_vec && !use_mul_mat_vec_q;
      }

Check the hardware and set parameter in initial stage:

In ggml-sycl.cpp

info.devices[i].max_wg_per_cu = info.max_work_group_sizes[i] / prop.get_max_compute_units();
        info.devices[i].hw_info = get_device_hw_info(&device);

In this case, you could add info.devices[i].xxx = 128 for xe, 256 for xe2.
In the code, call info.devices[i].xxx to get the right value.

Thank you!

[arthw]

@Titaniumtown
Could you check the review comments?

[Titaniumtown]

Yup! I will get on it, sorry, life has been busy.

[arthw]

It's good job!

Thank you!

[arthw]

@Titaniumtown
This PR make some LLM error on non-BMG.
Like llama-2-7b.Q4_0.gguf on Arc770:

./examples/sycl/test.sh -m ../models/llama-2-7b.Q4_0.gguf

...
Exception caught at file:/hd1/jianyuzh/llama.cpp/arthw/ggml/src/ggml-sycl/ggml-sycl.cpp, line:5236
Error OP FLASH_ATTN_EXT

Could you check it?
And I will check it too.

Thank you!

[Titaniumtown]

@arthw has someone opened an issue for that? I don't have any Alchemist hardware to test with.

[arthw]

No, I found it.
OK, I will check it.
Hope fix it as soon!

[arthw]

@Titaniumtown
I fixed it by #26441.