-
Notifications
You must be signed in to change notification settings - Fork 9.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SYCL] Add support for SYCL Nvidia target #5738
[SYCL] Add support for SYCL Nvidia target #5738
Conversation
@NeoZhangJianyu, @abhilash1910, @Alcpz, feel free to review |
Just curious, does this kind of build time selection mean that it's not possible to run an Nvidia and an Intel GPU at the same time using SYCL? |
It is possible to target both Nvidia and Intel GPUs with the same SYCL code, if you pass |
@AidanBeltonS But this PR don't provide the key feature to user. Another word, how to persuade user to use SYCL backend for NV GPU, instead of cuBlas backend? |
oneAPI by default does support a single binary file for multiple GPUs. oneMKL is the reason we cannot do it in this case. In the long term I think we should raise this issue and fix that rather than not support NVidia GPUs.
|
26be9e6
to
8e67d2d
Compare
adb139b
to
8f7c98b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM ! Lets wait for CI.
cc @ggerganov
* Add support for nvidia target in CMake * Update sycl read-me for Nvidia target * Fix errors
* Add support for nvidia target in CMake * Update sycl read-me for Nvidia target * Fix errors
* Add support for nvidia target in CMake * Update sycl read-me for Nvidia target * Fix errors
This PR adds CMake and instructions for a SYCL Nvidia target. This is done with the
LLAMA_SYCL_BACKEND
CMake option which defaults to INTEL but can be set to NVIDIA. This approach allows us in the future to expand this for AMD targets.This PR is dependent on #5591 as that resolves failing Nvidia
test-backend-ops
tests.