[libsycl] Add lit configuration files and basic test #177407

KseniyaTikhomirova · 2026-01-22T17:32:58Z

This PR brings initial version of lit configs for libsycl tests.
We plan to add more tests for libsycl integration with the compiler in the future.

github-actions · 2026-01-22T17:34:58Z

✅ With the latest revision this PR passed the Python code formatter.

KseniyaTikhomirova · 2026-01-23T17:21:12Z

Hi @tahonermann, @dvrogozh, @bader, @againull, @dm-vodopyanov

FYI, published for review.

a few notes:
This PR brings initial, cut version of lit configs for e2e test for libsycl project.
It has only basic stuff to run libsycl runtime tests. And it has a little bit more than I use now.
No compiler features integration yet - there is no code in RT to test it. So it is just plain compile & run of .cpp.
I use ShTest format since I don't need anything custom right now: we have only 1 type of device and only 1 backend, and what is more important - almost no features ready. So for test to run there is just 1 check - sycl-ls returns at least 1 device.
I also added a draft of documentation.

All extra features, parameters, configurations will be added as libsycl code base & feature coverage grow.

Also adding @uditagarwal97 who has an experience with lit feature implementation in intel/llvm and who kindly agreed to make a review of this PR.

P.S. this PR depends on device impl (#176972) to run lit (sycl-ls should be able to identify devices). check-sycl-e2e is not included into check-all so it won't break anything in buildbot for libsycl + liboffload if merged earlier. Once both PRs land in main check-sycl-e2e will be added into BB workflow. @sarnex agreed to help with integration to BB and it may be complicated because checl-sycl-e2e is not a top level target. If we won't find a way to run it as it is, I will consider adding it to the top level check-all (but really would like to avoid it).

Thank you.

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

libsycl/test_e2e/README.md

dvrogozh · 2026-01-23T17:40:06Z

libsycl/test_e2e/CMakeLists.txt

+  set(LIBSYCL_TEST_E2E_TARGETS "all")
+endif()
+
+if(NOT DEFINED LEVEL_ZERO_LIBS_DIR AND NOT DEFINED LEVEL_ZERO_INCLUDE_DIR)


Hm, I am confused. Per my understanding default way to find L0 should be via pkg-config. Do I understand it right that that's how L0 actually will be found in some upper level cmake list? And if it will be why this customization with LEVEL_ZERO_LIBS_DIR and LEVEL_ZERO_INCLUDE_DIR is needed?

note: removed that, it is not needed right now, I noted about PR that "And it has a little bit more than I use now." but since there are questions about this , I removed it.

the answer is: this code is not intended to find default L0 version, it was intended to provide interface to configure tests to some specific L0 setup/version, to override default one.

dvrogozh · 2026-01-23T17:41:59Z

libsycl/test_e2e/README.md

+## Prerequisites
+
+* Target runtime(s) to execute tests on devices.
+  TODO: add link to liboffload instruction once they add it.


Shouldn't liboffload just be there if you build clang/llvm with sycl support? I was expecting that here will be a list of runtimes such as L0 which must be available for the test to run. Should we list here these runtimes one by one?

your understanding is correct and the idea is to have an instruction about L0/CUDA/HIP runtimes setup here. And I believe this instruction must be provided on liboffload level since it is the runtime that directly works with them. This instruction is absent since liboffload has poor (and/or absent) documentation now. Intel liboffload team has docs in TODO list and will add it once it will be ready.

libsycl/test_e2e/README.md

dvrogozh · 2026-01-23T17:52:45Z

libsycl/test_e2e/README.md

+* **linux** - host OS;
+* **any-device-is-gpu** - device type to be available;
+* **any-device-is-level_zero** - backend to be available;


Suggested change

* **linux** - host OS;

* **any-device-is-gpu** - device type to be available;

* **any-device-is-level_zero** - backend to be available;

* `linux` - host OS;

* `any-device-is-gpu` - device type to be available;

* `any-device-is-level_zero` - backend to be available;

libsycl/test_e2e/README.md

dvrogozh · 2026-01-23T17:56:56Z

libsycl/test_e2e/README.md

+Following features can be checked in tests to limit test execution to the
+specific environment via REQUIRES, UNSUPPORTED, etc. filters.


Suggested change

Following features can be checked in tests to limit test execution to the

specific environment via REQUIRES, UNSUPPORTED, etc. filters.

Following features can be passed to LIT via `REQUIRES`, `UNSUPPORTED`, etc. filters.

to limit test execution to the specific environment.

libsycl/test_e2e/basic/platform_get_devices.cpp

bader · 2026-01-24T00:12:31Z

Let's drop "E2E" and just call these tests "libsycl tests" as all other runtime projects do.

Pull request title: "[SYCL] E2E LIT tests initial configs & test" -> "[libsycl] Add lit configuration files and basic test"

Pull request description:

"This PR brings initial, cut version of lit configs for e2e test for libsycl project." -> "This PR brings initial version of lit configs for libsycl tests."
"No compiler features integration yet - there is no code in RT to test it." -> "We plan to add more tests for libsycl integration with the compiler in the future."

libsycl/CMakeLists.txt

bader · 2026-01-24T00:15:56Z

libsycl/CMakeLists.txt

 add_subdirectory(tools)
+
+if(LLVM_INCLUDE_TESTS)
+  add_subdirectory(test_e2e)


Suggested change

add_subdirectory(test_e2e)

add_subdirectory(test)

I left e2e in the name intentionally. In the end we will have same test folders as other projects: test, unittests + test_e2e. As in intel/llvm unittests and test won't require backend runtimes to be present on system to do check. while e2e testing require and so can't be joined with test.

From my point of view, we have two top level test directories artificially created due to attempting to move some tests requiring third-party software/hardware to llvm-test-suite repository.
I see no reason to complicate the implementation with this separation. I don't see such complications in other runtime library projects depending on third-party software/hardware (e.g. liboffload, libc++, etc.)

we have different configs for them. lit config and features for e2e tests are way more complex that is needed for test. test require compilation + sometimes tools like objdump to be run. That's it. I lot of stuff related to device choice, runtimes, build-only/run-only has no sense for that kind of tests.
I would prefer 2 different setups doing exactly what is needed for each kind of tests than merge them and have a monstrous config that should handle everything.
plus it may be easier to avoid a mess when compile-only and e2e tests are separated. we can have ./test/compile-only and ./test/e2e if you want to keep it under 'test' top folder but I don't see why it is better.

Because it's simpler. I don't understand why we need to maintain to different LIT config files when "compile-only" tests are sub-set of regular tests. We can use single config for both types of tests.

bader · 2026-01-24T00:16:18Z

libsycl/test_e2e/CMakeLists.txt

@@ -0,0 +1,36 @@
+cmake_minimum_required(VERSION 3.20.0)
+
+message("Configuring SYCL End-to-End Tests")


Suggested change

message("Configuring SYCL End-to-End Tests")

message("Configuring libsycl tests")

Please, change "SYCL" to "libsycl" across the whole patch when we talk about the runtime library and drop end-to-end when refer to libsycl tests.

SYCl -> libsycl - ok, agree. But why should we drop end-to-end? I am adding e2e tests for now, but there are more tests coming (next phases) - unittests and tests that don't require offloading backends (abi, compile-only). I think we should differentiate them.

I think we should differentiate them.

Why?

libsycl/test_e2e/CMakeLists.txt

libsycl/test_e2e/basic/platform_get_devices.cpp

libsycl/test_e2e/lit.cfg.py

libsycl/test_e2e/README.md

bader · 2026-01-24T01:19:48Z

libsycl/test_e2e/lit.site.cfg.py.in

+config.libsycl_compiler = lit_config.params.get("libsycl_compiler", "@LIBSYCL_CXX_COMPILER@")
+config.libsycl_root_dir= os.path.dirname(os.path.dirname(config.libsycl_compiler))
+config.libsycl_bin_dir = os.path.join(config.libsycl_root_dir, 'bin')


From the code in CMake files I see that LIBSYCL_CXX_COMPILER points to the clang++ executiable inside the <compiler-root-dir>/bin, so I think this configuration will configure libsycl_bin_dir to <compiler-root-dir>/bin/bin. Am I right?

No:
with

print (config.libsycl_compiler) print (config.libsycl_root_dir) print (config.libsycl_bin_dir)

in lit.cfg.py I got:

<some dir>/kt_llvm_project/build/bin/clang++ <some dir>/kt_llvm_project/build <some dir>/kt_llvm_project/build/bin

l9: config.libsycl_root_dir= os.path.dirname(os.path.dirname(config.libsycl_compiler))
dirname is called twice and goes 2 levels up.

Does it make sense to code it like this:

config.libsycl_compiler = lit_config.params.get("libsycl_compiler", "@LIBSYCL_CXX_COMPILER@") config.libsycl_bin_dir = os.path.dirname(config.libsycl_compiler) config.libsycl_root_dir = os.path.dirname(config.libsycl_bin_dir)

?

github-actions · 2026-01-26T16:54:52Z

✅ With the latest revision this PR passed the C/C++ code formatter.

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

KseniyaTikhomirova force-pushed the add_lit_pr branch from 24ab14b to 6a8d6d8 Compare January 22, 2026 17:55

KseniyaTikhomirova added the SYCL https://registry.khronos.org/SYCL label Jan 23, 2026

KseniyaTikhomirova requested a review from tahonermann January 23, 2026 15:25

KseniyaTikhomirova force-pushed the add_lit_pr branch from 2533f18 to 6a8d6d8 Compare January 23, 2026 17:03

KseniyaTikhomirova requested review from againull and bader January 23, 2026 17:04

[SYCL] E2E LIT tests initial configs & test

904e8ff

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

KseniyaTikhomirova force-pushed the add_lit_pr branch from 6a8d6d8 to 904e8ff Compare January 23, 2026 17:24

dvrogozh reviewed Jan 23, 2026

View reviewed changes

bader reviewed Jan 24, 2026

View reviewed changes

KseniyaTikhomirova changed the title ~~[SYCL] E2E LIT tests initial configs & test~~ [libsycl] Add lit configuration files and basic test Jan 26, 2026

KseniyaTikhomirova force-pushed the add_lit_pr branch from ffc6ff4 to f2d669c Compare January 26, 2026 16:56

fix code-review comments

ebd8ced

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

KseniyaTikhomirova force-pushed the add_lit_pr branch from f2d669c to ebd8ced Compare January 26, 2026 17:32

KseniyaTikhomirova requested review from bader and dvrogozh February 2, 2026 13:45

remove extra code

02df274

Signed-off-by: Tikhomirova, Kseniya <[email protected]>

KseniyaTikhomirova force-pushed the add_lit_pr branch from c3b1e37 to 02df274 Compare February 2, 2026 13:52

		Following features can be checked in tests to limit test execution to the
		specific environment via REQUIRES, UNSUPPORTED, etc. filters.

		@@ -0,0 +1,36 @@
		cmake_minimum_required(VERSION 3.20.0)

		message("Configuring SYCL End-to-End Tests")

	message("Configuring SYCL End-to-End Tests")
	message("Configuring libsycl tests")

[libsycl] Add lit configuration files and basic test #177407

Are you sure you want to change the base?

[libsycl] Add lit configuration files and basic test #177407

Conversation

KseniyaTikhomirova commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KseniyaTikhomirova commented Jan 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bader commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KseniyaTikhomirova Feb 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

KseniyaTikhomirova commented Jan 22, 2026 •

edited

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading

bader commented Jan 24, 2026 •

edited

Loading

KseniyaTikhomirova Feb 2, 2026 •

edited

Loading

github-actions bot commented Jan 26, 2026 •

edited

Loading