Skip to content

[SYCL][Unit Tests] Fix regression after https://github.com/intel/llvm/pull/19687 #20534

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Nov 6, 2025
Merged

[SYCL][Unit Tests] Fix regression after https://github.com/intel/llvm/pull/19687 #20534

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
194e20b
[SYCL][Unit Tests] Fix regression after https://github.com/intel/llvm…
dm-vodopyanov Nov 3, 2025
360c60e
Fix clang format
dm-vodopyanov Nov 3, 2025
c8999d6
Update
dm-vodopyanov Nov 4, 2025
8d543d7
Update
dm-vodopyanov Nov 5, 2025
69b4775
Update
dm-vodopyanov Nov 5, 2025
02ae68a
Update
dm-vodopyanov Nov 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions sycl/cmake/modules/AddSYCLUnitTest.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,8 @@ endfunction()
#
# Will compile the list of files together to create two builds, with and without
# the SYCL preview features enabled.
# Produces two binaries, named `basename(test_name_prefix_non_preview)` and `basename(test_name_prefix_preview)`
# Produces two binaries, named `basename(test_name_prefix_Non_Preview_Tests)` and `basename(test_name_prefix_Preview_Tests)`
macro(add_sycl_unittest test_name_prefix link_variant)
add_sycl_unittest_internal(${test_name_prefix}_non_preview ${link_variant} FALSE ${ARGN})
add_sycl_unittest_internal(${test_name_prefix}_preview ${link_variant} TRUE ${ARGN})
add_sycl_unittest_internal(${test_name_prefix}_Non_Preview_Tests ${link_variant} FALSE ${ARGN})
add_sycl_unittest_internal(${test_name_prefix}_Preview_Tests ${link_variant} TRUE ${ARGN})
endmacro()
4 changes: 2 additions & 2 deletions sycl/unittests/compression/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
add_sycl_unittest(CompressionTests OBJECT
CompressionTests.cpp
)
target_compile_definitions(CompressionTests_non_preview PRIVATE SYCL_RT_ZSTD_AVAILABLE)
target_compile_definitions(CompressionTests_preview PRIVATE SYCL_RT_ZSTD_AVAILABLE __INTEL_PREVIEW_BREAKING_CHANGES)
target_compile_definitions(CompressionTests_Non_Preview_Tests PRIVATE SYCL_RT_ZSTD_AVAILABLE)
target_compile_definitions(CompressionTests_Preview_Tests PRIVATE SYCL_RT_ZSTD_AVAILABLE __INTEL_PREVIEW_BREAKING_CHANGES)
10 changes: 5 additions & 5 deletions sycl/unittests/config/ConfigTests.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
#include <fstream>
#include <regex>

TEST(ConfigTests, CheckConfigProcessing) {
TEST(ConfigTests, DISABLED_CheckConfigProcessing) {
Copy link
Contributor

@uditagarwal97 uditagarwal97 Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all the disabled unittests failing because they set some environment variable which gets unintentionally modified when we run preview and non-preview unittests parallelly?

IIRC, preview and non-preview unit tests are in different folders so there shouldn't be any race on filesystem...
Btw, in CI, when we run check-sycl-unitests, we run both preview and non-preview unittests and we don't see any failure there.

Copy link
Contributor

@uditagarwal97 uditagarwal97 Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference in our setup when we run unit tests via check-sycl-unitests and check-sycl? It's weird that unitests are failing with one but not another.

Copy link
Contributor

@AlexeySachkov AlexeySachkov Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the main difference between check-sycl-unittests and check-sycl is that check-sycl-unittests handles different tests at build system level, i.e. each test executable is a separate target which are all run sequentially:

https://github.com/intel/llvm/blob/e05e82b240afe298faca4f89ec448772e4123b12/sycl/cmake/modules/AddSYCLUnitTest.cmake#L104C42-L104C63

From CI logs:

cmake --build $GITHUB_WORKSPACE/build --target check-sycl-unittests

I don't think that cmake runs more than 1 thread by default.

check-sycl on the other hand is defined as a LIT test suite:

llvm/sycl/test/CMakeLists.txt

Lines 79 to 85 in e05e82b

add_lit_testsuite(check-sycl-combined-triples "Running device-agnostic SYCL regression tests for all available triples"
${CMAKE_CURRENT_BINARY_DIR}
ARGS ${RT_TEST_ARGS}
PARAMS "SYCL_TRIPLE=${TRIPLES}"
DEPENDS ${SYCL_TEST_DEPS}
${SYCL_TEST_EXCLUDE}
)

And therefore it is by default launched in highly parallel mode by LIT where each TEST is a separate LIT test and can be run in parallel with other tests.

So yes, our CI simply runs everything sequentially in one thread, hence no races. LIT runs everything in parallel - and I assume that working directory for all tests is the same, so conf.txt file that we create is shared by more then one tests (i.e. its preview and non-preview versions)

Copy link
Contributor

@uditagarwal97 uditagarwal97 Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes a lot of sense. Thanks!

#ifdef _WIN32
_putenv_s("SYCL_CONFIG_FILE_NAME", "conf.txt");
#else
Expand Down Expand Up @@ -244,7 +244,7 @@ TEST(ConfigTests, CheckConfigProcessing) {
// not a valid number, the disk cache tracing will be enabled (depreciated
// behavior). The default value is 0 and no tracing is enabled.
using namespace sycl::detail;
TEST(ConfigTests, CheckSyclCacheTraceTest) {
TEST(ConfigTests, DISABLED_CheckSyclCacheTraceTest) {

// Lambda to test parsing of SYCL_CACHE_TRACE
auto TestConfig = [](int expectedValue, int expectedDiskCache,
Expand Down Expand Up @@ -330,7 +330,7 @@ TEST(ConfigTests, CheckSyclCacheTraceTest) {
// Cache eviction is performed when the cache size exceeds the threshold.
// The thresholds are specified in bytes.
// The default value is "0" which means that eviction is disabled.
TEST(ConfigTests, CheckSyclCacheEvictionThresholdTest) {
TEST(ConfigTests, DISABLED_CheckSyclCacheEvictionThresholdTest) {

using InMemEvicType =
sycl::detail::SYCLConfig<SYCL_IN_MEM_CACHE_EVICTION_THRESHOLD>;
Expand Down Expand Up @@ -393,7 +393,7 @@ TEST(ConfigTests, CheckSyclCacheEvictionThresholdTest) {
// Cache eviction is performed when the cache size exceeds the threshold.
// The thresholds are specified in bytes.
// The default value is "0" which means that eviction is disabled.
TEST(ConfigTests, CheckPersistentCacheEvictionThresholdTest) {
TEST(ConfigTests, DISABLED_CheckPersistentCacheEvictionThresholdTest) {

using OnDiskEvicType = sycl::detail::SYCLConfig<SYCL_CACHE_MAX_SIZE>;

Expand Down Expand Up @@ -451,7 +451,7 @@ TEST(ConfigTests, CheckPersistentCacheEvictionThresholdTest) {
}

// SYCL_PARALLEL_FOR_RANGE_ROUNDING_PARAMS accepts ...
TEST(ConfigTests, CheckParallelForRangeRoundingParams) {
TEST(ConfigTests, DISABLED_CheckParallelForRangeRoundingParams) {

// Lambda to set SYCL_PARALLEL_FOR_RANGE_ROUNDING_PARAMS.
auto SetRoundingParams = [](const char *value) {
Expand Down
4 changes: 2 additions & 2 deletions sycl/unittests/kernel-and-program/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ add_sycl_unittest(KernelAndProgramTests OBJECT
KernelArgs.cpp
KernelInfoShortcuts.cpp
)
target_compile_definitions(KernelAndProgramTests_non_preview PRIVATE __SYCL_INTERNAL_API)
target_compile_definitions(KernelAndProgramTests_preview PRIVATE __SYCL_INTERNAL_API __INTEL_PREVIEW_BREAKING_CHANGES)
target_compile_definitions(KernelAndProgramTests_Non_Preview_Tests PRIVATE __SYCL_INTERNAL_API)
target_compile_definitions(KernelAndProgramTests_Preview_Tests PRIVATE __SYCL_INTERNAL_API __INTEL_PREVIEW_BREAKING_CHANGES)
24 changes: 12 additions & 12 deletions sycl/unittests/kernel-and-program/PersistentDeviceCodeCache.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ class PersistentDeviceCodeCache

/* Checks that key values with \0 symbols are processed correctly
*/
TEST_P(PersistentDeviceCodeCache, KeysWithNullTermSymbol) {
TEST_P(PersistentDeviceCodeCache, DISABLED_KeysWithNullTermSymbol) {
std::string Key{'1', '\0', '3', '4', '\0'};
std::vector<unsigned char> SpecConst(Key.begin(), Key.end());
std::string ItemDir = detail::PersistentDeviceCodeCache::getCacheItemPath(
Expand All @@ -316,7 +316,7 @@ TEST_P(PersistentDeviceCodeCache, KeysWithNullTermSymbol) {
ASSERT_NO_ERROR(llvm::sys::fs::remove_directories(ItemDir));
}

TEST_P(PersistentDeviceCodeCache, MultipleImages) {
TEST_P(PersistentDeviceCodeCache, DISABLED_MultipleImages) {
const char *ExtraEntryName = "ExtraEntry";
_sycl_offload_entry_struct ExtraEntryStruct = {
/*addr*/ nullptr, const_cast<char *>(ExtraEntryName),
Expand Down Expand Up @@ -381,21 +381,21 @@ TEST_P(PersistentDeviceCodeCache, MultipleImages) {
/* Do read/write for the same cache item to/from 300 threads for small device
* code size. Make sure that there is no data corruption or crashes.
*/
TEST_P(PersistentDeviceCodeCache, ConcurentReadWriteSmallItem) {
TEST_P(PersistentDeviceCodeCache, DISABLED_ConcurentReadWriteSmallItem) {
ConcurentReadWriteCache(0, 300);
}

/* Do read/write for the same cache item to/from 100 threads for medium device
* code size. Make sure that there is no data corruption or crashes.
*/
TEST_P(PersistentDeviceCodeCache, ConcurentReadWriteCacheMediumItem) {
TEST_P(PersistentDeviceCodeCache, DISABLED_ConcurentReadWriteCacheMediumItem) {
ConcurentReadWriteCache(1, 100);
}

/* Do read/write for the same cache item to/from 20 threads from big device
* code size. Make sure that there is no data corruption or crashes.
*/
TEST_P(PersistentDeviceCodeCache, ConcurentReadWriteCacheBigItem) {
TEST_P(PersistentDeviceCodeCache, DISABLED_ConcurentReadWriteCacheBigItem) {
ConcurentReadWriteCache(2, 20);
}

Expand All @@ -406,7 +406,7 @@ TEST_P(PersistentDeviceCodeCache, ConcurentReadWriteCacheBigItem) {
* - source file is corrupted;
* - binary file is corrupted.
*/
TEST_P(PersistentDeviceCodeCache, CorruptedCacheFiles) {
TEST_P(PersistentDeviceCodeCache, DISABLED_CorruptedCacheFiles) {
std::string BuildOptions{"--corrupted-file"};
std::string ItemDir = detail::PersistentDeviceCodeCache::getCacheItemPath(
*getSyclObjImpl(Dev), {&Img}, {}, BuildOptions);
Expand Down Expand Up @@ -491,7 +491,7 @@ TEST_P(PersistentDeviceCodeCache, CorruptedCacheFiles) {
* - new cache item is created if existing one is locked on write operation;
* - cache miss happens on read operation.
*/
TEST_P(PersistentDeviceCodeCache, LockFile) {
TEST_P(PersistentDeviceCodeCache, DISABLED_LockFile) {
std::string BuildOptions{"--obsolete-lock"};
std::string ItemDir = detail::PersistentDeviceCodeCache::getCacheItemPath(
*getSyclObjImpl(Dev), {&Img}, {}, BuildOptions);
Expand Down Expand Up @@ -541,7 +541,7 @@ TEST_P(PersistentDeviceCodeCache, LockFile) {
// llvm::sys::fs::setPermissions does not make effect on Windows
/* Checks cache behavior when filesystem read/write operations fail
*/
TEST_P(PersistentDeviceCodeCache, AccessDeniedForCacheDir) {
TEST_P(PersistentDeviceCodeCache, DISABLED_AccessDeniedForCacheDir) {
std::string BuildOptions{"--build-options"};
std::string ItemDir = detail::PersistentDeviceCodeCache::getCacheItemPath(
*getSyclObjImpl(Dev), {&Img}, {}, BuildOptions);
Expand Down Expand Up @@ -584,7 +584,7 @@ TEST_P(PersistentDeviceCodeCache, AccessDeniedForCacheDir) {
#endif //_WIN32

// Unit tests for testing eviction in persistent cache.
TEST_P(PersistentDeviceCodeCache, BasicEviction) {
TEST_P(PersistentDeviceCodeCache, DISABLED_BasicEviction) {

// Cleanup the cache directory.
std::string CacheRoot = detail::PersistentDeviceCodeCache::getRootDir();
Expand Down Expand Up @@ -646,7 +646,7 @@ TEST_P(PersistentDeviceCodeCache, BasicEviction) {
}

// Unit test for testing size file creation and update, concurrently.
TEST_P(PersistentDeviceCodeCache, ConcurentReadWriteCacheFileSize) {
TEST_P(PersistentDeviceCodeCache, DISABLED_ConcurentReadWriteCacheFileSize) {
// Cleanup the cache directory.
std::string CacheRoot = detail::PersistentDeviceCodeCache::getRootDir();
ASSERT_NO_ERROR(llvm::sys::fs::remove_directories(CacheRoot));
Expand All @@ -659,7 +659,7 @@ TEST_P(PersistentDeviceCodeCache, ConcurentReadWriteCacheFileSize) {
}

// Unit test for adding and evicting cache, concurrently.
TEST_P(PersistentDeviceCodeCache, ConcurentReadWriteCacheEviction) {
TEST_P(PersistentDeviceCodeCache, DISABLED_ConcurentReadWriteCacheEviction) {
// Cleanup the cache directory.
std::string CacheRoot = detail::PersistentDeviceCodeCache::getRootDir();
ASSERT_NO_ERROR(llvm::sys::fs::remove_directories(CacheRoot));
Expand All @@ -670,7 +670,7 @@ TEST_P(PersistentDeviceCodeCache, ConcurentReadWriteCacheEviction) {
}

// Unit test for ensuring that os_utils::getDirectorySize is thread-safe.
TEST_P(PersistentDeviceCodeCache, ConcurentDirectorySizeCalculation) {
TEST_P(PersistentDeviceCodeCache, DISABLED_ConcurentDirectorySizeCalculation) {
// Cleanup the cache directory.
std::string CacheRoot = detail::PersistentDeviceCodeCache::getRootDir();
ASSERT_NO_ERROR(llvm::sys::fs::remove_directories(CacheRoot));
Expand Down
8 changes: 4 additions & 4 deletions sycl/unittests/pipes/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ add_sycl_unittest(PipeTests OBJECT
host_pipe_registration.cpp
)

add_dependencies(PipeTests_non_preview sycl)
target_include_directories(PipeTests_non_preview PRIVATE SYSTEM ${sycl_inc_dir})
add_dependencies(PipeTests_preview sycl)
target_include_directories(PipeTests_preview PRIVATE SYSTEM ${sycl_inc_dir})
add_dependencies(PipeTests_Non_Preview_Tests sycl)
target_include_directories(PipeTests_Non_Preview_Tests PRIVATE SYSTEM ${sycl_inc_dir})
add_dependencies(PipeTests_Preview_Tests sycl)
target_include_directories(PipeTests_Preview_Tests PRIVATE SYSTEM ${sycl_inc_dir})
8 changes: 4 additions & 4 deletions sycl/unittests/ur/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ add_sycl_unittest(UrTests OBJECT
UrUtility.cpp
)

add_dependencies(UrTests_non_preview sycl)
target_include_directories(UrTests_non_preview PRIVATE SYSTEM ${sycl_inc_dir})
add_dependencies(UrTests_preview sycl)
target_include_directories(UrTests_preview PRIVATE SYSTEM ${sycl_inc_dir})
add_dependencies(UrTests_Non_Preview_Tests sycl)
target_include_directories(UrTests_Non_Preview_Tests PRIVATE SYSTEM ${sycl_inc_dir})
add_dependencies(UrTests_Preview_Tests sycl)
target_include_directories(UrTests_Preview_Tests PRIVATE SYSTEM ${sycl_inc_dir})
4 changes: 2 additions & 2 deletions sycl/unittests/xpti_trace/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@ add_sycl_unittest(XptiTraceTests OBJECT
NodeCreation.cpp
QueueIDCheck.cpp
)
target_link_libraries(XptiTraceTests_non_preview PRIVATE xpti xptitest_subscriber)
target_link_libraries(XptiTraceTests_preview PRIVATE xpti xptitest_subscriber)
target_link_libraries(XptiTraceTests_Non_Preview_Tests PRIVATE xpti xptitest_subscriber)
target_link_libraries(XptiTraceTests_Preview_Tests PRIVATE xpti xptitest_subscriber)
Loading