Skip to content

Fix UBSan errors: vptr check failures due to lack of RTTI#9384

Closed
ncelikNV wants to merge 1 commit intoshader-slang:masterfrom
ncelikNV:fix-ubsan-errors-15-llvm-rtti
Closed

Fix UBSan errors: vptr check failures due to lack of RTTI#9384
ncelikNV wants to merge 1 commit intoshader-slang:masterfrom
ncelikNV:fix-ubsan-errors-15-llvm-rtti

Conversation

@ncelikNV
Copy link
Contributor

@ncelikNV ncelikNV commented Dec 16, 2025

Fixes lots of member call on address [...] which does not point to an object of type '[...]' UBSan errors, e.g.:

.../source/compiler-core/slang-downstream-compiler-set.cpp:88:46: runtime error: member call on address 0x73ce982dc260 which does not point to an object of type 'Slang::IDownstreamCompiler'

UBSan's vptr/dynamic type checks require RTTI, as it assumes that a
vtable is invalid if the vtable prefix points to a null type info
pointer:

https://github.com/llvm/llvm-project/blob/ea9addae8336e92222214036bbec821e6b29d8bc/compiler-rt/lib/ubsan/ubsan_type_hash_itanium.cpp#L215-L217

But as LLVM is built without RTTI by default, slang-llvm was being built
without RTTI.

Related to #9099.

Fixes lots of "member call on address [...] which does not point to an
object of type '[...]'" UBSan errors, e.g.:

```
.../source/compiler-core/slang-downstream-compiler-set.cpp:88:46: runtime error: member call on address 0x73ce982dc260 which does not point to an object of type 'Slang::IDownstreamCompiler'
```

UBSan's vptr/dynamic type checks require RTTI, as it assumes that a
vtable is invalid if the vtable prefix points to a null type info
pointer:

https://github.com/llvm/llvm-project/blob/ea9addae8336e92222214036bbec821e6b29d8bc/compiler-rt/lib/ubsan/ubsan_type_hash_itanium.cpp#L215-L217

But as LLVM is built without RTTI by default, slang-llvm was being built
without RTTI.

Related to shader-slang#9099.
@ncelikNV ncelikNV requested a review from a team as a code owner December 16, 2025 20:13
@ncelikNV ncelikNV added the pr: non-breaking PRs without breaking changes label Dec 16, 2025
@ncelikNV
Copy link
Contributor Author

As far as I understand, this would require a new release of Slang/slang-llvm to avoid breaking SLANG_SLANG_LLVM_FLAVOR=FETCH_BINARY builds that would download an old build of slang-llvm without RTTI, so I'm not sure if this should have the pr: breaking change label or not. #8737 didn't have it.

Also, somehow my libslang-llvm.so with RTTI is smaller than the one from the v2025.24 release. slang-2025.24-linux-x86_64's libslang-llvm.so is 130 MiB. Mine (with RTTI) is 117 MiB. I built LLVM 21.1.2 with -DCMAKE_C_COMPILER=clang-20 -DCMAKE_CXX_COMPILER=clang++-20 -DLLVM_TARGETS_TO_BUILD="X86;ARM;AArch64" -DLLVM_ENABLE_PROJECTS="clang" -DCLANG_ENABLE_STATIC_ANALYZER=OFF -DCLANG_ENABLE_ARCMT=OFF.

Btw, that last CMake option, CLANG_ENABLE_ARCMT (which external/build-llvm.sh uses) is deprecated and it causes a CMake warning.

target_compile_options(slang-llvm PRIVATE -wd4244 /Zc:preprocessor)
endif()

if(NOT LLVM_ENABLE_RTTI)
Copy link
Contributor

@juliusikkala juliusikkala Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep this if(NOT LLVM_ENABLE_RTTI) block here for the USE_SYSTEM_LLVM option to work. Some distributions may have LLVM without RTTI, and removing this would cause build errors with that option on such distributions.

I think this block shouldn't be running on builds that used build-llvm.sh because of the flag that was added in this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I just noticed <LLVM install dir>/lib/cmake/llvm/LLVMConfig.cmake sets LLVM_ENABLE_RTTI to what it was set to when the LLVM CMake build was configured. I thought LLVM_ENABLE_RTTI wouldn't be set here so NOT LLVM_ENABLE_RTTI would always be true.

@ncelikNV ncelikNV marked this pull request as draft December 18, 2025 17:13
-DLLVM_BUILD_TOOLS=0
# slang-llvm is built with RTTI enabled to support UndefinedBehaviorSanitizer's vptr checks, so
# LLVM should be built with RTTI as well
-DLLVM_ENABLE_RTTI=1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if we built LLVM with RTTI enabled in Slang release packages, these won't ship a build of slang-llvm with sanitizer instrumentation enabled.

If we want to instrument slang-llvm with sanitizers as well, we have to build it as part of the sanitizer build instead of downloading the latest Slang release package. If we want to avoid having to build LLVM with RTTI as part of that, I guess we can just use https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#id16 to disable UBSan's vptr checks in slang-llvm (we're unlikely to run into a real type mismatch when casting these Slang COM API types).

@ncelikNV ncelikNV closed this Jan 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr: non-breaking PRs without breaking changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants