Build error due to incompatibility with _GLIBCXX_USE_CXX11_ABI=0

[maleadt]

For redistribution purposes, I'm building compute_runtime (as well as dependencies like libigc) both using _GLIBCXX_USE_CXX11_ABI=0 (cxx03 string ABI) and _GLIBCXX_USE_CXX11_ABI=1 (cxx11 string ABI), using a compiler wrapper script that sets this flag as late as possible.

Since 18c25e5, our cxx03 build has mysteriously broken, with ocloc segfaulting during compilation of builtins:

[13:24:30] [1099/4669] Generating ../../../../bin/built_ins/x64/gen12lp/bindful_copy_image_to_image3d_adls.spv
[13:24:30] ninja: job failed: cd /workspace/srcdir/compute-runtime/shared/source/built_ins/kernels && /usr/bin/cmake -E env LD_LIBRARY_PATH=/usr/lib/csl-musl-x86_64:/usr/lib/csl-glibc-x86_64:/usr/local/lib64:/usr/local/lib:/usr/lib64:/usr/lib:/lib64:/lib:/workspace/x86_64-linux-musl-cxx11/destdir/lib:/opt/x86_64-linux-musl/x86_64-linux-musl/lib64:/opt/x86_64-linux-musl/x86_64-linux-musl/lib:/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64:/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib:/workspace/destdir/lib64:/workspace/destdir/lib::/workspace/srcdir/compute-runtime/build/bin /workspace/srcdir/compute-runtime/build/bin/ocloc -q -file /workspace/srcdir/compute-runtime/shared/source/built_ins/kernels/fill_buffer_stateless.builtin_kernel -spv_only -device adln -cl-intel-greater-than-4GB-buffer-required -64 -output stateless_fill_buffer_stateless -out_dir /workspace/srcdir/compute-runtime/build/bin/built_ins/x64/gen12lp -options -cl-kernel-arg-info
[13:24:30] free(): invalid pointer
[13:24:30] Subprocess aborted

Looking at GDB, this indeed seems related to the string ABI:

free(): invalid pointer

Program received signal SIGABRT, Aborted.
__pthread_kill_internal (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:45
45	pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_internal (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:45
#1  0x00007ffff7a2ed0d in __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at pthread_kill.c:62
#2  0x00007ffff79e4ef2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007ffff79d043c in __GI_abort () at abort.c:79
#4  0x00007ffff7a237b8 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff7b55b22 "%s\n")
    at ../sysdeps/posix/libc_fatal.c:155
#5  0x00007ffff7a37fca in malloc_printerr (str=str@entry=0x7ffff7b53767 "free(): invalid pointer") at malloc.c:5543
#6  0x00007ffff7a3975c in _int_free (av=<optimized out>, p=<optimized out>, have_lock=have_lock@entry=0)
    at malloc.c:4326
#7  0x00007ffff7a3c231 in __GI___libc_free (mem=<optimized out>) at malloc.c:3278
#8  0x00007ffff7d33495 in operator delete (ptr=<optimized out>)
    at /workspace/srcdir/gcc-8.1.0/libstdc++-v3/libsupc++/del_op.cc:49
#9  0x00007ffff7d72fe5 in __gnu_cxx::new_allocator<char>::deallocate (this=<optimized out>, __p=<optimized out>)
    at /workspace/srcdir/gcc_build/x86_64-linux-gnu/libstdc++-v3/include/ext/new_allocator.h:116
#10 0x00007ffff7d73dcd in std::string::_Rep::_M_dispose (__a=..., this=<optimized out>)
    at /workspace/srcdir/gcc_build/x86_64-linux-gnu/libstdc++-v3/include/bits/basic_string.h:3251
#11 std::string::reserve (this=this@entry=0x7fffffffc288, __res=<optimized out>, __res@entry=29)
    at /workspace/srcdir/gcc_build/x86_64-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:961
#12 0x00007ffff7d73ffe in std::string::append (this=this@entry=0x7fffffffc288,
    __s=0x41d878 "cl_khr_byte_addressable_store cl_khr_device_uuid cl_khr_fp16 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_local_int32_base_atomics cl_khr_local_int32_extende"...,
    __n=__n@entry=29) at /workspace/srcdir/gcc_build/x86_64-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:758
#13 0x00007ffff7d4e142 in std::operator>><char, std::char_traits<char>, std::allocator<char> > (__in=..., __str=...)
    at /workspace/srcdir/gcc-8.1.0/libstdc++-v3/src/c++98/istream-string.cc:83
#14 0x00007ffff7e6d838 in NEO::convertEnabledExtensionsToCompilerInternalOptions(char const*, StackVec<_cl_name_version, 27ul, unsigned char>&) () from /workspace/srcdir/compute-runtime/build/bin/libocloc.so
#15 0x00007ffff7eb5c29 in NEO::appendExtensionsToInternalOptions(NEO::HardwareInfo const&, std::string const&, std::string&) () from /workspace/srcdir/compute-runtime/build/bin/libocloc.so
#16 0x00007ffff7f0cb5e in NEO::OfflineCompiler::initialize(unsigned long, std::vector<std::string, std::allocator<std::string> > const&, bool) () from /workspace/srcdir/compute-runtime/build/bin/libocloc.so
#17 0x00007ffff7f0d7bc in NEO::OfflineCompiler::create(unsigned long, std::vector<std::string, std::allocator<std::string> > const&, bool, int&, OclocArgHelper*) () from /workspace/srcdir/compute-runtime/build/bin/libocloc.so
#18 0x00007ffff7efe9a1 in Ocloc::Commands::compile(OclocArgHelper*, std::vector<std::string, std::allocator<std::string> > const&) () from /workspace/srcdir/compute-runtime/build/bin/libocloc.so
#19 0x00007ffff7eeaec8 in oclocInvoke () from /workspace/srcdir/compute-runtime/build/bin/libocloc.so
#20 0x0000000000401093 in main ()

... but I haven't been able to come up with an MWE. I'm not sure yet how limiting the list of symbols exported from libocloc could affect this; looking at the objects and libraries in the build I don't see any std::__cxx11::basic_string signatures. It's possible or even likely I'm doing something wrong, but would appreciate any thoughts here.

[maleadt]

The issue is on my side; there's a mismatch between the system glibc (alwayx cxx11 string ABI) and what I'm building.

EDIT: actually, scratch that, it looks like my glibc does support both string ABIs:

sandbox:${WORKSPACE}/srcdir/compute-runtime/build # nm -C /usr/lib/csl-glibc-x86_64/libstdc++.so | grep -w basic_string | grep 'reserve()'
0000000000145400 W std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::reserve()
000000000015b660 W std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::reserve()
00000000000f7880 W          std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::reserve()

[JablonskiMateusz]

Hi @maleadt
Could you share more details how to reproduce the issue?

[maleadt]

Building with _GLIBCXX_USE_CXX11_ABI=0 is what reproduces the error here.

[JablonskiMateusz]

@maleadt could you provide repro steps how you setup _GLIBCXX_USE_CXX11_ABI=0, what cmd line for cmake, what version of compiler?

I cannot reproduce the issue on my side