Get unexpected behavior on running clang.wasm

[swwind]

Describe the bug

The clang.wasm behave incorrectly on handling my -internal-isystem arguments.

wasmer 3.0.0-beta.2 | rustc 1.64.0 (a55dd71d5 2022-09-19) | x86_64

Steps to reproduce

// example.cpp
#include <stdio.h>
int main() {
  printf("hello world\n");
}

wapm install taybenlor/[email protected]
export SYSROOT=$(realpath ./wapm_package/taybenlor/[email protected]/sysroot)
# then try to run clang.wasm...

Expected behavior

We use wasmtime to compile example.cpp, everything works as expected.

wasmtime run ./wapm_packages/taybenlor/[email protected]/clang.wasm --allow-unknown-exports \
  --mapdir="/sys::$SYSROOT" --mapdir="/src::$PWD" -- \
  -cc1 -triple wasm32-unknown-wasi -emit-obj \
  -isysroot /sys \
  -internal-isystem /sys/include/c++/v1 \
  -internal-isystem /sys/include \
  -internal-isystem /sys/lib/clang/8.0.1/include \
  -x c++ /src/example.cpp -o /src/example.o -v

clang -cc1 version 8.0.1 based upon LLVM 8.0.1 default target wasm32-wasi
ignoring nonexistent directory "/sys/usr/local/include"
ignoring nonexistent directory "../lib/clang/8.0.1/include"
ignoring nonexistent directory "/sys/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /sys/include/c++/v1
 /sys/include
 /sys/lib/clang/8.0.1/include
End of search list.

Actual behavior

Then we try to pass the same arguments to wasmer, and we get a different result.

wasmer run ./wapm_packages/taybenlor/[email protected]/clang.wasm \
  --mapdir="/sys::$SYSROOT" --mapdir="/src::$PWD" -- \
  -cc1 -triple wasm32-unknown-wasi -emit-obj \
  -isysroot /sys \
  -internal-isystem /sys/include/c++/v1 \
  -internal-isystem /sys/include \
  -internal-isystem /sys/lib/clang/8.0.1/include \
  -x c++ /src/example.cpp -o /src/example.o -v

clang -cc1 version 8.0.1 based upon LLVM 8.0.1 default target wasm32-wasi
ignoring nonexistent directory "/sys/usr/local/include"
ignoring nonexistent directory "../lib/clang/8.0.1/include"
ignoring nonexistent directory "/sys/usr/include"
ignoring duplicate directory "/sys/include/c++/v1"
ignoring duplicate directory "/sys/include/c++/v1"
#include "..." search starts here:
#include <...> search starts here:
 /sys/include/c++/v1
End of search list.
In file included from /src/example.cpp:1:
/sys/include/c++/v1/stdio.h:108:15: fatal error: 'stdio.h' file not found
#include_next <stdio.h>
              ^~~~~~~~~
1 error generated.

We can see two unexpected ignoring duplicate directory "/sys/include/c++/v1" lines there, and missing two important include paths in #include <...> search starts here:.

Additional context

It looks like the vector that stores all include paths (like vec![a, b, c]) in memory has become vec![a, a, a] for no reason, thus we got two duplicate error and lost the last two paths.

I'm guessing there might be some memory leak, or this may related to #3125 or #3203.

[ptitSeb]

I do reproduce the issue on my side (but I don't have the sysroot/lib/clang/8.0.1/include installed after the wapm installation.

[ptitSeb]

Also the behaviour is the same using singlepass or the default cranelift. And valgrind doesn't detect anything (and doesn't change the behaviour)

[SuperSodaSea]

I encountered the same problem as you when I try to compile the latest clang to WASI. It seems that this problem is caused by invalid st_dev and st_ino returned by stat(), see #3583.

[ptitSeb]

Interesting, thanks for mentionning this.