build_llvm.py: Speed up llvm build with multiple procs (#3497) #3512
Conversation
build-scripts/build_llvm.py
Outdated
| @@ -156,11 +156,14 @@ def build_llvm(llvm_dir, platform, backends, projects, use_clang=False, extra_fl | |||
| CONFIG_CMD += " -A x64" | |||
| else: | |||
| CONFIG_CMD += " -G'Ninja'" | |||
| print("Config command: " + CONFIG_CMD) | |||
There was a problem hiding this comment.
| print("Config command: " + CONFIG_CMD) | |
| print(f"Config command: {CONFIG_CMD}") |
build-scripts/build_llvm.py
Outdated
| subprocess.check_call(shlex.split(CONFIG_CMD), cwd=build_dir) | ||
|
|
||
| BUILD_CMD = "cmake --build . --target package" + ( | ||
| " --config Release" if "windows" == platform else "" | ||
| ) | ||
| BUILD_CMD += " --parallel " + str(os.cpu_count()) | ||
| print("Build command: " + BUILD_CMD) |
There was a problem hiding this comment.
| print("Build command: " + BUILD_CMD) | |
| print(f"Build command: {BUILD_CMD}") |
build-scripts/build_llvm.py
Outdated
| subprocess.check_call(shlex.split(CONFIG_CMD), cwd=build_dir) | ||
|
|
||
| BUILD_CMD = "cmake --build . --target package" + ( | ||
| " --config Release" if "windows" == platform else "" | ||
| ) | ||
| BUILD_CMD += " --parallel " + str(os.cpu_count()) |
There was a problem hiding this comment.
i guess it's simpler to leave it to ninja when it's used.
There was a problem hiding this comment.
Yes, not sure whether ninja will set the cpu count when -j or --parallel isn't set? I uploaded this PR since it took long time for the CI to compile the llvm library when creating a new release, e.g.
https://github.com/bytecodealliance/wasm-micro-runtime/actions/runs/9394335353/job/25877128841
It will be good if we can speed it up.
BTW, in windows, it doesn't add -G'Ninja', instead it adds -G'Unix Makefiles' or -A x64. Maybe adding --parellel n option also benefits the compilation for windows.
There was a problem hiding this comment.
iirc, ninja decides the concurrency by itself, based on the number of processors, unless you override it with -j N.
No description provided.