Skip to content

common: fix missing exports in llama-common#22340

Merged
ggerganov merged 3 commits into
ggml-org:masterfrom
qualcomm:llama-common-missing-exports
Apr 27, 2026
Merged

common: fix missing exports in llama-common#22340
ggerganov merged 3 commits into
ggml-org:masterfrom
qualcomm:llama-common-missing-exports

Conversation

@max-krasnyansky
Copy link
Copy Markdown
Member

Overview

The recent change to make llama-common the dynamic library broke some of the builds that use LTO.
The LTO (at least in llvm/clang-22) is dropping those explicit instantiations for common_debug_cb_eval

Those callbacks are basically LLAMA_API now since they are used by other dyn libs.
So this PR marks them as such.

See detailed error here.

Details
[367/419] Linking CXX executable bin\llama-eval-callback.exe
FAILED: [code=1] bin/llama-eval-callback.exe
C:\Windows\system32\cmd.exe /C "cd . && C:\PROGRA~1\LLVM\bin\CLANG_~1.EXE --target=arm64-pc-windows-msvc -nostartfiles -nostdlib -march=armv8.7a+fp16 -fvectorize -ffp-model=fast -flto -D_GNU_SOURCE -O3 -DNDEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrt -Xlinker /subsystem:console -fuse-ld=lld-link examples/eval-callback/CMakeFiles/llama-eval-callback.dir/eval-callback.cpp.obj -o bin\llama-eval-callback.exe -Xlinker /MANIFEST:EMBED -Xlinker /implib:examples\eval-callback\llama-eval-callback.lib -Xlinker /pdb:bin\llama-eval-callback.pdb -Xlinker /version:0.0   common/llama-common.lib  src/llama.lib  ggml/src/ggml.lib  ggml/src/ggml-cpu.lib  ggml/src/ggml-opencl/ggml-opencl.lib  ggml/src/ggml-hexagon/ggml-hexagon.lib  ggml/src/ggml-base.lib  common/llama-common-base.lib  -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -loldnames  && cd ."
lld-link: error: undefined symbol: bool __cdecl common_debug_cb_eval<0>(struct ggml_tensor *, bool, void *)
>>> referenced by C:/Users/maxk/src/llama.cpp/examples/eval-callback/eval-callback.cpp
>>>               examples/eval-callback/CMakeFiles/llama-eval-callback.dir/eval-callback.cpp.obj
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

[368/419] Linking CXX executable bin\llama-debug.exe
FAILED: [code=1] bin/llama-debug.exe
C:\Windows\system32\cmd.exe /C "cd . && C:\PROGRA~1\LLVM\bin\CLANG_~1.EXE --target=arm64-pc-windows-msvc -nostartfiles -nostdlib -march=armv8.7a+fp16 -fvectorize -ffp-model=fast -flto -D_GNU_SOURCE -O3 -DNDEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrt -Xlinker /subsystem:console -fuse-ld=lld-link examples/debug/CMakeFiles/llama-debug.dir/debug.cpp.obj -o bin\llama-debug.exe -Xlinker /MANIFEST:EMBED -Xlinker /implib:examples\debug\llama-debug.lib -Xlinker /pdb:bin\llama-debug.pdb -Xlinker /version:0.0   common/llama-common.lib  src/llama.lib  ggml/src/ggml.lib  ggml/src/ggml-cpu.lib  ggml/src/ggml-opencl/ggml-opencl.lib  ggml/src/ggml-hexagon/ggml-hexagon.lib  ggml/src/ggml-base.lib  common/llama-common-base.lib  -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -loldnames  && cd ."
lld-link: error: undefined symbol: bool __cdecl common_debug_cb_eval<0>(struct ggml_tensor *, bool, void *)
>>> referenced by C:/Users/maxk/src/llama.cpp/examples/debug/debug.cpp
>>>               examples/debug/CMakeFiles/llama-debug.dir/debug.cpp.obj
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

Requirements

@max-krasnyansky max-krasnyansky requested a review from a team as a code owner April 24, 2026 23:07
@max-krasnyansky
Copy link
Copy Markdown
Member Author

Here is the LLVM/clang version info

PS C:\Users\maxk\src\llama.cpp>  C:\PROGRA~1\LLVM\bin\CLANG_~1.EXE --version
clang version 22.1.3 (https://github.com/llvm/llvm-project e9846648fd6183ee6d8cbdb4502213fcf902a211)
Target: aarch64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin

Installed with winget.
Rest of MSVC stuff is Visual Studio CE 2026

@ggerganov
Copy link
Copy Markdown
Member

ggerganov commented Apr 25, 2026

It's not OK to mix the libllama exports with libllama-common.

The proper fix is to refactor the common/debug implementation. Passing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates this problem for no reason. It should be a member of the base_callback_data instead.

cc @pwilkin

…_data

Passing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.
It should just be a member of the base_callback_data instead.
@max-krasnyansky max-krasnyansky force-pushed the llama-common-missing-exports branch from 6166475 to 38d762d Compare April 25, 2026 23:50
@max-krasnyansky max-krasnyansky requested review from a team and ggerganov as code owners April 25, 2026 23:50
@max-krasnyansky
Copy link
Copy Markdown
Member Author

max-krasnyansky commented Apr 25, 2026

It's not OK to mix the libllama exports with libllama-common.

The proper fix is to refactor the common/debug implementation. Passing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates this problem for no reason. It should be a member of the base_callback_data instead.

cc @pwilkin

Sounds good. I updated the implementation to do just that.
Android builds are good. Checking my Windows on Snapdragon setup ...

@max-krasnyansky
Copy link
Copy Markdown
Member Author

@ggerganov @pwilkin
My latest update resolves the original LTO issue with MSVC/LLVM-22 builds.
Let me know if we need further updates.

Move common_debug_cb_user_data's data members (std::regex,
std::vector<uint8_t>) into a private impl struct in debug.cpp.

This removes the includes of common.h and <regex> from debug.h,
reducing transitive dependencies for any translation unit that
includes the header.

Assisted-by: llama.cpp:local pi
@ggerganov
Copy link
Copy Markdown
Member

@max-krasnyansky I have a few edits in branch: https://github.com/ggml-org/llama.cpp/commits/pr/22340-gg/

I don't have permission to push to your branch, nor to open PRs to the qualcomm repo, so please cherry-pick the commits.

@max-krasnyansky
Copy link
Copy Markdown
Member Author

max-krasnyansky commented Apr 26, 2026

@max-krasnyansky I have a few edits in branch: https://github.com/ggml-org/llama.cpp/commits/pr/22340-gg/

I don't have permission to push to your branch, nor to open PRs to the qualcomm repo, so please cherry-pick the commits.

@ggerganov Done

Next time for updates like this I'll start PRs with branches in the main repo to simplify iterations.

@ggerganov ggerganov added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Apr 26, 2026
@ggerganov ggerganov merged commit 5594d13 into ggml-org:master Apr 27, 2026
44 of 46 checks passed
IntelNav pushed a commit to IntelNav/llama.cpp that referenced this pull request Apr 29, 2026
* common: refactor common/debug to move abort_on_nan into base_callback_data

Passing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.
It should just be a member of the base_callback_data instead.

* cont : cleanup

* common : use pimpl in debug.h to reduce header dependencies

Move common_debug_cb_user_data's data members (std::regex,
std::vector<uint8_t>) into a private impl struct in debug.cpp.

This removes the includes of common.h and <regex> from debug.h,
reducing transitive dependencies for any translation unit that
includes the header.

Assisted-by: llama.cpp:local pi

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
IntelNav pushed a commit to IntelNav/llama.cpp that referenced this pull request Apr 29, 2026
* common: refactor common/debug to move abort_on_nan into base_callback_data

Passing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.
It should just be a member of the base_callback_data instead.

* cont : cleanup

* common : use pimpl in debug.h to reduce header dependencies

Move common_debug_cb_user_data's data members (std::regex,
std::vector<uint8_t>) into a private impl struct in debug.cpp.

This removes the includes of common.h and <regex> from debug.h,
reducing transitive dependencies for any translation unit that
includes the header.

Assisted-by: llama.cpp:local pi

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
rsenthilkumar6 pushed a commit to rsenthilkumar6/llama.cpp that referenced this pull request May 1, 2026
* common: refactor common/debug to move abort_on_nan into base_callback_data

Passing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.
It should just be a member of the base_callback_data instead.

* cont : cleanup

* common : use pimpl in debug.h to reduce header dependencies

Move common_debug_cb_user_data's data members (std::regex,
std::vector<uint8_t>) into a private impl struct in debug.cpp.

This removes the includes of common.h and <regex> from debug.h,
reducing transitive dependencies for any translation unit that
includes the header.

Assisted-by: llama.cpp:local pi

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
samuraieng pushed a commit to samuraieng/llama.cpp that referenced this pull request May 6, 2026
* common: refactor common/debug to move abort_on_nan into base_callback_data

Passing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.
It should just be a member of the base_callback_data instead.

* cont : cleanup

* common : use pimpl in debug.h to reduce header dependencies

Move common_debug_cb_user_data's data members (std::regex,
std::vector<uint8_t>) into a private impl struct in debug.cpp.

This removes the includes of common.h and <regex> from debug.h,
reducing transitive dependencies for any translation unit that
includes the header.

Assisted-by: llama.cpp:local pi

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
* common: refactor common/debug to move abort_on_nan into base_callback_data

Passing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.
It should just be a member of the base_callback_data instead.

* cont : cleanup

* common : use pimpl in debug.h to reduce header dependencies

Move common_debug_cb_user_data's data members (std::regex,
std::vector<uint8_t>) into a private impl struct in debug.cpp.

This removes the includes of common.h and <regex> from debug.h,
reducing transitive dependencies for any translation unit that
includes the header.

Assisted-by: llama.cpp:local pi

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
meh pushed a commit to meh/llama.cpp that referenced this pull request May 10, 2026
* common: refactor common/debug to move abort_on_nan into base_callback_data

Passing bool abort_on_nan as template parameter for common_debug_cb_eval is unnecessary and creates an issue with LTO.
It should just be a member of the base_callback_data instead.

* cont : cleanup

* common : use pimpl in debug.h to reduce header dependencies

Move common_debug_cb_user_data's data members (std::regex,
std::vector<uint8_t>) into a private impl struct in debug.cpp.

This removes the includes of common.h and <regex> from debug.h,
reducing transitive dependencies for any translation unit that
includes the header.

Assisted-by: llama.cpp:local pi

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

examples merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants