Commit 4d8a9a0
committed
Fix aarch64/arm64 Linux wheel build failure due to missing libatomic linking
Problem
The tokenizers library fails to build on aarch64 Linux systems during CI with:
/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/aarch64-redhat-linux/13/ld: cannot find -latomic: No such file or directory
Root Cause
- On aarch64, certain atomic operations require explicit linking with libatomic
- The sentencepiece dependency detects libatomic (Found atomic: /usr/lib64/libatomic.so.1) but incorrectly adds just the string "atomic" to link flags instead of the actual library path
- This causes the linker to fail when building the Python extension
Solution
Added proper libatomic detection and linking for aarch64/arm64 systems in two places:
1. For the tokenizers static library
2. For the pytorch_tokenizers_cpp Python extension module
The fix uses CMake's find_library to locate libatomic and explicitly links it when building on aarch64/arm64 architectures.1 parent 348c748 commit 4d8a9a0
1 file changed
+21
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
97 | 106 | | |
98 | 107 | | |
99 | 108 | | |
| |||
160 | 169 | | |
161 | 170 | | |
162 | 171 | | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
163 | 184 | | |
164 | 185 | | |
165 | 186 | | |
| |||
0 commit comments