Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] can't link c++ so if python is compiled with ccache since 72.2.0 #4748

Open
trim21 opened this issue Nov 13, 2024 · 2 comments
Open

[BUG] can't link c++ so if python is compiled with ccache since 72.2.0 #4748

trim21 opened this issue Nov 13, 2024 · 2 comments

Comments

@trim21
Copy link

trim21 commented Nov 13, 2024

setuptools version

setuptools >= 72.2.0

Python version

python 3.10

OS

debian 12

Additional environment information

No response

Description

I have a python compiled by pyenv with ccache, and since 72.2.0, setuptools can't link so as expected.

(bencode2) trim21@omv ~/proj/bencode2 $ python setup.py build_ext -i --force
running build_ext
building 'bencode2.__bencode' extension
ccache /usr/bin/c++ -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -march=native -mtune=native -fPIC -DFMT_HEADER_ONLY= -I./src/bencode2 -I./vendor/fmt/include -I./vendor/small_vector/source/include -I/home/trim21/proj/bencode2/.venv/lib/python3.10/site-packages/pybind11/include -I/home/trim21/proj/bencode2/.venv/include -I/home/trim21/.pyenv/versions/3.10.15/include/python3.10 -c ./src/bencode2/bencode.cpp -o build/temp.linux-x86_64-cpython-310/./src/bencode2/bencode.o -fvisibility=hidden -g0 -std=c++17
ccache /usr/bin/c++ -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -march=native -mtune=native /usr/bin/c++ -shared build/temp.linux-x86_64-cpython-310/./src/bencode2/bencode.o -L/home/trim21/.pyenv/versions/3.10.15/lib -o build/lib.linux-x86_64-cpython-310/bencode2/__bencode.cpython-310-x86_64-linux-gnu.so
/usr/bin/ld: cannot use executable file '/usr/bin/c++' as input to a link
collect2: error: ld returned 1 exit status
error: command '/usr/bin/ccache' failed with exit code 1

(bencode2) trim21@omv ~/proj/bencode2 $ python -m sysconfig|grep ccache
        BLDSHARED = "ccache /usr/bin/cc -shared -L/home/trim21/.pyenv/versions/3.10.15/lib -Wl,-rpath,/home/trim21/.pyenv/versions/3.10.15/lib -L/home/trim21/.pyenv/versions/3.10.15/lib -Wl,-rpath,/home/trim21/.pyenv/versions/3.10.15/lib"
        CC = "ccache /usr/bin/cc"
        CONFIG_ARGS = "'--prefix=/home/trim21/.pyenv/versions/3.10.15' '--libdir=/home/trim21/.pyenv/versions/3.10.15/lib' '--enable-shared' 'CC=ccache /usr/bin/cc' 'CFLAGS= -march=native -mtune=native' 'LDFLAGS=-L/home/trim21/.pyenv/versions/3.10.15/lib -Wl,-rpath,/home/trim21/.pyenv/versions/3.10.15/lib' 'LIBS=-L/home/trim21/.pyenv/versions/3.10.15/lib -Wl,-rpath,/home/trim21/.pyenv/versions/3.10.15/lib' 'CPPFLAGS=-I/home/trim21/.pyenv/versions/3.10.15/include'"
        CXX = "ccache /usr/bin/c++"
        LDCXXSHARED = "ccache /usr/bin/c++ -shared"
        LDSHARED = "ccache /usr/bin/cc -shared -L/home/trim21/.pyenv/versions/3.10.15/lib -Wl,-rpath,/home/trim21/.pyenv/versions/3.10.15/lib -L/home/trim21/.pyenv/versions/3.10.15/lib -Wl,-rpath,/home/trim21/.pyenv/versions/3.10.15/lib"
        LINKCC = "ccache /usr/bin/cc"
        MAINCC = "ccache /usr/bin/cc"

Expected behavior

it should build so as expected

How to Reproduce

you will need to install pyenv, compile python with https://github.com/pyenv/pyenv-ccache

then use this python to build binary module,

for example

git clone https://github.com/trim21/bencode-py
cd bencode-py
pyenv local 3.10
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python setup.py build_ext -i --force

Output

(bencode2) trim21@omv ~/proj/bencode2 $ python setup.py build_ext -i --force -g
running build_ext
building 'bencode2.__bencode' extension
ccache /usr/bin/c++ -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -march=native -mtune=native -fPIC -g -DFMT_HEADER_ONLY= -I./src/bencode2 -I./vendor/fmt/include -I./vendor/small_vector/source/include -I/home/trim21/proj/bencode2/.venv/lib/python3.10/site-packages/pybind11/include -I/home/trim21/proj/bencode2/.venv/include -I/home/trim21/.pyenv/versions/3.10.15/include/python3.10 -c ./src/bencode2/bencode.cpp -o build/temp.linux-x86_64-cpython-310/./src/bencode2/bencode.o -fvisibility=hidden -g0 -std=c++17
ccache /usr/bin/c++ -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -march=native -mtune=native /usr/bin/c++ -shared -g build/temp.linux-x86_64-cpython-310/./src/bencode2/bencode.o -L/home/trim21/.pyenv/versions/3.10.15/lib -o build/lib.linux-x86_64-cpython-310/bencode2/__bencode.cpython-310-x86_64-linux-gnu.so
/usr/bin/ld: cannot use executable file '/usr/bin/c++' as input to a link
collect2: error: ld returned 1 exit status
error: command '/usr/bin/ccache' failed with exit code 1
@trim21 trim21 added bug Needs Triage Issues that need to be evaluated for severity and status. labels Nov 13, 2024
@trim21 trim21 changed the title [BUG] can't link so if python is compiled with ccache since 72.2.0 [BUG] can't link c++ so if python is compiled with ccache since 72.2.0 Nov 13, 2024
@trim21
Copy link
Author

trim21 commented Nov 13, 2024

some testing:

54936c7 is good

64f10c5 is bad

this is caused by #4539

@abravalheri abravalheri added upstream and removed Needs Triage Issues that need to be evaluated for severity and status. labels Nov 13, 2024
@bjodah
Copy link

bjodah commented Jan 15, 2025

I ran into this problem myself today (cpython 3.13.1), the symptom is somewhat different though, even though ccache was used to compile cpython (manual build and install of cpython from source), the linking command does not reference ccache, but it repeats the name of the compiler twice:

      clang++ -isystem /opt-3/sundials-6.7.0-asan/include -fsanitize=address -stdlib++-isystem /opt-2/libcxx19-asan/include/c++/v1 -ferror-limit=5 -fPIC -DPYCVODES_NO_KLU=0 -DPYCVODES_NO_LAPACK=0 -DANYODE_NO_LAPACK=0 -I/tmp/pip-build-env-uzzybwax/overlay/lib/python3.13/site-packages/numpy/_core/include -Ipycvodes/include -Iexternal/anyode/include -I/opt-3/cpython-v3.13.1-asan/include/python3.13d -c pycvodes/_cvodes.cpp -o build/temp.linux-x86_64-cpython-313-pydebug/pycvodes/_cvodes.o -DVERSION_INFO=\"0.15.0.dev0+git\" -std=c++17
      In file included from pycvodes/_cvodes.cpp:1280:
      In file included from /tmp/pip-build-env-uzzybwax/overlay/lib/python3.13/site-packages/numpy/_core/include/numpy/arrayobject.h:5:
      In file included from /tmp/pip-build-env-uzzybwax/overlay/lib/python3.13/site-packages/numpy/_core/include/numpy/ndarrayobject.h:12:
      In file included from /tmp/pip-build-env-uzzybwax/overlay/lib/python3.13/site-packages/numpy/_core/include/numpy/ndarraytypes.h:1913:
      /tmp/pip-build-env-uzzybwax/overlay/lib/python3.13/site-packages/numpy/_core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with "          "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
         17 | #warning "Using deprecated NumPy API, disable it with " \
            |  ^
      1 warning generated.
      clang++ -isystem /opt-3/sundials-6.7.0-asan/include -fsanitize=address -stdlib++-isystem /opt-2/libcxx19-asan/include/c++/v1 -ferror-limit=5 clang++ -shared -fsanitize=address -Wl,-rpath=/opt-3/cpython-v3.13.1-asan/lib -lsundials_nvecserial -lsundials_cvodes -lsundials_sunlinsolspgmr -lsundials_sunlinsolspbcgs -lsundials_sunlinsolsptfqmr -lsundials_sunmatrixdense -lsundials_sunmatrixband -lsundials_sunlinsollapackdense -lsundials_sunlinsollapackband -lsundials_sunlinsolklu -Wl,--disable-new-dtags -Wl,-rpath,/opt-3/sundials-6.7.0-asan/lib -L/opt-3/sundials-6.7.0-asan/lib -lopenblas -fsanitize=address -Wl,-rpath,/opt-2/libcxx19-asan/lib -L/opt-2/libcxx19-asan/lib -lc++ -lc++abi -stdlib=libc++ -isystem /opt-3/sundials-6.7.0-asan/include -fsanitize=address -stdlib++-isystem /opt-2/libcxx19-asan/include/c++/v1 -ferror-limit=5 build/temp.linux-x86_64-cpython-313-pydebug/pycvodes/_cvodes.o -lsundials_nvecserial -lsundials_cvodes -lsundials_sunlinsolspgmr -lsundials_sunlinsolspbcgs -lsundials_sunlinsolsptfqmr -lsundials_sunmatrixdense -lsundials_sunmatrixband -lsundials_sunlinsollapackdense -lsundials_sunlinsollapackband -lsundials_sunlinsolklu -o build/lib.linux-x86_64-cpython-313-pydebug/pycvodes/_cvodes.cpython-313d-x86_64-linux-gnu.so
      clang++: error: no such file or directory: 'clang++'
      [end of output]

If I pin "setuptools==72.1.0" in pyproject.toml, the extension compiles and links just fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants