Fails to install FindPythonLibNew.cmake when specifying DESTDIR

[simonschmeisser]

I'm trying to build pybind11_catkin master using catkin tools on ROS melodic/ubuntu bionic. Is that expected to fail? The workspace which get's turned into a huge deb contains MoveIt Noetic and similar so I thought it might be right to use master instead of melodic-devel?

Error is:

Errors << pybind11_catkin:install /home/jenkins/workspace/mikado_mikado_ros_version-2.1/Entwicklung/../catkinworkspace-tmp/log/pybind11_catkin/build.install.000.log
CMake Error at cmake_install.cmake:155 (file):
  file INSTALL cannot find
  "/home/jenkins/workspace/mikado_mikado_ros_version-2.1/catkinworkspace-tmp/devel/.private/pybind11_catkin/share/pybind11_catkin/cmake/FindPythonLibsNew.cmake".


make[2]: *** [install] Error 1

earlier in the logs I found

Linked: (/home/jenkins/workspace/mikado_mikado_ros_version-2.1/Entwicklung/../catkinworkspace-test-tmp/devel/.private/pybind11_catkin/share/pybind11_catkin/cmake/FindPythonLibsNew.cmake, /home/jenkins/workspace/mikado_mikado_ros_version-2.1/Entwicklung/../catkinworkspace-test-tmp/devel/share/pybind11_catkin/cmake/FindPythonLibsNew.cmake)

which is in a different workspace that we use for running the unit tests

[wxmerkt]

Yes, you are correct in using master. Indeed the melodic-devel branch is not meant for usage - it was a test to see if we could use the system-packaged pybind11-dev and just overlay some CFG_EXTRAS. However for us the latest pybind11 was crucial to get memory sharing between Eigen and Numpy and other features. I'll go ahead and delete it now to avoid confusion.

I'd expect the installation to still work on 18.04. It's tested on CI and passes the buildfarm, so I don't think we should have issues there. Since you are planning to package, I reckon you are using an install space? Are you sourcing the devel or install space while building? Is it from clean on Jenkins every time?

This Dockerfile builds fine and I just ran it for --install and --no-install:

FROM ros:melodic-ros-core
RUN apt-get update && apt-get install -y python-catkin-tools python-rosdep git g++ && sudo rosdep init && rosdep update
RUN cd && mkdir -p catkin_ws/src && cd catkin_ws && catkin config --extend /opt/ros/melodic && catkin config --install
RUN cd ~/catkin_ws/src && git clone https://github.com/ipab-slmc/pybind11_catkin.git
RUN cd ~/catkin_ws/src/pybind11_catkin && rosdep install --from-paths ./ -iry
RUN cd ~/catkin_ws && catkin build -s

Is there an easy way to reproduce the failure you're seeing?

[simonschmeisser]

ok, found the files:

Entwicklung/debian/mikado-ros  /home/jenkins/workspace/mikado_mikado_ros_version-2.1/catkinworkspace-tmp/build/pybind11_catkin /install/share/cmake/pybind11/

(spaces added for clarity) So the first part is the install prefix which I set for having stuff end up in the debian package
second part is the BINARY_DIR
third part is some install space

this means the

COMMAND ${CMAKE_COMMAND} -E copy_directory ${PROJECT_BINARY_DIR}/install/share/cmake/pybind11/ ${CATKIN_DEVEL_PREFIX}/share/${PROJECT_NAME}/cmake

doesn't know about the DESTDIR but it also means that my resulting Debian package would contain a folder /home/jenkins/...
(or I would need to exclude it, which is fine actually)

here's the relevant part of debian/rules:

catkin config --build-space ../catkinworkspace-tmp/build
catkin config --devel-space ../catkinworkspace-tmp/devel
catkin config --log-space ../catkinworkspace-tmp/log
catkin config --install-space /opt/isys-vision/mikado --install
catkin config --source-space .
DESTDIR=$(shell pwd)/debian/mikado-ros/ catkin build --no-status --summary --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCATKIN_ENABLE_TESTING="OFF" -DCMAKE_CXX_COMPILER_LAUNCHER=ccache

[simonschmeisser]

With the workaround in the PR It successfully build without a DESTDIR/install space set and also finished with those set. Now unfortunately I get a new error:

Errors << moveit_core:make /home/jenkins/workspace/mikado_mikado_ros_version-2.1/Entwicklung/../catkinworkspace-tmp/log/moveit_core/build.make.000.log
In file included from /home/jenkins/workspace/mikado_mikado_ros_version-2.1/Entwicklung/extern/moveit/moveit_core/python/tools/src/pybind_rosmsg_typecasters.cpp:37:0:
/home/jenkins/workspace/mikado_mikado_ros_version-2.1/Entwicklung/extern/moveit/moveit_core/python/tools/include/moveit/python/pybind_rosmsg_typecasters.h:39:10: fatal error: pybind11/pybind11.h: No such file or directory
 #include <pybind11/pybind11.h>
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[4]: *** [python/tools/CMakeFiles/moveit_python_tools.dir/src/pybind_rosmsg_typecasters.cpp.o] Error 1
make[3]: *** [python/tools/CMakeFiles/moveit_python_tools.dir/all] Error 2

There is no pybind11 folder in the install space and the pybind11_catkin one is empty

The files actually ended up here:
Entwicklung/debian/mikado-ros/home/jenkins/workspace/mikado_mikado_ros_version-2.1/catkinworkspace-tmp/devel/.private/pybind11_catkin/include/pybind11_catkin/pybind11/

[wxmerkt]

Hi @simonschmeisser
Thank you for the pointers above. I have been able to replicate the issue as you've indicated. Using DESTDIR different from the install space is a use case that we don't often have - we primarily use default devel or install spaces and make sure it works on the ROS buildfarm in the way this package is released there. Is the workaround to re-use the binary package from the ROS buildfarm sufficient for you? Looking into it more closely will require cycles I don't have right now so won't be able to offer an ETA for a fix.