Add catkin package(s) to provide the default version of Gazebo

[meyerj]

As a follow-up of the discussion in tu-darmstadt-ros-pkg/hector_gazebo#30 (@mikepurvis, @tfoote, @meyerj), I created a proof-of-concept catkin package gazebo_dev.

The new package contains no code and its main purpose is to depend on the respective ROS distro's default version of the Gazebo development library package. Other packages that want to link to the Gazebo libraries can in turn depend on gazebo_dev, removing the need to maintain separate branches for different ROS and Gazebo versions as long as the source code is compatible. The current practice of depending on gazebo_ros pulls in some unnecessary dependencies (see original discussion).

This patch is for ROS indigo and Gazebo 2 and there is no separate libgazebo2-dev package/key, but it is quite straight-forward to adapt gazebo_dev/package.xml for newer releases by replacing the <build_export_depend> line with the appropriate libgazeboX-dev rosdep key (jade, kinetic).

Secondary, the package provides a cmake script exposed via catkin's CFG_EXTRAS mechanism so that the gazebo_dev package can be used in the same way as any other catkin package without the need to find the Gazebo libraries or apply C++ flags explicitly. I modified the other packages in this repository to only depend on gazebo_dev and removed the obsolete cmake code. It should be noted that all packages that directly depend on gazebo_dev will compile in C++11 mode because of the addition of GAZEBO_CXX_FLAGS to CMAKE_CXX_FLAGS. That's why this dependency has not been exported via the CATKIN_DEPENDS key of catkin_package() in the gazebo_plugins package. Derived packages have to depend on gazebo_dev explicitly.

Possible extensions would be a check in gazebo_dev-extras.cmake whether the found Gazebo version is indeed the expected one or even to provide a way to override the required version by means of cmake variables.

The gazebo_dev package does not declare a run-time dependency for the same Gazebo version at the moment. I am not sure if this is required. A rosdep key gazebo_default could be established that points to the correct gazebo binary package, so that derived packages like gazebo_plugins can build_depend on gazebo_dev and exec_depend on gazebo_default. Or is there a need for a separate run-time only package with that name, which would provide the per-distro gazebo_default key? Like

gazebo <----------\
libgazebo5-dev <---+--(build_export_depend)-- gazebo_dev (ROS) <------\
libgazebo7-dev <--/                                                 (build_depend)
                                                                        +---------- gazebo_plugins (ROS)
gazebo <---\                                                        (exec_depend)
gazebo5 <---+----------(exec_depend)--------- gazebo_default (ROS) <--/
gazebo7 <--/

This patch also converts all package manifest to format 2 according to REP 140 to be able to use the new <build_export_depend> tag. Furthermore it removes the unnecessary dependency message_generation from package gazebo_ros.

I also created a branch indigo-gazebo-dev in hector_gazebo (diff) which applies a patch to use the new gazebo_dev package and which compiles successfully in ROS indigo, jade and kinetic with the respective default Gazebo versions.

[meyerj]

This line had no effect anyway. catkin_package(DEPENDS ...) is case-sensitive, but gazebo-config.cmake only sets the capitalized GAZEBO_INCLUDE_DIRS and GAZEBO_LIBRARIES variables (@PKG_NAME@ is GAZEBO).

Basically catkin adds ${depend_name}_INCLUDE_DIRS and ${depend_name}_LIBRARIES for each dependency to the exported include directories and libraries for this package (source code in catkin_package.cmake:163). There will be no error or warning message because gazebo_FOUND is set by cmake before. Maybe this is worth a patch in catkin, like adding a check that at least one of the two variables is non-empty or making DEPENDS case insensitive by checking the lower- and upper-case variants, too (@dirk-thomas)? Or at least improve the documentation and mention the case dependency explicitly?

For gazebo_plugins I think that not exporting the Gazebo dependency at all is the better option because simply adding include directories and libraries would not be sufficient anyway.

[dirk-thomas]

Since all CMake variables are case sensitive I don't think it would be good to make anything in catkin case insensitive.

catkin is supposed to already check the arguments and ensure the passed name as been found before. I tried this specific example and the condition checking this was incorrect. ros/catkin#813 addresses this and with it applied the above CMake will result in the intendend error message that gazebo was not found (because gazebo_FOUND is not set).

[mikepurvis]

Looks fantastic. I'm not over the moon about the name. I don't know if this is possible, but I pictured that a single package named gazebo_default would behave like so:

• <build_depend>gazebo_default</build_depend> would pull in libgazeboX-dev package and make the headers available to catkin, while
• <exec_depend>gazebo_default</exec_depend> would pull in only the runtime.

That would make the most compelling case for users, if the majority of Gazebo-using ROS packages could simply <depend>gazebo_default</depend> and get the right thing, which is headers at build time, application at runtime.

[meyerj]

Unfortunately having a single wrapper package only for build- and run-time dependencies is not sufficient, at least not for releases. The catkin gazebo_dev package would be released as ros-distro-gazebo-dev and depends on the respective libgazeboX-dev package and the catkin package gazebo_default (if we would add it as such) would be released as ros-distro-gazebo-default and only depends on the binary Gazebo package.

A derived package like gazebo_plugins would only depend on ros-distro-gazebo-default (or directly on gazeboX) because gazebo_dev it not a run-time dependency, but rosdep would install the development version because it checks the build dependencies.

I have not yet verified if my assumptions are true and whether bloom correctly resolves these dependencies but I do not think that there is a way to achieve the same binary package hierarchy without a separate gazebo_dev catkin package.

However, we could rename the gazebo_dev package to gazebo_default_dev, if this is preferred.

[scpeters]

The travis and ROS builds failed during rosdep install; I assume this PR will need a rosdistro pull request as well?

[scpeters]

I really like this idea.

On a related note that I expect is beyond the scope of this PR, we have also been dealing with the issue of supporting extra gazebo versions through debian packages hosted on http://packages.osrfoundation.org (ie. ros-indigo-gazebo7-ros-pkgs and this tutorial). One way is to remap the rosdep keys (for example). I guess I don't have any specific questions about this at the moment; just mentioning it because it seems relevant.

cc: @j-rivero

[meyerj]

I am aware of the alternative gazebo packages in the http://packages.osrffoundation.org repository and we are using them for Gazebo 6 in ROS indigo at Intermodalics with a custom rosdep file. Works quite well. I do not think that this PR would break these alternative packages if the gazebo_dev package is also released as ros-<distro>-gazebo<version>-dev with the correct dependencies. Which build system generates these binary packages and from which release repository or have they been released and built manually?

By the way, the latest ros-indigo-gazebo6-ros-control package released at Aug 15 2015 is broken, either because of ABI-incompatible changes in Gazebo 6 or in ros_control since then. I don't remember the details, but we would appreciate a re-release of gazebo_ros_pkgs for version 6 in indigo.

[meyerj]

The travis and ROS builds failed during rosdep install; I assume this PR will need a rosdistro pull request as well?

Travis calls rosdep install without the --ignore-src flag and gazebo_dev cannot be found in the rosdistro index (yet). Adding one of both should hopefully fix the Travis check. I always wondered why rosdep is not ignoring packages found in one of the source folders by default (not necessarily for other packages in the ROS_PACKAGE_PATH).

The job at http://build.ros.org/ (Ipr__gazebo_ros_pkgs__ubuntu_trusty_amd64/22) does not recognize the transitive build dependency gazebo exported by the gazebo_dev package. Either I misinterpreted the section about the <build_export_depend> tag in REP-140 or this is a bug in the ROS build infrastructure. However, adding gazebo (or libgazeboX-dev) as a build dependency to gazebo_dev would probably also fix this.

The jobs at http://build.osrfoundation.org/ seem to always install a certain version of Gazebo independent of the declared package.xml dependencies and rosdep.

[j-rivero]

Thanks Johannes (et all) for the idea and the PR. I think that is a good improvement over the current setup.

The gazebo_dev package does not declare a run-time dependency for the same Gazebo version at the moment. I am not sure if this is required.

Maybe not mandatory but since we have the distinction between build/run-time I think that an user should expect to get the gazebo application as a run-time dependency of gazebo_dev.

A rosdep key gazebo_default could be established that points to the correct gazebo binary package, so that derived packages like gazebo_plugins can build_depend on gazebo_dev and exec_depend on gazebo_default.

AFAIK, we already have two different rosdep keys that map the libgazeboX-dev and gazebo.

Or is there a need for a separate run-time only package with that name, which would provide the per-distro gazebo_default key

The gazebo rosdep key seems to be unique for the different gazeboX (at least for gazebo, gazebo2 and gazebo7). What would be the benefit of this package?

I do not think that this PR would break these alternative packages if the gazebo_dev package is also released as ros--gazebo-dev with the correct dependencies. Which build system generates these binary packages and from which release repository or have they been released and built manually?

We are building them in the osrfoundation.org build farm (managed by Gazebo team). The process is mainly to overwrite rosdep rules (and some magic to rename packages) and run bloom on top, publishing the results in an alternative -release repos (i.e.gazebo6_ros_pkgs-release). And generate the packages using that metadata. I can not see a problem with the introduction of this new gazebo-dev package.

The job at http://build.ros.org/ (Ipr__gazebo_ros_pkgs__ubuntu_trusty_amd64/22) does not recognize the transitive build dependency gazebo exported by the gazebo_dev package. Either I misinterpreted the section about the <build_export_depend> tag in REP-140 or this is a bug in the ROS build infrastructure

I understand the same. @dirk-thomas any idea about what we are doing wrong?

[dirk-thomas]

@meyerj That looks indeed like a bug in the build farm. While for a single package it makes sense to:

• install the build dependencies
• build (without tests)
• install the run and test dependencies
• build with tests and run the tests
  that is insufficient for your case with multiple packages.

While gazebo is only a run dependency of gazebo_dev the problem is that gazebo_dev is a build dependency for other packages in the repo and therefore the workspace. Hence gazebo must be installed as part of the build dependencies (instead of later as part of the run dependencies).

I will look into a fix for the build farm tomorrow so you shouldn't need to change the dependency type.

[dirk-thomas]

I have temporarily modified the PR job to use the branch of the ros_buildfarm repo which contains the referenced fix (http://build.ros.org/job/Ipr__gazebo_ros_pkgs__ubuntu_trusty_amd64/jobConfigHistory/showDiffFiles?timestamp1=2016-07-14_09-54-00&timestamp2=2016-07-14_09-54-01).

I was wondering why the PR failed in the pre-merge step until I noticed that it currently has merge conflicts. @meyerj Can you please fix them? The following PR build should pass then.

[meyerj]

I was wondering why the PR failed in the pre-merge step until I noticed that it currently has merge conflicts. @meyerj Can you please fix them? The following PR build should pass then.

Apparently I did not fetch the newest versions of the indigo-devel, jade-devel and kinetic-devel branches before I started with this. Rebased now and the job seems to build successfully. Thanks for the quick fix, Dirk!

The gazebo_dev package does not declare a run-time dependency for the same Gazebo version at the moment. I am not sure if this is required.
Maybe not mandatory but since we have the distinction between build/run-time I think that an user should expect to get the gazebo application as a run-time dependency of gazebo_dev.

I added the run-time dependency in 06d2dc3 (indigo), 809c3c0 (jade) and 08b1feb (kinetic).

A rosdep key gazebo_default could be established that points to the correct gazebo binary package, so that derived packages like gazebo_plugins can build_depend on gazebo_dev and exec_depend on gazebo_default.

AFAIK, we already have two different rosdep keys that map the libgazeboX-dev and gazebo.

Yes, but in the general rosdep/base.yaml file. What would be needed is a key which resolves to different binary packages depending on the distribution. rosdep would allow this by adding a tag to the sources list file, but with the current 20-default.list file it only looks at the distribution.yaml file for released ROS packages. Not sure, but I think there is no way to achieve something like

$ rosdep resolve gazebo_default --rosdistro indigo
#apt
gazebo2
$ rosdep resolve gazebo_default --rosdistro jade
#apt
gazebo5
$ rosdep resolve gazebo_default --rosdistro kinetic
#apt
gazebo7

without changes to the 20-default.list or even rosdep, the underlying Python packages or the REPs they are based on.

Or is there a need for a separate run-time only package with that name, which would provide the per-distro gazebo_default key

The gazebo rosdep key seems to be unique for the different gazeboX (at least for gazebo, gazebo2 and gazebo7). What would be the benefit of this package?

Currently the rosdep keys in base.yaml can only differentiate between OS versions, but not between ROS distributions. If every OS would only support one Gazebo version, there would be no problem. But because for example trusty can run all kind of different versions and the default version depends on whether ROS indigo or jade is installed (or even both), the gazebo rosdep key cannot serve as a pointer for the default version and there was a need to introduce the gazebo5 key for jade. The key gazebo7 does not even exist and kinetic reuses the gazebo key, based on the assumptions that indigo and kinetic can never run on the same OS.

Releasing a separate package gazebo_default with nothing else than a run-time dependency on the respective ROS distro's default version would partially solve this problem, but unlike the other packages in this repo gazebo_default must not be released separately for different Gazebo versions (e.g. no ros-indigo-gazebo7-default package). Ideally, this package would simply be named gazebo and could be released as ros-indigo-gazebo (depends on gazebo), ros-indigo-gazebo5 (depends on gazebo5), ros-kinetic-gazebo (depends on gazebo7) and so on, but I understood that it is problematic to have ROS package names with the same name as a rosdep key in base.yaml. Perhaps it is an option to remove the gazebo key from that file in favor of having only gazeboX keys, which are not ROS distro specific.

[j-rivero]

Thanks for the explanation Johannes. Looks like the rosdep change would imply a good bunch of modifications out of the scope of this PR, so let's focus on merge the changes are they are now. Looks to me that the changes proposed are fully backward compatible with third party code out there, please correct me if you see any risk. My plan:

• Migrate this changes first to Jade and Kinetic. Release and wait for feedback so we can get any major issue or bugs in these distributions.
• If we don't detect any problem after some time in Jade and Kinetic, merge this PR into the good old Indigo.

Sounds good?

[meyerj]

Sounds good?

+1

I already linked them above, but here are the diff's for jade and kinetic again:
jade-devel...meyerj:jade-gazebo-dev
kinetic-devel...meyerj:kinetic-gazebo-dev

I did not create three separate pull requests, but they should be equivalent. The only differences are the dependencies declared in gazebo_dev/package.xml.

[kev-the-dev]

Looks like this was merged in all releases we would want to touch and can close the issue