add meson

[christianrauch]

Please add the following dependency to the rosdep database.

Package name:

meson

Purpose of using this:

"Meson is an open source build system meant to be both extremely fast, and, even more importantly, as user friendly as possible."

[christianrauch]

I replaced the distribution packages with pip since this 1) will make it available on all platforms and 2) provides always the latest version.

[mabelzhang]

I think that's not the right format. You'll still have to list the name of the platforms, specifically, ubuntu, debian, fedora, and opensuse are encouraged. Even if that's valid syntax, we definitely cannot use a wildcard for all distros since we don't want pip for Gentoo. Please see other examples with pip packages in the file for reference.

[christianrauch]

I think that's not the right format. You'll still have to list the name of the platforms, specifically, ubuntu, debian, fedora, and opensuse are encouraged.

This syntax was suggested for rosdep/base.yaml in #31273 (comment). I assume it is working in this yaml file too?

Even if that's valid syntax, we definitely cannot use a wildcard for all distros since we don't want pip for Gentoo. Please see other examples with pip packages in the file for reference.

Is pip not supposed to work on Gentoo? In any case, I set this to null. See the above example too for this syntax.

[cottsay]

This syntax was suggested for rosdep/base.yaml in #31273 (comment). I assume it is working in this yaml file too?

The wildcards currently only work for releases within an OS. I only just recently updated rosdep to support OS-level wildcards (specifically to make pip easier), but that enhancement hasn't been released yet. Best thing to do for now is list a handful of distros specifically as is done in other examples in this file.

Even if that's valid syntax, we definitely cannot use a wildcard for all distros since we don't want pip for Gentoo.

This is actually handled by rosdep itself, see ros-infrastructure/rep#329 (comment)

[christianrauch]

The wildcards currently only work for releases within an OS.

Right. The comment I mentioned was only for distribution names. I set some distributions manually now.

This is actually handled by rosdep itself, see ros-infrastructure/rep#329 (comment)

Cool. I want that feature :-)

[christianrauch]

Feel free to squash all my commits into one. I just wanted to preserve the history through the PR discussions.

[nuclearsandwich]

I replaced the distribution packages with pip since this 1) will make it available on all platforms and 2) provides always the latest version.

In general, if packages are available for a target platform, then the platform package is strongly preferred to pip because it allows the package to be released via the official buildfarm. pip dependencies cannot be used by binary releases on the build farm. Since this package is for a build system that is itself not supported on the build farm, that reasoning lacks some weight.
However, I think it still applies especially if there's community interest in eventual support for meson and ament_meson build types on the build farm, that will only be possible with the platform-packaged distribution of meson.

Lastly, pip-only packages in the official rosdep database use a different package naming convention and the key name should then be python3-meson-pip.

[christianrauch]

In general, if packages are available for a target platform, then the platform package is strongly preferred to pip because it allows the package to be released via the official buildfarm. pip dependencies cannot be used by binary releases on the build farm. Since this package is for a build system that is itself not supported on the build farm, that reasoning lacks some weight.

This generally makes sense, since the bloomed packages cannot depend on pip.

In this case, meson is just a build dependency. The build farm can check out that pip package to build the ROS package. I am using this in a "cmake wrapper" that checks out a git repo and runs meson on it. There is no runtime dependency that has to be satisfied when the bloomed package

The meson project is moving very fast and Ubuntu LTS is always stuck with 2 year old packages. After 1 year into an LTS, I rarely encounter packages that will build with the Ubuntu supplied version, so I always have to resort to the pip version.

Lastly, pip-only packages in the official rosdep database use a different package naming convention and the key name should then be python3-meson-pip.

In the python.yaml it looks as if python libraries are prefixed with python- but regular applications that are installed via pip are just suffixed with -pip. So in this case it should be meson-pip correct?

[nuclearsandwich]

In this case, meson is just a build dependency. The build farm can check out that pip package to build the ROS package. I am using this in a "cmake wrapper" that checks out a git repo and runs meson on it. There is no runtime dependency that has to be satisfied when the bloomed package

The build farm builds do not directly use rosdep. All binary package dependencies must be satisfiable with system dependencies. Bloom does the work of using rosdep to generate valid platform build and runtime dependency and the build farm jobs install only the packages resolved by bloom (and their dependencies) in the containers which perform these package builds.

I think my advice on the matter depends heavily on whether the goal is to get packages using meson onto the official ROS build farm or whether they are just meant for personal use. In my strong opinion packages on the ROS build farm need to be reproducible solely with packages from the platform and ROS package repositories. Packages which rely on third party repositories should be subject to additional scrutiny.

In the python.yaml it looks as if python libraries are prefixed with python- but regular applications that are installed via pip are just suffixed with -pip. So in this case it should be meson-pip correct?

My understanding of the current conventions is that the python3- prefix is unconditional on new pip keys but I'll seek second opinions from @tfoote and @cottsay.

[christianrauch]

The build farm builds do not directly use rosdep.

Is this a technical limitation or is this just not implemented yet? If the pip dependencies are not used at runtime, then building with pip packages should be fine.

I think my advice on the matter depends heavily on whether the goal is to get packages using meson onto the official ROS build farm or whether they are just meant for personal use.

For my immediate need, I want to have packages build in a CI using rosdep for dependency resolution. In the longer term, I also wanted to release the packages via bloom. If the latter is not possible via pip build dependencies, then I will only be able to release for platforms that have a recent enough version of meson packaged. Alternatively, I have to implement an ugly hack that includes all python build dependencies in the package itself, or even more horrifying -- package meson as a ros/bloom package :-)

[nuclearsandwich]

Is this a technical limitation or is this just not implemented yet? If the pip dependencies are not used at runtime, then building with pip packages should be fine.

It's an explicit objective not to use pip at either build or runtime for the packaged ROS binaries. One of the prime benefits of building our packaging systems on top of the platform's packaging system is our ability to rely on the integration and stability work done by that platform. This comes with the trade-off of not always having access to the latest releases of packages.

Vendoring or embedding dependencies is something that is occasionally done out of necessity but even in that case packages which are available via the base platform, even if in an older form lacking some features, must always be used except in extraordinary circumstances.

[christianrauch]

@nuclearsandwich @tfoote @cottsay What is the status of this? Can we go on with it?

[cottsay]

Two things:

1. To be clear, this key will not be usable on the buildfarm. Using rosdep to install dependencies locally will work fine, but Bloom will give you an error when it tries to resolve all of the dependencies to system packages (deb/rpm). This is by design, not a bug.
2. This rule should follow the documented conventions, and as a pip-only package, should be named python3-meson-pip. This is particularly important to distinguish from a possible future key which represents the system's meson package.

[christianrauch]

I renamed it to python3-meson-pip.

I have no idea how to approach the pip vs. packages problem. The packaged meson would be a separate key, but that means that ROS packages have to decide which version to choose and if they want to support the build server.

IMHO the build server should support pip build dependencies. Since meson is not a runtime dependency, it should not be added as a dependency to the bloomed package.

[christianrauch]

To make this complete, I am also adding the meson key for the distribution packages.

[christianrauch]

@cottsay @nuclearsandwich @mabelzhang Are you ok with these changes now? The rosdep key names should now follow conventions.

[cottsay]

As a Python package, I think that the rules still belong in python.yaml. Other than that, I think we're good.

[christianrauch]

As a Python package, I think that the rules still belong in python.yaml. Other than that, I think we're good.

Right. That was an oversight. I think in this case it also has to be called python3-meson, which sounds a bit cumbersome. Is this correct that everything that is implemented in python will have to carry the python3-, python4-, ... prefix?

[cottsay]

Is this correct that everything that is implemented in python will have to carry the python3-, python4-, ... prefix?

In general, rosdep keys refer to a "resource" and not necessarily a package. In this case, I believe that the "resource" backed by the key is the meson executable, which may be provided for any interpreter. So IMO, simply calling the key meson to match the Ubuntu package which provides /usr/bin/meson seems appropriate.

That said, python3-meson seems fine too. Thoughts from other @ros/rosdeputies?

[sloretz]

That was an oversight. I think in this case it also has to be called python3-meson, which sounds a bit cumbersome. Is this correct that everything that is implemented in python will have to carry the python3-, python4-, ... prefix?

I swear there was a rule about the preferred rosdep key name being the Debian package name, (meson in this case), but I cannot find that rule written down anywhere. I would be happy to approve with this key named meson.

While it's implemented in Python, if it's not intended to be imported by another Python program then it seems like it could be in base.yaml. That said, I don't know why python.yaml is separated in the first place.

[christianrauch]

Ok, I renamed the key to meson but kept it in the python.yaml because it's implemented in python.

[nuclearsandwich]

Thanks for iterating with us @christianrauch.