Apaka allows you to create Debian packages for a given autoproj based workspace. A description of the general architecture is part for the following publication:
Clone the repository into an existing autoproj installation
git clone https://github.com/rock-core/tools-apaka tools/apaka
Add "- tools/apaka" to your autoproj/manifest under the layout section
call
aup apaka
amake apaka
Start a new shell and source the env.sh.
The command line tool apaka provides a way to generate Debian packages from autoproj information
for use with the rock_osdeps package set.
When creating a new release, the reprepro repository and build environment need to be prepared first, i.e. required dependencies will be installed, including among others pbuilder, cowdancer, and apache2. A new site will be added to your apache2 configuration under /etc/apache/sites-enabled/100_apaka-reprepro.conf
apaka prepare
Creating a new Rock release requires some adaption to existing packages, so that an overlay can be applied, here using the
optional --patch-dir. The current recipes for Rock can be found in apaka-rock_patches.
To build all packages that are bootstrapped with a currently active autoproj manifest:
apaka build --patch-dir tools/apaka-rock_patches --architecture amd64 --distribution bionic --release-name master-20.06
To build only a particular package or package_set add it as parameter, e.g., here for base/cmake:
apaka build --patch-dir tools/apaka-rock_patches --architecture amd64 --distribution bionic --release-name master-20.06 base/cmake
The package repository can be browsed under:
http://<hostname>/apaka-releases
As a final step, the yaml descriptions known as *.osdeps file can be generated and integrated into a packages set such as rock-osdeps
apaka osdeps --release-name master-20.06 <rock_osdeps yaml dir>
-
Deregistration of a package in reprepro
apaka reprepro --deregister --distribution bionic --release-name master-20.06 *orocos.rb* -
Registration of a debian package using the dsc file, which needs to be in the same folder as the deb and orig.tar.gz
apaka reprepro --register --distribution bionic --release-name master-20.06 build/rock-packager/rock-master-20.06-ruby-tools-orocos.rb/xenial-amd64/rock-master-20.06-ruby-tools-orocos.rb_0.1.0-1~xenial.dsc -
Preparing for building
apaka prepare --architecture amd64 --distribution bionic --release-name master-20.06 -
Building a package
apaka build --architecture amd64 --distribution bionic --release-name master-20.06 control/vispThis includes creating the .dsc file and orig.tar.gz using deb_package, building using cowbuilder and registering the package in reprepro.
-
Generating .dsc source package description and orig.tar.gz
apaka package --architectures amd64 --distributions bionic --release-name master-20.06 tools/service_discovery -
Generate osdeps description file for use in rock_osdeps package set
apaka osdeps --release-name master-20.06 --dest-dir /tmp/osdep-files
If autoproj meta packages should be represented as debian meta packages, you can use:
apaka build_meta --release-name master-20.06 --distribution bionic --architecture amd64
This will only create meta packages of packages it finds currently on reprepro for the corresponding release, distribution, architecture combination.
Either start with a fresh bootstrap:
sudo apt install ruby ruby-dev wget
wget http://www.rock-robotics.org/autoproj...
ruby autoproj_bootstrap
If you want to use an already defined build configuration then replace the last step with something like:
ruby autoproj_bootstrap git git://github.com/yourownproject/yours...
or remove the install folder in order to get rid of old packages.
If a release has been created with default settings all its Debian Packages install their files into /opt/rock/release-name and now to activate debian packages for your autoproj workspace:
adapt the autoproj/manifest to contain only the packages in the layout that you require as source packages. However, the layout section should not be empty, e.g. to bootstrap all precompiled packages of the rock-core package set add:
layout:
- rock.core
After the package_sets that you would require for a normal bootstrap, you require a package set that contains the overrides for your release. You can find an example at https://github.com/rock-core/rock-osdeps-package_set The package set has to contain the required osdeps definition and setup of environment variables: Hence, a minimal package set could look like the following:
package_sets:
- github: rock-core/package_set
- github: rock-core/rock-osdeps-package_set
layout:
- rock.core
After adding the package set use autoproj as usual:
source env.sh
autoproj update
Follow the questions for configuration and select your prepared release for the Debian packages. Finally start a new shell, reload the env.sh and call amake. This should finally install all required Debian packages and remaining required packages, which might have not been packaged.
- packages have separate installation folder and each provide and env.sh to setup the environment, but note that this env.sh does not include the settings for a packages dependencies
- multiple autoproj workspace can reuse the existing set of Rock Debian packages
- multiple releases of the Rock Debian packages can be installed in parallel, the target folder is typically /opt/rock/release-name
- in order to enforce the usage of a source package in a workspace create a file autoproj/deb_blacklist.yml containing the name of the particular package. This will disable automatically the use of this Debian package and all that depend on that package, e.g., to disable base/types add all packages that start with simulation/ create a deb_blacklist.yml with the following content:
---
- base/types
- simulation/.*
- ^orogen$
You will be informed about the disabled packages:
Triggered regeneration of rock-osdeps.osdeps: /opt/workspace/rock_autoproj_v2/.autoproj/remotes git_git_github.com_rock_core_rock_osdeps_git/lib/../rock-osdeps.osdeps, blacklisted packages: ["base/types"] Disabling osdeps: ["base/types", "tools/service_discovery", "tools/pocolog_cpp", ...
In order to add a new distributions a few things have to be done. Firstly, the default configuration file should be extended with the particular distribution, i.e., add the desired distribution here. Call 'deb_local --show-current-os' to retrieve the corrensponding labels for your current operating system. Examples:
distributions:
bionic:
type: ubuntu,debian
labels: 18.04,bionic,beaver,default
stretch:
type: debian
labels: 9.4,stretch,default
ruby_version: ruby23
Adapt the template for /etc/pbuilderrc i.e., lib/apaka/packaging/templates/etc-pbuilderrc In order to bootstrap new images, pbuilder has to be informed, whether an distribution label such as 'bionic' or 'stretch' has to be interpreted as Ubuntu or Debian distribution (currently apaka consider only these two). New distributions should be added to the list in the above mentioned template of /etc/pbuilderrc. This update will not be automatically applied to existing installations, hence, you will have change existing installations of apaka manually. Furthermore, reprepro only accounts for the new distribution, if you create a new release. To add it to existing releases you have to add the distribution (architecture) manually to /var/www/apaka-releases/rock-release/conf/distributions.
-
If you get a message like
error loading "/opt/rock/master-18.01/lib/ruby/vendor_ruby/hoe/yard.rb": yard is not part of the bundle. Add it to Gemfile.. skipping...Then add the following to install/gems/Gemfile (in the corresponding autoproj installation)
gem 'yard'
You can call
rake doc
to autogenerate the documentation, which can then be found in a doc/ subfolder.
To run the test suite you can either call
rake test
or
export BUNDLE_GEMFILE=$PWD/test/workspace
ruby -Ilib test/test_packaging.rb
To run only individual tests, use the -n option, e.g., for the test_canonize
export BUNDLE_GEMFILE=$PWD/test/workspace
ruby -Ilib test/test_packaging.rb -n test_canonize
Generate key pair gpg2 (use RSA and RSA)
gpg2 --full-gen-key
List proper key-ID-format of the generated key pair. For the next step use the 16 character long key ID that is shown in the line starting with pub after rsa4096/.
gpg2 --list-key --keyid-format long
Edit /var/www/akala-releases/releaseName/config/distributions and add the following line to the block matching the distribution that is to be signed (e.g. bionic).
SignWith: insert_short_pub_key_ID
Rebuild one new package that has not been build before, in order to have deb_local sign the entire release (of this distribution). If all packages have already been build, remove a small package from the release with reprepro and rebuild it. E.g. search for base/cmake and remove matching packages on the bionic distribution.
reprepro -b . listmatched bionic '*base-cmake*'
reprepro -b . removematched bionic '*base-cmake*'
Rebuild base/cmake with deb_local or using apaka-make-deb with --package base/cmake option.
Export public key to file. Replace releaseName and bionic as needed.
gpg2 --armor --output /var/www/apaka-releases/_releaseName_/dists/_bionic_/Release_pub --export insert_short_pub_key_ID
Get the public key file Release_pub and make it known to apt and update.
sudo apt-key add Release_pub && apt-get update
Create an apt entry for your-release. Open /etc/apt/sources.list.d/your-release.list (sudo required) and add deb URL distribution:
deb http://<package-server>/apaka-releases/your-release bionic main
To see how to create an advanced setup particularly with autoproj (here for the Rock framework), have a look at the rock-osdeps-package_set.
The main interface is the 'apaka' binary which can be used with a number of modes:
$>apaka --help [23:11:47]
Commands:
apaka build [PackageName] # Build a (debian) package from a given autoproj package, or a gem
apaka build_meta [PackageName] # Build a (debian) meta package
apaka config # Show the current configuration
apaka help [COMMAND] # Describe available commands or one specific command
apaka osdeps # Generate osdeps files for a package release
apaka package [PackageName] # Prepare the artifact to build a (debian) package from a given autoproj package
apaka package_meta [PackageName] # Create artifacts required to build a (debian) meta package
apaka query [Package] # Query the current database
apaka reprepro [Package] # Manipulate the reprepro instance for a particular release
Options:
[--verbose], [--no-verbose] # turns verbose output
[--debug], [--no-debug] # turns debug output on of off
[--config-file=CONFIG_FILE] # Configuration file to use
[--release-name=RELEASE_NAME] # Release name to use
# Default: release-20.06
The help for each mode can be called by:
$>apaka help package
Usage:
apaka package [PackageName]
Options:
[--package-set-dir=PACKAGE_SET_DIR] # Directory with the binary-package set to update
[--version=VERSION] # Version of the package to create for
[--architecture=ARCHITECTURE] # Architecture to build for
[--distribution=DISTRIBUTION] # Distribution to build for
[--build-dir=BUILD_DIR] # Build folder of the source package -- needs to be within
[--dest-dir=DEST_DIR] # Destination folder of the source package
[--patch-dir=PATCH_DIR] # Overlay directory to patch existing packages
[--pkg-set-dir=PKG_SET_DIR] # Package set directory
[--rebuild], [--no-rebuild] # Force rebuilding / repackaging
[--no-deps], [--no-no-deps] # Do not package dependencies
[--ancestor-blacklist=one two three] # Packages added to the ancestor blacklist, i.e., if needed as dependency, use a package from the current release name
[--verbose], [--no-verbose] # turns verbose output
[--debug], [--no-debug] # turns debug output on of off
[--config-file=CONFIG_FILE] # Configuration file to use
[--release-name=RELEASE_NAME] # Release name to use
# Default: release-20.06
Prepare the artifact to build a (debian) package from a given autoproj package
A typical workflow to build all packages for drivers/orogen/iodrivers_base will look as follows:
$>apaka build --patch-dir tools/apaka-rock_patches --config-file apaka-master-20.06.yml --release-name master-20.06 drivers/orogen/iodrivers_base
If you want to create a meta package for your release which will be named
rock-master-20.06-meta-full, then you can use the following call:
$>apaka build_meta --config-file apaka-master-20.06.yml --release-name master-20.06
Please refer to the following publication when citing Apaka:
Binary software packaging for the Robot Construction Kit
Thomas M. Roehr, Pierre Willenbrock
In Proceedings of the 14th International Symposium on Artificial Intelligence, (iSAIRAS-2018), 04.6.-06.6.2018, Madrid, ESA, Jun/2018.
Github will be used for pull requests and issue tracking: https://github.com/rock-core/tools-apaka
This software is distributed under the New/3-clause BSD license
Copyright (c) 2014-2021, DFKI GmbH Robotics Innovation Center