CGImap is a C++ implementation of some parts of the OpenStreetMap API as an FastCGI process.
The rails implementation of the "map" call had a few problems with memory - it uses a lot of it and there is a leak which makes it annoying to use in long-running environments, such as the main OSM server.
CGImap attempts to address these memory problems and makes it easier to optimise the queries, something which is also a bit of a pain to do in Rails.
Over time, many more API 0.6 endpoints were added.
Currently, CGImap implements:
- the "map" API call,
- single node, way and relation fetches,
- multiple node, way and relation fetches,
- the "full" way and relation calls and
- changset metadata downloads, including discussions,
- single node, way and relation history calls,
- single node, way and relation specific version fetches,
- multiple node, way and relation specific version fetches,
- relations for element (node, way, relation),
- ways for node,
- create changeset,
- update changeset,
- close changeset,
- changeset downloads,
- changeset uploads.
Note that changeset metadata isn't the same thing as the changeset
download (i.e: osmChange data).
CGImap uses a PostgreSQL server for the APIDB backend. A minimum server version of 9.5 is required.
CGImap depends on the following libraries. Versions used during development are in brackets. Respective library versions for Ubuntu 20.04 and Ubuntu 22.04 are supported at this time. Other versions may work, but YMMV.
On Ubuntu 20.04:
- libxml2-dev (2.9.10+dfsg-5ubuntu0.20.04.3)
- libpqxx-dev (6.4.5-2build1)
- libfcgi-dev (2.4.0-10build1)
- zlib1g-dev (1:1.2.11.dfsg-2ubuntu1.3)
- libboost-dev (1.71.0.0ubuntu2)
- libboost-program-options-dev (1.71.0.0ubuntu2)
- libfmt-dev (6.1.2+ds-2)
- libmemcached-dev (1.0.18-4.2ubuntu2)
- libcrypto++-dev (5.6.4-9build1)
- libargon2-dev (0~20171227-0.2)
- libyajl-dev (2.1.0-3)
If you're running a Debian or Ubuntu system these can be installed using the following command:
sudo apt-get install libxml2-dev libpqxx-dev libfcgi-dev zlib1g-dev \
libboost-dev libboost-program-options-dev libfmt-dev \
libmemcached-dev libcrypto++-dev libargon2-dev libyajl-dev
Note that C++17 is required to build CGImap.
The build system used is GNU Make, using pkg-config to provide some of the flags.
Note that the full set of packages needed from a fresh install (tested with Ubuntu 20.04 and 22.04) - you may already have many or all of these - is:
sudo apt-get install git build-essential automake autoconf libtool
To build the system from scratch, first check out the source code (skip this step if you've already got the source):
git clone https://github.com/zerebubuth/openstreetmap-cgimap.git
Then change to the source code directory to configure and build:
cd openstreetmap-cgimap/
./autogen.sh
./configure
make
You should now have a "./openstreetmap-cgimap" executable in the current directory. For system-wide installation you still need to run:
sudo make install
For historic reasons, CGImap provides a number of shared libraries which were
originally intended for reuse by other applications. As there are no real world
consumers of these libraries, you might want to run the configure
script with the following parameters to create a static build instead:
./configure --enable-static --disable-shared
To run CGImap binary, use the command:
./openstreetmap-cgimap --socket :54321 --backend apidb \
--dbname openstreetmap
Note: CGImap has to be used with a FastCGI enabled server like lighttpd, apache2 etc. See the instructions below to use CGImap with lighttpd or apache2.
A sample lighttd.conf file is provided, which can be used for testing purposes only. To test CGImap with lighttpd you will need to install lighttpd:
sudo apt-get install lighttpd
Edit the supplied lighttpd.config file to include your CGImap path and run it with the lighttpd like
/usr/sbin/lighttpd -f lighttpd.conf
You can then access the running instance at http://localhost:31337/api/0.6/map?bbox=...
The api.osm.org instance runs CGImap as a daemon and Apache with mod_fastcgi_handler.
Typically you will need to modify the database connection parameters and path
to the executable. See ./openstreetmap-cgimap --help for a list of options.
To convert a command line option to an environment variable append CGIMAP_ to
the option and capatalize it. For example, the option --dbname becomes the
environment variable CGIMAP_DBNAME.
Besides environment variables and command line options, CGImap configuration settings can also be provided in an INI-style config file. Use command line parameter --configfile to define the file location.
Example:
dbname=openstreetmap
host=localhost
username=user
password=pass
#update-host=127.0.0.1
#update-dbname=openstreetmap
#ratelimit=100000
# Expert settings (should be left to their default values in most cases)
# see --help for further details
#
#map-area=10
#disable-api-write=
#max-payload=50000000
An init.d script to run CGImap as a daemon is supplied in
scripts/cgimap.init. To use it modify the paths and environment
variables to suit your installation, copy it to /etc/init.d/cgibin and
change the mode to 755, and owner:group to root:root.
For more recent operating systems using systemd instead of init
scripts/cgimap.service is provided to run CGImap as a daemon. To
use it modify the paths, set environment variables or the settings
in the config file, copy it to /etc/systemd/system/cgimap.service.
Change the mode to 755, and owner:group to root:root. To enable the service use
systemctl enable cgimap and to start the service systemctl start cgimap.
An example of this can be found in OSM Chef.
Fcgi programs can be deployed with Apache using mod_fastcgi_handler,
mod_fcgid, mod_fastcgi, and on recent versions mod_proxy_fcgi. A sample
Apache configuration file that will work in conjunction with CGImap as a
daemon is supplied in scripts/cgimap.conf. To use this on a Ubuntu-based
system you need to copy the configuration to where Apache will read it and
create an api directory:
sudo cp scripts/cgimap.conf /etc/apache2/sites-available/cgimap
sudo chmod 644 /etc/apache2/sites-available/cgimap
sudo chown root:root /etc/apache2/sites-available/cgimap
sudo mkdir /var/www/api
sudo a2ensite cgimap
sudo service apache2 restart
The apache modules mod_proxy and mod_fastcgi_handler must also be enabled.
This repository includes a Dockerfile and Dockerfile2204, which can be used to build a cgimap image based on Ubuntu 20.04 / 22.04 respectively:
docker build -f Dockerfile . -t cgimap
Due to the multi-stage build process, Docker version 17.05 or higher is required. The resulting cgimap image has a total size of about 140MB. openstreetmap-cgimap is built as a static binary.
zerebubuth/openstreetmap-cgimap#213 has additional configuration details on how to use the cgimap image in a complete development environment, which includes the Rails port, a Postgresql DB, lighttpd as reverse proxy, and openstreetmap-cgimap.
The read only apidb backend requires permissions to SELECT on the Postgres server. OAuth and update database connections require additional permissions to INSERT/UPDATE/DELETE data, as well as creating temporary tables.
It is recommended that a separate account is created for CGImap to avoid any possibility of data corruption. Care has been taken programming CGImap but, as with most C++ applications, there is the chance of an exploitable flaw leading to complete pwnage.
To run the test suite using make check you will need additional packages installed:
sudo apt-get install postgresql-all postgresql-common postgresql-server-dev-all
Test cases are executed under pg_virtualenv, i.e. no further steps are needed to create databases as your user.
The CGImap code is formatted using
clang-format in a style
which is based on the "LLVM style" which ships with
clang-format. Note that version 3.6 or later is needed to avoid
introducing problems with formatting try-catch blocks. To enable an
automatic reformatting option, provide the --with-clang-format
option to configure and then reformatting can be done across the
whole set of source files by running:
make clang-format
Ideally, this should be done before committing each set of changes.
CGImap contains code from and is partly based on the following:
- modosmapi by d40cht and japplebyalis.
- quad_tile.c by TomH.
- GNU CGICC by Stephen F. Booth and Sebastien Diaz.
- libxml++ by The libxml++ development team