| Authors: | Christian Tanzer
<[email protected]>
Ralf Schlatterbeck <[email protected]> |
|---|
The FFG web app is an application that serves data about the network nodes deployed by Funkfeuer Graz.
It uses the tapyr framework and the common node database (CNDB).
Linux or OS X
- its best to set up a separate account that runs FFG
Apache, with mod-fcgid installed
- or another webserver, e.g. nginx...
PostgreSQL (preferred) or mySQL
- an account for the web app that can create databases and tables
git
Python (> 2.6, < 3)
- virtualenv, distribute
- Depending on the OS (I'm looking at you, Debian), some packages, e.g., werkzeug, should be installed into virtualenv to get versions a bit younger than a couple of years
- This might require the installation of a build environment (for python packages that need the C compiler)
Python packages
- Babel
- BeautifulSoup
- docutils
- flup
- jinja2
- m2crypto
- passlib
- plumbum
- psycopg2
- py-bcrypt
- pyOpenSSL
- pyasn1
- pyquery
- python-dateutil
- pytz
- rcssmin, rjsmin (for minimization of CSS and Javascript files)
- rsclib
- sqlalchemy
- werkzeug
Most packages are available via the Python Package Index
The following is an example installation on Debian Wheezy. It contains some information that is applicable to other distributions but is quite Debian-specific in other parts.
If you are running in a virtual machine, you need at least 384 MB of RAM, 256 MB isn't enough.
Some of the needed Packages are either not in Debian or are too old to be useful. The following packages can be installed via the Debian installer:
$ apt-get install \
apache2-mpm-worker build-essential git libapache2-mod-fcgid \
postgresql python-pip python-babel python-bs4 python-dateutil \
python-dev python-distribute python-docutils python-flup \
python-jinja2 python-m2crypto python-openssl python-passlib \
python-psycopg2 python-pyasn1 python-pyquery python-sqlalchemy \
python-tz python-virtualenv python-werkzeug swig
Other packages can be installed using pip — note that you may want
to install some of these into a virtual python environment (virtualenv),
see later in sectioni How to install — depending on your
estimate how often you want to change external packages:
$ pip install plumbum py-bcrypt rcssmin rjsmin rsclib pyspkac
Create user and database user permitted to create databases:
$ adduser ffg $ createuser -d ffg -P
Assuming an account ffg located in /home/ffg, you'll need something like the following:
### Logged in as `ffg`
$ cd /home/ffg
### Define config
$ vi .ffg.config
### Add the lines (using the appropriate values for **your** install)::
cookie_salt = 'some random value, e.g., the result of uuid.uuid4 ()'
db_name = "ffg"
db_url = "postgresql://<account>:<password>@localhost"
languages = "de", "en"
locale_code = "de"
smtp_server = "localhost"
target_db_url = db_url
time_zone = "Mars/Olympos Mons"
### create a virtual environment for Python
$ mkdir bin
$ mkdir PVE
$ python -m virtualenv --system-site-packages PVE/std
$ (cd PVE ; ln -s std active)
Depending on the packages you have already installed system-wide, you may want to install some packages into the virtual environment if you anticipate that these will change:
### install Python packages into the virtualenv ### if one of these packages is already installed in the system ### Python, you'll need to say `pip install --upgrade`, not `pip install` $ source PVE/active/bin/activate $ pip install plumbum pytz py-bcrypt rcssmin rjsmin rsclib pyspkac
Then we continue with the setup of an active and a passive branch of the web application. With this you can upgrade the passive application while the active application is running without risking a non-functional system should something go wrong during the upgrade:
### create a directory with an `active` and `passive` branch of the
### web application
###
### * the active branch will be the one that serves apache requests
###
### * the passive branch can be used for updating the software and
### testing it. It all works will the branches can be switched
###
$ mkdir fcgi
$ mkdir v
$ mkdir v/1
$ mkdir v/1/www
$ mkdir v/1/www/media
$ ln -s v/1 active
$ ln -s v/2 passive
$ git clone git://github.com/Tapyr/tapyr.git v/1/tapyr
$ git clone git://github.com/CNDB/CNDB.git v/1/cndb
$ git clone git://github.com/FFM/FFG.git v/1/www/app
$ cp -a v/1 v/2
$ vi active/www/.ffg.config
### Add the lines (using the appropriate values for **your** install)::
db_name = "ffg1"
$ vi passive/www/.ffg.config
db_name = "ffg2"
### Define PYTHONPATH
$ export PYTHONPATH=/home/ffg/active/cndb:/home/ffg/active/tapyr
With a small config-file, the deploy-app can automatically create an Apache configuration file and a fcgi script. You can find sample config-files in active/www/app/httpd_config/. For instance, active/www/app/httpd_config/ffg_gg32_com__443.config contains:
config_path = "~/fcgi/ffg_gg32_com__443.config" host_macro = "gtw_host_ssl" port = "443" script_path = "~/fcgi/ffg_gg32_com__443.fcgi" server_admin = "[email protected]" server_name = "ffg.gg32.com" ssl_key_name = "srvr1-gg32-com-2048"
Create a config:
### Create a fcgi script and config for Apache
$ python active/www/app/deploy.py create_config \
-HTTP_Config <your-config> -input_encoding=utf-8
You can use the created Apache configuration as is, or modify it manually or by modifiying the template.
For Debian, the apache configuration should be placed into
/etc/apache2/sites-available/, e.g., into the file
nodedb2.example.com, and enabled. You probably will have to disable
the default site installed. We used the following commands — we
also enable some needed modules:
$ a2ensite nodedb2.example.com $ a2dissite default $ a2enmod mod_expires $ a2enmod fcgid $ /etc/init.d/apache2 restart
For https sites, you'll also need the modules:
$ a2enmod rewrite $ a2enmod ssl
Finally we create a database and populate it with data:
### Create a database $ python active/www/app/deploy.py create ### Put some data into the database
During the testing phase: Whenever we upgrade the software, the converter (from the old Graz database to the new one) will be run again. The following steps will prepare the "passive" deployment version and run the converter. This is similar to migrating to a new version (see below) except that we're running the converter instead of migrating the old version (because usually the Graz database has moved on and we want the latest version from the latest dump):
### Set correct virtual environment and PYTHONPATH, note that we ### need to explicitly set the PYTHONPATH to the passive environment $ source PVE/active/bin/activate $ export PYTHONPATH=/home/ffg/passive/cndb:/home/ffg/passive/tapyr ### Update source code $ python passive/www/app/deploy.py update ### Byte compile python files $ python passive/www/app/deploy.py pycompile ### Compile translations $ python passive/www/app/deploy.py babel compile ### Run the converter with database dump version XXXX $ python passive/www/app/convert_graz.py ffgraz_XXXX ### Setup app cache $ python passive/www/app/deploy.py setup_cache ### Switch active and passive branches $ python passive/www/app/deploy.py switch $ sudo /etc/init.d/apache2 restart
Whenever we need to upgrade the installation, we can update the passive configuration, set up everything, migrate the data from the active to the passive configuration, and if everything went OK, enable it by exchanging the symbolic links to the active and passive configuration:
### Test deployment script and generate some needed files ### Update source code $ python passive/www/app/deploy.py update ### Byte compile python files $ python passive/www/app/deploy.py pycompile ### Compile translations $ python passive/www/app/deploy.py babel compile ### Migrate database from active to passive $ python passive/www/app/deploy.py migrate -Active -Passive -verbose ### Setup app cache $ python passive/www/app/deploy.py setup_cache ### Switch active and passive branches $ python passive/www/app/deploy.py switch $ sudo /etc/init.d/apache2 restart
Christian Tanzer <[email protected]> and Ralf Schlatterbeck <[email protected]>