esdc Development

esdc-ce Development

• Quick start
• Programming rules and style conventions

Quick start

The esdc-ce project requires following application to be up and running, ideally on a CentOS 7 machine:

• PostgreSQL/PgBouncer
• RabbitMQ
• Redis

Your workstation will also require some packages to be installed (dependencies for esdc-ce python requirements):

** WIP: Migrationg to Python3**

• python, virtualenv -> Not required in future release
• python3, virtualenv
• python-devel on Centos/Fedora (python-pip, python-virtualenv, libffi-dev and python-dev in Ubuntu) -> Not required in future release
• python3-devel on Centos/Fedora (python3-pip, python3-virtualenv, libffi-dev and python3-dev in Ubuntu)
• libffi-devel
• openssl-devel
• gcc, make, patch
• libevent and libevent-devel in CentOS/Fedora (libevent-dev in Ubuntu) for gevent
• PostgreSQL postgresql-devel in CentOS/Fedora (libpq-dev in Ubuntu) for psycopg2
• zlib-devel and libjpeg-turbo-devel (zlib1g-dev in Ubuntu) for PIL
• GeoIP package (GeoIP in CentoOS/Fedora, geoip-bin in Ubuntu). Ubuntu users need to update setting.py (local_settings.py) value for GEOIP_LIBRARY_PATH to '/usr/lib/x86_64-linux-gnu/libGeoIP.so.1' and in Fedora GEOIP_LIBRARY_PATH to '/usr/lib64/libGeoIP.so.1'
• libxslt-devel libxml2-devel (libxslt1-dev libxml2-dev in Ubuntu)

With all the required applications running and dependencies installed you should be able to deploy the project on your workstation:

bin/ctl.sh init_envs
bin/ctl.sh deploy

And run the development web server:

bin/ctl.sh runserver

NOTE: Any custom configuration should be placed in core/local_settings.py or core/celery/local_config.py.

EXAMPLE local_settings.py

DEBUG = True

NOTE: If you intend to integrate some bigger functionality into esdc, please create a separate esdc third party app for that.

Of course, this kind of setup does not allow you to work with virtual machines or other objects on a compute node. For this, you will also require a running ErigonOS compute node.

Programming Rules

Coding Conventions

Python

• Python 2.7 and >=3.5 compatible code
• PEP8
• 120 character line length
• 4 spaces indentation

JavaScript

• 2 spaces indentation
• style similar to Python (underscores)

HTML/CSS

• 2 spaces indentation

Shell

• bash >= 3.1
• tabs indentation
• shellcheck

Imports and Dependencies

We may split this project into smaller microservice-like projects in the future. Because of that, it is important to keep some parts (modules) separated.

• core namespace: do not import anything from other namespaces here. This module should not depend on any other module in the project.

• que namespace: do not import anything from other namespaces here, except the core module. This module may depend only on the core module in the project.

• Django models: do not import api, gui and other high-level functions/classes in models.

Testing

To handle regressions, there is the esdc-tests project for API integration tests. Also the Python default unittest module may be used. It is always welcome to provide tests together with the pull request to demonstrate that a change or new feature works as intended.

Best practice

Due to the nature of project, we prefer to preserve the database (and the rest of the environment) between tests. Therefore, tests should clean after themselves and should not behave destructively towards the environment.

The unittest-based tests are usually run with the following command:

    bin/ctl.sh test --keepdb

You have to alter your local_settings.py so that database configuration contain the following line:

        'TEST': {'NAME': 'esdc'}

For example:

DATABASES = {
    'esdc': {
        'ENGINE': 'transaction_hooks.backends.postgresql_psycopg2',
        'NAME': 'esdc',
        'USER': 'esdc',
        'PASSWORD': 'asfNASFlakfnlaf',
        'HOST': '127.0.0.1',
        'PORT': '6432',
        'TEST': {
            'NAME': 'esdc'
        }
    },
    'pdns': {
        'ENGINE': 'transaction_hooks.backends.postgresql_psycopg2',
        'NAME': 'pdns',
        'USER': 'esdc',
        'PASSWORD': 'AFnaLKNGalknfslkfn',
        'HOST': '127.0.0.1',
        'PORT': '6432',
        'TEST': {
            'NAME': 'pdns'
        }
    },
}

Yes, this is your local development environment, not ad-hoc created testing environment that you are using to run tests. For example zabbix api is not mocked so the tests are really communicating with it.