diff --git a/docs/source/_static/img/ChangeDetection_screenshot.jpeg b/docs/source/_static/img/ChangeDetection_screenshot.jpeg new file mode 100644 index 000000000..5b05207f4 Binary files /dev/null and b/docs/source/_static/img/ChangeDetection_screenshot.jpeg differ diff --git a/docs/source/_static/img/footprint_screenshot.jpeg b/docs/source/_static/img/footprint_screenshot.jpeg new file mode 100644 index 000000000..684894d2e Binary files /dev/null and b/docs/source/_static/img/footprint_screenshot.jpeg differ diff --git a/docs/source/cli.md b/docs/source/cli.md index 083a58c5c..66365798c 100644 --- a/docs/source/cli.md +++ b/docs/source/cli.md @@ -1,10 +1,10 @@ # Command Line Interface -THIS DOCUMENT IS OUTDATED: Please use the `--help` flag of the CLI. +The Mapswipe Backend provides a Command Line Interface(CLI) with which the users can interact with the program. +They can be used for example to create projects, which were uploaded to the [manager-dashboard](for_mapswipe_managers.html), +or to export statistics on the finished projects. To get a comprehensible lists of the available commands use the ```--help``` flag. ---- - -This document describes how to use the command line interface of MapSwipe Worker. +```mapswipe_workers --help``` would get you all possible commands, while e.g. ```mapswipe_workers archive --help``` would get you additional information on how to use that command. In our current deployment setup the commands of the MapSwipe Workers CLI are hard-coded in the Docker-Compose File. @@ -13,148 +13,3 @@ You can run these commands also using docker-compose: ``` docker-compose run mapswipe_workers mapswipe_workers --help ``` - - -``` -Usage: mapswipe_workers [OPTIONS] COMMAND [ARGS]... - -Options: - -v, --verbose - --version Show the version and exit. - --help Show this message and exit. - -Commands: - create-projects - create-tutorial - firebase-to-postgres - generate-stats - run - user-management -``` - - -## Create projects from submitted project drafts - -``` -Usage: mapswipe_workers create-projects [OPTIONS] - -Options: - -s, --schedule [m|h|d] Will create projects every 10 minutes (m), every - hour (h) or every day (d). - --help Show this message and exit. -``` - - -## Transfer data from Firebase to Postgres - -``` -Usage: mapswipe_workers firebase-to-postgres [OPTIONS] - -Options: - -s, --schedule [m|h|d] Will update and transfer relevant data (i.a. users - and results) from Firebase into Postgres every 10 - minutes (m), every hour (h) or every day (d). - --help Show this message and exit. -``` - - -## Generate Statistics - -``` -Usage: mapswipe_workers generate-stats [OPTIONS] - -Options: - -s, --schedule [m|h|d] Generate stats every 10 minutes (m), every hour (h) - or every day (d). - --project_id_list TEXT provide project id strings as a list stats will be - generated only for these projects. - Use it like '["project_a", "project_b"]' - --help Show this message and exit. -``` - -## Generate Statistics for all projects - -Ideally you run this using a separate docker container. e.g. like this: - -``` -docker-compose run mapswipe_workers mapswipe_workers generate-stats-all-projects -``` - -``` -Usage: mapswipe_workers generate-stats-all-projects [OPTIONS] - -Options: - -s, --schedule [m|h|d] Generate stats every 10 minutes (m), every hour (h) - or every day (d). - --help Show this message and exit. -``` - -## User Management - -``` -Usage: mapswipe_workers user-management [OPTIONS] - -Options: - --email TEXT The email of the MapSwipe user. [required] - --manager BOOLEAN Set option to grant or remove project manager - credentials. Use true to grant credentials. Use false to - remove credentials. - --help Show this message and exit. -``` - -## Archive Projects - -``` -Usage: mapswipe_workers archive [OPTIONS] - - Archive projects in Postgres. Delete groups, tasks and results from - Firebase. - -Options: - -i, --project-id TEXT Archive project with giving project id - --project-ids TEXT Archive multiple projects. Provide project id strings - as a list: '["project_a", "project_b"]' - - --help Show this message and exit. - -``` - -You can get the project ids for the projects to be archive either from the manager dashboard or query directly in postgres. For example: - -``` -SELECT project_id -FROM projects -WHERE status = 'finished' AND created < '2020-09-01' -``` - -## Delete Projects - -``` -Usage: mapswipe_workers delete [OPTIONS] - - Delete tasks, groups, project and results. - -Options: - -i, --project-id TEXT Delete project with giving project id - --project-ids TEXT Delete multiple projects. Provide project id strings - as a list: '["project_a", "project_b"]' - - --help Show this message and exit. -``` - - -## Create Tutorial from json file (e.g. provided in sample data) - -``` -Usage: mapswipe_workers create-tutorial [OPTIONS] - -Options: - --input_file TEXT The json file with your tutorial information. [required] - --help Show this message and exit. - -``` - -## Run a script which requires mapswipe_workers library -``` -docker-compose run mapswipe_workers python3 python_scripts/add_project_geometries_to_api.py -``` diff --git a/docs/source/configuration.md b/docs/source/configuration.md index 7b5e737fe..894cfaac1 100644 --- a/docs/source/configuration.md +++ b/docs/source/configuration.md @@ -13,15 +13,24 @@ In following chapters configuration values and keys are discussed for each part All configuration values for MapSwipe Workers are stored in environment variables. Required environment variables are: +### Firebase - FIREBASE_API_KEY - FIREBASE_DB +- FIREBASE_TOKEN +- GOOGLE_APPLICATION_CREDENTIALS + +### Postgres DB - POSTGRES_DB - POSTGRES_HOST - POSTGRES_PASSWORD - POSTGRES_PORT - POSTGRES_USER -Optional environment variables are: +### OSMCha + +- OSMCHA_API_KEY + +### Optional environment variables: - SLACK_CHANNEL - SLACK_TOKEN - SENTRY_DSN @@ -34,19 +43,16 @@ For satellite imagery access to at least one provider is needed. Define the API - IMAGE_ESRI_API_KEY - IMAGE_ESRI_BETA_API_KEY -In addition to get access to Firebase a Service Account Key is required. -The path the Service Account Key is defined in: -- GOOGLE_APPLICATION_CREDENTIALS - > Notes: When deploying using `docker` or `docker-compose` `POSTGRES_HOST` should have the value `postgres` and the Service Account Key (`serviceAccountKey.json`) should be copied to `mapswipe_workers/serviceAccountKey.json` so that during the build of the image the file can by copied by Docker. - ### Elaboration **Firebase**: MapSwipe Workers use the Firebase Python SDK and the Firebase REST API. Both require the database name (`FIREBASE_DB`) and the API-Key from the Firebase instance. The Firebase Python SDK does also need a Service Account Key. The path to this file is set in the `GOOGLE_APPLICATION_CREDENTIALS` environment variable. **Postgres**: MapSwipe Workers writes data to a Postgres database and generate files for the API based data in Postgres. +**OSMCha**: MapSwipe Workers enriches some Projects with data from OSM changelogs which are requested from OSMCha. Create an account, you will find you api key in your profile e.g. `Token 589adf125234a` + **Sentry (optional)**: MapSwipe workers use sentry to capture exceptions. You can find your project’s DSN in the “Client Keys” section of your “Project Settings” in Sentry. Check [Sentry's documentation](https://docs.sentry.io/error-reporting/configuration/?platform=python) for more information. **Slack (optional)**: The MapSwipe workers send messages to slack when a project has been created successfully, the project creation failed or an exception gets raised. refer to [Python slackclient's documentation](https://github.com/slackapi/python-slackclient) how to get a Slack Token. @@ -80,13 +86,26 @@ The Service Account Key (`serviceAccountKey.json`) should be saved to `postgres/ ## Manager Dashboard +Please refer to the official [documentation](https://firebase.google.com/docs/web/learn-more#config-object) if you set up your own firebase. +Otherwise you can request guidance on the settings from the mapswipe team. The structure of your app.js should look like below. + `manager_dashboard/manager_dashboard/js/app.js` ``` -TODO +// Your web app's Firebase configuration +var firebaseConfig = { + apiKey: "", + authDomain: "", + databaseURL: "", + projectId: "", + storageBucket: "", + messagingSenderId: "", + appId: "" +}; +// Initialize Firebase +firebase.initializeApp(firebaseConfig); ``` - ## NGINX `nginx/nginx.conf`: diff --git a/docs/source/contributing.md b/docs/source/contributing.md deleted file mode 100644 index 526d9b8bf..000000000 --- a/docs/source/contributing.md +++ /dev/null @@ -1,43 +0,0 @@ -# Contributing - -This document describes how to setup Python MapSwipe Workers locally without using the provided Dockerfile for development purposes. - - -## Feature Branch - -To contribute to the MapSwipe back-end please create dedicated feature branches based on the `dev` branch: - -```bash -git checkout dev -git checkout -b featureA -# Hack away ... -git commit -am 'Describe changes.' -git push -u origin featureA -# Create a Pull Request from feature branch into the dev branch on GitHub. -``` - -> Note: If a bug in production (master branch) needs fixing before a new versions of MapSwipe Workers gets released (merging dev into master branch), a hotfix branch should be created. In the hotfix branch the bug should be fixed and then merged with the master branch (and also dev). - - -## Style Guide - -This project uses [black](https://github.com/psf/black), [flake8](https://gitlab.com/pycqa/flake8) and [isort](https://github.com/PyCQA/isort) to achive a constisted style across the project. -The configuration of flake8 and isort is stored in `setup.cfg`. - -Use [pre-commit](https://pre-commit.com/) to run `black` and `flake8` prior to any git commit. `pre-commit`, `black` and `flake8` should already be installed in your virtual environment since they are listed in `requirements.txt`. To setup pre-commit simply run: - -``` -pre-commit install -``` - -From now on `black`, `flake8` and `isort` should run automatically whenever `git commit` is executed. - -When running those tools manually please make sure the right version is used. The version can be looked up in `.pre-commit-config.yaml`. -To update to newer version please make sure to change version numbers in `.pre-commit-config.yaml`, `.travis.yml` and `requirements.txt`. - - -### Tips - -Ignore a hook: `SKIP=flake8 git commit -m "foo"` -Mark in code that flake8 should ignore the line: `print() # noqa` - diff --git a/docs/source/data.md b/docs/source/data.md index 700b9ba37..be315aebe 100644 --- a/docs/source/data.md +++ b/docs/source/data.md @@ -31,7 +31,7 @@ Attributes: ## Aggregated Results -This gives you the unfiltered MapSwipe results. This is most suited if you want to apply some custom data processing with the MapSwipe data, e.g. select only specific tasks for machine learning. If you want to use MapSwipe data in the Tasking Manager you might look for the data explained below. +This gives you the unfiltered MapSwipe results. This is most suited if you want to apply some custom data processing with the MapSwipe data, e.g. select only specific tasks for machine learning. If you want to use MapSwipe data in the Tasking Manager you might look for the data described below. Files: - `aggregated_results_{project_id}.csv`, e.g. [agg\_results\_-M56eeMCZ5VeOHjJN4Bx.csv](https://apps.mapswipe.org/api/agg_results/agg_results_-M56eeMCZ5VeOHjJN4Bx.csv) @@ -51,11 +51,13 @@ Files: | 2_share | float | 2_count divived by total_count. This gives you the share of all users who marked as 2. | | 3_share | float | 3_count divived by total_count. This gives you the share of all users who marked as 3. | | agreement | float | This is defined as [Scott's Pi](https://en.wikipedia.org/wiki/Scott%27s_Pi) and gives you an understanding of inter-rater reliability. The value is 1.0 if all users agree, e.g. all users classify as "building". If users disagree this value will be lower. | - | geom | string | The geometry of this task as WKT geometry. | +| geom | string | The geometry of this task as WKT geometry. | +Additionally, project type specific data can be found here. E.g. footprint projects which were created based on OSM data, will have data describing the original OSM object included. - ## HOT Tasking Manager Geometries - This gives you filtered MapSwipe data ready to be imported to the HOT Tasking Manager. Currently, the geometries in this dataset consist of maximum 15 MapSwipe Tasks, where at least 35% of all users indicated the presence of a building by classifying as "yes" or "maybe". + +## HOT Tasking Manager Geometries +This gives you filtered MapSwipe data ready to be imported to the HOT Tasking Manager. Currently, the geometries in this dataset consist of maximum 15 MapSwipe Tasks, where at least 35% of all users indicated the presence of a building by classifying as "yes" or "maybe". Files: - `hot_tm_{project_id}.geojson`, e.g. [hot\_tm\_-M56eeMCZ5VeOHjJN4Bx.geojson](https://apps.mapswipe.org/api/hot_tm/hot_tm_-M56eeMCZ5VeOHjJN4Bx.geojson) @@ -67,4 +69,15 @@ Files: - \ No newline at end of file +## Users + +This gives you data on the users which contributed to a project. + +| Name | Type | Description | +|---------------------------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| idx | integer | - | +| groups_completed | integer | Number of groups completed | +| total_contributions | integer | Number of tasks completed | +| agreeing_contributions | integer | Tasks with the same result as the final result (e.g. Tile has buildings). | +| disagreeing_contributions | integer | Tasks with other result as the final result. | +| simple_agreement_score | float | Share of tasks which had the same result as the final result. E.g. 0.8 would mean that the user labeled 80% of the tiles the same way as the majority of voters. | diff --git a/docs/source/deployment_overview.md b/docs/source/deployment_overview.md deleted file mode 100644 index db79edaf8..000000000 --- a/docs/source/deployment_overview.md +++ /dev/null @@ -1,33 +0,0 @@ -# Deployment Overview - -The MapSwipe Back-End is deployed using Docker. -Every part of the back-end has a Dockerfile. -All parts/ Dockerfiles come together in the docker-compose file. - -To setup the whole ecosystem of MapSwipe Workers it is easier to first make sure every part is configured and all keys are in place. - -MapSwipe utilizes a bunch of Google Cloud services: - -- [Firebase](https://firebase.google.com/) project with Realtime Database and Functions - - Create a project - - Create a database: `> Develop > Database > Create Database` -- [Google Cloud Storage](https://cloud.google.com/storage/) for backup of Postgres - -[Configuration](configuration.md) describes all needed configuration and credentials. - -[Installation](installation.md) describes step-by-step how to setup the backend for the first time. - - -## Continuous deployment using Ansible and Travis - -Travis is setup to automatically deploy a new version of MapSwipe Back-End to the server once it run successfully. -This is done by using Travis script deployment (https://docs.travis-ci.com/user/deployment/script/). Travis simply calls the `deploy.sh` script found at the root directory of MapSwipe Workers. -To be able to connect to the MapSwipe server the Travis instance uses an encrypted SSH private key (Which can be found in the directory `travis/`). - -In the `deploy.sh` script an Ansible Playbook is run (https://docs.ansible.com/ansible/latest/index.html). Ansible is an automation tool which utilizes a SSH connection (`ansible/ansible.cfg`) to run commands defined in the Playbook (`ansible/playbook.yml`) on hosts defined in the Inventory (`ansible/inventory.yml`). - -## Continuous Deployment with Github Actions -We use an encrypted service account key file. The file has been generated with this commands: - -`openssl enc -aes-256-cbc -e -p -nosalt -in=ci-mapswipe-firebase-adminsdk-80fzw-ebce84bd5b.json -out=ci-mapswipe-firebase-adminsdk-80fzw-ebce84bd5b.json.enc` - diff --git a/docs/source/dev_setup.md b/docs/source/dev_setup.md index 8af7f4cb5..7185ba8c6 100644 --- a/docs/source/dev_setup.md +++ b/docs/source/dev_setup.md @@ -1,23 +1,21 @@ # Development Setup -In this document some tips and workflows for development are loosely collected. Those are independent of the production setup using Docker-Compose. A working Firebase Project (Including Firebase Functions and Database Rules) is presupposed. Get in touch with the MapSwipe team (e.g. in Slack) to get access to an existing Firebase Instance for development purposes. +In this document some tips and workflows for development are loosely collected. +Those are independent of the production setup using Docker-Compose. +A working Firebase Project (Including Firebase Functions and Database Rules) is presupposed. +Get in touch with the MapSwipe team (e.g. in Slack) to get access to an existing Firebase Instance for development purposes. Check list: -1. Clone repo from GitHub. -2. Install GDAL/OGR and GDAL for Python on your machine. -3. Set environment variables and get a Service Account Key File. -4. Set up local Postgres database using Docker. -5. Install MapSwipe Workers Python package. +1. Github: Clone repo from GitHub. +2. Requirements: Install GDAL/OGR and GDAL for Python on your machine. +3. Configuration: Set environment variables and get a Service Account Key File. +4. Database: Set up local Postgres database using Docker. +5. Python-Package: Install MapSwipe Workers Python package. 6. Run MapSwipe Workers. ## Installation -### Requirements - -MapSwipe Workers requires GDAL/OGR (`gdal-bin`) and GDAL for Python (`libgdal-dev`, `python-gdal`) to be installed. Furthermore, we rely on Docker to set up Postgres. - - ### Clone from GitHub ... and switch to development branch. @@ -28,15 +26,29 @@ cd python-mapswipe-workers git checkout dev ``` +### Requirements + +MapSwipe Workers requires GDAL/OGR (`gdal-bin`) and GDAL for Python (`libgdal-dev`, `python-gdal`) to be [installed](https://mothergeo-py.readthedocs.io/en/latest/development/how-to/gdal-ubuntu-pkg.html). +Furthermore, we rely on Docker to set up Postgres. + ### Configuration All configurations values are stored in environment variables. Please refer to the documentation on [Configuration](configuration.html) for further details. +#### Service Account Key + +The MapSwipe Workers requires a Service Account Key (`serviceAccountKey.json`) to access Firebase database. +Request yours from the MapSwipe working group. + +The path to the Service Account Key is defined in the `GOOGLE_APPLICATION_CREDENTIALS` environment variable. -### Directories +You could also set up your own Firebase instance. However, this is not recommended. +If you still want to do it, get your Service Account Key from Firebase from [Google Cloud Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts). -MapSwipe Workers needs access to a data directory for logs and data for data for the API: +#### Directories + +MapSwipe Workers needs access to a data directory for logs and data for the API: To create this directories run: ``` @@ -46,18 +58,10 @@ mkdir --parents ~/.local/share/mapswipe_workers > Note: XDG Base Directory Specification is respected -### Service Account Key - -The MapSwipe Workers requires a Service Account Key (`serviceAccountKey.json`) to access Firebase database. Request yours from the MapSwipe working group. - -The path the Service Account Key is defined in the `GOOGLE_APPLICATION_CREDENTIALS` environment variable. +### Database -You could also set up your own Firebase instance. However, this is not recommended. If you still want to do it, get your Service Account Key from Firebase from [Google Cloud Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts). - - -### Postgres - -Setup a local Postgres instance for MapSwipe Workers using the Dockerfile provided for development purposes (`postgres/Dockerfile-dev`). The Dockerfile for production (`postgres/Dockerfile`) does need additional setup for build-in backup to Google Cloud Storage, which is not needed for local development. That is why a simplified Dockerfile for development is provided. +Setup a local Postgres instance for MapSwipe Workers using the Dockerfile provided for development purposes (`postgres/Dockerfile-dev`). +The Dockerfile for production (`postgres/Dockerfile`) does need additional setup for build-in backup to Google Cloud Storage, which is not needed for local development. That is why a simplified Dockerfile for development is provided. Make sure that the specified port is not in use already. If so, adjust the port (also in the `.env` file). ```bash @@ -68,7 +72,7 @@ docker run -d -p 5432:5432 --name mapswipe_postgres -e POSTGRES_DB="$POSTGRES_DB Or set up Postgres using the `initdb.sql` file in the `postgres/` folder. -### Install MapSwipe Workers Python Package +### Mapswipe-Workers Python Package 1. Export environment variables to current shell. 2. Create a Python virtual environment with `system-site-packages` option enabled to get access to GDAL/OGR Python packages @@ -87,10 +91,12 @@ mapswipe_workers --help > Yeah! If you reached this point, you are ready to get into coding. Below you find some more information on Logging, Firebase Functions and Database Backup. However, you don't need this to get started for now. +# Further Information ## Logging -Mapswipe workers logs are generated using the Python logging module of the standard library (See [Official docs](https://docs.python.org/3/library/logging.html) or this [Tutorial](https://realpython.com/python-logging/#the-logging-module). To use the logger object import the it from the `definitions` module: +Mapswipe workers logs are generated using the Python logging module of the standard library (See [Official docs](https://docs.python.org/3/library/logging.html) or this [Tutorial](https://realpython.com/python-logging/#the-logging-module). +To use the logger object import it from the `definitions` module: ```python from mapswipe_workers.definitions import logger @@ -104,9 +110,11 @@ except Exception: logger.exception('Additional information.') ``` -Default logging level is Info. To change the logging level edit the logging configuration which is found in the module `definitions.py`. Logs are written to STDOUT and `~/.local/share/mapswipe_workers/mapswipe_workers.log`. +Default logging level is Info. To change the logging level edit the logging configuration which is found in the module `definitions.py`. +Logs are written to STDOUT and `~/.local/share/mapswipe_workers/mapswipe_workers.log`. -Per default logging of third-party packages is disabled. To change this edit the definition module (`mapswipe_workers/defintions.py`). Set the `disable_existing_loggers` parameter of the `logging.config.fileConfig()` function to `False`. +Per default logging of third-party packages is disabled. To change this edit the definition module (`mapswipe_workers/defintions.py`). +Set the `disable_existing_loggers` parameter of the `logging.config.fileConfig()` function to `False`. ## Firebase Functions @@ -128,37 +136,12 @@ Firebase functions are used to increment/decrement or calculate various attribut Those functions will be directly or indirectly triggered by incoming results from the MapSwipe App. -By using Firebase functions those attributes can be calculated in real-time and be accessed by users immediately. The use of those functions also reduces the data-transfer between the Firebase Realtime Database and MapSwipe Workers. - -On how to setup development environment and how to deploy functions to the Firebase instance please refer to the official [Guide on Cloud Function for Firebase](https://firebase.google.com/docs/functions/get-started). -For more information refer to the official [Reference on Cloud Function for Firebase](https://firebase.google.com/docs/reference/functions/). For example function take a look at this [GitHub repository](https://github.com/firebase/functions-samples). - - -## Travis Setup - -A Travis instance is used to build MapSwipe Workers and run tests. -There exists a Firebase instance only for Travis. -For the configuration of Travis following environment variables are used: - -- FIREBASE_API_KEY -- FIREBASE_DB -- FIREBASE_TOKEN -- POSTGRES_DB -- POSTGRES_HOST -- POSTGRES_PASSWORD -- POSTGRES_PASSWORD -- POSTGRES_USER -- WALG_GS_PREFIX: empty -- GOOGLE_APPLICATION_CREDENTIALS: mapswipe_workers/serviceAccountKey.json - -Those variables can be definied directly in the repository settings of Travis. For more inofmration refer to: https://docs.travis-ci.com/user/environment-variables/#defining-variables-in-repository-settings - -Additionaly a Service Account Key in JSON format is encrypted and added to the GitHub repository using the travis CLI. Once Travis runs it will decrypt the Service Account Key. Read more on that process in the Travis docs: https://docs.travis-ci.com/user/encrypting-files/ - -Once a Travis build succeeds Travis executes an Ansible Playbook to deploy MapSwipe Workers to an already installed and configured server. -For this to work an SSH-Key (with access rights to the server) is also encrypted and added to the GitHub repository. Travis will decrypt the key and Ansible will use it to execute commands defined in the Playbook on the server. +By using Firebase functions those attributes can be calculated in real-time and be accessed by users immediately. +The use of those functions also reduces the data-transfer between the Firebase Realtime Database and MapSwipe Workers. -All files encrypted for Travis (Service Account Key, SSH-Key) are stored in the `travis` directory. +On how to setup the development environment and how to deploy functions to the Firebase instance please refer to the official [Guide on Cloud Function for Firebase](https://firebase.google.com/docs/functions/get-started). +For more information refer to the official [Reference on Cloud Function for Firebase](https://firebase.google.com/docs/reference/functions/). +For example function take a look at this [GitHub repository](https://github.com/firebase/functions-samples). ## Database Backup diff --git a/docs/source/for_mapswipe_managers.md b/docs/source/for_mapswipe_managers.md index 71c4fef74..14332a231 100644 --- a/docs/source/for_mapswipe_managers.md +++ b/docs/source/for_mapswipe_managers.md @@ -22,7 +22,10 @@ For BuildArea and ChangeDetection projects: For Footprint projects: * The GeoJSON file should contain only simple Polygons. We currently don't support complex Multipolygon geometries (e.g. [polygon with holes](https://developers.google.com/maps/documentation/javascript/examples/polygon-hole)) -Once you submit, the task should appear relatively quickly in the manager dashboard. You will receive a message in Slack. But it's still not active and not visible to the MapSwipe app users. You need to set the project status to `active` through the manager dashboard. Just click on the respective button. If the new project does not appear in the app after about 1 hour, check Slack for an error message, and see Troubleshooting below. +Once you submit, the task should appear relatively quickly in the manager dashboard. You will receive a message in Slack. +But it's still not active and not visible to the MapSwipe app users. +You need to set the project status to `active` through the manager dashboard. Just click on the respective button. +If the new project does not appear in the app after about 1 hour, check Slack for an error message, and see Troubleshooting below. ![Project Management](_static/img/manager_dashboard_manage_screenshot.png) diff --git a/docs/source/index.rst b/docs/source/index.rst index 5f267370d..cd6f55681 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -13,41 +13,35 @@ Welcome to MapSwipe Back-End's documentation! readme_link overview - contributing + data + dev_setup + testing + + +.. toctree:: + :maxdepth: 2 + :caption: Project Types: + + project_type + project_type-buildArea + project_type-changeDetection + project_type-footprint .. toctree:: :maxdepth: 2 - :caption: Using: + :caption: Usage: for_mapswipe_managers + cli use_cases tile_size - data .. toctree:: :maxdepth: 2 :caption: Deployment: - deployment_overview configuration installation - cli debugging backup - - -.. toctree:: - :maxdepth: 2 - :caption: Development: - - dev_setup - testing_and_travis - - -Indices and tables -================== - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` diff --git a/docs/source/project_type-buildArea.md b/docs/source/project_type-buildArea.md index 235f850b4..fa1a2a17e 100644 --- a/docs/source/project_type-buildArea.md +++ b/docs/source/project_type-buildArea.md @@ -2,81 +2,137 @@ ![Build Area](_static/img/data_structure-firebase-1.svg) -| Name | ID | Description | Screenshot | -| ---- | -- | ----------- | ---------- | -| BuildArea | 1 | A 6 squares layout is used for this project type. By tapping you can classify a tile of satellite imagery as *yes*, *maybe* or *bad_imagery*. Project managers can define which objects to look for, e.g. "buildings". Furthermore, they can specify the tile server of the background satellite imagery, e.g. "bing" or a custom tile server. | | - - -## Data Model -The MapSwipe crowdsourcing workflow is designed following an approach already presented by [Albuquerque et al. (2016)](http://www.mdpi.com/2072-4292/8/10/859). Five concepts are important in the following: -* project drafts -* projects -* groups -* tasks -* results. - -| - -As a project manager you have to care about the **Project Drafts** only. The information you provide through the **Manager Dashboard** will be used to set up your project. You should provide the following information. - ### Project Drafts -The project drafts contain all information needed to set up your project. Only MapSwipe user accounts with dedicated project manager role can create projects. Make sure to get the rights before submitting project drafts. - -| Parameter | Description | -| --- | --- | -| _Basic Project Information_ | -| **Name** | The name of your project (25 chars max) | -| **Look For** | What should the users look for (e.g. buildings, cars, trees)? (15 chars max). | -| **Project Type** | Is `1` for all Build Area projects. | -| **Direct Image Link** | An url to an image. Make sure you have the rights to use this image. It should end with .jpg or .png. | -| **Project Details** | The description for your project. (3-5 sentences). | -| **Verification Number** | How many people do you want to see every tile before you consider it finished? (default is 3 - more is recommended for harder tasks, but this will also make project take longer) | -| **Group Size** | How big should a mapping session be? Group size refers to the number of tasks per mapping session. | -| _Project Type Specific Information_ | -| **Zoom Level** | We use the Tile Map Service zoom levels. Please check for your area which zoom level is available. For example, Bing imagery is available at zoomlevel 18 for most regions. If you use a custom tile server you may be able to use even higher zoom levels. | -| **KML file Content** | The content of a KML file. Make sure that you provide a single polygon geometry. | -| **Tile Server Name** | Select the tile server providing satellite imagery tiles for your project. Make sure you have permission. You can choose: `Bing`, `Digital Globe`, `Sinergise`, `Custom` | -| **Custom Tile Server URL** (optional) | A custom tile server URL that uses {z}, {x} and {y} as placeholders and that already includes the api key. This is only needed if you choose `Custom` as the _Tile Server Name_ | -| **WMTS Layer Name** (optional) | Enter the name of the layer of the Web Map Tile Service (WMTS). This is only needed if you choose `Sinergise` as the _Tile Server Name_. | -| **API Key required** (optional) | Do you need an api key for the imagery? | -| **API Key** (optional)| Insert the api key if required. | +To initialize a Build Area Project as a Project Manager you only need to upload a bounding polygon as well as fill in some information about your mission. +Details on the basic information you need to fill in to describe you mission can be found on the [main project type site](project_type.html). + +```json +{ + "createdBy": "TestCreator", + "geometry": { + "type": "FeatureCollection", + "features": [{ + "type": "Feature", + "geometry": { + "type": "Polygon", + "coordinates": [ + [ + [34.975833892822266, -15.899098066386088], + [35.089302062988274, -15.899098066386088], + [35.089302062988274, -15.820002241903946], + [34.975833892822266, -15.820002241903946], + [34.975833892822266, -15.899098066386088] + ] + ] + }, + "properties": {} + }] + }, + "image": "", + "lookFor": "buildings", + "name": "test - Malawi (1)\ntest", + "projectDetails": "This is a test project", + "verificationNumber": 3, + "groupSize": 120, + "tileServer": { + "name": "bing", + "credits": "© 2019 Microsoft Corporation, Earthstar Geographics SIO" + }, + "projectType": 1 +} +``` ### Projects -Projects get created from _Project Drafts_ by the Mapswipe workers. The workers extend the information by the following parameters. - - -### Tasks -To create a new mapping task, the overall project extent is split up into many single tasks. Tasks are the smallest unit in the MapSwipe data model. They are derived from the area of interest by gridding it into many small equal-sized rectangular polygons. Each task corresponds to a specific tile coordinate from a tile map service (TMS) using a web Mercator projection as its geographical reference system. Therefore, each task is characterized by a geometry and its tile coordinates, which describe its x, y and z position. For the projects analysed in this project, the tiles for all tasks are generated at zoom level 18. Taking the latitude of each task location into account the satellite imagery has a maximum spatial resolution of ~ 0.6 meter at the equator. - -| Parameter | Description | -| --- | --- | -| **Id** | Each task can be identified by its Id. The Id is a composition of its position in the corresponding tile map system, which can be described by the x, y and z coordinates. | -| **Tile Z** | The z coordinate of the tile defines the zoom level. Greater values for z will correspond to higher spatial resolution of the corresponding image. For most regions Bing provides images up to zoom level 18. For aerial imagery or images captured by UAVs even higher z values are valid. | -| **Tile X**| The x coordinate characterises the longitudinal position of the tile in the overall tile map system taken the zoom level into account. The x coordinates increase from west to east starting at a longitude of -180 degrees. | -| **Tile Y** | The y coordinate characterises the latitudinal position of the tile in the overall tile map system taken the zoom level into account. The latitude is clipped to range from circa -85 to 85 degrees. The y coordinates increase from north to south starting at a latitude of around 85 degrees. | -| **Geometry** | Each task has a polygon geometry, which can be generated by its x, y and z coordinates. At the equator the task geometry is a square with an edge length of around 150 metres covering circa 0.0225 square kilometres. Due to the web Mercator projector the task geometry will be clinched with increasing distance to the equator. At the same time the area per task will decrease. | -| **Tile URL** | The tile URL points to the specific tile image described by the x, y, and z coordinates. Usually, the image has a resolution of 256 x 256 pixels. However, some providers also generate image tiles with higher resolution (e.g. 512 x 512 pixels). | +Below you can find an example for a created Build Area project in firebase. + +```json +{ + "contributorCount": 1, + "created": "2021-12-23T13:47:27.346088Z", + "createdBy": "X0zTSyvY0khDfRwc99aQfIjTEPK2", + "groupMaxSize": 0, + "groupSize": 25, + "image": "https://firebasestorage.googleapis.com/v0/b/dev-mapswipe.appspot.com/o/projectImages%2Fbuildarea.png?alt=media&token=07505c0e-0f80-454c-b446-9b82a73d9d3e", + "isFeatured": false, + "lookFor": "Buildings", + "name": "Build Area with Bing Imagery Z18 - Kenya (1)\nMapSwipe Devs", + "progress": 0, + "projectDetails": "This is a \"normal\" Build Area project. The project uses Bing Imagery at zoom level 18", + "projectId": "-MrbXgHx8YJDt6cTIyGA", + "projectNumber": "1", + "projectRegion": "Kenya", + "projectTopic": "Build Area with Bing Imagery Z18", + "projectType": 1, + "requestingOrganisation": "MapSwipe Devs", + "requiredResults": 148158, + "resultCount": 0, + "status": "active", + "tileServer": { + "apiKey": "", + "credits": "imagery credits of project", + "name": "bing", + "url": "https://ecn.t0.tiles.virtualearth.net/tiles/a{quad_key}.jpeg?g=1&token={key}" + }, + "tutorialId": "tutorial_-MnNaUEShyefFtMG6_5-", + "verificationNumber": 3, + "zoomLevel": 18 +} +``` ### Groups -Single MapSwipe projects can contain up to several hundred thousand tasks. This can pose a challenge to fast and performant communication between clients and server if many volunteers contribute data at the same time. Therefore, groups have been introduced to reduce the amount of client requests on the backend server. Groups consists of several tasks, that will be shown to the user in one mapping session. The grouping algorithm uses the extent of a project as an input and generates chunks of tasks lying next to each other. Each group has a height of three tasks and a width of approximately 40 tasks. +The grouping algorithm uses the extent of a project as an input and generates chunks of tasks lying next to each other. +Each group has a height of three tasks and a width of approximately 40 tasks. + +| Parameter | Description | +|--------------|------------------------------------------------------------------------------------------------------| +| **Geometry** | The Build Area groups save the bounding box coordinates in fields labeled xMax, xMin, yMax and yMin. | + +```json +{ + "finishedCount" : 0, + "groupId" : "g101", + "numberOfTasks" : 54, + "progress" : 0, + "projectId" : "-MrbXgHx8YJDt6cTIyGA", + "requiredCount" : 3, + "xMax" : "160239", + "xMin" : "160222", + "yMax" : "129763", + "yMin" : "129761" +} +``` -| Parameter | Description | -| --- | --- | -| **Id** | Each group can be identified by its Id. | -| **Tasks** | Each group contains several tasks. The information for all tasks of the group will be stored in an array. | -| **Geometry** | The group geometry is defined by the union of all assigned task geometries. | -| **Completed Count** | Once a group has been completely mapped by a volunteer the completed count of the corresponding group will be raised by one. The completed count of the group is used to assess the overall progress of each project. For doing so the completed count is compared to the redundancy required (see Table 2). During the mapping process groups will be served in ascending completed count order. Thus, groups with low completed count will be served first. | +### Tasks -### Results +Tasks are only saved for tutorials, since their spatial inforamtion can be derived from the spatial extent of the corresponding group. + +| Parameter | Description | +|-------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| *Project Type Specific Information* | | +| **Tile X** | The x coordinate characterises the longitudinal position of the tile in the overall tile map system taken the zoom level into account. The x coordinates increase from west to east starting at a longitude of -180 degrees. | +| **Tile Y** | The y coordinate characterises the latitudinal position of the tile in the overall tile map system taken the zoom level into account. The latitude is clipped to range from circa -85 to 85 degrees. The y coordinates increase from north to south starting at a latitude of around 85 degrees. | +| **Geometry** | Each task has a polygon geometry, which can be generated by its x, y and z coordinates. At the equator the task geometry is a square with an edge length of around 150 metres covering circa 0.0225 square kilometres. Due to the web Mercator projector the task geometry will be clinched with increasing distance to the equator. At the same time the area per task will decrease. | +| **Tile URL** | The tile URL points to the specific tile image described by the x, y, and z coordinates. Usually, the image has a resolution of 256 x 256 pixels. However, some providers also generate image tiles with higher resolution (e.g. 512 x 512 pixels). | + +Below is an example json for a tutorial project, as can be seen on the three extra attributes screen, referenceAnswer and taskID_real. +```json +{ + "groupId" : 101, + "projectId" : "tutorial_-MGwrwsP9cTYf6c_Nbg3", + "referenceAnswer" : 0, + "screen" : 1, + "taskId" : "18-100-131072", + "taskId_real" : "18-65040-120545", + "taskX" : 100, + "taskY" : 131072, + "url" : "https://ecn.t0.tiles.virtualearth.net/tiles/a023313133022210002.jpeg?g=7505&mkt=en-US" +} +``` -Results contain information on the user classifications. However, only “Yes”, “Maybe” and “Bad Imagery” classifications are stored as results. Whenever users indicate “No building” by just swiping to the next set of tasks, no data entry is created. “No Building” classifications can only be modelled retrospectively for groups where a user also submitted at least one “Yes”, “Maybe” or “Bad Imagery” classification. +### Results -| Parameter | Description | -| --- | --- | -| **Id** | Each result can be identified by its Id. The Id is a combination of task Id and user Id. | -| **Task Id** | Each result corresponds to a specific task, which can be described by its Id. | -| **User Id** | Each result is contributed by a specific user. Users can be identified by their Id. | -| **Timestamp** | The timestamp (in milliseconds since 01-01-1970) provides information on the time the user completed the group and uploaded the result data. Results within the same group are assigned the same timestamp. | -| **Result** | This parameter describes the given answer. 1 corresponds to “Yes”, 2 corresponds to “Maybe” and 3 corresponds to “Bad Imagery”. Each user can only submit one result per task. | +Results contain information on the user classifications. However, only “Yes” (1), “Maybe” (2) and “Bad Imagery” (3) classifications are stored as results. +Whenever users indicate “No building” by just swiping to the next set of tasks, no data entry is created. +“No Building” classifications can only be modelled retrospectively for groups where a user also submitted at least one “Yes”, “Maybe” or “Bad Imagery” classification. \ No newline at end of file diff --git a/docs/source/project_type-changeDetection.md b/docs/source/project_type-changeDetection.md index e69de29bb..2fdb26a92 100644 --- a/docs/source/project_type-changeDetection.md +++ b/docs/source/project_type-changeDetection.md @@ -0,0 +1,127 @@ +# Change Detection + +## Project Draft + +The Change detection project type is initialized in the same way as the standard buildArea project. + +Project Draft example for a Change Detection project: +```json +{ + "createdBy": "Sample Manager", + "geometry": {"type":"FeatureCollection","features":[{"type":"Feature","properties":{},"geometry":{"type":"Polygon","coordinates":[[[-175.21785736083984,-21.122295110595505],[-175.2367401123047,-21.148873980682744],[-175.21339416503906,-21.152716314425852],[-175.19931793212888,-21.15239612375494],[-175.19588470458984,-21.147913381670975],[-175.1931381225586,-21.136385707660683],[-175.1934814453125,-21.129660817021904],[-175.21785736083984,-21.122295110595505]]]}}]}, + "image": "http://www.redcrosseth.org/media/k2/items/cache/5a05a447acfdf6fcc40548cc4c1cea8d_L.jpg", + "lookFor": "DESTROYED BUILDINGS", + "name": "Change Detection Sample Project", + "projectDetails": "This project uses Bing as the tile server and zoom level 18 for the before image. For after we use imagery from open aerial map.", + "verificationNumber": 3, + "groupSize": 15, + "projectType": 3, + "tileServerA": { + "name": "bing", + "apiKeyRequired": true, + "caption": "Before", + "credits": "© 2019 Microsoft Corporation, Earthstar Geographics SIO" + }, + "tileServerB": { + "name": "custom", + "url": "https://tiles.openaerialmap.org/5b3541802b6a08001185f8b1/0/5b3541802b6a08001185f8b2/{z}/{x}/{y}.png", + "apiKeyRequired": false, + "apiKey": "", + "caption": "After", + "date": "2018-02-21", + "credits": "© OpenAerialMap" + } +} +``` +Examples of other initialization options can be found in the mapswipe-backend repository at mapswipe_workers/tests/integration/fixtures/change_detection/project_drafts.json. + +## Project structure + +Project Structure example for a project which was created via HOT Tasking Manager Project ID. +```json +{ + "contributorCount" : 0, + "created" : "2021-12-23T14:14:52.179930Z", + "createdBy" : "X0zTSyvY0khDfRwc99aQfIjTEPK2", + "groupMaxSize" : 0, + "groupSize" : 25, + "image" : "https://firebasestorage.googleapis.com/v0/b/dev-mapswipe.appspot.com/o/projectImages%2FEQ%2BEarthquake.png?alt=media&token=6e82ba52-8adb-4214-8f81-4b7030c00946", + "isFeatured" : false, + "lookFor" : "damaged buildings", + "name" : "Earthquake - Experimental Damage Assessment - Les Cayes (Haiti) (1)\nSimon BA", + "progress" : 0, + "projectDetails" : "In attempt to provide a rapid damage assessment for the 7.2 magnitude earthquake on August 14, please slowly compare the images to determine if damage is visible in the post-event scene. This methodology is still being tested and should not replace traditional damage assessment methods. Imagery is provided through [Maxar's Open Data Programm](https://www.maxar.com/open-data) and hosted by [MapBox](https://www.mapbox.com/).", + "projectId" : "-Mrbd5ArF4lb_GoYG2I5", + "projectNumber" : "1", + "projectRegion" : "Les Cayes (Haiti)", + "projectTopic" : "Earthquake - Experimental Damage Assessment", + "projectType" : 3, + "requestingOrganisation" : "Simon BA", + "requiredResults" : 3636, + "resultCount" : 0, + "status" : "inactive", + "tileServer" : { + "apiKey" : "", + "credits" : "© 2019 Maxar", + "name" : "maxar_premium", + "url" : "https://services.digitalglobe.com/earthservice/tmsaccess/tms/1.0.0/DigitalGlobe%3AImageryTileService@EPSG%3A3857@jpg/{z}/{x}/{y}.jpg?connectId={key}" + }, + "tileServerB" : { + "credits" : "© Maxar, MapBox", + "name" : "custom", + "url" : "https://api.mapbox.com/v4/mapboxsatellite.haiti-post-2021/{z}/{x}/{y}.webp?sku=101Fw3jtBuWI5" + }, + "tutorialId" : "tutorial_-MhJtd9ePFOw8Vs6xwZ2", + "verificationNumber" : 3, + "zoomLevel" : 19 +} +``` + +## Group structure + +| Parameter | Description | +|--------------|------------------------------------------------------------------------------------------------------------| +| **Geometry** | The Change Detection groups save the bounding box coordinates in fields labeled xMax, xMin, yMax and yMin. | + +```json +{ + "finishedCount" : 0, + "groupId" : "g101", + "numberOfTasks" : 24, + "progress" : 0, + "projectId" : "-Mrbd5ArF4lb_GoYG2I5", + "requiredCount" : 3, + "xMax" : "154722", + "xMin" : "154715", + "yMax" : "235151", + "yMin" : "235149" +} +``` + +## Task structure + +| Parameter | Description | +|-------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| *Project Type Specific Information* | | +| **Task X** | The x coordinate characterises the longitudinal position of the tile in the overall tile map system taken the zoom level into account. The x coordinates increase from west to east starting at a longitude of -180 degrees. | +| **Task Y** | The y coordinate characterises the latitudinal position of the tile in the overall tile map system taken the zoom level into account. The latitude is clipped to range from circa -85 to 85 degrees. The y coordinates increase from north to south starting at a latitude of around 85 degrees. | +| **Geometry** | Each task has a polygon geometry, which can be generated by its x, y and z coordinates. At the equator the task geometry is a square with an edge length of around 150 metres covering circa 0.0225 square kilometres. Due to the web Mercator projector the task geometry will be clinched with increasing distance to the equator. At the same time the area per task will decrease. | +| **URL** | Image for the tile at timestamp A. The tile URL points to the specific tile image described by the x, y, and z coordinates. | +| **URL 2** | Image for the tile after timestamp A. The tile URL points to the specific tile image described by the x, y, and z coordinates. | + +```json +{ + "groupId" : "g101", + "projectId" : "-Mrbd5ArF4lb_GoYG2I5", + "taskId" : "19-154715-235149", + "taskX" : "154715", + "taskY" : "235149", + "url" : "https://services.digitalglobe.com/earthservice/tmsaccess/tms/1.0.0/DigitalGlobe%3AImageryTileService@EPSG%3A3857@jpg/19/154715/289138.jpg", + "urlB" : "https://api.mapbox.com/v4/mapboxsatellite.haiti-post-2021/19/154715/235149.webp?sku=101Fw3jtBuWI5" +} +``` + +## Result Structure + +Results contain information on the user classifications. However, only “Yes” (1), “Maybe” (2) and “Bad Imagery” (3) classifications are stored as results. +Whenever users indicate “No Change” by just swiping to the next task, no data entry is created. diff --git a/docs/source/project_type-footprint.md b/docs/source/project_type-footprint.md index fad1b7940..ae907abb9 100644 --- a/docs/source/project_type-footprint.md +++ b/docs/source/project_type-footprint.md @@ -1,74 +1,111 @@ # Footprint -![Footprint](_static/img/data_structure-firebase-2.svg) +## Project Draft +Footprint projects can be supplied with geometries in three seperate ways. +1. by specifying a HOT Tasking Manager Project ID and an object [filter](https://docs.ohsome.org/ohsome-api/v1/filter.html) +2. by specifying an url to the data (e.g. an [ohsomeAPI](https://docs.ohsome.org/ohsome-api/v1/) call) +3. by uploading an aoi and an object [filter](https://docs.ohsome.org/ohsome-api/v1/filter.html) -## Import structure +Project Draft example for a footprint project which was initialized with an aoi and a filter: ```json { - "inputGeometries" : "https://heibox.uni-heidelberg.de/f/7a61e549b6/?dl=1", - "project" : { - "image" : "http://www.fragosus.com/test/Javita.jpg", - "lookFor" : "Buildings", - "name" : "Mapping to end FGM in North Monduli", - "projectDetails" : "Swipe slowly through the satellite imagery and mark anything that looks like it could be a building or village. This area has high levels of girls being subjected to FGM and child marriage.", - "verificationCount" : "3" + "createdBy" : "Sample Admin", + "filter" : "building=* and geometry:polygon", + "geometry" : { + "type": "FeatureCollection", + "features": [ + { + "type": "Feature", + "properties": {}, + "geometry": { + "type": "Polygon", + "coordinates": [[[9.18032169342041, 48.790552471542284],[9.187102317810059,48.790552471542284],[9.187102317810059,48.79407236257656],[9.18032169342041,48.79407236257656],[9.18032169342041,48.790552471542284]]]} + } + ] }, + "groupSize" : 25, + "lookFor": "Buildings", + "image": "http://www.fragosus.com/test/Javita.jpg", + "projectDetails": "This is a template for a GeoJSON AOI project. We use Bing as the tile server.", + "inputType" : "aoi_file", + "name" : "Test Footprint GeoJSON AOI", + "projectTopic" : "Test Footprint GeoJSON AOI", "projectType" : 2, - "tileServer" : "bing" + "verificationNumber": 3, + "tileServer" : { + "credits" : "© 2019 Microsoft Corporation, Earthstar Geographics SIO", + "name" : "bing", + "url" : "", + "wmtsLayerName" : "" + } } ``` - -The `tileserver` attribute can have the following values: `bing`, `custom`. If a custom tileserver is chosen, you need to provide a `custom_tileserver_url` attribute which links to a TMS using x, y, z placeholders. - -Imports which have been imported successfully will have a `complete` attribute set to `true`. - +Examples for the other initialization options can be found in the mapswipe-backend repository at mapswipe_workers/tests/integration/fixtures/footprint/projectDrafts. ## Project structure +Project Structure example for a project which was created via HOT Tasking Manager Project ID. ```json { - "contributors" : 0, - "groupAverage" : 0, - "id" : 13564, - "image" : "http://www.fragosus.com/test/Javita.jpg", - "importKey" : "-LNOgRd0szBM2HJBX27B", - "info" : { - "api_key" : "your_bing_api_key", - "group_size" : 50, - "input_geometries_file" : "data/valid_geometries_13564.geojson", - "tileserver" : "bing" - }, + "TMId" : "11193", + "contributorCount" : 1, + "created" : "2021-12-10T18:05:26.090515Z", + "createdBy" : "X0zTSyvY0khDfRwc99aQfIjTEPK2", + "filter" : "building=* and geometry:polygon", + "groupMaxSize" : 0, + "groupSize" : 30, + "image" : "https://firebasestorage.googleapis.com/v0/b/dev-mapswipe.appspot.com/o/projectImages%2Fimage.jpeg?alt=media", + "inputType" : "TMId", "isFeatured" : false, "lookFor" : "Buildings", - "name" : "Mapping to end FGM in North Monduli", + "name" : "OSM Building Validation - Indonesia (1)\nAmerican Red Cross", "progress" : 0, - "projectDetails" : "Swipe slowly through the satellite imagery and mark anything that looks like it could be a building or village. This area has high levels of girls being subjected to FGM and child marriage.", + "projectDetails" : "The Red Cross Climate Centre, Indonesian Red Cross (Palang Merah Indonesia/PMI), IFRC, British Red Cross and Australian Red Cross are implementing a programme where the data contributed will be used by the Red Cross to assist in forecasting future disaster impacts, by knowing in advance what is likely to be impacted and its exposure and vulnerability. The information will help implementation of early action activities to take place before a disaster strikes, contributing to reduce risk, prepare for effective response and ultimately to strengthen community resilience.", + "projectId" : "-Mq_IVluLteQRS75gWej", + "projectNumber" : "1", + "projectRegion" : "Indonesia", + "projectTopic" : "OSM Building Validation", "projectType" : 2, - "state" : 3, - "verificationCount" : 3 + "requestingOrganisation" : "American Red Cross", + "requiredResults" : 286302, + "resultCount" : 0, + "status" : "private_active", + "teamId" : "-Mq_EQlzqmYytCspuFSq", + "tileServer" : { + "apiKey" : "ca613e76-811f-46e7-9e1d-84f6795441c2", + "credits" : "© 2019 Maxar", + "name" : "maxar_premium", + "url" : "https://services.digitalglobe.com/earthservice/tmsaccess/tms/1.0.0/DigitalGlobe%3AImageryTileService@EPSG%3A3857@jpg/{z}/{x}/{y}.jpg?connectId={key}" + }, + "tutorialId" : "tutorial_-MO3ky5z--RY8PC1lONa", + "verificationNumber" : 3 } ``` ## Group structure -``` +The footprint groups follow the standard group structure. +```json { - "completedCount" : 0, - "count" : 50, - "id" : 100, - "neededCount" : 3, - "project_id" : 13564, - "tasks" : {...} + "finishedCount" : 0, + "groupId" : "g100", + "numberOfTasks" : 30, + "progress" : 0, + "projectId" : "-Mq_FxTdV2QJHsxQcvFk", + "requiredCount" : 3 } - ``` - ## Task structure +| Parameter | Description | +|-------------------------------------|----------------------------------------------------------------------------------------| +| *Project Type Specific Information* | | +| **GeoJSON** | Each task has a polygon geometry, which usually outlines a building or another object. | + ```json { "feature_id" : 0, @@ -77,22 +114,10 @@ Imports which have been imported successfully will have a `complete` attribute s "type" : "Polygon" }, "id" : "13564_100_0", - "project_id" : 13564 + "properties": "feature_geometries, e.g. attributes from osm" } ``` ## Result Structure -``` -{"OzDyJJ8su8TQtctYVIg9w5tplaX2": - {"data": - {"device":"6a39e920622c4fbb", - "id":"13555_105_298", - "item":"Buildings", - "projectId":13555, - "result":1, - "timestamp":1544613097198, - "user":"OzDyJJ8su8TQtctYVIg9w5tplaX2" - } - } -} -``` + +The Result for a footprint project are explicitly given via the "yes", "no" and "not sure" buttons. \ No newline at end of file diff --git a/docs/source/project_type.md b/docs/source/project_type.md index ce3e31e57..b650d07fa 100644 --- a/docs/source/project_type.md +++ b/docs/source/project_type.md @@ -10,109 +10,140 @@ The MapSwipe back end currently supports 3 **project types**. Each project type | Name | ID | Description | Screenshot | | ---- | -- | ----------- | ---------- | -| BuildArea | 1 | A 6 squares layout is used for this project type. By tapping you can classify a tile of satellite imagery as *yes*, *maybe* or *bad_imagery*. Project managers can define which objects to look for, e.g. "buildings". Furthermore, they can specify the tile server of the background satellite imagery, e.g. "bing" or a custom tile server. | | -| Footprint | 2 | add description. | | -| ChangeDetection | 3 | add description. | | +| [BuildArea](project_type-buildArea.html) | 1 | A 6 squares layout is used for this project type. By tapping you can classify a tile of satellite imagery as *yes*, *maybe* or *bad_imagery*. Project managers can define which objects to look for, e.g. "buildings". Furthermore, they can specify the tile server of the background satellite imagery, e.g. "bing" or a custom tile server. | | +| [Footprint](project_type-footprint.html) | 2 | An image with a footprint overlay. The question is whether this footprint is correctly approximating a structure on the shown image, which can be answered with *yes*, *no* or *Not sure*. Additionally, a button is shown which hides the footprint overlay. | | +| [ChangeDetection](project_type-changeDetection.html) | 3 | Two images are shown, the upper picture shows a scene before e.g. a disaster, while the lower picture shows the scene afterwards. By tapping you can classify the scene as *yes*, *maybe* or *bad_imagery*. Project managers can define which objects to look for, e.g. "buildings". Furthermore, they can specify the tile server of the background satellite imagery, e.g. "bing" or a custom tile server. | | ## Data Model -This way of formulating the overall crowdsourcing challenge and it's sub components shapes the **data model** we use. The data model is depicted in *Figure 1* and consists of the following parts: +This way of formulating the overall crowdsourcing challenge and it's subcomponents shapes the **data model** we use. +The data model is depicted in *Figure 1* and consists of the following parts: * project drafts * projects * groups * tasks * results -* solutions * users +* announcement (can be set manually in firebase -> banner at the top of the app) +| | -| | +Below you can see the structure on the example of a Build Area project. The project manager supplies a bounding polygon, +which is then divided into multiple groups, which are in turn divided into multiple tasks. +Results are always bound to a task and what a result means differs by project type. -### Project Drafts -After project managers defined their mapping challenges in the very first step, they can generate **project drafts** through the manager dashboard. The project drafts contain all information about your mapping challenge that you need to initialize a project in MapSwipe. For instance, the project draft defines which area you want to get mapped and how many users should work on each task. +| -| Parameter | Description | -| --- | --- | -| _Basic Information_ | These parameters are the same across all project types. | -| **Name** | The name of your project (25 chars max) | -| **Look For** | What should the users look for (e.g. buildings, cars, trees)? (15 chars max). | -| **Project Type** | The type of your mapping challenge. | -| **Direct Image Link** | An url to an image. Make sure you have the rights to use this image. It should end with .jpg or .png. | -| **Project Details** | The description for your project. (3-5 sentences). | -| **Verification Number** | How many people do you want to see every tile before you consider it finished? (default is 3 - more is recommended for harder tasks, but this will also make project take longer) | -| **Group Size** | How big should a mapping session be? Group size refers to the number of tasks per mapping session. | -| _Project Type Specific Information_ | There will be varying parameters defined by the individual project types. You can find this information at the page for each project type. | +As a project manager you have to care about the **Project Drafts** only. The information you provide through the **Manager Dashboard** will be used to set up your project. You should provide the following information. -### Projects -The **project** holds all information provided by the project drafts, but adds additional information which are needed for the MapSwipe app such as progress and number of users who contributed. A project consists of several groups. +### Project Drafts +The project drafts contain all information needed to set up your project. +Only MapSwipe user accounts with dedicated **project manager role** can create projects. +Make sure to get the rights before submitting project drafts. + +After project managers defined their mapping challenges in the very first step, they can generate **project drafts** through the manager dashboard. +The project drafts contain all information about your mapping challenge that you need to initialize a project in MapSwipe. +For instance, the project draft defines which area you want to get mapped and how many users should work on each task. + +| Parameter | Description | +|-------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| _Basic Information_ | These parameters are the same across all project types. | +| **Name** | The name of your project (25 chars max) | +| **Look For** | What should the users look for (e.g. buildings, cars, trees)? (15 chars max). | +| **Project Type** | The type of your mapping challenge. | +| **Direct Image Link** | An url to an image. Make sure you have the rights to use this image. It should end with .jpg or .png. | +| **Project Details** | The description for your project. (3-5 sentences). | +| **Verification Number** | How many people do you want to see every tile before you consider it finished? (default is 3 - more is recommended for harder tasks, but this will also make project take longer) | +| **Group Size** | How big should a mapping session be? Group size refers to the number of tasks per mapping session. | +| _Project Type Specific Information_ | There will be varying parameters defined by the individual project types. You can find this information at the page for each project type. | -| Parameter | Description | -| --- | --- | -| *Basic Information* | -| **Name** | The name of your project (25 chars max) | -| **Look For** | What should the users look for (e.g. buildings, cars, trees)? (15 chars max). | -| **Project Type** | Is `1` for all Build Area projects. | -| **Direct Image Link** | An url to an image. Make sure you have the rights to use this image. It should end with .jpg or .png. | -| **Project Details** | The description for your project. (3-5 sentences). | -| **Verification Number** | How many people do you want to see every tile before you consider it finished? (default is 3 - more is recommended for harder tasks, but this will also make project take longer) | -| **Group Size** | How big should a mapping session be? Group size refers to the number of tasks per mapping session. | -| **progress** | | -| **isFeatured** | | -| **projectId** | | -| **contributorCount** | | -| **resultCount** | | -| **numberOfTasks** | | -| **status** | | -| *Project Type Specific Information* | There will be varying parameters defined by the individual project types. You can find this information at the page for each project type. | +### Projects +The **project** holds all information provided by the project drafts, +but adds additional information which are needed for the MapSwipe app such as progress and number of users who contributed. +A project consists of several groups. + +| Parameter | Description | +|-------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------| +| *Basic Information* | | +| **Name** | The name of your project (25 chars max) | +| **Look For** | What should the users look for (e.g. buildings, cars, trees)? (15 chars max). | +| **Project Type** | Is `1` for all Build Area projects. | +| **Direct Image Link** | An url to an image. Make sure you have the rights to use this image. It should end with .jpg or .png. | +| **Project Details** | The description for your project. (3-5 sentences). | +| **Verification Number** | How many people do you want to see every tile before you consider it finished? (default is 3 - more is recommended for harder tasks, but this will also make project take longer) | +| **Group Size** | How big should a mapping session be? Group size refers to the number of tasks per mapping session. | +| **progress** | | +| **isFeatured** | If true the project will appear bigger in the app. | +| **projectId** | ID of the project. | +| **contributorCount** | How many volunteers contributed. | +| **resultCount** | | +| **numberOfTasks** | Number of tasks in project. | +| **status** | Active vs inactive. | +| *Project Type Specific Information* | There will be varying parameters defined by the individual project types. You can find this information at the page for each project type. | ### Groups -The **groups** are an intermediary between projects and tasks. Each group belongs to a single project and consists of several tasks. Groups are the key to distribute tasks to MapSwipe users in a way that we can ensure that everything gets mapped as often as required in an efficient manner. - -| Parameter | Description | -| --- | --- | -| *Basic Information* | | -| groupId | | -| numberOfTasks | | -| progress | | -| projectId | | -| finishedCount | | -| requiredCount | | -| *Project Type Specific Information* | There will be varying parameters defined by the individual project types. You can find this information at the page for each project type. | +The **groups** are an intermediary between projects and tasks. +Each group belongs to a single project and consists of several tasks. + +Single MapSwipe projects can contain up to several hundred thousand tasks. +This can pose a challenge to fast and performant communication between clients and server if many volunteers contribute data at the same time. +Therefore, groups have been introduced to reduce the amount of client requests on the backend server. + +Groups consists of several tasks, that will be shown to the user in one mapping session. +They are the key to distribute tasks to MapSwipe users in a way that we can ensure that everything gets mapped as often as required in an efficient manner. + +| Parameter | Description | +|-------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| *Basic Information* | | +| groupId | ID of the group. | +| numberOfTasks | How many tasks are in each group. | +| projectId | ID of the project the group belongs to. | +| finishedCount | Once a group has been completely mapped by a volunteer the completed count of the corresponding group will be raised by one. The completed count of the group is used to assess the overall progress of each project.
For doing so the completed count is compared to the redundancy required (see Table 2). During the mapping process groups will be served in ascending completed count order. Thus, groups with low completed count will be served first | +| requiredCount | How many volunteers have to map a group. | +| *Project Type Specific Information* | There will be varying parameters defined by the individual project types. You can find this information at the page for each project type. | ### Tasks -The **tasks** are the smallest component in our data model. Each task formulates an easy and quick to solve mapping challenge. In many cases this challenge can be put into a simple question, e.g. *Can you see a building in this satellite imagery tile*. Tasks always belong to a specific group and project. - -| Parameter | Description | -| --- | --- | -| *Basic Information* | | -| groupId | | -| projectId | | -| taskId | | +The **tasks** are the smallest component in our data model. +Each task formulates an easy and quick to solve mapping challenge. +In many cases this challenge can be put into a simple question, e.g. *Can you see a building in this satellite imagery tile*. +Tasks always belong to a specific group and project. + +Tasks are usually gzip compressed on firebase to save space. That is why this information is not readable by humans in firebase. + +| Parameter | Description | +|-------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------| +| *Basic Information* | | +| **taskId** | ID of the task. | +| **groupId** | ID of the group the task belongs to. | +| **projectId** | ID of the project the task belongs to. | | *Project Type Specific Information* | There will be varying parameters defined by the individual project types. You can find this information at the page for each project type. | ### Results -The **results** hold the information you wanted in the very beginning. For each task you will receive several results by different users. A result is the simple answer to your initial question. For instance, it's a simple "yes" to the question "can you see a building in this satellite imagery tile". +The **results** hold the information you wanted in the very beginning. +For each task you will receive several results by different users. +A result is the simple answer to your initial question. +For instance, it's a simple "yes" to the question "can you see a building in this satellite imagery tile". | Parameter | Description | -| --- | --- | -| timestamp | | -| startTime | | -| endTime | | -| result | | | +|-----------|-------------| +| timestamp | | +| startTime | | +| endTime | | +| result | | -timestamptimestampstartTimetimestampendTimetimestamp - ### Users -The **users** provide the results to your tasks. They are the key to solve your mapping challenge. For each user we generate mapping related statistics, e.g. the number of projects a user has been worked on. - -| Parameter | Description | -| --- | --- | -| created | | -| projectContributionsCount | | -| groupContributionCount | | -| taskContributionCount | | -| timeSpentMapping | | -| username | | +The **users** provide the results to your tasks. +They are the key to solve your mapping challenge. +For each user we generate mapping related statistics, e.g. the number of projects a user has been worked on. + +| Parameter | Description | +|---------------------------|-------------| +| created | | +| projectContributionsCount | | +| groupContributionCount | | +| taskContributionCount | | +| timeSpentMapping | | +| username | | diff --git a/docs/source/testing.md b/docs/source/testing.md index 6c2196e8c..01ac47e38 100644 --- a/docs/source/testing.md +++ b/docs/source/testing.md @@ -1,40 +1,9 @@ -# Operational Tests -The operational tests should check the usual workflows we are having with MapSwipe: -* project creation -* mapping -* firebase to postgres transfer of results -* generate statistics based on postgres tables +# Testing -To run all tests you can simply run `bash test_00_main.sh`. There are still some uncovered bits, you find them in #158. Once you created some projects, you also run the `test_02_mapping.py` script several times. Make sure to clean up after you finish testing. +## Tests +* run tests locally during development -## Test Project Creation -* `test_01_create_projects.py` -* creates a user in firebase with project manager credentials -* login as project manager through firebase REST api -* set `projectDrafts` in firebase through firebase REST api, authenticated as project manager -* save project_ids to disk (to be able to delete those specific projects at a later stage of testing) -* run mapswipe workers project creation workflow through `_run_create_projects()` -* delete project manager in firebase - -## Test Mapping -* `test_02_mapping.py` -* create X users (without project manager credentials) -* save users to disk (to be able to delete those specific users at a later stage of testing) -* get projects which have been created by `test_01_create_projects.py` -* get firebase projects, groups and results before mapping and save to disk -* for each user and project set random results for X groups in firebase through REST api, authenticated as normal user -* get firebase projects, groups and results after mapping and save to disk -* compare firebase before and after data - -## Test Firebase to Postgres -* `test_03_firebase_to_postgres.py` -* copy new users from firebase to postgres -* copy all results from firebase to postgres - -## Generate Stats -* `test_04_generate_stats.py` -* generate csv files for all projects and users for which we got results since a timestamp defined in `last_update.txt` - -## Other -* we don't have broader checks of firebase database rules -* some database rules are checked indirectly, e.g. firebase read, write rules through REST api get, set requests during `test_02_mapping.py` +``` +python -m unittest discover --verbose --start-directory mapswipe_workers/tests/unittests/ +python -m unittest discover --verbose --start-directory mapswipe_workers/tests/integration/ +``` diff --git a/docs/source/testing_and_travis.md b/docs/source/testing_and_travis.md deleted file mode 100644 index 7c6de697e..000000000 --- a/docs/source/testing_and_travis.md +++ /dev/null @@ -1,20 +0,0 @@ -# Testing and Travis - -## Tests -* run tests locally during development - -``` -python -m unittest discover --verbose --start-directory mapswipe_workers/tests/unittests/ -python -m unittest discover --verbose --start-directory mapswipe_workers/tests/integration/ -``` - - -## Travis -* set environment variables in travis -* travis then sets up the docker containers -* test are run inside the mapswipe_workers docker container - -``` -docker-compose run mapswipe_workers python -m unittest discover --verbose --start-directory tests/unittests/ -docker-compose run mapswipe_workers python -m unittest discover --verbose --start-directory tests/integration/ -``` \ No newline at end of file diff --git a/docs/source/updating.md b/docs/source/updating.md deleted file mode 100644 index 623937431..000000000 --- a/docs/source/updating.md +++ /dev/null @@ -1,16 +0,0 @@ -# Updating - -## Mapswipe Workers - -``` -git pull -docker-compose up --build -d --force-recreate mapswipe_workers -``` - - -## Database Rules and Functions - -``` -git pull -docker-compose up --build -d --force-recreate firebase_deploy -``` \ No newline at end of file