Setup: Fix docker configuration to set up evalai-ngx in dev environment(Cloud-CV#129)

Author: RishabhJain2018

README.md

@@ -35,60 +35,28 @@ Our ultimate goal is to build a centralized platform to host, participate and co
 
 Some background: Last year, the [Visual Question Answering Challenge (VQA) 2016](http://www.visualqa.org/vqa_v1_challenge.html) was hosted on some other platform, and on average evaluation would take **~10 minutes**. EvalAI hosted this year's [VQA Challenge 2017](https://evalai.cloudcv.org/featured-challenges/1/overview). This year, the dataset for the [VQA Challenge 2017](http://www.visualqa.org/challenge.html) is twice as large. Despite this, we’ve found that our parallelized backend only takes **~130 seconds** to evaluate on the whole test set VQA 2.0 dataset.
 
-## Installation Instructions
+## Development setup
 
-Setting up EvalAI-ngx on your local machine is really easy.
-Follow this guide to setup your development machine.
+Use [Docker Compose](https://docs.docker.com/compose/install/) to run all the components of EvalAI-ngx together. The steps are:
 
-Get the source code on your machine via git
-```
-git clone [email protected]:Cloud-CV/EvalAI-ngx.git
-```
-If you have not added [ssh key](https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/) to your GitHub account then get the source code by running the following command
-```
-git clone https://github.com/Cloud-CV/EvalAI-ngx
-```
-
-```
-npm install -g @angular/cli
-cd EvalAI-ngx/
-npm install
-```
-
-## Development
-
-### For Running on localhost:
-
-Run `ng serve` for a dev server. Navigate to `http://localhost:4200/`. The app will automatically reload if you change any of the source files.
-
-### Backend for localhost:
-
-Setting up EvalAI on your local machine is really easy. You can setup EvalAI using docker:
-The steps are:
-
-1. Install [docker](https://docs.docker.com/install/linux/docker-ce/ubuntu/) and [docker-compose](https://docs.docker.com/compose/install/) on your machine.
-
-2. Get the source code on to your machine via git.
+1. Get the source code on to your machine via git.
 
     ```shell
-    git clone https://github.com/Cloud-CV/EvalAI.git evalai && cd evalai
+    git clone https://github.com/Cloud-CV/EvalAI-ngx.git && cd evalai-ngx
     ```
 
-3. Build and run the Docker containers. This might take a while.
+2. Build and run the Docker containers. This might take a while. You should be able to access EvalAI at `localhost:8888`.
 
     ```
-    docker-compose up --build
+    docker-compose up
     ```
 
-4. That's it. Open web browser and hit the url [http://127.0.0.1:8888](http://127.0.0.1:8888). Three users will be created by default which are listed below -
-    
+3. That's it. Open web browser and hit the url [http://127.0.0.1:8888](http://127.0.0.1:8888). Three users will be created by default which are listed below -
+
     **SUPERUSER-** username: `admin` password: `password`  
     **HOST USER-** username: `host` password: `password`  
     **PARTICIPANT USER-** username: `participant` password: `password`
 
-If you are facing any issue during installation, please see our [common errors during installation page](https://evalai.readthedocs.io/en/latest/faq(developers).html#common-errors-during-installation).
-
-
 ### For deploying with [Surge](https://surge.sh/):
 
 Surge will automatically generate deployment link whenever a pull request passes Travis CI. 
@@ -131,22 +99,6 @@ Run `ng test` to execute the unit tests via [Karma](https://karma-runner.github.
 
 Run `ng e2e` to execute the end-to-end tests via [Protractor](http://www.protractortest.org/).
 
-### Setup using Docker
-
-You can also use Docker Compose to run all the components of EvalAI-ngx together. The steps are:
-
-1. Get the source code on to your machine via git.
-
-    ```shell
-    git clone https://github.com/Cloud-CV/EvalAI-ngx.git && cd EvalAI-ngx
-    ```
-
-2. Build and run the Docker containers. This might take a while. You should be able to access EvalAI at `localhost:8888`.
-
-    ```
-    docker-compose -f docker-compose.dev.yml up -d --build
-    ```
-
 ## The Team
 
 EvalAI-ngx is currently maintained by [Shekhar Rajak](http://s-hacker.info/), [Mayank Lunayach](https://github.com/lunayach), [Shivani Prakash Gupta](https://www.behance.net/shivaniprakash19), [Rishabh Jain](https://rishabhjain2018.github.io/) and [Deshraj Yadav](https://deshraj.github.io).

docker-compose.dev.yml renamed to docker-compose.yml

@@ -1,44 +1,42 @@
-version: "2"
+version: "3"
 services:
 
   db:
-    image: postgres
+    image: postgres:10.4
     ports:
       - "5432:5432"
 
-  rabbitmq:
-    image: rabbitmq
-    hostname: rabbitmq
+  sqs:
+    image: pakohan/elasticmq
+    hostname: sqs
     ports:
-      - "5672:5672"
-      - "15672:15672"  # here, we can access rabbitmq management plugin
+      - 9324:9324
 
   django:
     container_name: ngx_django
     hostname: django
     env_file:
-      - docker/dev.env
+      - docker/dev/docker.env
     build:
       context: ./
       dockerfile: docker/dev/django/Dockerfile
     ports:
       - "8000:8000"
-    command: ["./docker/wait-for-it.sh", "db:5432", "--", "sh", "/code/docker/dev/django/container-start.sh"]
     depends_on:
       - "db"
+      - "sqs"
 
-  submission-worker:
+  worker:
     env_file:
-      - docker/dev.env
+      - docker/dev/docker.env
     build:
       context: ./
-      dockerfile: docker/dev/rabbitmq/Dockerfile
-    command: ["./docker/wait-for-it.sh", "django:8000", "--", "python", "scripts/workers/submission_worker.py"]
+      dockerfile: docker/dev/worker/Dockerfile
     depends_on:
       - "django"
 
   angularjs:
-    container_name: ngx_angularjs
+    container_name: evalai_ngx
     hostname: angularjs
     build:
       context: ./
@@ -48,5 +46,8 @@ services:
     ports:
       - "8888:4200"
       - "4200:4200"
+    depends_on:
+      - "django"
     volumes:
-     - .:/code
+    - .:/code
+    - /code/node_modules

docker/dev/angularjs/Dockerfile

@@ -1,20 +1,18 @@
-FROM node:9.4.0
-MAINTAINER CloudCV Team
+FROM node:12.2.0
 
-RUN apt-get update -qq && apt-get install -y build-essential git curl libfontconfig
+RUN apt-get update
+RUN apt-get install -y build-essential git curl libfontconfig
 RUN apt-get install nodejs-legacy -y
-RUN apt-get install npm -y
-RUN npm cache verify
-RUN npm install -g @angular/[email protected] --unsafe
+RUN npm install -g @angular/[email protected]
 
 RUN mkdir /code
 
 # Copy codebase
 COPY . /code
 
 WORKDIR /code
-
 RUN npm install
+RUN npm audit fix
 
 EXPOSE 8888
 

docker/dev/django/Dockerfile

@@ -1,30 +1,19 @@
-FROM python:2.7
-MAINTAINER CloudCV
-ENV PYTHONUNBUFFERED 1
+FROM python:3.6.5
 
-ENV DEBIAN_FRONTEND noninteractive
+ENV PYTHONUNBUFFERED 1
 
 RUN apt-get update \
     && apt-get upgrade -y \
     && apt-get install -y
 
-RUN apt-get install -y apt-utils
-
-RUN groupadd webapps
-RUN useradd evalai -G webapps
-RUN mkdir -p /var/log/evalai/ && chown -R evalai /var/log/evalai/ && chmod -R u+rX /var/log/evalai/
-RUN mkdir -p /var/run/evalai/ && chown -R evalai /var/run/evalai/ && chmod -R u+rX /var/run/evalai/
-
 RUN mkdir /code
 
 RUN git clone https://github.com/Cloud-CV/EvalAI.git /code/
 
 WORKDIR /code
 
-RUN cp /code/settings/dev.sample.py /code/settings/dev.py
-
 RUN pip install -r requirements/dev.txt
 
-CMD ["sh", "/code/docker/dev/django/container-start.sh"]
+CMD ["./docker/wait-for-it.sh", "db:5432", "--", "sh", "/code/docker/dev/django/container-start.sh"]
 
 EXPOSE 8000

docker/dev/django/container_start.sh

@@ -0,0 +1,5 @@
+#!/bin/sh
+cd /code && \
+python manage.py migrate --noinput && \
+python manage.py seed && \
+python manage.py runserver 0.0.0.0:8000

docker/dev.env renamed to docker/dev/docker.env

@@ -1,11 +1,14 @@
+AWS_ACCESS_KEY_ID=x
+AWS_SECRET_ACCESS_KEY=x
+AWS_DEFAULT_REGION=us-east-1
+AWS_ACCOUNT_ID=x
+
 DEBUG=True
+DJANGO_SERVER=django
+DJANGO_SETTINGS_MODULE=settings.dev
 
 POSTGRES_NAME=postgres
 POSTGRES_USER=postgres
 POSTGRES_PASSWORD=postgres
 POSTGRES_HOST=db
 POSTGRES_PORT=5432
-
-DJANGO_SERVER=django
-
-RABBITMQ_HOST=rabbitmq

docker/dev/rabbitmq/Dockerfile renamed to docker/dev/worker/Dockerfile

@@ -1,5 +1,5 @@
-FROM python:2.7
-MAINTAINER CloudCV
+FROM python:3.6.5
+
 ENV PYTHONUNBUFFERED 1
 
 RUN apt-get update && apt-get install -y python python-pip python-dev libpq-dev libjpeg-dev libyaml-dev libffi-dev
@@ -10,10 +10,10 @@ RUN git clone https://github.com/Cloud-CV/EvalAI.git /code/
 
 WORKDIR /code
 
-RUN cp settings/dev.sample.py settings/dev.py
-
-RUN pip install -U cffi service_identity
-
+RUN pip install -U cffi service_identity cython==0.29 numpy==1.14.5
 RUN pip install -r requirements/dev.txt
+RUN pip install -r requirements/worker.txt
+
+ADD . /code
 
-CMD ["python", "scripts/workers/submission_worker.py"]
+CMD ["python", "-m", "scripts.workers.submission_worker"]