Merge pull request #594 from Epistimio/release-v0.1.14rc

Release candidate v0.1.14

Author: bouthilx

.github/release-drafter.yml

@@ -4,6 +4,8 @@ categories:
   - title: '🚀 Features'
     labels:
       - 'feature'
+  - title: '🏗  Enhancements'
+    labels:
       - 'enhancement'
   - title: '🐛 Bug Fixes'
     labels:

.github/workflows/build.yml

@@ -47,12 +47,6 @@ jobs:
       uses: actions/setup-python@v2
       with:
         python-version: ${{ matrix.python-version }}
-    - name: Setup MongoDB
-      uses: supercharge/[email protected]
-      with:
-          mongodb-version: 4.2
-    - name: Configure MongoDB
-      run: mongo orion_test --eval 'db.createUser({user:"user",pwd:"pass",roles:["readWrite"]});'
     - name: Install dependencies
       run: |
           python -m pip install --upgrade pip
@@ -67,6 +61,35 @@ jobs:
         env_vars: PLATFORM,PYTHON
         name: codecov-umbrella
         fail_ci_if_error: false
+  mongodb:
+    needs: pretest
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v1
+      - name: Set up Python 3.7
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.7
+      - name: Setup MongoDB
+        uses: supercharge/[email protected]
+        with:
+          mongodb-version: 4.2
+      - name: Configure MongoDB
+        run: mongo orion_test --eval 'db.createUser({user:"user",pwd:"pass",roles:["readWrite"]});'
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install tox tox-gh-actions
+      - name: Test with tox
+        run: tox -e mongodb
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v1
+        with:
+          file: ./coverage.xml
+          flags: backward
+          env_vars: PLATFORM,PYTHON
+          name: codecov-umbrella
+          fail_ci_if_error: false
   backward-compatibility:
     needs: pretest
     runs-on: ubuntu-latest

README.rst

@@ -124,7 +124,7 @@ If you use Oríon for published work, please cite our work using the following b
      month        = nov,
      year         = 2020,
      publisher    = {Zenodo},
-     version      = {v0.1.11},
+     version      = {v0.1.14},
      doi          = {10.5281/zenodo.3478592},
      url          = {https://doi.org/10.5281/zenodo.3478592}
    }

ROADMAP.md

@@ -3,41 +3,16 @@ Last update December 3rd, 2020
 
 ## Next releases - Short-Term
 
-### v0.1.12
+### v0.1.15
 
 #### Quick release for bug fixes
 
-### v0.1.13
-
-#### Default database backend
-PickledDB will become the default database backend instead of MongoDB. This initiative reduces
-initial configuration, allowing users to get started faster.
-
 ### v0.2
-#### Journal Protocol Plugins
-Offering:
-- No need to setup DB, can use one's existing backend
-- Can re-use tools provided by backend for visualizations, etc.
-#### Python API
-
-Traditional `suggest`/`observe` interface
-
-```python
-experiment = orion.client.register(
-    experiment='fct-dummy',
-    x='loguniform(0.1, 1)', verbose=True, message='running trial {trial.hash_name}')
-
-trial = experiment.suggest()
-results = dummy(**trial.arguments)
-experiment.observe(trial, results)
-```
-
-### Algorithms
-Introducing new algorithms: [TPE](https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf), [HyperBand](https://arxiv.org/abs/1603.06560)
+#### Native Multi-Processing support
 
-## Next releases - Mid-Term
+Added support for parallelism with auto-scaling. No more need to launch multiple workers 
+(though still supported) for parallelism, simply pass `--n-workers` or `workon(n_workers)`.
 
-### v0.3
 #### Generic `Optimizer` interface supporting various types of algorithms
 
 Change interface to support trial object instead of curated lists. This is necessary to support algorithms such as PBT.

codecov.yml

@@ -1,3 +1,6 @@
+ignore:
+  - "tests/**/*"
+
 coverage:
   precision: 2
   round: down

conda/meta.yaml

@@ -27,6 +27,8 @@ requirements:
     - tabulate
     - AppDirs
     - plotly
+    - python-kaleido
+    - requests
     - pandas
     - falcon
     - gunicorn

docs/src/index.rst

@@ -19,6 +19,7 @@
    user/algorithms
    user/searchspace
    user/monitoring
+   user/viz
    user/benchmark
    user/evc
    user/storage

docs/src/install/database.rst

@@ -10,8 +10,8 @@ to enable fast, seamless, and scalable storage integration.
 Out of the box, we support three database backend:
 
 #. :ref:`EphemeralDB Config`, an in-memory database
-#. :ref:`PickledDB Config`, a file-based database
-#. :ref:`MongoDB Config`, a document-oriented database (default)
+#. :ref:`PickledDB Config`, a file-based database (default)
+#. :ref:`MongoDB Config`, a document-oriented database
 
 In this document, we'll review the different methods by which you can configure which database Oríon
 will use during its execution. This page also contains the :ref:`installation instructions for
@@ -28,10 +28,10 @@ This will create a yaml file of the following format.
 
    .. code-block:: yaml
 
-      database:
-        type: 'mongodb'
-        name: 'orion_test'
-        host: 'mongodb://user:pass@localhost'
+      storage:
+        database:
+          type: 'pickleddb'
+          host: '/path/to/a/file.pkl'
 
 The file is typically located at ``$HOME/.config/orion.core/orion_config.yaml`` but it may differ
 based on your operating system.
@@ -40,10 +40,8 @@ The second way of configuring the database backend is to use environment variabl
 
    .. code-block:: sh
 
-       ORION_DB_ADDRESS=mongodb://user:pass@localhost
-       ORION_DB_NAME=orion_test
-       ORION_DB_TYPE=MongoDB
-       ORION_DB_PORT=27017
+       ORION_DB_ADDRESS=/path/to/a/file.pkl
+       ORION_DB_TYPE=PickledDB
 
 Note that both configuration methods can be used together, environment variables that are set will
 overwrite the corresponding values in the global configuration. This is useful if you need to define
@@ -79,37 +77,27 @@ stage.
    $ orion db test
 
    Check for a configuration inside the default paths...
-       {'type': 'mongodb', 'name': 'mydb', 'host': 'localhost'}
+       {'name': 'orion', 'type': 'pickleddb', 'host': '', 'port': 27017}
+   Success
+   Check if configuration file has valid database configuration...
+       {'name': 'orion', 'type': 'pickleddb', 'host': '', 'port': 27017}
    Success
-
-   Check for a configuration inside the environment variables... Skipping
-   No environment variables found.
-
-   Check if configuration file has valid database configuration... Skipping
-   Missing configuration file.
 
    [...]
 
-In the last example, we can observe that the last two tests were skipped because there were no
-environment variables or local configuration file specified.
-
-Alternatively, here's an example including all three configuration methods.
+Alternatively, here's an example including all configuration methods.
+This is with MongoDB since there are more options to play with.
 
 .. code-block:: sh
 
-   $ ORION_DB_PORT=27018
+   $ ORION_DB_NAME=test
    $ orion db test --config local.yaml
 
    Check for a configuration inside the global paths...
-       {'type': 'mongodb', 'name': 'mydb', 'host': 'localhost'}
-   Success
-
-   Check for a configuration inside the environment variables...
-       {'type': 'mongodb', 'name': 'mydb', 'host': 'localhost', 'port': '27018'}
+       {'name': 'test', 'type': 'pickleddb', 'host': '', 'port': 27017}
    Success
-
    Check if configuration file has valid database configuration...
-       {'type': 'mongodb', 'name': 'mydb', 'host': 'localhost', 'port': '27017'}
+       {'type': 'mongodb', 'host': 'localhost'}
    Success
 
    [...]
@@ -123,9 +111,9 @@ that will be used and then prints the instance created to confirm the database t
 
    [...]
 
-   Using configuration: {'type': 'mongodb', 'name': 'mydb', 'host': 'localhost'}
+   Using configuration: {'name': 'orion', 'type': 'pickleddb', 'host': '', 'port': 27017}
    Check if database of specified type can be created... Success
-   DB instance <orion.core.io.database.mongodb.MongoDB object at 0x7f86d70067f0>
+   DB instance <orion.core.io.database.pickleddb.PickledDB object at 0x7f86d70067f0>
 
    [...]
 
@@ -234,7 +222,7 @@ To install MongoDB locally, follow the `official instructions
 <https://docs.mongodb.com/manual/administration/install-community/>`_ for your operating system.
 Alternatively, use :ref:`MongoDB Atlas <mongodb-atlas>` to create a database in the cloud.
 
-Once MondoDB is installed, create the database using:
+Once MongoDB is installed, create the database using:
 
 .. code-block:: sh
 

docs/src/tutorials/pytorch-mnist.rst

@@ -7,8 +7,7 @@ This is a simple tutorial on running hyperparameter search with Oríon on Pytorc
 Installation and setup
 ======================
 
-Make sure Oríon is installed (:doc:`/install/core`) and the database is configured
-(:doc:`/install/database`).
+Make sure Oríon is installed (:doc:`/install/core`).
 
 Then install ``pytorch`` and ``torchvision`` and clone the
 PyTorch `examples repository`_:

docs/src/tutorials/pytorch_a2c_ppo.rst

@@ -6,13 +6,6 @@ PyTorch A2C PPO ACKTR
 
     If Oríon not installed: pip install orion
 
-    If the database is not setup, you can follow the instructions here:
-    :doc:`/install/database`.
-
-    Alternatively, you can test the example without setting up a database by
-    using the option `--debug`, but note that all data gathered during an
-    execution will be lost at the end of it.
-
 
 Intro
 =====

docs/src/tutorials/pytorch_cifar.rst

@@ -6,12 +6,6 @@ PyTorch CIFAR10
 
     If Oríon not installed: pip install orion
 
-    If the database is not setup, you can follow the instructions here:
-    :doc:`/install/database`.
-
-    Alternatively, you can test the example without setting up a database by
-    using the option `--debug`, but note that all data gathered during an
-    execution will be lost at the end of it.
 
 Set up
 

docs/src/user/config.rst

@@ -88,10 +88,8 @@ Full Example of Global Configuration
 .. code-block:: yaml
 
     database:
-        host: localhost
-        name: orion
-        port: 27017
-        type: mongodb
+        host: orion_db.pkl
+        type: pickleddb
 
     experiment:
         algorithms:
@@ -136,10 +134,8 @@ Database
 .. code-block:: yaml
 
     database:
-        host: localhost
-        name: orion
-        port: 27017
-        type: mongodb
+        host: orion_db.pkl
+        type: pickleddb
 
 
 .. _config_database_name:
@@ -161,7 +157,7 @@ type
 ~~~~
 
 :Type: str
-:Default: MongoDB
+:Default: PickledDB
 :Env var: ORION_DB_TYPE
 :Description:
     Type of database. Builtin backends are ``mongodb``, ``pickleddb`` and ``ephemeraldb``.
@@ -174,10 +170,12 @@ host
 ~~~~
 
 :Type: str
-:Default: 127.0.1.1
+:Default: ""
 :Env var: ORION_DB_ADDRESS
 :Description:
-    URI for ``mongodb``, or file path for ``pickleddb``.
+    URI for ``mongodb``, or file path for ``pickleddb``.  An empty
+    string will set the value depending on the database type
+    (localhost or <user_data_dir>/orion/orion_db.pkl).