Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent output in orion status and orion list #342

Merged
merged 6 commits into from
Feb 28, 2020

Conversation

donglinjy
Copy link
Collaborator

@donglinjy donglinjy commented Feb 26, 2020

After run pytest, there are some invalid expriments and run orion status will get below error but orion list works well to return only the valid experiments, so I change this a little bit to make them consistent at this moment.

$ orion status
Traceback (most recent call last):
  File "/Users/donglin/App/anaconda2/envs/orion/bin/orion", line 11, in <module>
    load_entry_point('orion', 'console_scripts', 'orion')()
  File "/Users/donglin/gitspace/orion/src/orion/core/cli/__init__.py", line 39, in main
    return orion_parser.execute(argv)
  File "/Users/donglin/gitspace/orion/src/orion/core/cli/base.py", line 74, in execute
    function(args)
  File "/Users/donglin/gitspace/orion/src/orion/core/cli/status.py", line 56, in main
    experiments = get_experiments(args)
  File "/Users/donglin/gitspace/orion/src/orion/core/cli/status.py", line 113, in get_experiments
    for exp in experiments]
  File "/Users/donglin/gitspace/orion/src/orion/core/cli/status.py", line 113, in <listcomp>
    for exp in experiments]
  File "/Users/donglin/gitspace/orion/src/orion/core/io/experiment_builder.py", line 255, in build_view
    experiment = create_experiment(**db_config)
  File "/Users/donglin/gitspace/orion/src/orion/core/io/experiment_builder.py", line 289, in create_experiment
    experiment.algorithms = _instantiate_algo(experiment.space, kwargs.get('algorithms'))
  File "/Users/donglin/gitspace/orion/src/orion/core/io/experiment_builder.py", line 374, in _instantiate_algo
    return PrimaryAlgo(space, config)
  File "/Users/donglin/gitspace/orion/src/orion/core/worker/primary_algo.py", line 40, in __init__
    super(PrimaryAlgo, self).__init__(space, algorithm=algorithm_config)
  File "/Users/donglin/gitspace/orion/src/orion/algo/base.py", line 111, in __init__
    space, **subalgo_kwargs)
  File "/Users/donglin/gitspace/orion/src/orion/core/utils/__init__.py", line 159, in __call__
    raise NotImplementedError(error)
NotImplementedError: Could not find implementation of BaseAlgorithm, type = 'dumbalgo'
Currently, there is an implementation for types:
['asha', 'random']

There may be some other to-do related to this issue

  1. Instead of print the whole error stack, may be can consider to only give the meaningful error message.
  2. Not sure the purpose of 'clean_db' in our test set, but seems it won't clean but insert more.
@pytest.fixture()
def clean_db(database, exp_config):
    """Clean insert example experiment entries to collections."""
    database.experiments.drop()
    database.experiments.insert_many(exp_config[0])
    database.lying_trials.drop()
    database.trials.drop()
    database.trials.insert_many(exp_config[1])
    database.workers.drop()
    database.workers.insert_many(exp_config[2])
    database.resources.drop()
    database.resources.insert_many(exp_config[3])

@bouthilx

@donglinjy donglinjy closed this Feb 27, 2020
@donglinjy donglinjy reopened this Feb 27, 2020
@bouthilx
Copy link
Member

  1. Instead of print the whole error stack, may be can consider to only give the meaningful error message.

Agreed!

  1. Not sure the purpose of 'clean_db' in our test set, but seems it won't clean but insert more.

The name may be misleading. The goal is to get db reset to a fresh initial state for the tests. (with some initial data)

@codecov-io
Copy link

codecov-io commented Feb 27, 2020

Codecov Report

Merging #342 into develop will not change coverage.
The diff coverage is 85.79%.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop     #342   +/-   ##
========================================
  Coverage    43.57%   43.57%           
========================================
  Files           64       64           
  Lines        11343    11343           
  Branches       271      271           
========================================
  Hits          4943     4943           
  Misses        6375     6375           
  Partials        25       25
Impacted Files Coverage Δ
...sts/unittests/core/io/test_orion_cmdline_parser.py 19.35% <ø> (ø) ⬆️
tests/unittests/core/cli/test_info.py 20.22% <100%> (ø) ⬆️
src/orion/core/__init__.py 95.12% <100%> (ø) ⬆️
tests/unittests/client/test_client.py 100% <100%> (ø) ⬆️
tests/functional/client/test_cli_client.py 100% <100%> (ø) ⬆️
tests/unittests/core/io/test_cmdline_parser.py 22.91% <17.85%> (ø) ⬆️
tests/unittests/storage/test_legacy.py 36.66% <33.33%> (ø) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 007e7fc...410f51a. Read the comment docs.

@donglinjy donglinjy changed the title Error in orion status cmd Inconsistent output in orion status and orion list Feb 27, 2020
@bouthilx
Copy link
Member

Copy link
Member

@bouthilx bouthilx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @donglinjy !

@bouthilx bouthilx merged commit 6badd44 into Epistimio:develop Feb 28, 2020
This was referenced Jun 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants