Release v0.1.12rc #537
Merged
Release v0.1.12rc #537
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add tests for v0.1.10 backward compatibility
Merge master back to dev
Use pypi Token to release
Streamlining the release steps
Automatically update backward test versions
Experiment branching in particular was difficult to debug based on logs.
Why: The LPI requires the objective. If we don't filter the non-completed trials, the computation fails. How: Filter out non-completed trials in the plotting function. The analysis function assumes trials with objectives, otherwise it would required passing data frames including status.
Why:
%s format is harder to read than f'{}' format.
Add debugging verbosity
Ignore non-completed trials when computing LPI
Test that precision isn't messed up by linearization
Why: Hyperband and ASHA would fail if they are passed trials that they did not sample. How: Save sampled ids in algos and only observe points that were previously sampled, otherwise ignore. This way we could still salvages points from EVC if the current algo sample points that are in EVC (with same fidelity), otherwise it ignores the points.
Support EVC in all algos
Why: The code version was not infered properly when `orion hunt` is called with only experiment name, not passing the user script cmdline. How: Re-infer code version based on merge of old metadata and new metadata.
Add more logging when branching
Why: If the cmdline call is empty (no script), the metadata won't contain the user script and thus code version will not be inferred. We fetching from DB to optimize, the code version should be re-inferred to make sure there was no code change between executions.
Add missing plotting doc in API section
Detect code change when no CMDLINE passed
Why: Changing the version of Oríon could impact the behavior of the HPO, therefore we should branch to avoid corrupting results. How: Add Conflict, Resolution and Adapter to detect and handle changes of orion version.
…_change Branch if Oríon version changes
Why: The experiment view is useful to query the database and make plots, but sometimes we want to add new trials or make modifications to trials (like status) without executing the experiments. Making such edit should not require building the experiment with checks for branching, as there is no intention in executing trials. There should be read/write/execution rights instead of just providing a view. Read can: Make any calls that only require reading the DB. No writes on DB Read/Write can: Make any calls that read or write the DB as long as it is not updating trial results. Also the producer should not be accessible in Read/Write mode, the algorithm should not be executed in an unchecked environment. Read/Write/Exec can: Do everything. How: Remove ExperimentView and add modes directly in Experiment. All methods in Experiment now check it access rights of the experiment allows them to execute.
Why: More coherent with build_experiment/get_experiment of the client.
Add Read/Write/Execution modes for Experiment
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🚀 Features
🐛 Bug Fixes
📜 Documentation
🧰 Maintenance