Restoring past trials from storage fails when experiment has no tunables #884
Assignees
Labels
Comments
bpkroth
pushed a commit
that referenced
this issue
Dec 5, 2024
…bles (#889) # Pull Request ## Fix storage API failing to return DF for experiments without any tunables Addresses #884 --- ## Description When an experiment is run without any tunables (benchmarking with default parameters), the storage API fails to return a dataframe of the results. Example error: ``` >>> exp = storage.experiments["eujingchua-bench-57-02"] >>> exp.results_df Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/workspaces/MySQL-Autotuning/MLOS/mlos_bench/mlos_bench/storage/sql/experiment_data.py", line 212, in results_df return common.get_results_df(self._engine, self._schema, self._experiment_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspaces/MySQL-Autotuning/MLOS/mlos_bench/mlos_bench/storage/sql/common.py", line 183, in get_results_df ExperimentData.CONFIG_COLUMN_PREFIX + row.param_id, ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~ TypeError: can only concatenate str (not "NoneType") to str ``` Going to the relevant line and inserting a breakpoint, the relevant columns are `["trial_id", "tunable_config_id", "param", "value"]`. We can see the data is full of NULLs / Nones ``` (Pdb) results = configs.fetchall() (Pdb) results [(1, 6541, None, None), (2, 6541, None, None), (3, 6541, None, None), (4, 6541, None, None), (5, 6541, None, None), (6, 6541, None, None), (7, 6541, None, None), (8, 6541, None, None), (9, 6541, None, None), (10, 6541, None, None), (11, 6541, None, None), (12, 6541, None, None), (13, 6541, None, None), (14, 6541, None, None), (15, 6541, None, None), (16, 6541, None, None), (17, 6541, None, None), (18, 6541, None, None), (19, 6541, None, None), (20, 6541, None, None), (21, 6541, None, None), (22, 6541, None, None), (23, 6541, None, None), (24, 6541, None, None), (25, 6541, None, None), (26, 6541, None, None), (27, 6541, None, None), (28, 6541, None, None), (29, 6541, None, None), (30, 6541, None, None), (31, 6541, None, None), (32, 6541, None, None), (33, 6541, None, None), (34, 6541, None, None), (35, 6541, None, None), (36, 6541, None, None), (37, 6541, None, None), (38, 6541, None, None), (39, 6541, None, None), (40, 6541, None, None), (41, 6541, None, None), (42, 6541, None, None), (43, 6541, None, None), (44, 6541, None, None), (45, 6541, None, None), (46, 6541, None, None), (47, 6541, None, None), (48, 6541, None, None), (49, 6541, None, None), (50, 6541, None, None), (51, 6541, None, None), (52, 6541, None, None), (53, 6541, None, None), (54, 6541, None, None), (55, 6541, None, None), (56, 6541, None, None), (57, 6541, None, None), (58, 6541, None, None), (59, 6541, None, None), (60, 6541, None, None), (61, 6541, None, None), (62, 6541, None, None), (63, 6541, None, None), (64, 6541, None, None), (65, 6541, None, None), (66, 6541, None, None), (67, 6541, None, None), (68, 6541, None, None), (69, 6541, None, None), (70, 6541, None, None), (71, 6541, None, None), (72, 6541, None, None), (73, 6541, None, None), (74, 6541, None, None), (75, 6541, None, None), (76, 6541, None, None), (77, 6541, None, None), (78, 6541, None, None), (79, 6541, None, None), (80, 6541, None, None), (81, 6541, None, None), (82, 6541, None, None), (83, 6541, None, None), (84, 6541, None, None), (85, 6541, None, None), (86, 6541, None, None), (87, 6541, None, None), (88, 6541, None, None), (89, 6541, None, None), (90, 6541, None, None), (91, 6541, None, None), (92, 6541, None, None), (93, 6541, None, None), (94, 6541, None, None), (95, 6541, None, None), (96, 6541, None, None), (97, 6541, None, None), (98, 6541, None, None), (99, 6541, None, None), (100, 6541, None, None)] ``` --- ## Type of Change _Indicate the type of change by choosing one (or more) of the following:_ - 🛠️ Bug fix --- ## Testing Unit test added covering this case, and also manual testing. --- ## Additional Notes (optional) _Add any additional context or information for reviewers._ --- --------- Co-authored-by: Eu Jing Chua <[email protected]> Co-authored-by: Sergiy Matusevych <[email protected]>
|
Fixed by #889 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When experiment has no tunables at all, restoring data from the storage crashes with the following exception:
We need to handle this situation gracefully and add unit tests to check for this condition.
The text was updated successfully, but these errors were encountered: