normalize_state broke a few things

[mwcvitkovic]

On a fresh install, with the config/experiment.json file reading
{ "a3c.json": { "a3c_conv_shared_breakout": "dev" } }
running python run_lab.py crashes with

...
[2018-09-05 07:23:19,207 INFO logger.py info] Initialized session 0
[2018-09-05 07:23:19,208 INFO logger.py info] Initialized DistSession 0
Process w0:
Traceback (most recent call last):
  File "/Users/mwcvitkovic/miniconda3/envs/lab/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/control.py", line 149, in run
    return self.session.run()
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/control.py", line 76, in run
    self.run_episode()
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/control.py", line 59, in run_episode
    action = self.agent.act(state)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/agent/__init__.py", line 66, in act
    action = self.algorithm.act(state)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/agent/algorithm/reinforce.py", line 108, in act
    if self.normalize_state:
AttributeError: 'ActorCritic' object has no attribute 'normalize_state'
[2018-09-05 07:23:20,034 INFO analysis.py analyze_trial] Analyzing trial
Traceback (most recent call last):
  File "run_lab.py", line 73, in <module>
    main()
  File "run_lab.py", line 69, in main
    run_by_mode(spec_file, spec_name, lab_mode)
  File "run_lab.py", line 60, in run_by_mode
    Trial(spec, info_space).run()
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/control.py", line 238, in run
    self.data = analysis.analyze_trial(self)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/analysis.py", line 483, in analyze_trial
    trial_fitness_df = calc_trial_fitness_df(trial)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/analysis.py", line 273, in calc_trial_fitness_df
    all_session_fitness_df = pd.concat(list(trial.session_data_dict.values()))
  File "/Users/mwcvitkovic/miniconda3/envs/lab/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 212, in concat
    copy=copy)
  File "/Users/mwcvitkovic/miniconda3/envs/lab/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 245, in __init__
    raise ValueError('No objects to concatenate')
ValueError: No objects to concatenate

Presumably this is because the spec if missing the normalize_state key. However, if you add the line "normalize_state": true to the a3c_conv_shared_breakout algorithm spec, running python run_lab.py crashes with

...
[2018-09-05 07:25:38,543 INFO logger.py info] Initialized session 0
[2018-09-05 07:25:38,543 INFO logger.py info] Initialized DistSession 0
Process w0:
Traceback (most recent call last):
  File "/Users/mwcvitkovic/miniconda3/envs/lab/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/control.py", line 149, in run
    return self.session.run()
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/control.py", line 76, in run
    self.run_episode()
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/control.py", line 59, in run_episode
    action = self.agent.act(state)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/agent/__init__.py", line 66, in act
    action = self.algorithm.act(state)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/agent/algorithm/reinforce.py", line 109, in act
    state = policy_util.update_online_stats_and_normalize_state(body, state)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/agent/algorithm/policy_util.py", line 481, in update_online_stats_and_normalize_state
    update_online_stats(body, state)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/agent/algorithm/policy_util.py", line 416, in update_online_stats
    assert state.size == body.state_dim
AssertionError
[2018-09-05 07:25:39,357 INFO analysis.py analyze_trial] Analyzing trial
Traceback (most recent call last):
  File "run_lab.py", line 73, in <module>
    main()
  File "run_lab.py", line 69, in main
    run_by_mode(spec_file, spec_name, lab_mode)
  File "run_lab.py", line 60, in run_by_mode
    Trial(spec, info_space).run()
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/control.py", line 238, in run
    self.data = analysis.analyze_trial(self)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/analysis.py", line 483, in analyze_trial
    trial_fitness_df = calc_trial_fitness_df(trial)
  File "/Users/mwcvitkovic/Projects/kengz-SLM-Lab/slm_lab/experiment/analysis.py", line 273, in calc_trial_fitness_df
    all_session_fitness_df = pd.concat(list(trial.session_data_dict.values()))
  File "/Users/mwcvitkovic/miniconda3/envs/lab/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 212, in concat
    copy=copy)
  File "/Users/mwcvitkovic/miniconda3/envs/lab/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 245, in __init__
    raise ValueError('No objects to concatenate')
ValueError: No objects to concatenate

Running with "normalize_state": false doesn't crash, though it hangs without doing anything. This may be a separate issue.

[kengz]

Thanks for catching, found it! a3c.json and dppo.json were new spec files; forgot to update them. Just added those keys back in PR #157 tho it defaults to true.

also FYI the last error start from Analyzing trial ... ValueError: No objects to concatenate is just there because the trial failed. so the root cause is in the first stack trace.

[kengz]

also I'll improve the distributed stack trace later to make it clearer (i.e. pointing to the actual cause)