Structure of ReplayBuffer._storage

[FHainzl]

In deepq.ReplayBuffer, the docstring for ReplayBuffer._storage says
[(np.ndarray, float, float, np.ndarray, bool)]
but shouldn't it actually be
"""[(np.ndarray, [float], float, np.ndarray, bool)]?

I think this would be correct for two reasons:

• Actions could be multi-dimensional
• The docstring for ReplayBuffer.add claims
  :param action: ([float]) the action,
  which I think is an inconsistency with the storage docstring.

Am I missing something here?

[araffin]

Hello,

Good point.

in fact, it depends on whether the buffer is used with discrete actions (e.g. DQN) or continuous actions (e.g. SAC).

For discrete: action should be int
For continuous: action should be np.ndarray (in fact a numpy array of type float32)

Overall, np.ndarray would be the most correct approximation, feel free to submit a PR that solves this inconsistency ;)