[RLlib] "TypeError: 'int' object is not iterable when using from_jsonable with nested Discrete in Dict" #50131
Labels
bug
Something that is supposed to be working; but isn't
P1
Issue that should be fixed within a few weeks
rllib
RLlib related issues
rllib-newstack
rllib-offline-rl
Offline RL problems
What happened + What you expected to happen
When calling the _map_to_episodes function in offline_prelearner.py, an issue arises during the creation of SingleAgentEpisode. The problem occurs in the from_jsonable_if_needed function when providing observation_space and action_space. It checks with is_composite_space and then uses the from_jsonable method from gymnasium.spaces, which expects sample_n to be of type dict[str, list[Any]]. However, when combining a Dict (composite space) with a Discrete (primitive space), the resulting structure does not conform to the expected list[Any] format, leading to an error.
Offline learning should be possible when providing observation_space and action_space. The implementation must work correctly even when combining composite spaces and primitive spaces (Dict + Discrete).
Versions / Dependencies
ray[rllib]==2.41.0
Reproduction script
Issue Severity
None
The text was updated successfully, but these errors were encountered: