-
Notifications
You must be signed in to change notification settings - Fork 8.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add documentation for Vectorized environments #2327
Conversation
1026844
to
6ad2ef9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me! Could you add another section on recording the episodic returns with the vector env using the following code?
import gym
envs = gym.vector.make("CartPole-v1", num_envs=3)
envs = gym.wrappers.RecordEpisodeStatistics(envs)
envs.reset()
for i in range(100):
_, _, _, infos = envs.step(envs.action_space.sample())
for info in infos:
if "episode" in info.keys():
print(f"i, episode_reward={info['episode']['r']}")
break
|
||
>>> envs = gym.vector.make("CartPole-v1", num_envs=3) | ||
>>> envs.action_space | ||
MultiDiscrete([2 2 2]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm. What happens when the origin env's action space is MultiDiscrete
, do you return a tuple of MultiDiscrete
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The rules for batching all the standard Gym spaces are available here. For example a MultiDiscrete
space becomes a Box
with an appropriate shape. Do you think it is worth mentioning these conversion rules in the documentation?
... ]) | ||
RuntimeError: Some environments have an observation space different from `Box([-4.8 ...], [4.8 ...], (4,), float32)`. In order to batch observations, the observation spaces from all environments must be equal. | ||
|
||
However, sometimes it may be handy to have access to the observation and action spaces of a sub-environment, and not the batched spaces. You can access those with the properties :attr:`~VectorEnv.single_observation_space` and :attr:`~VectorEnv.single_action_space` of the vectorized environment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!! This is super helpful. I'd suggest highlighting this section with a heading like
Working with the spaces of a sub-environment
Hey, I really appreciate all the time you put into this. However, per #2276 this isn't compatible with how the documentation website is going to be done. The way to include these docs would be to add elements of this to the api.md docs where applicable, and then to add the remainder to the dev_docs.md page once we have that (or just create it know and the person working on it can add the rest to that file). |
Is the documentation going to be just a collection of Markdown documents? How do you plan on having cross-references and auto-generated documentation from docstrings? |
I think a page on wrappers for vectorized environments ( |
There's a way of hooking the code into the MD files. When a PR for the new Gym website comes in a few weeks you'll see. I'll leave this PR open until then. |
Any progress on the documentation of vectorized environments? What is the status of the documentation? |
I was reading up on Vectorised Environments and I happened to stumble upon this PR. Any chance I can view the HTML version of the docs right now? Is there a beta version of the new documentation website that's mentioned in the README? |
@nikhilweee There is a HTML version of this documentation available here: https://tristandeleu.github.io/gym/vector/ |
Thank you very much, just what I was looking for! |
gym.vector.make
,AsyncVectorEnv
, andSyncVectorEnv
AsyncVectorEnv
and exception handling) and advanced (custom spaces) usages of vectorized environments.gym.vector.make
,VectorEnv
,AsyncVectorEnv
, andSyncVectorEnv
for automatic generation of the API reference.