[Bug Report] Lunar Lander Runs Slowly

[ian-hendo]

Describe the bug

The Lunar Lander environment creates particles and updates their physics, even when not being rendered. This slows down wall-time training on the environment by ~5x-10x.

Wrapping said code within the step(.) function with if self.render_mode == "human": resolves the issue.

Code example

if self.render_mode == "human":
    p = self._create_particle(
        3.5,  # 3.5 is here to make particle speed adequate
        impulse_pos[0],
        impulse_pos[1],
        m_power,
    )  # particles are just a decoration
    p.ApplyLinearImpulse(
        (ox * MAIN_ENGINE_POWER * m_power, oy * MAIN_ENGINE_POWER * m_power),
        impulse_pos,
        True,
    )

System info

N/A

Additional context

N/A

Checklist

• I have checked that there is no similar issue in the repo

[pseudo-rnd-thoughts]

Wow, if this is true that would be a significant speed up. Could you make a PR with the results and show that the observations don't change as a well

[ian-hendo]

Hi Mark, Thanks for the quick reply. I will try and do as you say, but finding time may be difficult at the moment unfortunately. Also - off topic, but I see your bio on github says you are a PhD student at UoS studying RL. Incidentally, I am also a PhD student at UoS studying RL! Perhaps it would be good to have a chat to each other outside of this issue report? Kind Regards, Ian

…

On Thu, 1 Dec 2022, 12:04 Mark Towers, ***@***.***> wrote: Wow, if this is true that would be a significant speed up. Could you make a PR with the results and show that the observations don't change as a well — Reply to this email directly, view it on GitHub <#170 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A4QNXSKYVGG4PSSDG73KHM3WLCH3NANCNFSM6AAAAAASQV56HA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[pseudo-rnd-thoughts]

Yes, I'm at the Uni of Southampton. I will send you an email to your soton address

[mbertheau]

One issue seems to be that replays won't include particles, if they are only added in render_mode "human".

[pseudo-rnd-thoughts]

We can include the optimising in the render_mode=="rgb_array"

[mbertheau]

I'm currently at if self.render_mode is not None. That seems to cover all cases.

[pseudo-rnd-thoughts]

Would someone make PR with this fix along with a test to show that this doesn't break anything

[mbertheau]

What did you have in mind for the test?

[pseudo-rnd-thoughts]

If the rendering is deterministic, could you save the rendered frames then make the change and see if the rendered frames are the same (using the same actions and reset seed). Don't think we want this added to the actual tests, could include the test code for someone to replicate.
Then add a quick performance test to show the difference in fps?

I don't think we can add anything to the actual tests in the project as the old version won't exist after the fix.

[PaulMest]

I've been going through the HuggingFace RL course over the holidays and also discovered this. Submitted the fix for it here: #235. Results appear to be pretty great for such a small change!