Add REINFORCE implementation tutorial

[siddarth-c]

Description

Created a new tutorial depicting the new .step() function of gymnasium v26 using PyTorch. REINFORCE is employed to solve Mujoco's Reacher.

Type of change

Please delete options that are not relevant.

• This change requires a documentation update

Checklist:

• I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[pseudo-rnd-thoughts]

Thanks for the tutorial, it looks very helpful.
Could you fix the pre-commit issues and address the comments

[pseudo-rnd-thoughts]

There is a number of issues when I build the tutorial, could you build the tutorial (see the readme.md) and look at the previous tutorials to fix the issues.

reinforce_reacher_gym_v26.rst:2: WARNING: Field list ends without a blank line; unexpected unindent.
reinforce_reacher_gym_v26.rst:14: ERROR: Unexpected indentation.
reinforce_reacher_gym_v26.rst:25: WARNING: Block quote ends without a blank line; unexpected unindent.
reinforce_reacher_gym_v26.rst:220: ERROR: Unexpected indentation.
reinforce_reacher_gym_v26.rst:224: WARNING: Definition list ends without a blank line; unexpected unindent.

[siddarth-c]

I have updated the files as per the readme.md
Will do better the next time, thanks for bearing with me!

[pseudo-rnd-thoughts]

No worries, currently when I build the tutorial, on the left hand side, all of the titles appear here.
I believe it is the way that you have structured the sections that causes this. could you fix
In addition, the same issues that get reported when building still appear and the final figure doesn't appear

[siddarth-c]

I have modified the structure of the titles and code following the Blackjack tutorial. And running through the tests mentioned in readme.md I do not get any warnings.

[pseudo-rnd-thoughts]

@siddarth-c I have made a number of upgrades to the tutorials. There are only a couple more thing before we can merge

1. Replace the top image of the agent with a gif of the trained agent completing the environment
2. Could you add an image for the policy network should the data flow. i.e., input data -> shared network -> split to mean network and std network -> output

[siddarth-c]

Done with the mentioned changes

[pseudo-rnd-thoughts]

That policy network figure is very nice, to confirm, this is not copied from someone else and is your own creation.
Also for the top gif, is this from the final agent? The agent doesn't seem to do very well in the environment

[siddarth-c]

The policy learned via REINFORCE is not optimal in Reacher (despite extensive hyperparameter searches).
So I have changed the environment to the Inverted Pendulum, where it is able to learn the optimal policy (achieves max reward of 1000). I hope this change is acceptable.

And yes, the policy network was designed by me

[pseudo-rnd-thoughts]

Amazing, thank you for the tutorial, we would be interested in anymore tutorials that you create. Probably more on the gym environment side than training though they are always helpful