.NET Reinforcement learning (e.g. OpenAI Gym) using Godot as 3D engine.

[GeorgeS2019]

Hackathon Idea

A Godot template project for OpenAI Gym using only .NET machine learning framework(s)

Your name

• Jim (aka GeorgeFFM at Discord)
• Shehroze Malik (github.com/shehrozeee: Discord: Mini_Ninja) <= Real one not mini Ninja
• praveenraghuvanshi

Team name

GymRogueOne

Brief Description

Currently a few attempts have been made to do Gym in .NET. In almost all cases, the 3D engine is Unity3D. Here we would like to do that in a MIT community popular 3D engine: Godot. In almost all of these attempts, the RL part stays with python (both Unity and Godot). This requires the 3D engine to communicate with a python server running the RL service written in python.

Here, we want to evaluate the feasibility to do all that in .NET (e.g. ML.NET or TorchSharp path WITHOUT python or PythonNET environment).

We will use example(s) provided by Gym to do a proof of concept to deliver a working Godot template project for Gym using .NET core framework.

Other

Are you looking for team members?

• Yes
• No

[praveenraghuvanshi]

@GeorgeS2019 Interested in joining the project. Any prior experience in RL is required or any other pre-requisite?

[GeorgeS2019]

@praveenraghuvanshi Please share what you like to have fun, what you good at and what you would like to learn. So I know how you can have fun next few days, what you have experience

[praveenraghuvanshi]

Fun - Solving a real problem
Good at - .Net, Deep learning, Computer Vision, EDA, Azure.
Like to Learn - unexplored area of RL.
For more details, pls visit links at https://linktr.ee/praveenraghuvanshi

Let me know in case you need further information.

[GeorgeS2019]

@praveenraghuvanshi Great, let us aim for the Sky :-) Real problem.

Which real problem U have in mind, not explored RL .. I am keen to learn, so as Microsoft experts

[GeorgeS2019]

@praveenraghuvanshi I have a quick glance. Impressive. Can you export a (challenging) RL python to ONNX and figure out how to import that back using ML.NET? using ONNX runtime of ML.NET?

[praveenraghuvanshi]

Sure... I'll give a shot

[GeorgeS2019]

Can you try doing Elon Musk, landing StarShip on Mars?

is that challenging enough for you???

[praveenraghuvanshi]

I am not aware about that. Could you help me with some links?

[GeorgeS2019]

[GeorgeS2019]

give me sometimes, i need to organize. Please wait :-)

[GeorgeS2019]

https://www.youtube.com/watch?v=NofGYVdL49o

[edbeeching]

Hi, I am the author of Godot RL Agents
I am also a final year PhD student in Deep Reinforcement Learning for Robotic Control and Navigation.

This is a really interesting project and I wish you the best of luck.

The challenge you will face will be implementing RL algorithms in C#. This is the main reason all other implementations interface with python, practically all Deep RL research is conducted with python with wrappers for environments written in C++... etc. This had led to many robust and well maintained python Deep RL frameworks .

I have been looking to implement the neural network side in Godot for inference only, which would be a good first step and aleviates the need to implement the RL algorithm in python. Where I struggled was related to the import of the onnx runtime in a godot project, whenever I tried to do this I had issues. I managed to get ML.NET working inside a Godot project, however.
I did try torchsharp, but it does not support loading from onnx format, yet.

I will keep an eye on this hackathon project. Good luck!

[GeorgeS2019]

@edbeeching TorchSharp loading ONNX and instead of inference but translating ONNX into TorchSharp codes is currently being evaluated.