Convert Q-learning (and/or Sarsa) into an example agent in reinforce-zoo

[stites]

Build an example using current code. Use reinforce-algorithms to come up with an example of using the current algorithm interfaces (Reinforce.Algorithms), and the Q-Table "backend" (Reinforce.Agents). This would go into the reinforce-zoo folder and would be a good introduction to current internals. You can open new ticket for this if it takes a long time.

[msaroufim]

Hey @stites I'd like to take a stab at this. I have a bunch of experience with RL in python but am terrible at Haskell, I've read a bunch but wanted to get my hands a bit dirtier. Do you have 5-10min to talk about what this issue would entail?

[stites]

I would love to chat! I believe this repo is pretty beginner-friendly as well (meaning no fancy types). My email address [email protected] (rot13-encoded), send me a ping!

As you may be aware, reinforce just consists of RL algorithms with tabular agents and the idea was to stabilize hasktorch before jumping back into function approximation. My attention has since shifted a bit more to maintaining hasktorch, however.

I should also mention that there's a performance bottleneck coming from the openai gym dependency (openai/gym-http-api). TBH it's pretty miserable and OpenAI no longer maintains that library (so they don't really support other languages anymore).

The options are using the call-python-via-msgpack repo (see #6) which I haven't tried and don't know what the performance hit looks like, or writing haskell bindings that directly interface with the ALE (all their C++ seems extern'd, so would be straight forward). I was also starting to rewrite some of Sutton's old environments in haskell.