Cogment Verse

Cogment is an innovative open source AI platform designed to leverage the advent of AI to benefit humankind through human-AI collaboration developed by AI Redefined. Cogment enables AI researchers and engineers to build, train and operate AI agents in simulated or real environments shared with humans. For the full user documentation visit https://docs.cogment.ai

🚧 This repository is under construction, it propose a library of environments and agent implementations to get started with Human In the Loop Learning (HILL) and Reinforcement Learning (RL) with Cogment in minutes. Cogment Verse is designed both for practionners discovering the field as well as for experienced researchers or engineers as an framework to develop and benchmark new approaches.

Cogment verse includes environments from:

• OpenAI Gym,
• Petting Zoo.
• MinAtar.

Documentation table of contents

• Getting started
• Tutorials 🚧
  • Simple Behavior Cloning
• Experimental results 🚧
  • A2C
  • REINFORCE
• Develop 🚧
  • Development Setup
  • Debug
  • Environment development
• Changelog
• Contributors guide
• Community code of conduct
• Acknowledgments

Getting started

Setup, Build and Run

1. Clone this repository
2. Install the following dependencies:
   • Python 3.9 or above,
   • Node.JS v14 or above,
   • parallel, on ubuntu it is installable using apt-get install parallel, on mac it is available through brew install parallel,
   • unrar, on ubuntu it is installable using apt-get install unrar, on mac it is available through brew install unrar.
3. ./run.sh build
4. ./run.sh services_start
5. In a different terminal, start the trials with ./run.sh client start <run-name>. Different run names can be found in run_params.yaml
6. (Optional) To launch webclient, run ./run.sh web_client_start in a different terminal. Open http://localhost:8000 to join or visualize trials

Run monitoring

You can monitor ongoing run using mlflow. By default a local instance of mlflow is started by cogment-verse and is accessible at http://localhost:3000.

Human player

Some of the availabe run involve a human player, for example benchmark_lander_hill enables a human player to momentarily take control of the lunar lander to help the AI agents during the training process.

Then start the run

./run.sh client start benchmark_lander_hill

Access the playing interface by launching a webclient with ./run.sh web_client_start and navigating to http://localhost:8000

Play

The play run implementation can be used to have any actor play in any environment. 3 example run parameters are provided:

headless_play instanciates any agents and start a number of trials.

./run.sh client start headless_play

observe instanciates any agents and start a number of trials with a human observer through the webclient.

./run.sh client start observe

play instanciates let a human player play in a supported environment.

./run.sh client start play

They can be inspected and adapted to your needs in run_params.yaml:

Acknowledgements

The subdirectories /tf_agents/cogment_verse_tf_agents/third_party and /torch_agents/cogment_verse_torch_agents/third_party contains code from third party sources

• hive: Taken from the Hive library by MILA/CRL
• td3: Taken form the authors' implementation