You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We should populate our public registry w/ training and evaluation Runs of the most popular OSS agent x environment pairs. We'll start with 2 agents and one env., and then add more agents and more envs.
We currently have SB3 PPO playing cartpole.
FINE GRAINED TODOS:
Factor out CartPoleEnv into a common component whose GitHubRepo points to OpenAI's repo.
add PAPAG agent playing cartpole env.
add Ilya PAPAG agent playing gym-pong env.
add SB3 agent playing gym-pong env.
Below are our current shortlists of other agents and envs to benchmark:
Agents/algos
SB3
Ilya PAPAG PPO agent
RLlib
Acme
RLkit
Open-clip
Environments
Atari games (one easy, one of the hardest)
PyBullet or Mujoco simulator (humanoid? half cheetah? soccer?)
Robotic arm simulator
Classic control?
DeepmindLab?
Minecraft?
[STRETCH] Hottest NLP (??) or vision benchmarks (ImageNet)
We should publish our code for these runs as new scripts in documentation/demos/ so that they double as helpful learning resources & examples of how to use the AgentOS & PCS APIs.
andyk
changed the title
Launch official Registry (setup url, populate first few envs/agents)
Improve public Agent Registry (setup url, populate first few envs/agents)
Mar 12, 2022
andyk
changed the title
Improve public Agent Registry (setup url, populate first few envs/agents)
Populate first agent runs at agentos.org/registry
Mar 22, 2022
andyk
changed the title
Populate first agent runs at agentos.org/registry
Generate first agent runs for agentos.org benchmark
Apr 13, 2022
andyk
changed the title
Generate first agent runs for agentos.org benchmark
Generate first benchmark agent runs for agentos.org leaderboard
Apr 13, 2022
We should populate our public registry w/ training and evaluation Runs of the most popular OSS agent x environment pairs. We'll start with 2 agents and one env., and then add more agents and more envs.
We currently have SB3 PPO playing cartpole.
FINE GRAINED TODOS:
Below are our current shortlists of other agents and envs to benchmark:
Agents/algos
Environments
We should publish our code for these runs as new scripts in
documentation/demos/
so that they double as helpful learning resources & examples of how to use the AgentOS & PCS APIs.Note the registry is currently hosted at https://aos-web.herokuapp.com, after #331 it will be hosted at agentos.org/registry.
The text was updated successfully, but these errors were encountered: