Final performance of the trained agents can be found in the table below.
This was computed by running python -m rl_zoo3.benchmark:
it runs the trained agent (trained on n_timesteps) for eval_timesteps and then reports the mean episode reward
during this evaluation.
It uses the deterministic policy except for Atari games.
You can view each model card (it includes video and hyperparameters) on our Huggingface page: https://huggingface.co/sb3
NOTE: this is not a quantitative benchmark as it corresponds to only one run (cf issue #38). This benchmark is meant to check algorithm (maximal) performance, find potential bugs and also allow users to have access to pretrained agents.
"M" stands for Million (1e6)
| algo | env_id | mean_reward | std_reward | n_timesteps | eval_timesteps | eval_episodes |
|---|---|---|---|---|---|---|
| a2c | Acrobot-v1 | -83.353 | 17.213 | 500k | 149979 | 1778 |
| a2c | Ant-v3 | -44.023 | 63.206 | 1M | 149469 | 761 |
| a2c | AntBulletEnv-v0 | 2497.147 | 37.359 | 2M | 150000 | 150 |
| a2c | AsteroidsNoFrameskip-v4 | 1286.550 | 423.750 | 10M | 614138 | 258 |
| a2c | BeamRiderNoFrameskip-v4 | 2890.298 | 1379.137 | 10M | 591104 | 47 |
| a2c | BipedalWalker-v3 | 299.754 | 23.459 | 5M | 149287 | 208 |
| a2c | BipedalWalkerHardcore-v3 | 96.171 | 122.943 | 200M | 149704 | 113 |
| a2c | BreakoutNoFrameskip-v4 | 279.793 | 122.177 | 10M | 604115 | 82 |
| a2c | CartPole-v1 | 500.000 | 0.000 | 500k | 150000 | 300 |
| a2c | EnduroNoFrameskip-v4 | 0.000 | 0.000 | 10M | 599040 | 45 |
| a2c | HalfCheetah-v3 | 3041.174 | 157.265 | 1M | 150000 | 150 |
| a2c | HalfCheetahBulletEnv-v0 | 2107.384 | 36.008 | 2M | 150000 | 150 |
| a2c | Hopper-v3 | 733.454 | 376.574 | 1M | 149987 | 580 |
| a2c | HopperBulletEnv-v0 | 815.355 | 313.798 | 2M | 149541 | 254 |
| a2c | Humanoid-v3 | 388.321 | 92.652 | 2M | 149942 | 1944 |
| a2c | LunarLander-v2 | 155.751 | 80.419 | 200k | 149443 | 297 |
| a2c | LunarLanderContinuous-v2 | 84.225 | 145.906 | 5M | 149305 | 256 |
| a2c | MountainCar-v0 | -111.263 | 24.087 | 1M | 149982 | 1348 |
| a2c | MountainCarContinuous-v0 | 91.166 | 0.255 | 100k | 149923 | 1659 |
| a2c | MsPacmanNoFrameskip-v4 | 1671.730 | 612.918 | 10M | 602450 | 185 |
| a2c | Pendulum-v1 | -162.965 | 103.210 | 1M | 150000 | 750 |
| a2c | PongNoFrameskip-v4 | 17.292 | 3.214 | 10M | 594910 | 65 |
| a2c | QbertNoFrameskip-v4 | 3882.345 | 1223.327 | 10M | 610670 | 194 |
| a2c | ReacherBulletEnv-v0 | 14.968 | 10.978 | 2M | 150000 | 1000 |
| a2c | RoadRunnerNoFrameskip-v4 | 31671.512 | 6364.085 | 10M | 606710 | 172 |
| a2c | SeaquestNoFrameskip-v4 | 1721.493 | 105.339 | 10M | 599691 | 67 |
| a2c | SpaceInvadersNoFrameskip-v4 | 627.160 | 201.974 | 10M | 604848 | 162 |
| a2c | Swimmer-v3 | 200.627 | 2.544 | 1M | 150000 | 150 |
| a2c | Walker2DBulletEnv-v0 | 858.209 | 333.116 | 2M | 149156 | 173 |
| a2c | Walker2d-v3 | 581.835 | 127.597 | 1M | 149782 | 593 |
| ars | Acrobot-v1 | -82.884 | 23.825 | 500k | 149985 | 1788 |
| ars | Ant-v3 | 2333.773 | 20.597 | 75M | 150000 | 150 |
| ars | CartPole-v1 | 500.000 | 0.000 | 50k | 150000 | 300 |
| ars | HalfCheetah-v3 | 4815.192 | 1340.752 | 12M | 150000 | 150 |
| ars | Hopper-v3 | 3343.919 | 5.730 | 7M | 150000 | 150 |
| ars | LunarLanderContinuous-v2 | 167.959 | 147.071 | 2M | 149883 | 562 |
| ars | MountainCar-v0 | -122.000 | 33.456 | 500k | 149938 | 1229 |
| ars | MountainCarContinuous-v0 | 96.672 | 0.784 | 500k | 149990 | 621 |
| ars | Pendulum-v1 | -212.540 | 160.444 | 2M | 150000 | 750 |
| ars | Swimmer-v3 | 355.267 | 12.796 | 2M | 150000 | 150 |
| ars | Walker2d-v3 | 2993.582 | 166.289 | 75M | 149821 | 152 |
| ddpg | AntBulletEnv-v0 | 2399.147 | 75.410 | 1M | 150000 | 150 |
| ddpg | BipedalWalker-v3 | 197.486 | 141.580 | 1M | 149237 | 227 |
| ddpg | HalfCheetahBulletEnv-v0 | 2078.325 | 208.379 | 1M | 150000 | 150 |
| ddpg | HopperBulletEnv-v0 | 1157.065 | 448.695 | 1M | 149565 | 346 |
| ddpg | LunarLanderContinuous-v2 | 230.217 | 92.372 | 300k | 149862 | 556 |
| ddpg | MountainCarContinuous-v0 | 93.512 | 0.048 | 300k | 149965 | 2260 |
| ddpg | Pendulum-v1 | -152.099 | 94.282 | 20k | 150000 | 750 |
| ddpg | ReacherBulletEnv-v0 | 15.582 | 9.606 | 300k | 150000 | 1000 |
| ddpg | Walker2DBulletEnv-v0 | 1387.591 | 736.955 | 1M | 149051 | 208 |
| dqn | Acrobot-v1 | -76.639 | 11.752 | 100k | 149998 | 1932 |
| dqn | AsteroidsNoFrameskip-v4 | 782.687 | 259.247 | 10M | 607962 | 134 |
| dqn | BeamRiderNoFrameskip-v4 | 4295.946 | 1790.458 | 10M | 600832 | 37 |
| dqn | BreakoutNoFrameskip-v4 | 358.327 | 61.981 | 10M | 601461 | 55 |
| dqn | CartPole-v1 | 500.000 | 0.000 | 50k | 150000 | 300 |
| dqn | EnduroNoFrameskip-v4 | 830.929 | 194.544 | 10M | 599040 | 14 |
| dqn | LunarLander-v2 | 154.382 | 79.241 | 100k | 149373 | 200 |
| dqn | MountainCar-v0 | -100.849 | 9.925 | 120k | 149962 | 1487 |
| dqn | MsPacmanNoFrameskip-v4 | 2682.929 | 492.567 | 10M | 599952 | 140 |
| dqn | PongNoFrameskip-v4 | 20.602 | 0.613 | 10M | 598998 | 88 |
| dqn | QbertNoFrameskip-v4 | 9496.774 | 5399.633 | 10M | 605844 | 124 |
| dqn | RoadRunnerNoFrameskip-v4 | 40396.350 | 7069.131 | 10M | 603257 | 137 |
| dqn | SeaquestNoFrameskip-v4 | 2000.290 | 606.644 | 10M | 599505 | 69 |
| dqn | SpaceInvadersNoFrameskip-v4 | 622.742 | 201.564 | 10M | 604311 | 155 |
| ppo | Acrobot-v1 | -73.506 | 18.201 | 1M | 149979 | 2013 |
| ppo | Ant-v3 | 1327.158 | 451.577 | 1M | 149572 | 175 |
| ppo | AntBulletEnv-v0 | 2865.922 | 56.468 | 2M | 150000 | 150 |
| ppo | AsteroidsNoFrameskip-v4 | 2156.174 | 744.640 | 10M | 602092 | 149 |
| ppo | BeamRiderNoFrameskip-v4 | 3397.000 | 1662.368 | 10M | 598926 | 46 |
| ppo | BipedalWalker-v3 | 287.939 | 2.448 | 5M | 149589 | 123 |
| ppo | BipedalWalkerHardcore-v3 | 122.374 | 117.605 | 100M | 148036 | 105 |
| ppo | BreakoutNoFrameskip-v4 | 398.033 | 33.328 | 10M | 600418 | 60 |
| ppo | CarRacing-v0 | 153.938 | 74.465 | 4M | 179000 | 179 |
| ppo | CartPole-v1 | 500.000 | 0.000 | 100k | 150000 | 300 |
| ppo | EnduroNoFrameskip-v4 | 996.364 | 176.090 | 10M | 572416 | 11 |
| ppo | HalfCheetah-v3 | 5819.099 | 663.530 | 1M | 150000 | 150 |
| ppo | HalfCheetahBulletEnv-v0 | 2924.721 | 64.465 | 2M | 150000 | 150 |
| ppo | Hopper-v3 | 2410.435 | 10.026 | 1M | 150000 | 150 |
| ppo | HopperBulletEnv-v0 | 2575.054 | 223.301 | 2M | 149094 | 152 |
| ppo | LunarLander-v2 | 242.119 | 31.823 | 1M | 149636 | 369 |
| ppo | LunarLanderContinuous-v2 | 270.863 | 32.072 | 1M | 149956 | 526 |
| ppo | MiniGrid-DoorKey-5x5-v0 | 0.964 | 0.006 | 100k | 149994 | 14956 |
| ppo | MiniGrid-Empty-Random-5x5-v0 | 0.968 | 0.013 | 100k | 149999 | 41965 |
| ppo | MiniGrid-FourRooms-v0 | 0.573 | 0.311 | 5M | 149927 | 3270 |
| ppo | MiniGrid-GoToDoor-5x5-v0 | 0.525 | 0.476 | 5M | 149991 | 16918 |
| ppo | MiniGrid-KeyCorridorS3R1-v0 | 0.949 | 0.011 | 500k | 149990 | 9733 |
| ppo | MiniGrid-LockedRoom-v0 | 0.000 | 0.000 | 10M | 149910 | 789 |
| ppo | MiniGrid-MultiRoom-N4-S5-v0 | 0.000 | 0.000 | 10M | 150000 | 1875 |
| ppo | MiniGrid-ObstructedMaze-2Dlh-v0 | 0.000 | 0.000 | 10M | 149760 | 260 |
| ppo | MiniGrid-PutNear-6x6-N2-v0 | 0.612 | 0.295 | 10M | 149998 | 14170 |
| ppo | MiniGrid-RedBlueDoors-6x6-v0 | 0.860 | 0.313 | 1M | 149997 | 7846 |
| ppo | MiniGrid-Unlock-v0 | 0.963 | 0.016 | 100k | 149999 | 12646 |
| ppo | MountainCar-v0 | -110.423 | 19.473 | 1M | 149954 | 1358 |
| ppo | MountainCarContinuous-v0 | 88.343 | 2.572 | 20k | 149983 | 633 |
| ppo | MsPacmanNoFrameskip-v4 | 1754.356 | 172.783 | 10M | 600822 | 163 |
| ppo | Pendulum-v1 | -172.225 | 104.159 | 100k | 150000 | 750 |
| ppo | PongNoFrameskip-v4 | 20.989 | 0.105 | 10M | 599902 | 90 |
| ppo | QbertNoFrameskip-v4 | 15627.108 | 3313.538 | 10M | 600248 | 83 |
| ppo | ReacherBulletEnv-v0 | 17.091 | 11.048 | 1M | 150000 | 1000 |
| ppo | RoadRunnerNoFrameskip-v4 | 40680.645 | 6675.058 | 10M | 605786 | 155 |
| ppo | SeaquestNoFrameskip-v4 | 1783.636 | 34.096 | 10M | 598243 | 66 |
| ppo | SpaceInvadersNoFrameskip-v4 | 960.331 | 425.355 | 10M | 603771 | 136 |
| ppo | Swimmer-v3 | 281.561 | 9.671 | 1M | 150000 | 150 |
| ppo | Walker2DBulletEnv-v0 | 2109.992 | 13.899 | 2M | 150000 | 150 |
| ppo | Walker2d-v3 | 3478.798 | 821.708 | 1M | 149343 | 171 |
| ppo_lstm | CarRacing-v0 | 862.549 | 97.342 | 4M | 149588 | 156 |
| ppo_lstm | CartPoleNoVel-v1 | 500.000 | 0.000 | 100k | 150000 | 300 |
| ppo_lstm | MountainCarContinuousNoVel-v0 | 91.469 | 1.776 | 300k | 149882 | 1340 |
| ppo_lstm | PendulumNoVel-v1 | -217.933 | 140.094 | 100k | 150000 | 750 |
| qrdqn | Acrobot-v1 | -69.135 | 9.967 | 100k | 149949 | 2138 |
| qrdqn | AsteroidsNoFrameskip-v4 | 2185.303 | 1097.172 | 10M | 599784 | 66 |
| qrdqn | BeamRiderNoFrameskip-v4 | 17122.941 | 10769.997 | 10M | 596483 | 17 |
| qrdqn | BreakoutNoFrameskip-v4 | 393.600 | 79.828 | 10M | 579711 | 40 |
| qrdqn | CartPole-v1 | 500.000 | 0.000 | 50k | 150000 | 300 |
| qrdqn | EnduroNoFrameskip-v4 | 3231.200 | 1311.801 | 10M | 585728 | 5 |
| qrdqn | LunarLander-v2 | 70.236 | 225.491 | 100k | 149957 | 522 |
| qrdqn | MountainCar-v0 | -106.042 | 15.536 | 120k | 149943 | 1414 |
| qrdqn | MsPacmanNoFrameskip-v4 | 997.867 | 877.130 | 10M | 604914 | 225 |
| qrdqn | PongNoFrameskip-v4 | 20.492 | 0.687 | 10M | 597443 | 63 |
| qrdqn | QbertNoFrameskip-v4 | 14799.728 | 2917.629 | 10M | 600773 | 92 |
| qrdqn | RoadRunnerNoFrameskip-v4 | 42325.424 | 8361.161 | 10M | 591016 | 59 |
| qrdqn | SeaquestNoFrameskip-v4 | 2557.576 | 76.951 | 10M | 596275 | 66 |
| qrdqn | SpaceInvadersNoFrameskip-v4 | 1899.928 | 823.488 | 10M | 597218 | 69 |
| sac | Ant-v3 | 4615.791 | 1354.111 | 1M | 149074 | 165 |
| sac | AntBulletEnv-v0 | 3073.114 | 175.148 | 1M | 150000 | 150 |
| sac | BipedalWalker-v3 | 297.668 | 33.060 | 500k | 149530 | 136 |
| sac | BipedalWalkerHardcore-v3 | 4.423 | 103.910 | 10M | 149794 | 88 |
| sac | HalfCheetah-v3 | 9535.451 | 100.470 | 1M | 150000 | 150 |
| sac | HalfCheetahBulletEnv-v0 | 2792.170 | 12.088 | 1M | 150000 | 150 |
| sac | Hopper-v3 | 2325.547 | 1129.676 | 1M | 149841 | 236 |
| sac | HopperBulletEnv-v0 | 2603.494 | 164.322 | 1M | 149724 | 151 |
| sac | Humanoid-v3 | 6232.287 | 279.885 | 2M | 149460 | 150 |
| sac | LunarLanderContinuous-v2 | 260.390 | 65.467 | 500k | 149634 | 672 |
| sac | MountainCarContinuous-v0 | 94.679 | 1.134 | 50k | 149966 | 1443 |
| sac | Pendulum-v1 | -156.995 | 88.714 | 20k | 150000 | 750 |
| sac | ReacherBulletEnv-v0 | 18.062 | 9.729 | 300k | 150000 | 1000 |
| sac | Swimmer-v3 | 345.568 | 3.084 | 1M | 150000 | 150 |
| sac | Walker2DBulletEnv-v0 | 2292.266 | 13.970 | 1M | 149983 | 150 |
| sac | Walker2d-v3 | 3863.203 | 254.347 | 1M | 149309 | 150 |
| td3 | Ant-v3 | 5813.274 | 589.773 | 1M | 149393 | 151 |
| td3 | AntBulletEnv-v0 | 3300.026 | 54.640 | 1M | 150000 | 150 |
| td3 | BipedalWalker-v3 | 305.990 | 56.886 | 1M | 149999 | 224 |
| td3 | BipedalWalkerHardcore-v3 | -98.116 | 16.087 | 10M | 150000 | 75 |
| td3 | HalfCheetah-v3 | 9655.666 | 969.916 | 1M | 150000 | 150 |
| td3 | HalfCheetahBulletEnv-v0 | 2821.641 | 19.722 | 1M | 150000 | 150 |
| td3 | Hopper-v3 | 3606.390 | 4.027 | 1M | 150000 | 150 |
| td3 | HopperBulletEnv-v0 | 2681.609 | 27.806 | 1M | 149486 | 150 |
| td3 | Humanoid-v3 | 5566.687 | 14.544 | 2M | 150000 | 150 |
| td3 | LunarLanderContinuous-v2 | 207.451 | 67.562 | 300k | 149488 | 337 |
| td3 | MountainCarContinuous-v0 | 93.483 | 0.075 | 300k | 149976 | 2275 |
| td3 | Pendulum-v1 | -151.855 | 90.227 | 20k | 150000 | 750 |
| td3 | ReacherBulletEnv-v0 | 17.114 | 9.750 | 300k | 150000 | 1000 |
| td3 | Swimmer-v3 | 359.127 | 1.244 | 1M | 150000 | 150 |
| td3 | Walker2DBulletEnv-v0 | 2213.672 | 230.558 | 1M | 149800 | 152 |
| td3 | Walker2d-v3 | 4717.823 | 46.303 | 1M | 150000 | 150 |
| tqc | Ant-v3 | 3339.362 | 1969.906 | 1M | 149583 | 202 |
| tqc | AntBulletEnv-v0 | 3456.717 | 248.733 | 1M | 150000 | 150 |
| tqc | BipedalWalker-v3 | 329.808 | 45.083 | 500k | 149682 | 254 |
| tqc | BipedalWalkerHardcore-v3 | 235.226 | 110.569 | 2M | 149032 | 131 |
| tqc | FetchPickAndPlace-v1 | -9.331 | 6.850 | 1M | 150000 | 3000 |
| tqc | FetchPush-v1 | -8.799 | 5.438 | 1M | 150000 | 3000 |
| tqc | FetchReach-v1 | -1.659 | 0.873 | 20k | 150000 | 3000 |
| tqc | FetchSlide-v1 | -29.210 | 11.387 | 3M | 150000 | 3000 |
| tqc | HalfCheetah-v3 | 12089.939 | 127.440 | 1M | 150000 | 150 |
| tqc | HalfCheetahBulletEnv-v0 | 3675.299 | 17.681 | 1M | 150000 | 150 |
| tqc | Hopper-v3 | 3754.199 | 8.276 | 1M | 150000 | 150 |
| tqc | HopperBulletEnv-v0 | 2662.373 | 206.210 | 1M | 149881 | 151 |
| tqc | Humanoid-v3 | 7239.320 | 1647.498 | 2M | 149508 | 165 |
| tqc | LunarLanderContinuous-v2 | 277.956 | 25.466 | 500k | 149928 | 706 |
| tqc | MountainCarContinuous-v0 | 63.641 | 45.259 | 50k | 149796 | 186 |
| tqc | PandaPickAndPlace-v1 | -8.024 | 6.674 | 1M | 150000 | 3000 |
| tqc | PandaPush-v1 | -6.405 | 6.400 | 1M | 150000 | 3000 |
| tqc | PandaReach-v1 | -1.768 | 0.858 | 20k | 150000 | 3000 |
| tqc | PandaSlide-v1 | -27.497 | 9.868 | 3M | 150000 | 3000 |
| tqc | PandaStack-v1 | -96.915 | 17.240 | 1M | 150000 | 1500 |
| tqc | Pendulum-v1 | -151.340 | 87.893 | 20k | 150000 | 750 |
| tqc | ReacherBulletEnv-v0 | 18.255 | 9.543 | 300k | 150000 | 1000 |
| tqc | Swimmer-v3 | 339.423 | 1.486 | 1M | 150000 | 150 |
| tqc | Walker2DBulletEnv-v0 | 2508.934 | 614.624 | 1M | 149572 | 159 |
| tqc | Walker2d-v3 | 4380.720 | 500.489 | 1M | 149606 | 152 |
| tqc | parking-v0 | -6.762 | 2.690 | 100k | 149983 | 7528 |
| trpo | Acrobot-v1 | -83.114 | 18.648 | 100k | 149976 | 1783 |
| trpo | Ant-v3 | 4982.301 | 663.761 | 1M | 149909 | 153 |
| trpo | AntBulletEnv-v0 | 2560.621 | 52.064 | 2M | 150000 | 150 |
| trpo | BipedalWalker-v3 | 182.339 | 145.570 | 1M | 148440 | 148 |
| trpo | CartPole-v1 | 500.000 | 0.000 | 100k | 150000 | 300 |
| trpo | HalfCheetah-v3 | 1785.476 | 68.672 | 1M | 150000 | 150 |
| trpo | HalfCheetahBulletEnv-v0 | 2758.752 | 327.032 | 2M | 150000 | 150 |
| trpo | Hopper-v3 | 3618.386 | 356.768 | 1M | 149575 | 152 |
| trpo | HopperBulletEnv-v0 | 2565.416 | 410.298 | 1M | 149640 | 154 |
| trpo | LunarLander-v2 | 133.166 | 112.173 | 200k | 149088 | 230 |
| trpo | LunarLanderContinuous-v2 | 262.387 | 21.428 | 200k | 149925 | 501 |
| trpo | MountainCar-v0 | -107.278 | 13.231 | 100k | 149974 | 1398 |
| trpo | MountainCarContinuous-v0 | 92.489 | 0.355 | 50k | 149971 | 1732 |
| trpo | Pendulum-v1 | -174.631 | 127.577 | 100k | 150000 | 750 |
| trpo | ReacherBulletEnv-v0 | 14.741 | 11.559 | 300k | 150000 | 1000 |
| trpo | Swimmer-v3 | 365.663 | 2.087 | 1M | 150000 | 150 |
| trpo | Walker2DBulletEnv-v0 | 1483.467 | 823.468 | 2M | 149860 | 197 |
| trpo | Walker2d-v3 | 4933.148 | 1452.538 | 1M | 149054 | 163 |