Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Non-functional objectives (PPO, A2C, Reinforce) #1804

Merged
merged 7 commits into from
Jan 23, 2024
Merged

[Feature] Non-functional objectives (PPO, A2C, Reinforce) #1804

merged 7 commits into from
Jan 23, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jan 15, 2024 •
edited

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jan 15, 2024 •
edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1804

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 3 Unrelated Failures

As of commit 55127fe with merge base a10cdbf (image):

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 15, 2024
@vmoens vmoens changed the title [Feature] Non-functional objectives [Feature] Non-functional objectives (PPO, A2C, Reinforce) Jan 19, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 20, 2024 •
edited

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 62.8155ms 62.5357ms 15.9909 Ops/s 16.2559 Ops/s $\color{#d91a1a}-1.63\%$
test_sync 33.2868ms 32.8937ms 30.4010 Ops/s 29.2596 Ops/s $\color{#35bf28}+3.90\%$
test_async 0.1037s 32.4802ms 30.7880 Ops/s 29.8330 Ops/s $\color{#35bf28}+3.20\%$
test_simple 0.4854s 0.4337s 2.3060 Ops/s 2.2698 Ops/s $\color{#35bf28}+1.59\%$
test_transformed 0.6511s 0.5969s 1.6754 Ops/s 1.6780 Ops/s $\color{#d91a1a}-0.15\%$
test_serial 1.4176s 1.3629s 0.7337 Ops/s 0.7357 Ops/s $\color{#d91a1a}-0.27\%$
test_parallel 1.2438s 1.1864s 0.8429 Ops/s 0.8520 Ops/s $\color{#d91a1a}-1.07\%$
test_step_mdp_speed[True-True-True-True-True] 0.1702ms 22.0287μs 45.3953 KOps/s 46.2909 KOps/s $\color{#d91a1a}-1.93\%$
test_step_mdp_speed[True-True-True-True-False] 0.1381ms 13.8800μs 72.0462 KOps/s 75.8701 KOps/s $\textbf{\color{#d91a1a}-5.04\%}$
test_step_mdp_speed[True-True-True-False-True] 48.6620μs 12.8000μs 78.1251 KOps/s 79.2066 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-True-True-False-False] 38.3210μs 7.7675μs 128.7421 KOps/s 130.9634 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[True-True-False-True-True] 62.9780μs 23.7283μs 42.1437 KOps/s 43.2283 KOps/s $\color{#d91a1a}-2.51\%$
test_step_mdp_speed[True-True-False-True-False] 48.0900μs 14.5564μs 68.6985 KOps/s 68.9183 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-True-False-False-True] 44.6240μs 14.0767μs 71.0395 KOps/s 71.6067 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-True-False-False-False] 40.9660μs 9.0497μs 110.5015 KOps/s 111.6522 KOps/s $\color{#d91a1a}-1.03\%$
test_step_mdp_speed[True-False-True-True-True] 54.1720μs 24.8028μs 40.3181 KOps/s 40.8407 KOps/s $\color{#d91a1a}-1.28\%$
test_step_mdp_speed[True-False-True-True-False] 49.6520μs 16.0921μs 62.1423 KOps/s 62.1067 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[True-False-True-False-True] 52.3780μs 14.3731μs 69.5745 KOps/s 72.1904 KOps/s $\color{#d91a1a}-3.62\%$
test_step_mdp_speed[True-False-True-False-False] 50.8550μs 9.0906μs 110.0041 KOps/s 111.4049 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[True-False-False-True-True] 0.2259ms 27.8257μs 35.9380 KOps/s 38.6939 KOps/s $\textbf{\color{#d91a1a}-7.12\%}$
test_step_mdp_speed[True-False-False-True-False] 44.9540μs 17.3343μs 57.6890 KOps/s 57.8054 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[True-False-False-False-True] 50.6650μs 15.6122μs 64.0526 KOps/s 63.4697 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-False-False-False-False] 42.6400μs 10.2891μs 97.1902 KOps/s 99.2506 KOps/s $\color{#d91a1a}-2.08\%$
test_step_mdp_speed[False-True-True-True-True] 66.1540μs 24.9600μs 40.0641 KOps/s 41.0483 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[False-True-True-True-False] 58.8400μs 16.0195μs 62.4238 KOps/s 63.1192 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-True-True-False-True] 50.5650μs 16.4620μs 60.7460 KOps/s 61.3698 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[False-True-True-False-False] 44.2530μs 10.3286μs 96.8189 KOps/s 98.6111 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[False-True-False-True-True] 66.8150μs 26.4080μs 37.8673 KOps/s 38.5344 KOps/s $\color{#d91a1a}-1.73\%$
test_step_mdp_speed[False-True-False-True-False] 49.8840μs 17.2824μs 57.8622 KOps/s 58.3916 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[False-True-False-False-True] 44.2420μs 17.5835μs 56.8714 KOps/s 57.1242 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-True-False-False-False] 45.7850μs 11.5839μs 86.3266 KOps/s 87.9143 KOps/s $\color{#d91a1a}-1.81\%$
test_step_mdp_speed[False-False-True-True-True] 0.2512ms 27.8428μs 35.9159 KOps/s 37.3378 KOps/s $\color{#d91a1a}-3.81\%$
test_step_mdp_speed[False-False-True-True-False] 52.8190μs 18.5679μs 53.8563 KOps/s 54.2497 KOps/s $\color{#d91a1a}-0.73\%$
test_step_mdp_speed[False-False-True-False-True] 53.2000μs 18.1166μs 55.1980 KOps/s 57.2392 KOps/s $\color{#d91a1a}-3.57\%$
test_step_mdp_speed[False-False-True-False-False] 39.3130μs 11.6430μs 85.8886 KOps/s 88.0231 KOps/s $\color{#d91a1a}-2.42\%$
test_step_mdp_speed[False-False-False-True-True] 66.3740μs 28.7079μs 34.8336 KOps/s 35.7217 KOps/s $\color{#d91a1a}-2.49\%$
test_step_mdp_speed[False-False-False-True-False] 44.5030μs 19.9170μs 50.2084 KOps/s 50.9015 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-False-False-False-True] 62.8980μs 18.9667μs 52.7240 KOps/s 54.0744 KOps/s $\color{#d91a1a}-2.50\%$
test_step_mdp_speed[False-False-False-False-False] 49.1320μs 12.7419μs 78.4809 KOps/s 80.6907 KOps/s $\color{#d91a1a}-2.74\%$
test_values[generalized_advantage_estimate-True-True] 20.9442ms 12.6085ms 79.3116 Ops/s 82.2894 Ops/s $\color{#d91a1a}-3.62\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.2051ms 28.2723ms 35.3704 Ops/s 36.3480 Ops/s $\color{#d91a1a}-2.69\%$
test_values[td0_return_estimate-False-False] 0.3096ms 0.1909ms 5.2370 KOps/s 5.5716 KOps/s $\textbf{\color{#d91a1a}-6.01\%}$
test_values[td1_return_estimate-False-False] 26.4515ms 26.0113ms 38.4448 Ops/s 38.5701 Ops/s $\color{#d91a1a}-0.32\%$
test_values[vec_td1_return_estimate-False-False] 36.0684ms 28.0647ms 35.6319 Ops/s 33.9991 Ops/s $\color{#35bf28}+4.80\%$
test_values[td_lambda_return_estimate-True-False] 38.4149ms 36.6934ms 27.2529 Ops/s 27.4436 Ops/s $\color{#d91a1a}-0.69\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.8966ms 28.1814ms 35.4843 Ops/s 35.6888 Ops/s $\color{#d91a1a}-0.57\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2654ms 8.1250ms 123.0775 Ops/s 122.6726 Ops/s $\color{#35bf28}+0.33\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 10.5946ms 2.0267ms 493.4020 Ops/s 495.1946 Ops/s $\color{#d91a1a}-0.36\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 8.8962ms 0.4395ms 2.2751 KOps/s 2.3071 KOps/s $\color{#d91a1a}-1.39\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 60.4866ms 40.1979ms 24.8769 Ops/s 24.7622 Ops/s $\color{#35bf28}+0.46\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.1333ms 2.6162ms 382.2309 Ops/s 376.5380 Ops/s $\color{#35bf28}+1.51\%$
test_dqn_speed 11.2065ms 7.6272ms 131.1101 Ops/s 133.3552 Ops/s $\color{#d91a1a}-1.68\%$
test_ddpg_speed 21.6365ms 14.6142ms 68.4266 Ops/s 69.9129 Ops/s $\color{#d91a1a}-2.13\%$
test_sac_speed 37.5683ms 29.3203ms 34.1061 Ops/s 34.1990 Ops/s $\color{#d91a1a}-0.27\%$
test_redq_speed 0.1195s 48.5842ms 20.5828 Ops/s 21.7622 Ops/s $\textbf{\color{#d91a1a}-5.42\%}$
test_redq_deprec_speed 33.7391ms 26.1123ms 38.2961 Ops/s 38.2048 Ops/s $\color{#35bf28}+0.24\%$
test_td3_speed 27.8706ms 20.3277ms 49.1939 Ops/s 48.1012 Ops/s $\color{#35bf28}+2.27\%$
test_cql_speed 94.6627ms 87.5783ms 11.4184 Ops/s 11.0110 Ops/s $\color{#35bf28}+3.70\%$
test_a2c_speed 30.2292ms 26.8257ms 37.2777 Ops/s 36.5726 Ops/s $\color{#35bf28}+1.93\%$
test_ppo_speed 34.6815ms 26.7947ms 37.3208 Ops/s 36.2709 Ops/s $\color{#35bf28}+2.89\%$
test_reinforce_speed 27.8611ms 25.8110ms 38.7432 Ops/s 38.8122 Ops/s $\color{#d91a1a}-0.18\%$
test_iql_speed 71.4988ms 63.0228ms 15.8673 Ops/s 15.6787 Ops/s $\color{#35bf28}+1.20\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 1.9958ms 1.3688ms 730.5431 Ops/s 691.7948 Ops/s $\textbf{\color{#35bf28}+5.60\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 8.7462ms 0.5172ms 1.9335 KOps/s 1.9372 KOps/s $\color{#d91a1a}-0.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 8.9695ms 0.4914ms 2.0350 KOps/s 2.0140 KOps/s $\color{#35bf28}+1.04\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.0593ms 1.3824ms 723.4044 Ops/s 688.0589 Ops/s $\textbf{\color{#35bf28}+5.14\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 9.9541ms 0.5185ms 1.9288 KOps/s 1.9345 KOps/s $\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 6.1487ms 0.4988ms 2.0049 KOps/s 2.0746 KOps/s $\color{#d91a1a}-3.36\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.1139s 1.7116ms 584.2462 Ops/s 604.2239 Ops/s $\color{#d91a1a}-3.31\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0551ms 0.6500ms 1.5386 KOps/s 1.5525 KOps/s $\color{#d91a1a}-0.90\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 10.0784ms 0.6261ms 1.5973 KOps/s 1.5794 KOps/s $\color{#35bf28}+1.14\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.2894ms 1.5038ms 664.9832 Ops/s 686.4557 Ops/s $\color{#d91a1a}-3.13\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 8.9565ms 0.5362ms 1.8649 KOps/s 1.9138 KOps/s $\color{#d91a1a}-2.55\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6737ms 0.4931ms 2.0278 KOps/s 2.0593 KOps/s $\color{#d91a1a}-1.53\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 0.1171s 1.6099ms 621.1449 Ops/s 687.7884 Ops/s $\textbf{\color{#d91a1a}-9.69\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6638ms 0.5086ms 1.9663 KOps/s 1.9599 KOps/s $\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 8.5404ms 0.4967ms 2.0135 KOps/s 2.0694 KOps/s $\color{#d91a1a}-2.70\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.1086s 1.8850ms 530.4970 Ops/s 607.0996 Ops/s $\textbf{\color{#d91a1a}-12.62\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8330ms 0.6492ms 1.5404 KOps/s 1.5412 KOps/s $\color{#d91a1a}-0.05\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 9.1140ms 0.6335ms 1.5785 KOps/s 1.6034 KOps/s $\color{#d91a1a}-1.55\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1184s 10.3167ms 96.9303 Ops/s 79.9235 Ops/s $\textbf{\color{#35bf28}+21.28\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.1891ms 13.6312ms 73.3613 Ops/s 73.8930 Ops/s $\color{#d91a1a}-0.72\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.0932ms 3.2443ms 308.2307 Ops/s 289.5269 Ops/s $\textbf{\color{#35bf28}+6.46\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 99.8390ms 11.7336ms 85.2252 Ops/s 99.1272 Ops/s $\textbf{\color{#d91a1a}-14.02\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.2767ms 13.7449ms 72.7545 Ops/s 64.4689 Ops/s $\textbf{\color{#35bf28}+12.85\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.3817ms 3.3003ms 303.0017 Ops/s 301.2696 Ops/s $\color{#35bf28}+0.57\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 93.5834ms 11.7510ms 85.0993 Ops/s 94.3217 Ops/s $\textbf{\color{#d91a1a}-9.78\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.3284ms 13.9164ms 71.8575 Ops/s 72.3154 Ops/s $\color{#d91a1a}-0.63\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.6968ms 3.5636ms 280.6179 Ops/s 270.4990 Ops/s $\color{#35bf28}+3.74\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 20, 2024 •
edited

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1160s 0.1157s 8.6447 Ops/s 8.8353 Ops/s $\color{#d91a1a}-2.16\%$
test_sync 0.1715s 0.1027s 9.7338 Ops/s 9.7579 Ops/s $\color{#d91a1a}-0.25\%$
test_async 0.1809s 91.2778ms 10.9556 Ops/s 10.9370 Ops/s $\color{#35bf28}+0.17\%$
test_single_pixels 0.1396s 0.1389s 7.2017 Ops/s 7.3320 Ops/s $\color{#d91a1a}-1.78\%$
test_sync_pixels 77.2159ms 74.5808ms 13.4083 Ops/s 13.2344 Ops/s $\color{#35bf28}+1.31\%$
test_async_pixels 0.1451s 71.2148ms 14.0420 Ops/s 14.0887 Ops/s $\color{#d91a1a}-0.33\%$
test_simple 0.9094s 0.8398s 1.1908 Ops/s 1.2164 Ops/s $\color{#d91a1a}-2.10\%$
test_transformed 1.1538s 1.0884s 0.9188 Ops/s 0.9513 Ops/s $\color{#d91a1a}-3.42\%$
test_serial 2.4202s 2.3533s 0.4249 Ops/s 0.4267 Ops/s $\color{#d91a1a}-0.41\%$
test_parallel 2.0625s 1.9144s 0.5224 Ops/s 0.5382 Ops/s $\color{#d91a1a}-2.94\%$
test_step_mdp_speed[True-True-True-True-True] 64.2810μs 34.5195μs 28.9691 KOps/s 30.7574 KOps/s $\textbf{\color{#d91a1a}-5.81\%}$
test_step_mdp_speed[True-True-True-True-False] 0.2183ms 20.7789μs 48.1258 KOps/s 50.2921 KOps/s $\color{#d91a1a}-4.31\%$
test_step_mdp_speed[True-True-True-False-True] 44.6600μs 20.2561μs 49.3679 KOps/s 53.2198 KOps/s $\textbf{\color{#d91a1a}-7.24\%}$
test_step_mdp_speed[True-True-True-False-False] 0.1990ms 11.7202μs 85.3228 KOps/s 89.3436 KOps/s $\color{#d91a1a}-4.50\%$
test_step_mdp_speed[True-True-False-True-True] 55.7610μs 35.5735μs 28.1108 KOps/s 28.9450 KOps/s $\color{#d91a1a}-2.88\%$
test_step_mdp_speed[True-True-False-True-False] 0.2119ms 22.1658μs 45.1145 KOps/s 45.7955 KOps/s $\color{#d91a1a}-1.49\%$
test_step_mdp_speed[True-True-False-False-True] 0.2124ms 20.9796μs 47.6653 KOps/s 48.8296 KOps/s $\color{#d91a1a}-2.38\%$
test_step_mdp_speed[True-True-False-False-False] 30.4910μs 13.3880μs 74.6940 KOps/s 76.1945 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[True-False-True-True-True] 0.2273ms 37.0973μs 26.9562 KOps/s 27.2775 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-False-True-True-False] 0.2079ms 24.1395μs 41.4259 KOps/s 42.1692 KOps/s $\color{#d91a1a}-1.76\%$
test_step_mdp_speed[True-False-True-False-True] 0.2170ms 21.2985μs 46.9517 KOps/s 48.6003 KOps/s $\color{#d91a1a}-3.39\%$
test_step_mdp_speed[True-False-True-False-False] 28.9300μs 13.4324μs 74.4469 KOps/s 75.7643 KOps/s $\color{#d91a1a}-1.74\%$
test_step_mdp_speed[True-False-False-True-True] 0.2345ms 39.0428μs 25.6129 KOps/s 25.9570 KOps/s $\color{#d91a1a}-1.33\%$
test_step_mdp_speed[True-False-False-True-False] 0.2150ms 25.8254μs 38.7215 KOps/s 38.3070 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[True-False-False-False-True] 0.2144ms 22.8239μs 43.8138 KOps/s 44.6820 KOps/s $\color{#d91a1a}-1.94\%$
test_step_mdp_speed[True-False-False-False-False] 40.5200μs 15.2323μs 65.6501 KOps/s 66.6532 KOps/s $\color{#d91a1a}-1.50\%$
test_step_mdp_speed[False-True-True-True-True] 0.2214ms 37.1150μs 26.9433 KOps/s 26.9636 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-True-True-True-False] 0.2129ms 23.9420μs 41.7675 KOps/s 41.6488 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-True-True-False-True] 0.2109ms 24.7349μs 40.4287 KOps/s 40.1869 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-True-True-False-False] 32.5110μs 15.1894μs 65.8354 KOps/s 66.2216 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-True-False-True-True] 0.2392ms 39.1616μs 25.5352 KOps/s 25.8501 KOps/s $\color{#d91a1a}-1.22\%$
test_step_mdp_speed[False-True-False-True-False] 0.2092ms 25.7436μs 38.8447 KOps/s 38.5449 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[False-True-False-False-True] 44.5310μs 26.6169μs 37.5701 KOps/s 37.3882 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-True-False-False-False] 0.2068ms 17.0231μs 58.7436 KOps/s 58.5018 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-False-True-True-True] 0.2422ms 41.1810μs 24.2831 KOps/s 24.3903 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-False-True-True-False] 48.7610μs 27.5497μs 36.2980 KOps/s 36.2289 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[False-False-True-False-True] 51.2510μs 26.5619μs 37.6479 KOps/s 37.6670 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[False-False-True-False-False] 0.2031ms 17.2043μs 58.1249 KOps/s 59.4635 KOps/s $\color{#d91a1a}-2.25\%$
test_step_mdp_speed[False-False-False-True-True] 0.2315ms 42.6936μs 23.4227 KOps/s 23.7035 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[False-False-False-True-False] 0.2189ms 29.4209μs 33.9895 KOps/s 33.9717 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-False-False-False-True] 0.2160ms 28.0164μs 35.6933 KOps/s 35.4146 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-False-False-False-False] 42.1300μs 18.8756μs 52.9784 KOps/s 53.2974 KOps/s $\color{#d91a1a}-0.60\%$
test_values[generalized_advantage_estimate-True-True] 24.7985ms 24.4127ms 40.9622 Ops/s 42.4147 Ops/s $\color{#d91a1a}-3.42\%$
test_values[vec_generalized_advantage_estimate-True-True] 90.2496ms 3.3691ms 296.8189 Ops/s 306.0717 Ops/s $\color{#d91a1a}-3.02\%$
test_values[td0_return_estimate-False-False] 89.1120μs 61.2666μs 16.3221 KOps/s 16.8359 KOps/s $\color{#d91a1a}-3.05\%$
test_values[td1_return_estimate-False-False] 53.0761ms 52.4097ms 19.0804 Ops/s 19.7446 Ops/s $\color{#d91a1a}-3.36\%$
test_values[vec_td1_return_estimate-False-False] 2.1074ms 1.7641ms 566.8654 Ops/s 560.6802 Ops/s $\color{#35bf28}+1.10\%$
test_values[td_lambda_return_estimate-True-False] 88.9023ms 84.1604ms 11.8821 Ops/s 12.5037 Ops/s $\color{#d91a1a}-4.97\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.1086ms 1.7777ms 562.5094 Ops/s 575.7919 Ops/s $\color{#d91a1a}-2.31\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.9429ms 24.6728ms 40.5305 Ops/s 45.6015 Ops/s $\textbf{\color{#d91a1a}-11.12\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8789ms 0.6981ms 1.4324 KOps/s 1.4750 KOps/s $\color{#d91a1a}-2.89\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7143ms 0.6770ms 1.4771 KOps/s 1.5781 KOps/s $\textbf{\color{#d91a1a}-6.40\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5370ms 1.4674ms 681.4543 Ops/s 695.1670 Ops/s $\color{#d91a1a}-1.97\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9240ms 0.6701ms 1.4922 KOps/s 1.5282 KOps/s $\color{#d91a1a}-2.35\%$
test_dqn_speed 14.0124ms 7.3739ms 135.6126 Ops/s 140.8917 Ops/s $\color{#d91a1a}-3.75\%$
test_ddpg_speed 15.3475ms 14.2829ms 70.0138 Ops/s 72.8401 Ops/s $\color{#d91a1a}-3.88\%$
test_sac_speed 30.2439ms 29.1877ms 34.2610 Ops/s 35.8443 Ops/s $\color{#d91a1a}-4.42\%$
test_redq_speed 49.1870ms 47.7553ms 20.9401 Ops/s 21.5767 Ops/s $\color{#d91a1a}-2.95\%$
test_redq_deprec_speed 24.8385ms 23.9409ms 41.7695 Ops/s 42.9955 Ops/s $\color{#d91a1a}-2.85\%$
test_td3_speed 29.2635ms 19.8336ms 50.4196 Ops/s 52.3446 Ops/s $\color{#d91a1a}-3.68\%$
test_cql_speed 84.6432ms 82.8677ms 12.0674 Ops/s 12.3875 Ops/s $\color{#d91a1a}-2.58\%$
test_a2c_speed 27.8706ms 26.8760ms 37.2079 Ops/s 39.0203 Ops/s $\color{#d91a1a}-4.64\%$
test_ppo_speed 27.8435ms 26.8323ms 37.2686 Ops/s 38.3686 Ops/s $\color{#d91a1a}-2.87\%$
test_reinforce_speed 27.1735ms 26.0230ms 38.4276 Ops/s 40.1472 Ops/s $\color{#d91a1a}-4.28\%$
test_iql_speed 58.5602ms 57.3796ms 17.4278 Ops/s 17.8513 Ops/s $\color{#d91a1a}-2.37\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.6029ms 1.8421ms 542.8492 Ops/s 552.7398 Ops/s $\color{#d91a1a}-1.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0537ms 0.8458ms 1.1823 KOps/s 1.1877 KOps/s $\color{#d91a1a}-0.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.0379ms 0.8236ms 1.2141 KOps/s 1.2220 KOps/s $\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5810ms 1.8201ms 549.4282 Ops/s 564.6449 Ops/s $\color{#d91a1a}-2.69\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0495ms 0.8330ms 1.2005 KOps/s 1.2059 KOps/s $\color{#d91a1a}-0.45\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.0182ms 0.8109ms 1.2332 KOps/s 1.2411 KOps/s $\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.9649ms 2.0736ms 482.2440 Ops/s 484.3446 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1672ms 0.9640ms 1.0374 KOps/s 1.0449 KOps/s $\color{#d91a1a}-0.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1450ms 0.9412ms 1.0624 KOps/s 1.0696 KOps/s $\color{#d91a1a}-0.67\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.2275ms 1.8389ms 543.8081 Ops/s 552.7225 Ops/s $\color{#d91a1a}-1.61\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0482ms 0.8464ms 1.1815 KOps/s 1.1870 KOps/s $\color{#d91a1a}-0.46\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9628ms 0.8243ms 1.2131 KOps/s 1.2203 KOps/s $\color{#d91a1a}-0.58\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5788ms 1.8257ms 547.7499 Ops/s 488.4158 Ops/s $\textbf{\color{#35bf28}+12.15\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9547ms 0.8365ms 1.1954 KOps/s 1.2058 KOps/s $\color{#d91a1a}-0.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.0397ms 0.8178ms 1.2228 KOps/s 1.2377 KOps/s $\color{#d91a1a}-1.20\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.1251s 2.3696ms 422.0124 Ops/s 484.7610 Ops/s $\textbf{\color{#d91a1a}-12.94\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1683ms 0.9701ms 1.0308 KOps/s 1.0435 KOps/s $\color{#d91a1a}-1.22\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1560ms 0.9507ms 1.0519 KOps/s 1.0654 KOps/s $\color{#d91a1a}-1.27\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1458s 10.6056ms 94.2896 Ops/s 101.0697 Ops/s $\textbf{\color{#d91a1a}-6.71\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.3201ms 14.6332ms 68.3376 Ops/s 71.2862 Ops/s $\color{#d91a1a}-4.14\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.8614ms 3.3951ms 294.5421 Ops/s 304.4775 Ops/s $\color{#d91a1a}-3.26\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1190s 9.9770ms 100.2305 Ops/s 100.0647 Ops/s $\color{#35bf28}+0.17\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.3409ms 14.5141ms 68.8983 Ops/s 71.2107 Ops/s $\color{#d91a1a}-3.25\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.1549ms 3.4378ms 290.8834 Ops/s 302.7580 Ops/s $\color{#d91a1a}-3.92\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1194s 10.2129ms 97.9158 Ops/s 98.4061 Ops/s $\color{#d91a1a}-0.50\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1239s 16.7475ms 59.7103 Ops/s 60.8496 Ops/s $\color{#d91a1a}-1.87\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.3446ms 3.5409ms 282.4171 Ops/s 286.7133 Ops/s $\color{#d91a1a}-1.50\%$

@vmoens vmoens added the enhancement New feature or request label Jan 23, 2024
@vmoens vmoens merged commit 5b67dd3 into main Jan 23, 2024
62 of 65 checks passed
@vmoens vmoens deleted the not-functional-objectives branch January 23, 2024 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@vmoens @facebook-github-bot