Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] replay_buffer_chunk #2388

Merged
merged 15 commits into from
Aug 13, 2024
Merged

[Feature] replay_buffer_chunk #2388

merged 15 commits into from
Aug 13, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 10, 2024 •
edited
Loading

@vmoens
Update
dd9043e 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Aug 10, 2024
Merged
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 10, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2388

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 13 Unrelated Failures

As of commit f6abfb2 with merge base 2b975da (image):

NEW FAILURE - The following job has failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Aug 10, 2024
Closed
Merged
vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Feature] replay_buffer_chunk
0538a9b 
ghstack-source-id: dd61faeee52c6b4db328821e75d186b91ba077bf
Pull Request resolved: #2388
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 10, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 10, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 77.8497ms 58.5105ms 17.0910 Ops/s 17.4981 Ops/s $\color{#d91a1a}-2.33\%$
test_sync 50.8175ms 33.6698ms 29.7002 Ops/s 31.1682 Ops/s $\color{#d91a1a}-4.71\%$
test_async 73.0156ms 31.3475ms 31.9005 Ops/s 32.2593 Ops/s $\color{#d91a1a}-1.11\%$
test_simple 0.4758s 0.4066s 2.4595 Ops/s 2.4386 Ops/s $\color{#35bf28}+0.86\%$
test_transformed 0.6224s 0.5569s 1.7957 Ops/s 1.7690 Ops/s $\color{#35bf28}+1.51\%$
test_serial 1.2954s 1.2354s 0.8095 Ops/s 0.8003 Ops/s $\color{#35bf28}+1.14\%$
test_parallel 1.1663s 1.1045s 0.9054 Ops/s 0.8981 Ops/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[True-True-True-True-True] 0.1625ms 25.3251μs 39.4866 KOps/s 38.1846 KOps/s $\color{#35bf28}+3.41\%$
test_step_mdp_speed[True-True-True-True-False] 59.2000μs 14.3247μs 69.8093 KOps/s 65.9513 KOps/s $\textbf{\color{#35bf28}+5.85\%}$
test_step_mdp_speed[True-True-True-False-True] 43.4010μs 14.2317μs 70.2657 KOps/s 68.8020 KOps/s $\color{#35bf28}+2.13\%$
test_step_mdp_speed[True-True-True-False-False] 52.5380μs 8.2131μs 121.7573 KOps/s 119.7426 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[True-True-False-True-True] 56.8860μs 26.6850μs 37.4742 KOps/s 37.0992 KOps/s $\color{#35bf28}+1.01\%$
test_step_mdp_speed[True-True-False-True-False] 68.4080μs 15.9549μs 62.6767 KOps/s 62.0592 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[True-True-False-False-True] 44.7640μs 15.8089μs 63.2555 KOps/s 62.3199 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[True-True-False-False-False] 41.9790μs 9.7631μs 102.4263 KOps/s 101.6170 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-False-True-True-True] 75.0200μs 28.3697μs 35.2488 KOps/s 34.8875 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[True-False-True-True-False] 84.2980μs 17.6597μs 56.6262 KOps/s 56.3889 KOps/s $\color{#35bf28}+0.42\%$
test_step_mdp_speed[True-False-True-False-True] 58.5790μs 15.8262μs 63.1865 KOps/s 62.4078 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-False-True-False-False] 59.4620μs 9.8787μs 101.2282 KOps/s 98.8972 KOps/s $\color{#35bf28}+2.36\%$
test_step_mdp_speed[True-False-False-True-True] 67.7170μs 29.7548μs 33.6080 KOps/s 33.2968 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[True-False-False-True-False] 67.3970μs 19.0498μs 52.4941 KOps/s 52.1081 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-False-False-False-True] 59.2110μs 17.2520μs 57.9644 KOps/s 56.7646 KOps/s $\color{#35bf28}+2.11\%$
test_step_mdp_speed[True-False-False-False-False] 53.6300μs 11.3102μs 88.4158 KOps/s 88.8606 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[False-True-True-True-True] 58.9200μs 28.3161μs 35.3156 KOps/s 34.6042 KOps/s $\color{#35bf28}+2.06\%$
test_step_mdp_speed[False-True-True-True-False] 74.7100μs 17.6431μs 56.6795 KOps/s 56.7278 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[False-True-True-False-True] 66.9850μs 18.6384μs 53.6526 KOps/s 53.2409 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[False-True-True-False-False] 61.0950μs 11.0936μs 90.1419 KOps/s 89.7707 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-True-False-True-True] 0.1156ms 29.9862μs 33.3486 KOps/s 33.3003 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-True-False-True-False] 61.8760μs 19.1197μs 52.3021 KOps/s 52.3653 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[False-True-False-False-True] 78.8170μs 19.9734μs 50.0667 KOps/s 49.8614 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-True-False-False-False] 40.8070μs 12.6120μs 79.2898 KOps/s 79.7903 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-False-True-True-True] 3.1034ms 31.6458μs 31.5998 KOps/s 31.4609 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-False-True-True-False] 49.4620μs 20.8038μs 48.0681 KOps/s 48.3121 KOps/s $\color{#d91a1a}-0.51\%$
test_step_mdp_speed[False-False-True-False-True] 63.9200μs 20.1375μs 49.6585 KOps/s 50.4979 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-False-True-False-False] 64.8920μs 12.6562μs 79.0129 KOps/s 79.4793 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-False-False-True-True] 87.1830μs 33.1228μs 30.1907 KOps/s 30.4228 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[False-False-False-True-False] 72.1150μs 22.1874μs 45.0707 KOps/s 45.7890 KOps/s $\color{#d91a1a}-1.57\%$
test_step_mdp_speed[False-False-False-False-True] 54.0610μs 21.2763μs 47.0006 KOps/s 47.2645 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[False-False-False-False-False] 64.4010μs 14.0864μs 70.9906 KOps/s 71.9569 KOps/s $\color{#d91a1a}-1.34\%$
test_values[generalized_advantage_estimate-True-True] 11.7448ms 9.4927ms 105.3444 Ops/s 104.8656 Ops/s $\color{#35bf28}+0.46\%$
test_values[vec_generalized_advantage_estimate-True-True] 38.3521ms 33.6204ms 29.7438 Ops/s 28.1281 Ops/s $\textbf{\color{#35bf28}+5.74\%}$
test_values[td0_return_estimate-False-False] 0.2246ms 0.1721ms 5.8101 KOps/s 6.0107 KOps/s $\color{#d91a1a}-3.34\%$
test_values[td1_return_estimate-False-False] 34.4234ms 23.9057ms 41.8311 Ops/s 41.9337 Ops/s $\color{#d91a1a}-0.24\%$
test_values[vec_td1_return_estimate-False-False] 35.4307ms 33.4750ms 29.8730 Ops/s 28.0037 Ops/s $\textbf{\color{#35bf28}+6.68\%}$
test_values[td_lambda_return_estimate-True-False] 36.4666ms 33.8276ms 29.5617 Ops/s 29.3759 Ops/s $\color{#35bf28}+0.63\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.6607ms 33.5368ms 29.8180 Ops/s 28.0083 Ops/s $\textbf{\color{#35bf28}+6.46\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.7513ms 8.3036ms 120.4300 Ops/s 121.1618 Ops/s $\color{#d91a1a}-0.60\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2176ms 1.8831ms 531.0304 Ops/s 562.8152 Ops/s $\textbf{\color{#d91a1a}-5.65\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4767ms 0.3460ms 2.8901 KOps/s 2.8575 KOps/s $\color{#35bf28}+1.14\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 42.0758ms 40.2551ms 24.8416 Ops/s 21.0268 Ops/s $\textbf{\color{#35bf28}+18.14\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.4509ms 3.0402ms 328.9208 Ops/s 327.6371 Ops/s $\color{#35bf28}+0.39\%$
test_dqn_speed 6.1494ms 1.2705ms 787.1020 Ops/s 797.0526 Ops/s $\color{#d91a1a}-1.25\%$
test_ddpg_speed 3.8850ms 2.6528ms 376.9672 Ops/s 374.2545 Ops/s $\color{#35bf28}+0.72\%$
test_sac_speed 9.7843ms 7.7786ms 128.5575 Ops/s 123.8817 Ops/s $\color{#35bf28}+3.77\%$
test_redq_speed 13.4621ms 12.7170ms 78.6348 Ops/s 77.7305 Ops/s $\color{#35bf28}+1.16\%$
test_redq_deprec_speed 13.7059ms 12.7771ms 78.2649 Ops/s 77.0552 Ops/s $\color{#35bf28}+1.57\%$
test_td3_speed 7.9578ms 7.6897ms 130.0442 Ops/s 128.2547 Ops/s $\color{#35bf28}+1.40\%$
test_cql_speed 36.3271ms 35.1123ms 28.4800 Ops/s 25.8037 Ops/s $\textbf{\color{#35bf28}+10.37\%}$
test_a2c_speed 8.0233ms 7.2669ms 137.6108 Ops/s 136.1217 Ops/s $\color{#35bf28}+1.09\%$
test_ppo_speed 9.0014ms 7.5374ms 132.6713 Ops/s 130.3522 Ops/s $\color{#35bf28}+1.78\%$
test_reinforce_speed 7.1816ms 6.4165ms 155.8484 Ops/s 152.8792 Ops/s $\color{#35bf28}+1.94\%$
test_iql_speed 34.0861ms 31.9992ms 31.2508 Ops/s 31.3395 Ops/s $\color{#d91a1a}-0.28\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.0579ms 4.8154ms 207.6674 Ops/s 203.8690 Ops/s $\color{#35bf28}+1.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8462ms 0.4795ms 2.0853 KOps/s 2.0831 KOps/s $\color{#35bf28}+0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6303ms 0.4515ms 2.2149 KOps/s 2.2006 KOps/s $\color{#35bf28}+0.65\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1295ms 4.8137ms 207.7410 Ops/s 203.1293 Ops/s $\color{#35bf28}+2.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7308ms 0.4673ms 2.1400 KOps/s 2.1094 KOps/s $\color{#35bf28}+1.45\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6076ms 0.4443ms 2.2506 KOps/s 2.2146 KOps/s $\color{#35bf28}+1.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2067ms 1.6618ms 601.7696 Ops/s 605.1186 Ops/s $\color{#d91a1a}-0.55\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3599ms 1.5758ms 634.6099 Ops/s 635.2480 Ops/s $\color{#d91a1a}-0.10\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.1942ms 4.9179ms 203.3396 Ops/s 198.5778 Ops/s $\color{#35bf28}+2.40\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1387s 0.7172ms 1.3943 KOps/s 1.6445 KOps/s $\textbf{\color{#d91a1a}-15.22\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7189ms 0.5766ms 1.7343 KOps/s 1.7080 KOps/s $\color{#35bf28}+1.54\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.9432ms 4.7867ms 208.9137 Ops/s 201.7414 Ops/s $\color{#35bf28}+3.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7051ms 0.4776ms 2.0936 KOps/s 2.0507 KOps/s $\color{#35bf28}+2.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.8713ms 0.5034ms 1.9866 KOps/s 2.0813 KOps/s $\color{#d91a1a}-4.55\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1960ms 4.8152ms 207.6751 Ops/s 199.8891 Ops/s $\color{#35bf28}+3.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.4505ms 0.4858ms 2.0585 KOps/s 2.0844 KOps/s $\color{#d91a1a}-1.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6394ms 0.4520ms 2.2122 KOps/s 2.1771 KOps/s $\color{#35bf28}+1.61\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5927ms 5.0322ms 198.7207 Ops/s 199.0429 Ops/s $\color{#d91a1a}-0.16\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.5840ms 0.6061ms 1.6499 KOps/s 1.6520 KOps/s $\color{#d91a1a}-0.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7971ms 0.5871ms 1.7032 KOps/s 1.6871 KOps/s $\color{#35bf28}+0.95\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1314s 6.4884ms 154.1220 Ops/s 162.3422 Ops/s $\textbf{\color{#d91a1a}-5.06\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1186s 15.0208ms 66.5745 Ops/s 71.8744 Ops/s $\textbf{\color{#d91a1a}-7.37\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.8006ms 1.2504ms 799.7200 Ops/s 885.7010 Ops/s $\textbf{\color{#d91a1a}-9.71\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1090s 5.9478ms 168.1288 Ops/s 165.7299 Ops/s $\color{#35bf28}+1.45\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.6770ms 13.0449ms 76.6585 Ops/s 78.6748 Ops/s $\color{#d91a1a}-2.56\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1031ms 1.3164ms 759.6242 Ops/s 858.9413 Ops/s $\textbf{\color{#d91a1a}-11.56\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1107s 6.1239ms 163.2956 Ops/s 125.7348 Ops/s $\textbf{\color{#35bf28}+29.87\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.4200ms 13.1063ms 76.2991 Ops/s 77.7886 Ops/s $\color{#d91a1a}-1.91\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.7191ms 1.5170ms 659.1866 Ops/s 849.3622 Ops/s $\textbf{\color{#d91a1a}-22.39\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 10, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1078s 0.1062s 9.4144 Ops/s 9.3455 Ops/s $\color{#35bf28}+0.74\%$
test_sync 99.6803ms 93.7218ms 10.6699 Ops/s 10.9576 Ops/s $\color{#d91a1a}-2.63\%$
test_async 0.1767s 88.1548ms 11.3437 Ops/s 11.0277 Ops/s $\color{#35bf28}+2.87\%$
test_single_pixels 0.1169s 0.1167s 8.5679 Ops/s 8.5205 Ops/s $\color{#35bf28}+0.56\%$
test_sync_pixels 76.7611ms 75.8065ms 13.1915 Ops/s 13.3856 Ops/s $\color{#d91a1a}-1.45\%$
test_async_pixels 0.1436s 71.1426ms 14.0563 Ops/s 13.8640 Ops/s $\color{#35bf28}+1.39\%$
test_simple 0.7578s 0.7554s 1.3238 Ops/s 1.2808 Ops/s $\color{#35bf28}+3.36\%$
test_transformed 1.0840s 1.0060s 0.9940 Ops/s 0.9989 Ops/s $\color{#d91a1a}-0.49\%$
test_serial 2.2406s 2.1634s 0.4622 Ops/s 0.4580 Ops/s $\color{#35bf28}+0.93\%$
test_parallel 1.9510s 1.8860s 0.5302 Ops/s 0.5282 Ops/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-True-True-True-True] 0.1379ms 38.8517μs 25.7389 KOps/s 25.7473 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[True-True-True-True-False] 91.8210μs 21.4953μs 46.5217 KOps/s 46.2699 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-True-True-False-True] 0.1238ms 21.3702μs 46.7942 KOps/s 47.1325 KOps/s $\color{#d91a1a}-0.72\%$
test_step_mdp_speed[True-True-True-False-False] 0.1268ms 12.4443μs 80.3578 KOps/s 83.0145 KOps/s $\color{#d91a1a}-3.20\%$
test_step_mdp_speed[True-True-False-True-True] 76.0720μs 40.5234μs 24.6771 KOps/s 24.8056 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[True-True-False-True-False] 58.9030μs 23.6234μs 42.3309 KOps/s 42.1951 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-True-False-False-True] 65.2710μs 23.7776μs 42.0564 KOps/s 41.7765 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[True-True-False-False-False] 0.1894ms 14.5473μs 68.7415 KOps/s 69.7556 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-False-True-True-True] 0.2450ms 42.9731μs 23.2704 KOps/s 23.7198 KOps/s $\color{#d91a1a}-1.89\%$
test_step_mdp_speed[True-False-True-True-False] 0.1069ms 26.1300μs 38.2702 KOps/s 38.6339 KOps/s $\color{#d91a1a}-0.94\%$
test_step_mdp_speed[True-False-True-False-True] 0.1869ms 23.3121μs 42.8962 KOps/s 42.6141 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[True-False-True-False-False] 44.2910μs 14.5082μs 68.9263 KOps/s 69.7645 KOps/s $\color{#d91a1a}-1.20\%$
test_step_mdp_speed[True-False-False-True-True] 83.8720μs 45.2737μs 22.0879 KOps/s 22.5119 KOps/s $\color{#d91a1a}-1.88\%$
test_step_mdp_speed[True-False-False-True-False] 0.2256ms 27.9562μs 35.7703 KOps/s 35.6754 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-False-False-False-True] 90.7020μs 25.7266μs 38.8703 KOps/s 38.7167 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-False-False-False-False] 42.5310μs 16.5864μs 60.2903 KOps/s 60.1158 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-True-True-True-True] 78.7620μs 42.9116μs 23.3037 KOps/s 23.5953 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-True-True-True-False] 48.6900μs 26.1556μs 38.2327 KOps/s 38.7276 KOps/s $\color{#d91a1a}-1.28\%$
test_step_mdp_speed[False-True-True-False-True] 0.1414ms 28.2872μs 35.3517 KOps/s 35.5669 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[False-True-True-False-False] 49.8120μs 16.5698μs 60.3507 KOps/s 61.6718 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-True-False-True-True] 84.1020μs 45.0104μs 22.2171 KOps/s 22.3054 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-False-True-False] 53.0010μs 28.0192μs 35.6898 KOps/s 35.7180 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-True-False-False-True] 64.8620μs 29.9515μs 33.3874 KOps/s 33.2981 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-True-False-False-False] 37.0610μs 18.6005μs 53.7621 KOps/s 54.1055 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-False-True-True-True] 4.0974ms 47.2163μs 21.1791 KOps/s 20.9926 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-False-True-True-False] 63.9410μs 30.6741μs 32.6008 KOps/s 32.6896 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-False-True-False-True] 61.4320μs 30.0656μs 33.2606 KOps/s 32.8253 KOps/s $\color{#35bf28}+1.33\%$
test_step_mdp_speed[False-False-True-False-False] 57.1210μs 18.4247μs 54.2749 KOps/s 54.1723 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[False-False-False-True-True] 90.9420μs 48.2267μs 20.7354 KOps/s 20.4138 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[False-False-False-True-False] 56.6510μs 33.0473μs 30.2597 KOps/s 30.6154 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[False-False-False-False-True] 56.7420μs 32.5311μs 30.7398 KOps/s 31.1529 KOps/s $\color{#d91a1a}-1.33\%$
test_step_mdp_speed[False-False-False-False-False] 0.2009ms 20.6706μs 48.3779 KOps/s 48.5733 KOps/s $\color{#d91a1a}-0.40\%$
test_values[generalized_advantage_estimate-True-True] 24.5170ms 24.1758ms 41.3636 Ops/s 40.3810 Ops/s $\color{#35bf28}+2.43\%$
test_values[vec_generalized_advantage_estimate-True-True] 94.9207ms 2.7947ms 357.8224 Ops/s 349.6681 Ops/s $\color{#35bf28}+2.33\%$
test_values[td0_return_estimate-False-False] 90.8830μs 63.8254μs 15.6677 KOps/s 15.1822 KOps/s $\color{#35bf28}+3.20\%$
test_values[td1_return_estimate-False-False] 57.4999ms 55.4435ms 18.0364 Ops/s 18.2316 Ops/s $\color{#d91a1a}-1.07\%$
test_values[vec_td1_return_estimate-False-False] 1.2851ms 1.0807ms 925.3160 Ops/s 918.5451 Ops/s $\color{#35bf28}+0.74\%$
test_values[td_lambda_return_estimate-True-False] 92.1077ms 86.4304ms 11.5700 Ops/s 11.5005 Ops/s $\color{#35bf28}+0.60\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2690ms 1.0756ms 929.6779 Ops/s 921.0527 Ops/s $\color{#35bf28}+0.94\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.8513ms 24.4486ms 40.9021 Ops/s 40.5907 Ops/s $\color{#35bf28}+0.77\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9582ms 0.7151ms 1.3985 KOps/s 1.3738 KOps/s $\color{#35bf28}+1.79\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8123ms 0.6645ms 1.5048 KOps/s 1.4936 KOps/s $\color{#35bf28}+0.75\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6220ms 1.4643ms 682.9246 Ops/s 678.5751 Ops/s $\color{#35bf28}+0.64\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8509ms 0.6785ms 1.4737 KOps/s 1.3996 KOps/s $\textbf{\color{#35bf28}+5.30\%}$
test_dqn_speed 7.3641ms 1.4131ms 707.6481 Ops/s 712.1486 Ops/s $\color{#d91a1a}-0.63\%$
test_ddpg_speed 3.1277ms 2.7914ms 358.2423 Ops/s 359.0197 Ops/s $\color{#d91a1a}-0.22\%$
test_sac_speed 0.1067s 8.7938ms 113.7166 Ops/s 125.0501 Ops/s $\textbf{\color{#d91a1a}-9.06\%}$
test_redq_speed 12.3624ms 10.5680ms 94.6251 Ops/s 95.6442 Ops/s $\color{#d91a1a}-1.07\%$
test_redq_deprec_speed 11.1800ms 10.8085ms 92.5200 Ops/s 90.1582 Ops/s $\color{#35bf28}+2.62\%$
test_td3_speed 8.3797ms 7.9210ms 126.2470 Ops/s 126.1106 Ops/s $\color{#35bf28}+0.11\%$
test_cql_speed 26.3206ms 25.5458ms 39.1455 Ops/s 39.3317 Ops/s $\color{#d91a1a}-0.47\%$
test_a2c_speed 6.1109ms 5.6105ms 178.2361 Ops/s 181.1975 Ops/s $\color{#d91a1a}-1.63\%$
test_ppo_speed 6.3958ms 5.9638ms 167.6796 Ops/s 171.6369 Ops/s $\color{#d91a1a}-2.31\%$
test_reinforce_speed 4.7993ms 4.5253ms 220.9822 Ops/s 223.8889 Ops/s $\color{#d91a1a}-1.30\%$
test_iql_speed 20.3600ms 19.8383ms 50.4075 Ops/s 50.7635 Ops/s $\color{#d91a1a}-0.70\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.0272ms 6.7126ms 148.9742 Ops/s 149.6243 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2335ms 0.5231ms 1.9117 KOps/s 1.9073 KOps/s $\color{#35bf28}+0.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7761ms 0.5016ms 1.9935 KOps/s 1.9858 KOps/s $\color{#35bf28}+0.39\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9916ms 6.5688ms 152.2349 Ops/s 151.2130 Ops/s $\color{#35bf28}+0.68\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7170ms 0.5165ms 1.9360 KOps/s 1.9187 KOps/s $\color{#35bf28}+0.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6788ms 0.4967ms 2.0132 KOps/s 2.0199 KOps/s $\color{#d91a1a}-0.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1676ms 2.0000ms 499.9884 Ops/s 505.1622 Ops/s $\color{#d91a1a}-1.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1685ms 1.9083ms 524.0250 Ops/s 531.3279 Ops/s $\color{#d91a1a}-1.37\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0431ms 6.7675ms 147.7659 Ops/s 147.3427 Ops/s $\color{#35bf28}+0.29\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1345s 0.7993ms 1.2511 KOps/s 1.4793 KOps/s $\textbf{\color{#d91a1a}-15.42\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8529ms 0.6566ms 1.5229 KOps/s 1.5338 KOps/s $\color{#d91a1a}-0.71\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 9.3384ms 6.8567ms 145.8435 Ops/s 150.7229 Ops/s $\color{#d91a1a}-3.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.6372ms 0.5230ms 1.9120 KOps/s 1.8996 KOps/s $\color{#35bf28}+0.65\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7070ms 0.5029ms 1.9886 KOps/s 1.9744 KOps/s $\color{#35bf28}+0.72\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.0324ms 6.5866ms 151.8243 Ops/s 152.1400 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7601ms 0.5197ms 1.9240 KOps/s 1.9133 KOps/s $\color{#35bf28}+0.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.3539ms 0.5071ms 1.9719 KOps/s 2.0161 KOps/s $\color{#d91a1a}-2.19\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0504ms 6.8397ms 146.2050 Ops/s 146.3487 Ops/s $\color{#d91a1a}-0.10\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1731ms 0.6798ms 1.4709 KOps/s 1.4736 KOps/s $\color{#d91a1a}-0.18\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8344ms 0.6550ms 1.5267 KOps/s 1.5133 KOps/s $\color{#35bf28}+0.89\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1519s 8.3223ms 120.1585 Ops/s 124.8735 Ops/s $\color{#d91a1a}-3.78\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.1244ms 16.2701ms 61.4626 Ops/s 53.6934 Ops/s $\textbf{\color{#35bf28}+14.47\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2188ms 1.3127ms 761.7886 Ops/s 789.8060 Ops/s $\color{#d91a1a}-3.55\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1328s 10.5203ms 95.0547 Ops/s 129.9537 Ops/s $\textbf{\color{#d91a1a}-26.85\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.4355ms 16.3698ms 61.0881 Ops/s 60.2043 Ops/s $\color{#35bf28}+1.47\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.7884ms 1.4242ms 702.1552 Ops/s 755.6949 Ops/s $\textbf{\color{#d91a1a}-7.08\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1299s 8.0733ms 123.8644 Ops/s 127.2140 Ops/s $\color{#d91a1a}-2.63\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.3033ms 16.4828ms 60.6692 Ops/s 60.8976 Ops/s $\color{#d91a1a}-0.38\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 3.0246ms 1.6869ms 592.8198 Ops/s 641.6070 Ops/s $\textbf{\color{#d91a1a}-7.60\%}$

@vmoens
Update
0af4014 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Feature] replay_buffer_chunk
e7d7d43 
ghstack-source-id: 4abe903dc1d3643d793f54f93cb4fe147cce8d06
Pull Request resolved: #2388
vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Feature] replay_buffer_chunk
7e2d205 
ghstack-source-id: 4abe903dc1d3643d793f54f93cb4fe147cce8d06
Pull Request resolved: #2388
@vmoens
Update
f8e3a24 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Feature] replay_buffer_chunk
b788bad 
ghstack-source-id: 8c4663d44a228554a51ec153ceb11c7b0c355160
Pull Request resolved: #2388
@vmoens
Update
a587b30 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Aug 10, 2024
@vmoens
[Feature] replay_buffer_chunk
ea2fb87 
ghstack-source-id: 10f4a291dbd92019f9ad1c9f57681f69e39402a2
Pull Request resolved: #2388
@vmoens
Update
fd24684 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Aug 10, 2024
Open
vmoens added 10 commits August 10, 2024 17:58
@vmoens
Update
279fe81 
[ghstack-poisoned]
@vmoens
Update
e1b5551 
[ghstack-poisoned]
@vmoens
Update
96bb83e 
[ghstack-poisoned]
@vmoens
Update
6f04915 
[ghstack-poisoned]
@vmoens
Update
df6efa9 
[ghstack-poisoned]
@vmoens
Update
64b8898 
[ghstack-poisoned]
@vmoens
Update
f391838 
[ghstack-poisoned]
@vmoens
Update
c72fbcd 
[ghstack-poisoned]
@vmoens
Update
0f73687 
[ghstack-poisoned]
@vmoens
Update
f6abfb2 
[ghstack-poisoned]
@vmoens vmoens merged commit f6abfb2 into gh/vmoens/16/base Aug 13, 2024
33 of 41 checks passed
vmoens added a commit that referenced this pull request Aug 13, 2024
@vmoens
[Feature] replay_buffer_chunk
012cf74 
ghstack-source-id: e5f82a74f37dc66c16f595b53fe586d9fa43fc8a
Pull Request resolved: #2388
@vmoens vmoens deleted the gh/vmoens/16/head branch August 13, 2024 19:59
@vmoens vmoens added the enhancement New feature or request label Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot