-
Notifications
You must be signed in to change notification settings - Fork 285
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] pickle-free RB checkpointing #1733
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1733
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (5 Unrelated Failures)As of commit 4fc7c31 with merge base 841f8d9 ( FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Name | Max | Mean | Ops | Ops on Repo HEAD
|
Change |
---|---|---|---|---|---|
test_single | 62.9151ms | 62.2233ms | 16.0712 Ops/s | 15.2832 Ops/s | |
test_sync | 40.6082ms | 34.5239ms | 28.9655 Ops/s | 25.0034 Ops/s | |
test_async | 68.1264ms | 33.0288ms | 30.2766 Ops/s | 30.8303 Ops/s | |
test_simple | 0.4861s | 0.4328s | 2.3108 Ops/s | 2.3175 Ops/s | |
test_transformed | 0.6443s | 0.5980s | 1.6724 Ops/s | 1.6697 Ops/s | |
test_serial | 1.3708s | 1.3254s | 0.7545 Ops/s | 0.7447 Ops/s | |
test_parallel | 1.3470s | 1.2842s | 0.7787 Ops/s | 0.7800 Ops/s | |
test_step_mdp_speed[True-True-True-True-True] | 0.1627ms | 22.6787μs | 44.0942 KOps/s | 44.4613 KOps/s | |
test_step_mdp_speed[True-True-True-True-False] | 58.3160μs | 13.6311μs | 73.3617 KOps/s | 73.2030 KOps/s | |
test_step_mdp_speed[True-True-True-False-True] | 44.6430μs | 13.7151μs | 72.9125 KOps/s | 73.1714 KOps/s | |
test_step_mdp_speed[True-True-True-False-False] | 30.4770μs | 8.2356μs | 121.4243 KOps/s | 123.3177 KOps/s | |
test_step_mdp_speed[True-True-False-True-True] | 75.4130μs | 23.9349μs | 41.7801 KOps/s | 41.8433 KOps/s | |
test_step_mdp_speed[True-True-False-True-False] | 36.4680μs | 14.8831μs | 67.1902 KOps/s | 66.8146 KOps/s | |
test_step_mdp_speed[True-True-False-False-True] | 51.6060μs | 15.0438μs | 66.4725 KOps/s | 66.7706 KOps/s | |
test_step_mdp_speed[True-True-False-False-False] | 34.1030μs | 9.5265μs | 104.9702 KOps/s | 105.7078 KOps/s | |
test_step_mdp_speed[True-False-True-True-True] | 78.7890μs | 25.1035μs | 39.8350 KOps/s | 39.0040 KOps/s | |
test_step_mdp_speed[True-False-True-True-False] | 56.9340μs | 16.3464μs | 61.1757 KOps/s | 61.4928 KOps/s | |
test_step_mdp_speed[True-False-True-False-True] | 45.6350μs | 14.8761μs | 67.2217 KOps/s | 66.7026 KOps/s | |
test_step_mdp_speed[True-False-True-False-False] | 71.3540μs | 9.5869μs | 104.3087 KOps/s | 105.0892 KOps/s | |
test_step_mdp_speed[True-False-False-True-True] | 74.4190μs | 26.4385μs | 37.8236 KOps/s | 37.6605 KOps/s | |
test_step_mdp_speed[True-False-False-True-False] | 62.8980μs | 17.4917μs | 57.1701 KOps/s | 57.1340 KOps/s | |
test_step_mdp_speed[True-False-False-False-True] | 44.0320μs | 16.1448μs | 61.9394 KOps/s | 62.2388 KOps/s | |
test_step_mdp_speed[True-False-False-False-False] | 33.7230μs | 10.5811μs | 94.5084 KOps/s | 93.5146 KOps/s | |
test_step_mdp_speed[False-True-True-True-True] | 75.2900μs | 25.6060μs | 39.0533 KOps/s | 39.3439 KOps/s | |
test_step_mdp_speed[False-True-True-True-False] | 55.2630μs | 16.1837μs | 61.7906 KOps/s | 60.5707 KOps/s | |
test_step_mdp_speed[False-True-True-False-True] | 54.8430μs | 17.5193μs | 57.0798 KOps/s | 57.5938 KOps/s | |
test_step_mdp_speed[False-True-True-False-False] | 33.3620μs | 10.7782μs | 92.7795 KOps/s | 91.5565 KOps/s | |
test_step_mdp_speed[False-True-False-True-True] | 88.7960μs | 26.9227μs | 37.1434 KOps/s | 37.6420 KOps/s | |
test_step_mdp_speed[False-True-False-True-False] | 66.6850μs | 17.5295μs | 57.0466 KOps/s | 56.7667 KOps/s | |
test_step_mdp_speed[False-True-False-False-True] | 57.0760μs | 18.5995μs | 53.7649 KOps/s | 54.4279 KOps/s | |
test_step_mdp_speed[False-True-False-False-False] | 75.4110μs | 11.9541μs | 83.6532 KOps/s | 83.3894 KOps/s | |
test_step_mdp_speed[False-False-True-True-True] | 91.3430μs | 27.6776μs | 36.1304 KOps/s | 35.4040 KOps/s | |
test_step_mdp_speed[False-False-True-True-False] | 55.8740μs | 18.6822μs | 53.5269 KOps/s | 53.2022 KOps/s | |
test_step_mdp_speed[False-False-True-False-True] | 64.6380μs | 18.2483μs | 54.7996 KOps/s | 53.2847 KOps/s | |
test_step_mdp_speed[False-False-True-False-False] | 54.8340μs | 11.9006μs | 84.0292 KOps/s | 82.6149 KOps/s | |
test_step_mdp_speed[False-False-False-True-True] | 76.6820μs | 28.7189μs | 34.8203 KOps/s | 34.5102 KOps/s | |
test_step_mdp_speed[False-False-False-True-False] | 68.2370μs | 19.8288μs | 50.4317 KOps/s | 50.4791 KOps/s | |
test_step_mdp_speed[False-False-False-False-True] | 42.5300μs | 19.3583μs | 51.6575 KOps/s | 50.8523 KOps/s | |
test_step_mdp_speed[False-False-False-False-False] | 64.2300μs | 12.9377μs | 77.2932 KOps/s | 77.2606 KOps/s | |
test_values[generalized_advantage_estimate-True-True] | 14.0804ms | 12.2214ms | 81.8237 Ops/s | 82.1451 Ops/s | |
test_values[vec_generalized_advantage_estimate-True-True] | 38.1451ms | 26.8332ms | 37.2673 Ops/s | 37.0595 Ops/s | |
test_values[td0_return_estimate-False-False] | 0.2331ms | 0.1857ms | 5.3840 KOps/s | 5.4859 KOps/s | |
test_values[td1_return_estimate-False-False] | 26.4852ms | 25.8370ms | 38.7042 Ops/s | 38.5905 Ops/s | |
test_values[vec_td1_return_estimate-False-False] | 36.8707ms | 26.6234ms | 37.5610 Ops/s | 36.4389 Ops/s | |
test_values[td_lambda_return_estimate-True-False] | 45.3492ms | 36.7656ms | 27.1994 Ops/s | 27.7371 Ops/s | |
test_values[vec_td_lambda_return_estimate-True-False] | 36.4647ms | 26.7146ms | 37.4327 Ops/s | 37.0874 Ops/s | |
test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.2687ms | 8.1493ms | 122.7102 Ops/s | 123.3221 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 12.8013ms | 1.9365ms | 516.3859 Ops/s | 554.8081 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 11.1474ms | 0.4534ms | 2.2055 KOps/s | 2.2799 KOps/s | |
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 48.8113ms | 40.1179ms | 24.9265 Ops/s | 25.2061 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 11.7456ms | 2.5509ms | 392.0158 Ops/s | 391.5953 Ops/s | |
test_dqn_speed | 8.4719ms | 1.6207ms | 616.9995 Ops/s | 598.9677 Ops/s | |
test_ddpg_speed | 72.3734ms | 3.9200ms | 255.1031 Ops/s | 275.3603 Ops/s | |
test_sac_speed | 11.1166ms | 10.1538ms | 98.4855 Ops/s | 97.2126 Ops/s | |
test_redq_speed | 29.9535ms | 19.7146ms | 50.7238 Ops/s | 51.8625 Ops/s | |
test_redq_deprec_speed | 26.6949ms | 15.7117ms | 63.6467 Ops/s | 64.2872 Ops/s | |
test_td3_speed | 21.7666ms | 10.5754ms | 94.5590 Ops/s | 94.4240 Ops/s | |
test_cql_speed | 50.5758ms | 38.7258ms | 25.8226 Ops/s | 25.5123 Ops/s | |
test_a2c_speed | 18.9132ms | 8.1665ms | 122.4518 Ops/s | 118.1227 Ops/s | |
test_ppo_speed | 19.1483ms | 8.5085ms | 117.5302 Ops/s | 113.2807 Ops/s | |
test_reinforce_speed | 9.0100ms | 7.2702ms | 137.5486 Ops/s | 133.8437 Ops/s | |
test_iql_speed | 44.9871ms | 34.3962ms | 29.0730 Ops/s | 28.8146 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.9376ms | 1.9700ms | 507.6251 Ops/s | 514.0858 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 3.2949ms | 2.1141ms | 473.0104 Ops/s | 476.5658 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.6043ms | 2.1231ms | 471.0018 Ops/s | 478.7991 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.1433ms | 2.0015ms | 499.6232 Ops/s | 508.0996 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 3.3815ms | 2.1182ms | 472.1009 Ops/s | 475.2456 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.5415ms | 2.1264ms | 470.2775 Ops/s | 472.1901 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 2.5659ms | 1.9540ms | 511.7614 Ops/s | 510.5988 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 3.4308ms | 2.1203ms | 471.6303 Ops/s | 478.6612 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.2113ms | 2.1195ms | 471.8109 Ops/s | 472.8666 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.1583ms | 1.9784ms | 505.4503 Ops/s | 497.1094 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 3.6753ms | 2.1578ms | 463.4261 Ops/s | 473.4800 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.6927ms | 2.1412ms | 467.0382 Ops/s | 464.3769 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.1670ms | 1.9806ms | 504.8937 Ops/s | 504.1485 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 3.2030ms | 2.1049ms | 475.0712 Ops/s | 464.7567 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.3591ms | 2.1179ms | 472.1624 Ops/s | 477.2363 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.6735ms | 1.9536ms | 511.8723 Ops/s | 509.5704 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 3.6365ms | 2.1072ms | 474.5563 Ops/s | 465.6575 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.9943ms | 2.1357ms | 468.2297 Ops/s | 472.0129 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1731s | 18.0667ms | 55.3505 Ops/s | 54.1027 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 0.1180s | 16.9381ms | 59.0384 Ops/s | 59.4888 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 0.1140s | 16.8184ms | 59.4588 Ops/s | 52.7334 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1118s | 16.7447ms | 59.7205 Ops/s | 59.2592 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1188s | 18.9676ms | 52.7215 Ops/s | 59.5743 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 0.1178s | 16.9409ms | 59.0286 Ops/s | 59.7943 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1160s | 16.9939ms | 58.8447 Ops/s | 59.5020 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 0.1132s | 16.8341ms | 59.4033 Ops/s | 59.9139 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 0.1150s | 16.8583ms | 59.3180 Ops/s | 59.1263 Ops/s |
|
Name | Max | Mean | Ops | Ops on Repo HEAD
|
Change |
---|---|---|---|---|---|
test_single | 0.1205s | 0.1195s | 8.3669 Ops/s | 7.9861 Ops/s | |
test_sync | 0.1026s | 0.1024s | 9.7645 Ops/s | 9.6781 Ops/s | |
test_async | 0.2709s | 0.1017s | 9.8288 Ops/s | 9.9660 Ops/s | |
test_single_pixels | 0.1431s | 0.1426s | 7.0137 Ops/s | 6.8878 Ops/s | |
test_sync_pixels | 98.9870ms | 96.3499ms | 10.3788 Ops/s | 10.5184 Ops/s | |
test_async_pixels | 0.1863s | 90.1381ms | 11.0941 Ops/s | 11.0860 Ops/s | |
test_simple | 0.9315s | 0.8641s | 1.1573 Ops/s | 1.1315 Ops/s | |
test_transformed | 1.1736s | 1.1075s | 0.9029 Ops/s | 0.8812 Ops/s | |
test_serial | 2.4834s | 2.4143s | 0.4142 Ops/s | 0.4035 Ops/s | |
test_parallel | 2.5331s | 2.4748s | 0.4041 Ops/s | 0.4022 Ops/s | |
test_step_mdp_speed[True-True-True-True-True] | 98.2140μs | 33.3779μs | 29.9599 KOps/s | 29.2569 KOps/s | |
test_step_mdp_speed[True-True-True-True-False] | 63.1330μs | 19.8401μs | 50.4030 KOps/s | 49.4269 KOps/s | |
test_step_mdp_speed[True-True-True-False-True] | 46.5620μs | 19.7651μs | 50.5942 KOps/s | 49.0509 KOps/s | |
test_step_mdp_speed[True-True-True-False-False] | 51.3830μs | 11.6575μs | 85.7820 KOps/s | 84.3114 KOps/s | |
test_step_mdp_speed[True-True-False-True-True] | 68.8030μs | 35.1269μs | 28.4682 KOps/s | 28.5071 KOps/s | |
test_step_mdp_speed[True-True-False-True-False] | 79.8740μs | 21.3821μs | 46.7680 KOps/s | 45.5486 KOps/s | |
test_step_mdp_speed[True-True-False-False-True] | 47.8720μs | 21.1618μs | 47.2549 KOps/s | 46.3383 KOps/s | |
test_step_mdp_speed[True-True-False-False-False] | 38.1810μs | 13.5105μs | 74.0165 KOps/s | 73.3348 KOps/s | |
test_step_mdp_speed[True-False-True-True-True] | 67.7930μs | 37.0224μs | 27.0107 KOps/s | 26.9799 KOps/s | |
test_step_mdp_speed[True-False-True-True-False] | 58.6130μs | 23.1678μs | 43.1633 KOps/s | 42.2529 KOps/s | |
test_step_mdp_speed[True-False-True-False-True] | 59.3930μs | 21.5460μs | 46.4124 KOps/s | 46.5639 KOps/s | |
test_step_mdp_speed[True-False-True-False-False] | 36.7220μs | 13.4554μs | 74.3194 KOps/s | 72.7338 KOps/s | |
test_step_mdp_speed[True-False-False-True-True] | 91.3540μs | 38.9625μs | 25.6657 KOps/s | 25.5189 KOps/s | |
test_step_mdp_speed[True-False-False-True-False] | 55.6530μs | 25.1462μs | 39.7674 KOps/s | 38.7271 KOps/s | |
test_step_mdp_speed[True-False-False-False-True] | 69.5930μs | 23.0974μs | 43.2948 KOps/s | 41.9656 KOps/s | |
test_step_mdp_speed[True-False-False-False-False] | 36.3110μs | 15.2978μs | 65.3688 KOps/s | 64.5861 KOps/s | |
test_step_mdp_speed[False-True-True-True-True] | 66.1430μs | 36.6634μs | 27.2751 KOps/s | 26.8271 KOps/s | |
test_step_mdp_speed[False-True-True-True-False] | 53.2220μs | 23.3674μs | 42.7946 KOps/s | 41.5047 KOps/s | |
test_step_mdp_speed[False-True-True-False-True] | 50.5020μs | 25.6738μs | 38.9502 KOps/s | 38.8456 KOps/s | |
test_step_mdp_speed[False-True-True-False-False] | 46.7220μs | 15.0498μs | 66.4460 KOps/s | 64.2572 KOps/s | |
test_step_mdp_speed[False-True-False-True-True] | 72.2040μs | 38.5857μs | 25.9163 KOps/s | 25.0134 KOps/s | |
test_step_mdp_speed[False-True-False-True-False] | 52.5720μs | 25.4382μs | 39.3110 KOps/s | 38.8786 KOps/s | |
test_step_mdp_speed[False-True-False-False-True] | 61.2230μs | 27.6824μs | 36.1240 KOps/s | 35.8732 KOps/s | |
test_step_mdp_speed[False-True-False-False-False] | 38.1820μs | 16.8521μs | 59.3399 KOps/s | 58.0550 KOps/s | |
test_step_mdp_speed[False-False-True-True-True] | 75.6330μs | 40.4051μs | 24.7494 KOps/s | 24.4361 KOps/s | |
test_step_mdp_speed[False-False-True-True-False] | 64.4130μs | 26.9024μs | 37.1714 KOps/s | 36.4839 KOps/s | |
test_step_mdp_speed[False-False-True-False-True] | 53.9120μs | 27.6087μs | 36.2204 KOps/s | 36.8488 KOps/s | |
test_step_mdp_speed[False-False-True-False-False] | 43.5120μs | 16.9089μs | 59.1403 KOps/s | 58.2549 KOps/s | |
test_step_mdp_speed[False-False-False-True-True] | 0.1183ms | 41.7540μs | 23.9498 KOps/s | 23.3405 KOps/s | |
test_step_mdp_speed[False-False-False-True-False] | 62.4130μs | 29.4041μs | 34.0089 KOps/s | 34.1385 KOps/s | |
test_step_mdp_speed[False-False-False-False-True] | 81.4640μs | 28.6735μs | 34.8754 KOps/s | 34.6057 KOps/s | |
test_step_mdp_speed[False-False-False-False-False] | 50.5030μs | 18.8222μs | 53.1287 KOps/s | 52.8140 KOps/s | |
test_values[generalized_advantage_estimate-True-True] | 27.2273ms | 25.4965ms | 39.2210 Ops/s | 37.5329 Ops/s | |
test_values[vec_generalized_advantage_estimate-True-True] | 83.8032ms | 3.2342ms | 309.1907 Ops/s | 292.4349 Ops/s | |
test_values[td0_return_estimate-False-False] | 0.1019ms | 65.0136μs | 15.3814 KOps/s | 14.8973 KOps/s | |
test_values[td1_return_estimate-False-False] | 57.8262ms | 56.1779ms | 17.8006 Ops/s | 17.3573 Ops/s | |
test_values[vec_td1_return_estimate-False-False] | 2.0832ms | 1.7517ms | 570.8724 Ops/s | 568.2741 Ops/s | |
test_values[td_lambda_return_estimate-True-False] | 92.4771ms | 89.2151ms | 11.2089 Ops/s | 10.7016 Ops/s | |
test_values[vec_td_lambda_return_estimate-True-False] | 2.0886ms | 1.7339ms | 576.7218 Ops/s | 565.7503 Ops/s | |
test_gae_speed[generalized_advantage_estimate-False-1-512] | 24.6283ms | 24.2220ms | 41.2847 Ops/s | 38.1233 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.8420ms | 0.7178ms | 1.3931 KOps/s | 1.3467 KOps/s | |
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7607ms | 0.6804ms | 1.4698 KOps/s | 1.4102 KOps/s | |
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5300ms | 1.4669ms | 681.7318 Ops/s | 668.5641 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.9648ms | 0.7095ms | 1.4095 KOps/s | 1.3816 KOps/s | |
test_dqn_speed | 7.6599ms | 1.4490ms | 690.1387 Ops/s | 676.0716 Ops/s | |
test_ddpg_speed | 4.6552ms | 3.2688ms | 305.9258 Ops/s | 305.5267 Ops/s | |
test_sac_speed | 10.2426ms | 9.1577ms | 109.1973 Ops/s | 99.8618 Ops/s | |
test_redq_speed | 17.0607ms | 16.4881ms | 60.6497 Ops/s | 59.6981 Ops/s | |
test_redq_deprec_speed | 0.1028s | 14.1721ms | 70.5609 Ops/s | 78.1444 Ops/s | |
test_td3_speed | 9.4806ms | 9.3063ms | 107.4545 Ops/s | 105.4723 Ops/s | |
test_cql_speed | 32.7444ms | 31.6499ms | 31.5957 Ops/s | 31.9509 Ops/s | |
test_a2c_speed | 8.5928ms | 7.2583ms | 137.7725 Ops/s | 142.7019 Ops/s | |
test_ppo_speed | 8.8619ms | 7.5385ms | 132.6530 Ops/s | 136.0143 Ops/s | |
test_reinforce_speed | 7.5583ms | 6.2503ms | 159.9935 Ops/s | 165.0519 Ops/s | |
test_iql_speed | 28.0903ms | 26.6232ms | 37.5613 Ops/s | 37.4979 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.6531ms | 2.4390ms | 410.0082 Ops/s | 394.5960 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 3.6170ms | 2.6094ms | 383.2257 Ops/s | 367.9250 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.0404ms | 2.6238ms | 381.1314 Ops/s | 366.3658 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 2.6632ms | 2.4397ms | 409.8859 Ops/s | 393.9723 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 3.9662ms | 2.6158ms | 382.2879 Ops/s | 365.7064 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.5388ms | 2.6164ms | 382.2091 Ops/s | 367.2982 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 2.8366ms | 2.4313ms | 411.3089 Ops/s | 395.1498 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 4.1000ms | 2.6111ms | 382.9862 Ops/s | 366.7949 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.7191ms | 2.6187ms | 381.8714 Ops/s | 367.6835 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.0573ms | 2.4438ms | 409.1953 Ops/s | 398.7138 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 3.6138ms | 2.6169ms | 382.1262 Ops/s | 366.4630 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.9909ms | 2.6151ms | 382.3935 Ops/s | 366.3323 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.0506ms | 2.4293ms | 411.6345 Ops/s | 398.4839 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 3.4945ms | 2.6126ms | 382.7534 Ops/s | 366.8575 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.2613ms | 2.6268ms | 380.6859 Ops/s | 365.5734 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.0619ms | 2.4304ms | 411.4549 Ops/s | 394.8091 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 3.7639ms | 2.6230ms | 381.2387 Ops/s | 366.5410 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.7988ms | 2.6199ms | 381.6893 Ops/s | 365.5442 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.2223s | 19.5068ms | 51.2641 Ops/s | 50.7880 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 0.1281s | 17.4053ms | 57.4537 Ops/s | 55.4753 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 0.1282s | 17.5897ms | 56.8513 Ops/s | 55.2736 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1276s | 15.2040ms | 65.7723 Ops/s | 55.2654 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1285s | 17.5812ms | 56.8790 Ops/s | 63.4691 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 0.1299s | 17.4911ms | 57.1718 Ops/s | 55.4492 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1293s | 17.4155ms | 57.4200 Ops/s | 55.6071 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 0.1282s | 17.6137ms | 56.7739 Ops/s | 55.3126 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 0.1282s | 17.5699ms | 56.9157 Ops/s | 55.1187 Ops/s |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
enhancement
New feature or request
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Allows for a pickle-free replay buffer checkpointing, where tensors are saved under mmap tensors and metadata in json files.
Not compatible with ListStorage.
Depends on pytorch/tensordict#582
Example:
The saved folder is printed as
The structure of the samples match and a random sample with the same seed provides the exact same samples with indices
TODO: