Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Solve recursion issue in losses hook #1897

Merged
merged 1 commit into from
Feb 10, 2024
Merged

[BugFix] Solve recursion issue in losses hook #1897

merged 1 commit into from
Feb 10, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 10, 2024 •
edited

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 10, 2024 •
edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1897

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (13 Unrelated Failures)

As of commit e778b90 with merge base 89213f9 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 10, 2024
@vmoens vmoens added bug Something isn't working Suitable for minor Suitable to be integrated in minor release (no new feature) labels Feb 10, 2024
@vmoens vmoens merged commit 2cfd9b6 into main Feb 10, 2024
40 of 53 checks passed
@vmoens vmoens deleted the fix-loss-recursion branch February 10, 2024 21:01
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1372s 67.3469ms 14.8485 Ops/s 16.0246 Ops/s $\textbf{\color{#d91a1a}-7.34\%}$
test_sync 44.2997ms 34.8257ms 28.7144 Ops/s 27.3207 Ops/s $\textbf{\color{#35bf28}+5.10\%}$
test_async 0.1211s 34.3677ms 29.0971 Ops/s 29.5828 Ops/s $\color{#d91a1a}-1.64\%$
test_simple 0.4969s 0.4343s 2.3026 Ops/s 2.2853 Ops/s $\color{#35bf28}+0.76\%$
test_transformed 0.6544s 0.5959s 1.6780 Ops/s 1.6777 Ops/s $\color{#35bf28}+0.02\%$
test_serial 1.4880s 1.4048s 0.7118 Ops/s 0.7020 Ops/s $\color{#35bf28}+1.40\%$
test_parallel 1.4364s 1.3699s 0.7300 Ops/s 0.7228 Ops/s $\color{#35bf28}+0.98\%$
test_step_mdp_speed[True-True-True-True-True] 0.1883ms 21.5256μs 46.4564 KOps/s 45.2579 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[True-True-True-True-False] 52.1470μs 13.2079μs 75.7123 KOps/s 75.2351 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-True-True-False-True] 31.0990μs 12.6995μs 78.7433 KOps/s 79.0568 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-True-True-False-False] 25.0570μs 7.6856μs 130.1141 KOps/s 128.3569 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-True-False-True-True] 83.4960μs 22.9379μs 43.5960 KOps/s 43.3645 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[True-True-False-True-False] 41.9890μs 14.4700μs 69.1084 KOps/s 68.7110 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[True-True-False-False-True] 68.5880μs 13.9519μs 71.6750 KOps/s 71.6611 KOps/s $\color{#35bf28}+0.02\%$
test_step_mdp_speed[True-True-False-False-False] 28.2630μs 8.9785μs 111.3772 KOps/s 109.9459 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-False-True-True-True] 92.8630μs 24.1689μs 41.3755 KOps/s 40.7991 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-False-True-True-False] 47.0580μs 15.6338μs 63.9641 KOps/s 62.8711 KOps/s $\color{#35bf28}+1.74\%$
test_step_mdp_speed[True-False-True-False-True] 45.7350μs 13.8172μs 72.3738 KOps/s 71.5591 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-False-True-False-False] 41.9880μs 8.9415μs 111.8382 KOps/s 109.5258 KOps/s $\color{#35bf28}+2.11\%$
test_step_mdp_speed[True-False-False-True-True] 78.0340μs 25.2474μs 39.6080 KOps/s 38.9029 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[True-False-False-True-False] 46.2460μs 16.8139μs 59.4745 KOps/s 57.8969 KOps/s $\color{#35bf28}+2.72\%$
test_step_mdp_speed[True-False-False-False-True] 34.4440μs 14.8454μs 67.3611 KOps/s 65.2349 KOps/s $\color{#35bf28}+3.26\%$
test_step_mdp_speed[True-False-False-False-False] 25.1170μs 10.0283μs 99.7182 KOps/s 96.9769 KOps/s $\color{#35bf28}+2.83\%$
test_step_mdp_speed[False-True-True-True-True] 67.8870μs 24.1353μs 41.4331 KOps/s 40.0070 KOps/s $\color{#35bf28}+3.56\%$
test_step_mdp_speed[False-True-True-True-False] 52.4480μs 15.6553μs 63.8763 KOps/s 61.3735 KOps/s $\color{#35bf28}+4.08\%$
test_step_mdp_speed[False-True-True-False-True] 56.9060μs 16.0808μs 62.1858 KOps/s 60.8407 KOps/s $\color{#35bf28}+2.21\%$
test_step_mdp_speed[False-True-True-False-False] 24.4760μs 10.2201μs 97.8465 KOps/s 94.9260 KOps/s $\color{#35bf28}+3.08\%$
test_step_mdp_speed[False-True-False-True-True] 52.8190μs 25.5959μs 39.0687 KOps/s 37.9479 KOps/s $\color{#35bf28}+2.95\%$
test_step_mdp_speed[False-True-False-True-False] 65.4220μs 16.9848μs 58.8763 KOps/s 56.8728 KOps/s $\color{#35bf28}+3.52\%$
test_step_mdp_speed[False-True-False-False-True] 46.4070μs 17.0815μs 58.5429 KOps/s 57.1436 KOps/s $\color{#35bf28}+2.45\%$
test_step_mdp_speed[False-True-False-False-False] 58.8400μs 11.2204μs 89.1237 KOps/s 86.4026 KOps/s $\color{#35bf28}+3.15\%$
test_step_mdp_speed[False-False-True-True-True] 77.6740μs 26.7093μs 37.4401 KOps/s 36.6169 KOps/s $\color{#35bf28}+2.25\%$
test_step_mdp_speed[False-False-True-True-False] 53.4800μs 18.2620μs 54.7585 KOps/s 53.1030 KOps/s $\color{#35bf28}+3.12\%$
test_step_mdp_speed[False-False-True-False-True] 65.9230μs 17.0541μs 58.6368 KOps/s 56.7929 KOps/s $\color{#35bf28}+3.25\%$
test_step_mdp_speed[False-False-True-False-False] 34.7650μs 11.2390μs 88.9756 KOps/s 84.7775 KOps/s $\color{#35bf28}+4.95\%$
test_step_mdp_speed[False-False-False-True-True] 77.0030μs 27.5556μs 36.2903 KOps/s 35.3895 KOps/s $\color{#35bf28}+2.55\%$
test_step_mdp_speed[False-False-False-True-False] 64.3680μs 19.2334μs 51.9929 KOps/s 50.5985 KOps/s $\color{#35bf28}+2.76\%$
test_step_mdp_speed[False-False-False-False-True] 78.1060μs 18.1909μs 54.9724 KOps/s 53.4348 KOps/s $\color{#35bf28}+2.88\%$
test_step_mdp_speed[False-False-False-False-False] 30.8380μs 12.2732μs 81.4782 KOps/s 77.8741 KOps/s $\color{#35bf28}+4.63\%$
test_values[generalized_advantage_estimate-True-True] 10.3914ms 9.5001ms 105.2615 Ops/s 106.6955 Ops/s $\color{#d91a1a}-1.34\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.5646ms 33.5318ms 29.8224 Ops/s 29.7195 Ops/s $\color{#35bf28}+0.35\%$
test_values[td0_return_estimate-False-False] 0.2389ms 0.1707ms 5.8579 KOps/s 5.9479 KOps/s $\color{#d91a1a}-1.51\%$
test_values[td1_return_estimate-False-False] 24.4400ms 23.7880ms 42.0380 Ops/s 43.0213 Ops/s $\color{#d91a1a}-2.29\%$
test_values[vec_td1_return_estimate-False-False] 35.6054ms 33.6561ms 29.7123 Ops/s 29.6814 Ops/s $\color{#35bf28}+0.10\%$
test_values[td_lambda_return_estimate-True-False] 37.5195ms 34.4298ms 29.0446 Ops/s 29.8783 Ops/s $\color{#d91a1a}-2.79\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.1990ms 33.5734ms 29.7855 Ops/s 29.7872 Ops/s $-0.01\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.9133ms 8.2927ms 120.5873 Ops/s 125.1855 Ops/s $\color{#d91a1a}-3.67\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2173ms 1.8430ms 542.5833 Ops/s 510.4130 Ops/s $\textbf{\color{#35bf28}+6.30\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5125ms 0.3551ms 2.8165 KOps/s 2.8887 KOps/s $\color{#d91a1a}-2.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 41.3684ms 39.5637ms 25.2757 Ops/s 24.9253 Ops/s $\color{#35bf28}+1.41\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.7591ms 3.0368ms 329.2919 Ops/s 328.1710 Ops/s $\color{#35bf28}+0.34\%$
test_dqn_speed 1.6862ms 1.3709ms 729.4663 Ops/s 676.3892 Ops/s $\textbf{\color{#35bf28}+7.85\%}$
test_ddpg_speed 3.1680ms 2.7204ms 367.5909 Ops/s 369.4949 Ops/s $\color{#d91a1a}-0.52\%$
test_sac_speed 9.0383ms 8.5450ms 117.0277 Ops/s 116.5655 Ops/s $\color{#35bf28}+0.40\%$
test_redq_speed 15.0091ms 13.3659ms 74.8174 Ops/s 73.6530 Ops/s $\color{#35bf28}+1.58\%$
test_redq_deprec_speed 15.7745ms 13.7629ms 72.6593 Ops/s 70.0206 Ops/s $\color{#35bf28}+3.77\%$
test_td3_speed 8.7779ms 8.5740ms 116.6315 Ops/s 115.0123 Ops/s $\color{#35bf28}+1.41\%$
test_cql_speed 38.0383ms 36.7317ms 27.2244 Ops/s 27.0886 Ops/s $\color{#35bf28}+0.50\%$
test_a2c_speed 8.0421ms 7.3416ms 136.2096 Ops/s 134.4601 Ops/s $\color{#35bf28}+1.30\%$
test_ppo_speed 8.8590ms 7.5233ms 132.9205 Ops/s 130.7150 Ops/s $\color{#35bf28}+1.69\%$
test_reinforce_speed 7.4723ms 6.5540ms 152.5795 Ops/s 150.5461 Ops/s $\color{#35bf28}+1.35\%$
test_iql_speed 34.8568ms 33.6466ms 29.7207 Ops/s 30.0659 Ops/s $\color{#d91a1a}-1.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.1827ms 2.8805ms 347.1565 Ops/s 356.8006 Ops/s $\color{#d91a1a}-2.70\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6101ms 0.5135ms 1.9473 KOps/s 1.9432 KOps/s $\color{#35bf28}+0.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8158ms 0.4873ms 2.0522 KOps/s 2.0237 KOps/s $\color{#35bf28}+1.41\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.3280ms 2.8607ms 349.5613 Ops/s 345.8986 Ops/s $\color{#35bf28}+1.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8263ms 0.5121ms 1.9526 KOps/s 1.9544 KOps/s $\color{#d91a1a}-0.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7908ms 0.4929ms 2.0286 KOps/s 2.0426 KOps/s $\color{#d91a1a}-0.68\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4293ms 2.9647ms 337.2968 Ops/s 346.3035 Ops/s $\color{#d91a1a}-2.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7402ms 0.6309ms 1.5851 KOps/s 1.5835 KOps/s $\color{#35bf28}+0.10\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1125ms 0.6029ms 1.6586 KOps/s 1.6610 KOps/s $\color{#d91a1a}-0.15\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.3061ms 2.8677ms 348.7055 Ops/s 351.9238 Ops/s $\color{#d91a1a}-0.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7489ms 0.5171ms 1.9337 KOps/s 1.9517 KOps/s $\color{#d91a1a}-0.92\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6519ms 0.4932ms 2.0276 KOps/s 2.0629 KOps/s $\color{#d91a1a}-1.71\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.3915ms 2.8589ms 349.7897 Ops/s 347.2608 Ops/s $\color{#35bf28}+0.73\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8494ms 0.5211ms 1.9190 KOps/s 1.9441 KOps/s $\color{#d91a1a}-1.29\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6264ms 0.4855ms 2.0597 KOps/s 2.0831 KOps/s $\color{#d91a1a}-1.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.2922ms 2.9441ms 339.6663 Ops/s 329.1077 Ops/s $\color{#35bf28}+3.21\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8688ms 0.6328ms 1.5803 KOps/s 1.5696 KOps/s $\color{#35bf28}+0.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7822ms 0.6052ms 1.6524 KOps/s 1.6510 KOps/s $\color{#35bf28}+0.08\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1050s 8.1835ms 122.1970 Ops/s 121.1574 Ops/s $\color{#35bf28}+0.86\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.3881ms 13.1595ms 75.9906 Ops/s 76.3090 Ops/s $\color{#d91a1a}-0.42\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.1307ms 2.5288ms 395.4425 Ops/s 397.3287 Ops/s $\color{#d91a1a}-0.47\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1075s 9.9337ms 100.6669 Ops/s 98.1336 Ops/s $\color{#35bf28}+2.58\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.9318ms 13.1556ms 76.0131 Ops/s 75.6151 Ops/s $\color{#35bf28}+0.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.6964ms 2.5316ms 395.0044 Ops/s 397.2109 Ops/s $\color{#d91a1a}-0.56\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1032s 8.2687ms 120.9378 Ops/s 93.5782 Ops/s $\textbf{\color{#35bf28}+29.24\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1103s 15.3365ms 65.2038 Ops/s 74.7806 Ops/s $\textbf{\color{#d91a1a}-12.81\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.0367ms 2.7853ms 359.0339 Ops/s 364.1627 Ops/s $\color{#d91a1a}-1.41\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1165s 0.1163s 8.5952 Ops/s 8.6720 Ops/s $\color{#d91a1a}-0.89\%$
test_sync 0.1705s 0.1025s 9.7532 Ops/s 9.6545 Ops/s $\color{#35bf28}+1.02\%$
test_async 0.2503s 91.5855ms 10.9188 Ops/s 10.9033 Ops/s $\color{#35bf28}+0.14\%$
test_single_pixels 0.1401s 0.1382s 7.2372 Ops/s 7.2638 Ops/s $\color{#d91a1a}-0.37\%$
test_sync_pixels 81.7069ms 79.1437ms 12.6352 Ops/s 12.4685 Ops/s $\color{#35bf28}+1.34\%$
test_async_pixels 0.2079s 74.8841ms 13.3540 Ops/s 12.7815 Ops/s $\color{#35bf28}+4.48\%$
test_simple 0.8993s 0.8287s 1.2067 Ops/s 1.2164 Ops/s $\color{#d91a1a}-0.80\%$
test_transformed 1.1455s 1.0800s 0.9259 Ops/s 0.9327 Ops/s $\color{#d91a1a}-0.73\%$
test_serial 2.5203s 2.4561s 0.4071 Ops/s 0.4156 Ops/s $\color{#d91a1a}-2.04\%$
test_parallel 2.1706s 2.0885s 0.4788 Ops/s 0.4875 Ops/s $\color{#d91a1a}-1.77\%$
test_step_mdp_speed[True-True-True-True-True] 0.1171ms 32.4741μs 30.7937 KOps/s 29.9171 KOps/s $\color{#35bf28}+2.93\%$
test_step_mdp_speed[True-True-True-True-False] 42.6920μs 19.4087μs 51.5233 KOps/s 50.3302 KOps/s $\color{#35bf28}+2.37\%$
test_step_mdp_speed[True-True-True-False-True] 43.8720μs 18.5494μs 53.9102 KOps/s 53.2789 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-True-True-False-False] 31.1320μs 10.9427μs 91.3848 KOps/s 89.4743 KOps/s $\color{#35bf28}+2.14\%$
test_step_mdp_speed[True-True-False-True-True] 71.3040μs 33.4403μs 29.9040 KOps/s 28.7785 KOps/s $\color{#35bf28}+3.91\%$
test_step_mdp_speed[True-True-False-True-False] 63.6540μs 21.1092μs 47.3726 KOps/s 46.6469 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[True-True-False-False-True] 44.8220μs 20.4678μs 48.8573 KOps/s 49.0044 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-True-False-False-False] 30.6720μs 12.7908μs 78.1815 KOps/s 77.0349 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[True-False-True-True-True] 64.4040μs 36.0190μs 27.7631 KOps/s 27.1772 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-False-True-True-False] 46.1140μs 23.0997μs 43.2907 KOps/s 42.7749 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-False-True-False-True] 54.8130μs 20.1178μs 49.7072 KOps/s 48.9329 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-False-True-False-False] 33.5620μs 12.8320μs 77.9301 KOps/s 75.5003 KOps/s $\color{#35bf28}+3.22\%$
test_step_mdp_speed[True-False-False-True-True] 69.2440μs 37.9109μs 26.3776 KOps/s 25.8005 KOps/s $\color{#35bf28}+2.24\%$
test_step_mdp_speed[True-False-False-True-False] 51.1230μs 24.9761μs 40.0383 KOps/s 39.4778 KOps/s $\color{#35bf28}+1.42\%$
test_step_mdp_speed[True-False-False-False-True] 50.8230μs 21.8500μs 45.7666 KOps/s 45.2192 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-False-False-False-False] 43.2120μs 14.6518μs 68.2510 KOps/s 67.1804 KOps/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[False-True-True-True-True] 77.1340μs 35.7617μs 27.9629 KOps/s 26.6612 KOps/s $\color{#35bf28}+4.88\%$
test_step_mdp_speed[False-True-True-True-False] 49.9330μs 22.9520μs 43.5692 KOps/s 42.0287 KOps/s $\color{#35bf28}+3.67\%$
test_step_mdp_speed[False-True-True-False-True] 52.7630μs 24.0441μs 41.5903 KOps/s 40.4110 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[False-True-True-False-False] 36.9320μs 14.6816μs 68.1127 KOps/s 65.7929 KOps/s $\color{#35bf28}+3.53\%$
test_step_mdp_speed[False-True-False-True-True] 72.8740μs 38.1549μs 26.2090 KOps/s 25.4798 KOps/s $\color{#35bf28}+2.86\%$
test_step_mdp_speed[False-True-False-True-False] 68.9240μs 25.2160μs 39.6573 KOps/s 38.7878 KOps/s $\color{#35bf28}+2.24\%$
test_step_mdp_speed[False-True-False-False-True] 60.0130μs 25.6988μs 38.9122 KOps/s 37.2963 KOps/s $\color{#35bf28}+4.33\%$
test_step_mdp_speed[False-True-False-False-False] 43.7420μs 16.6219μs 60.1615 KOps/s 60.0363 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-False-True-True-True] 70.5840μs 39.5622μs 25.2767 KOps/s 24.5773 KOps/s $\color{#35bf28}+2.85\%$
test_step_mdp_speed[False-False-True-True-False] 55.6330μs 26.8870μs 37.1927 KOps/s 36.4005 KOps/s $\color{#35bf28}+2.18\%$
test_step_mdp_speed[False-False-True-False-True] 54.1130μs 25.6961μs 38.9165 KOps/s 37.4192 KOps/s $\color{#35bf28}+4.00\%$
test_step_mdp_speed[False-False-True-False-False] 43.3130μs 16.7682μs 59.6366 KOps/s 59.4951 KOps/s $\color{#35bf28}+0.24\%$
test_step_mdp_speed[False-False-False-True-True] 74.7240μs 41.6649μs 24.0010 KOps/s 23.8143 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[False-False-False-True-False] 56.7630μs 28.7248μs 34.8131 KOps/s 33.9280 KOps/s $\color{#35bf28}+2.61\%$
test_step_mdp_speed[False-False-False-False-True] 56.3330μs 26.8184μs 37.2878 KOps/s 35.7414 KOps/s $\color{#35bf28}+4.33\%$
test_step_mdp_speed[False-False-False-False-False] 48.2530μs 18.3365μs 54.5360 KOps/s 54.9705 KOps/s $\color{#d91a1a}-0.79\%$
test_values[generalized_advantage_estimate-True-True] 24.4678ms 24.1711ms 41.3717 Ops/s 43.9022 Ops/s $\textbf{\color{#d91a1a}-5.76\%}$
test_values[vec_generalized_advantage_estimate-True-True] 82.0163ms 3.1923ms 313.2524 Ops/s 307.9140 Ops/s $\color{#35bf28}+1.73\%$
test_values[td0_return_estimate-False-False] 98.7450μs 58.9188μs 16.9725 KOps/s 17.1426 KOps/s $\color{#d91a1a}-0.99\%$
test_values[td1_return_estimate-False-False] 52.0378ms 50.7458ms 19.7061 Ops/s 20.2484 Ops/s $\color{#d91a1a}-2.68\%$
test_values[vec_td1_return_estimate-False-False] 2.0601ms 1.7494ms 571.6219 Ops/s 576.3981 Ops/s $\color{#d91a1a}-0.83\%$
test_values[td_lambda_return_estimate-True-False] 83.8167ms 83.1127ms 12.0319 Ops/s 12.5874 Ops/s $\color{#d91a1a}-4.41\%$
test_values[vec_td_lambda_return_estimate-True-False] 3.9597ms 1.7984ms 556.0357 Ops/s 563.8694 Ops/s $\color{#d91a1a}-1.39\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.1124ms 22.2318ms 44.9807 Ops/s 45.7151 Ops/s $\color{#d91a1a}-1.61\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8593ms 0.6776ms 1.4757 KOps/s 1.4805 KOps/s $\color{#d91a1a}-0.33\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7022ms 0.6298ms 1.5877 KOps/s 1.5837 KOps/s $\color{#35bf28}+0.26\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5511ms 1.4323ms 698.1951 Ops/s 698.3866 Ops/s $\color{#d91a1a}-0.03\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9114ms 0.6487ms 1.5415 KOps/s 1.5377 KOps/s $\color{#35bf28}+0.25\%$
test_dqn_speed 7.8856ms 1.4234ms 702.5447 Ops/s 713.8377 Ops/s $\color{#d91a1a}-1.58\%$
test_ddpg_speed 3.1664ms 2.7386ms 365.1543 Ops/s 367.9775 Ops/s $\color{#d91a1a}-0.77\%$
test_sac_speed 8.8162ms 8.3876ms 119.2243 Ops/s 121.0942 Ops/s $\color{#d91a1a}-1.54\%$
test_redq_speed 11.0785ms 10.3464ms 96.6519 Ops/s 96.8282 Ops/s $\color{#d91a1a}-0.18\%$
test_redq_deprec_speed 11.9178ms 11.3282ms 88.2753 Ops/s 86.9119 Ops/s $\color{#35bf28}+1.57\%$
test_td3_speed 17.6322ms 8.6718ms 115.3158 Ops/s 117.2975 Ops/s $\color{#d91a1a}-1.69\%$
test_cql_speed 25.8290ms 24.9177ms 40.1320 Ops/s 39.2336 Ops/s $\color{#35bf28}+2.29\%$
test_a2c_speed 6.0395ms 5.3236ms 187.8419 Ops/s 183.9202 Ops/s $\color{#35bf28}+2.13\%$
test_ppo_speed 6.5904ms 5.5683ms 179.5888 Ops/s 175.6970 Ops/s $\color{#35bf28}+2.22\%$
test_reinforce_speed 4.7575ms 4.3792ms 228.3510 Ops/s 223.8260 Ops/s $\color{#35bf28}+2.02\%$
test_iql_speed 19.9113ms 19.3718ms 51.6215 Ops/s 51.7147 Ops/s $\color{#d91a1a}-0.18\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7009ms 3.6030ms 277.5459 Ops/s 277.3702 Ops/s $\color{#35bf28}+0.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7015ms 0.5519ms 1.8119 KOps/s 1.8096 KOps/s $\color{#35bf28}+0.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6813ms 0.5291ms 1.8899 KOps/s 1.8979 KOps/s $\color{#d91a1a}-0.42\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.0007ms 3.6280ms 275.6349 Ops/s 280.8039 Ops/s $\color{#d91a1a}-1.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7652ms 0.5458ms 1.8322 KOps/s 1.8225 KOps/s $\color{#35bf28}+0.53\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7219ms 0.5245ms 1.9065 KOps/s 1.9153 KOps/s $\color{#d91a1a}-0.46\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.8232ms 3.7232ms 268.5844 Ops/s 267.8649 Ops/s $\color{#35bf28}+0.27\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9117ms 0.6754ms 1.4806 KOps/s 1.4799 KOps/s $\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8112ms 0.6487ms 1.5415 KOps/s 1.5430 KOps/s $\color{#d91a1a}-0.09\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.8318ms 3.5963ms 278.0654 Ops/s 277.0227 Ops/s $\color{#35bf28}+0.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6744ms 0.5506ms 1.8163 KOps/s 1.8077 KOps/s $\color{#35bf28}+0.48\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7296ms 0.5271ms 1.8973 KOps/s 1.8911 KOps/s $\color{#35bf28}+0.32\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8713ms 3.6470ms 274.1975 Ops/s 275.5099 Ops/s $\color{#d91a1a}-0.48\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8289ms 0.5472ms 1.8275 KOps/s 1.8282 KOps/s $\color{#d91a1a}-0.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6654ms 0.5201ms 1.9226 KOps/s 1.9103 KOps/s $\color{#35bf28}+0.64\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.0077ms 3.7384ms 267.4954 Ops/s 268.3115 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9103ms 0.6793ms 1.4722 KOps/s 1.4861 KOps/s $\color{#d91a1a}-0.93\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7930ms 0.6491ms 1.5406 KOps/s 1.5323 KOps/s $\color{#35bf28}+0.54\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1331s 10.2826ms 97.2515 Ops/s 97.4070 Ops/s $\color{#d91a1a}-0.16\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.6132ms 16.0097ms 62.4623 Ops/s 62.0227 Ops/s $\color{#35bf28}+0.71\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.1034ms 3.0043ms 332.8548 Ops/s 336.7235 Ops/s $\color{#d91a1a}-1.15\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1205s 9.9695ms 100.3055 Ops/s 99.8094 Ops/s $\color{#35bf28}+0.50\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.0813ms 16.0386ms 62.3497 Ops/s 54.7902 Ops/s $\textbf{\color{#35bf28}+13.80\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.7993ms 3.0330ms 329.7119 Ops/s 336.8596 Ops/s $\color{#d91a1a}-2.12\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1223s 10.3053ms 97.0372 Ops/s 97.0342 Ops/s $+0.00\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1295s 18.5782ms 53.8265 Ops/s 61.0765 Ops/s $\textbf{\color{#d91a1a}-11.87\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.3908ms 3.2972ms 303.2851 Ops/s 304.4696 Ops/s $\color{#d91a1a}-0.39\%$

vmoens added a commit that referenced this pull request Feb 28, 2024
@vmoens
[BugFix] Solve recursion issue in losses hook (#1897)
7eafe19
@vmoens vmoens mentioned this pull request Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Suitable for minor Suitable to be integrated in minor release (no new feature)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@vmoens @facebook-github-bot