[Feature] BatchSizeTransform

[vmoens]

Solves https://stackoverflow.com/questions/78198313/handling-batch-size-in-a-torchrl-environment/78199390#78199390

TODO:

• Tests

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2030

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [PREEMPTIVE] - We recently implemented changes in pull job linux-jammy-py3.8-gcc11 / build

❌ 4 New Failures

As of commit 687647c with merge base e57d0bc ():

NEW FAILURES - The following jobs have failed:

• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 0fe46360dec7e7533d440059c980c91bfff28cb2e244cd1dfce73af3271552e0 /exec failed with exit code 139
• Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 28ef41be04d03a81be87a59674aff0d4059024d0d1955a38f19f9c5481b9b5ab /exec failed with exit code 1
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  test/test_transforms.py::TestBatchSizeTransform::test_trans_parallel_env_check[False-reshape]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	98.3868ms	97.1837ms	10.2898 Ops/s	9.7537 Ops/s	$\textbf{\color{#35bf28}+5.50\%}$
test_sync	86.6367ms	86.0638ms	11.6193 Ops/s	11.6635 Ops/s	$\color{#d91a1a}-0.38\%$
test_async	0.1689s	84.4599ms	11.8399 Ops/s	14.4070 Ops/s	$\textbf{\color{#d91a1a}-17.82\%}$
test_single_pixels	0.1059s	0.1057s	9.4563 Ops/s	9.3191 Ops/s	$\color{#35bf28}+1.47\%$
test_sync_pixels	66.6528ms	65.4421ms	15.2807 Ops/s	15.3522 Ops/s	$\color{#d91a1a}-0.47\%$
test_async_pixels	0.1237s	55.7128ms	17.9492 Ops/s	18.4048 Ops/s	$\color{#d91a1a}-2.48\%$
test_simple	0.7225s	0.6521s	1.5336 Ops/s	1.5454 Ops/s	$\color{#d91a1a}-0.76\%$
test_transformed	0.9291s	0.8612s	1.1612 Ops/s	1.1372 Ops/s	$\color{#35bf28}+2.11\%$
test_serial	2.0710s	2.0019s	0.4995 Ops/s	0.4835 Ops/s	$\color{#35bf28}+3.33\%$
test_parallel	1.8202s	1.7290s	0.5784 Ops/s	0.5696 Ops/s	$\color{#35bf28}+1.53\%$
test_step_mdp_speed[True-True-True-True-True]	86.3510μs	33.2796μs	30.0485 KOps/s	30.8919 KOps/s	$\color{#d91a1a}-2.73\%$
test_step_mdp_speed[True-True-True-True-False]	36.6310μs	19.6238μs	50.9586 KOps/s	51.3105 KOps/s	$\color{#d91a1a}-0.69\%$
test_step_mdp_speed[True-True-True-False-True]	45.8210μs	18.6928μs	53.4964 KOps/s	55.3294 KOps/s	$\color{#d91a1a}-3.31\%$
test_step_mdp_speed[True-True-True-False-False]	27.0300μs	11.2308μs	89.0410 KOps/s	90.9216 KOps/s	$\color{#d91a1a}-2.07\%$
test_step_mdp_speed[True-True-False-True-True]	54.1210μs	34.7886μs	28.7451 KOps/s	29.1519 KOps/s	$\color{#d91a1a}-1.40\%$
test_step_mdp_speed[True-True-False-True-False]	47.3410μs	21.4872μs	46.5393 KOps/s	48.0765 KOps/s	$\color{#d91a1a}-3.20\%$
test_step_mdp_speed[True-True-False-False-True]	36.9100μs	20.4795μs	48.8294 KOps/s	49.9345 KOps/s	$\color{#d91a1a}-2.21\%$
test_step_mdp_speed[True-True-False-False-False]	0.2806ms	13.1287μs	76.1688 KOps/s	77.7136 KOps/s	$\color{#d91a1a}-1.99\%$
test_step_mdp_speed[True-False-True-True-True]	60.4710μs	36.8122μs	27.1649 KOps/s	27.9652 KOps/s	$\color{#d91a1a}-2.86\%$
test_step_mdp_speed[True-False-True-True-False]	47.6400μs	23.6250μs	42.3281 KOps/s	43.6983 KOps/s	$\color{#d91a1a}-3.14\%$
test_step_mdp_speed[True-False-True-False-True]	36.5200μs	20.8532μs	47.9543 KOps/s	49.8558 KOps/s	$\color{#d91a1a}-3.81\%$
test_step_mdp_speed[True-False-True-False-False]	34.6900μs	13.1181μs	76.2303 KOps/s	77.6609 KOps/s	$\color{#d91a1a}-1.84\%$
test_step_mdp_speed[True-False-False-True-True]	0.1126ms	38.5553μs	25.9368 KOps/s	26.4440 KOps/s	$\color{#d91a1a}-1.92\%$
test_step_mdp_speed[True-False-False-True-False]	68.8410μs	25.4400μs	39.3081 KOps/s	40.5616 KOps/s	$\color{#d91a1a}-3.09\%$
test_step_mdp_speed[True-False-False-False-True]	40.7010μs	22.3995μs	44.6438 KOps/s	45.8473 KOps/s	$\color{#d91a1a}-2.63\%$
test_step_mdp_speed[True-False-False-False-False]	31.1310μs	14.9801μs	66.7550 KOps/s	68.3343 KOps/s	$\color{#d91a1a}-2.31\%$
test_step_mdp_speed[False-True-True-True-True]	60.2810μs	37.0217μs	27.0112 KOps/s	27.6543 KOps/s	$\color{#d91a1a}-2.33\%$
test_step_mdp_speed[False-True-True-True-False]	46.5010μs	23.6036μs	42.3665 KOps/s	43.4437 KOps/s	$\color{#d91a1a}-2.48\%$
test_step_mdp_speed[False-True-True-False-True]	46.0910μs	24.8994μs	40.1616 KOps/s	41.3205 KOps/s	$\color{#d91a1a}-2.80\%$
test_step_mdp_speed[False-True-True-False-False]	29.2300μs	14.8738μs	67.2323 KOps/s	69.1859 KOps/s	$\color{#d91a1a}-2.82\%$
test_step_mdp_speed[False-True-False-True-True]	77.3910μs	38.7364μs	25.8155 KOps/s	25.8602 KOps/s	$\color{#d91a1a}-0.17\%$
test_step_mdp_speed[False-True-False-True-False]	48.8600μs	25.4525μs	39.2889 KOps/s	39.4145 KOps/s	$\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-False-False-True]	42.0010μs	26.3261μs	37.9851 KOps/s	37.9541 KOps/s	$\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-True-False-False-False]	41.7410μs	16.7214μs	59.8038 KOps/s	60.3943 KOps/s	$\color{#d91a1a}-0.98\%$
test_step_mdp_speed[False-False-True-True-True]	68.5410μs	40.7514μs	24.5390 KOps/s	24.6283 KOps/s	$\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-False-True-True-False]	45.9810μs	27.4469μs	36.4341 KOps/s	37.6204 KOps/s	$\color{#d91a1a}-3.15\%$
test_step_mdp_speed[False-False-True-False-True]	48.9010μs	26.5185μs	37.7096 KOps/s	37.6924 KOps/s	$\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-False-True-False-False]	33.8300μs	16.7264μs	59.7857 KOps/s	60.5837 KOps/s	$\color{#d91a1a}-1.32\%$
test_step_mdp_speed[False-False-False-True-True]	70.0220μs	42.3055μs	23.6376 KOps/s	24.3250 KOps/s	$\color{#d91a1a}-2.83\%$
test_step_mdp_speed[False-False-False-True-False]	50.4410μs	29.2169μs	34.2268 KOps/s	35.3471 KOps/s	$\color{#d91a1a}-3.17\%$
test_step_mdp_speed[False-False-False-False-True]	48.9410μs	28.0338μs	35.6712 KOps/s	36.4400 KOps/s	$\color{#d91a1a}-2.11\%$
test_step_mdp_speed[False-False-False-False-False]	37.1200μs	18.5442μs	53.9253 KOps/s	54.7481 KOps/s	$\color{#d91a1a}-1.50\%$
test_values[generalized_advantage_estimate-True-True]	22.0789ms	21.3261ms	46.8910 Ops/s	44.3364 Ops/s	$\textbf{\color{#35bf28}+5.76\%}$
test_values[vec_generalized_advantage_estimate-True-True]	86.3456ms	3.2791ms	304.9594 Ops/s	315.0766 Ops/s	$\color{#d91a1a}-3.21\%$
test_values[td0_return_estimate-False-False]	91.9110μs	60.4583μs	16.5403 KOps/s	16.1647 KOps/s	$\color{#35bf28}+2.32\%$
test_values[td1_return_estimate-False-False]	49.7956ms	48.9076ms	20.4467 Ops/s	19.9414 Ops/s	$\color{#35bf28}+2.53\%$
test_values[vec_td1_return_estimate-False-False]	1.9732ms	1.7329ms	577.0679 Ops/s	575.4404 Ops/s	$\color{#35bf28}+0.28\%$
test_values[td_lambda_return_estimate-True-False]	78.7800ms	78.0240ms	12.8166 Ops/s	12.3786 Ops/s	$\color{#35bf28}+3.54\%$
test_values[vec_td_lambda_return_estimate-True-False]	2.0388ms	1.7304ms	577.8904 Ops/s	575.5698 Ops/s	$\color{#35bf28}+0.40\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	21.3783ms	21.1951ms	47.1808 Ops/s	45.1472 Ops/s	$\color{#35bf28}+4.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.8783ms	0.6707ms	1.4910 KOps/s	1.4815 KOps/s	$\color{#35bf28}+0.64\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7213ms	0.6242ms	1.6021 KOps/s	1.5973 KOps/s	$\color{#35bf28}+0.30\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.4932ms	1.4284ms	700.0743 Ops/s	699.1085 Ops/s	$\color{#35bf28}+0.14\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.0103ms	0.6874ms	1.4548 KOps/s	1.5355 KOps/s	$\textbf{\color{#d91a1a}-5.26\%}$
test_dqn_speed	8.0367ms	1.4641ms	683.0256 Ops/s	699.4216 Ops/s	$\color{#d91a1a}-2.34\%$
test_ddpg_speed	3.0352ms	2.7426ms	364.6209 Ops/s	372.1033 Ops/s	$\color{#d91a1a}-2.01\%$
test_sac_speed	8.5979ms	7.9439ms	125.8832 Ops/s	127.0896 Ops/s	$\color{#d91a1a}-0.95\%$
test_redq_speed	17.7919ms	10.2686ms	97.3846 Ops/s	98.2427 Ops/s	$\color{#d91a1a}-0.87\%$
test_redq_deprec_speed	11.4235ms	10.8708ms	91.9900 Ops/s	93.0421 Ops/s	$\color{#d91a1a}-1.13\%$
test_td3_speed	7.9919ms	7.8527ms	127.3443 Ops/s	128.0098 Ops/s	$\color{#d91a1a}-0.52\%$
test_cql_speed	26.4972ms	24.8393ms	40.2588 Ops/s	40.8086 Ops/s	$\color{#d91a1a}-1.35\%$
test_a2c_speed	5.7631ms	5.5069ms	181.5905 Ops/s	187.8427 Ops/s	$\color{#d91a1a}-3.33\%$
test_ppo_speed	6.2434ms	5.9261ms	168.7453 Ops/s	175.3415 Ops/s	$\color{#d91a1a}-3.76\%$
test_reinforce_speed	5.1234ms	4.5226ms	221.1139 Ops/s	232.5049 Ops/s	$\color{#d91a1a}-4.90\%$
test_iql_speed	20.0716ms	19.2939ms	51.8299 Ops/s	52.5077 Ops/s	$\color{#d91a1a}-1.29\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.1464ms	2.9967ms	333.7003 Ops/s	349.1405 Ops/s	$\color{#d91a1a}-4.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.2014ms	0.5432ms	1.8409 KOps/s	1.8636 KOps/s	$\color{#d91a1a}-1.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6699ms	0.5182ms	1.9297 KOps/s	1.9530 KOps/s	$\color{#d91a1a}-1.19\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.3346ms	3.0136ms	331.8244 Ops/s	348.3886 Ops/s	$\color{#d91a1a}-4.75\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1223s	0.6502ms	1.5380 KOps/s	1.8900 KOps/s	$\textbf{\color{#d91a1a}-18.63\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6838ms	0.5121ms	1.9526 KOps/s	1.9752 KOps/s	$\color{#d91a1a}-1.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	4.2089ms	1.3933ms	717.7252 Ops/s	722.7399 Ops/s	$\color{#d91a1a}-0.69\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.5186ms	1.3195ms	757.8635 Ops/s	757.5056 Ops/s	$\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.5591ms	3.1192ms	320.5939 Ops/s	334.9536 Ops/s	$\color{#d91a1a}-4.29\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8198ms	0.6650ms	1.5038 KOps/s	1.5136 KOps/s	$\color{#d91a1a}-0.65\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	5.8811ms	0.6501ms	1.5383 KOps/s	1.5555 KOps/s	$\color{#d91a1a}-1.11\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.1784ms	3.0070ms	332.5623 Ops/s	350.4793 Ops/s	$\textbf{\color{#d91a1a}-5.11\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.5875ms	0.5468ms	1.8287 KOps/s	1.8577 KOps/s	$\color{#d91a1a}-1.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6865ms	0.5189ms	1.9272 KOps/s	1.9433 KOps/s	$\color{#d91a1a}-0.83\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.2409ms	3.0527ms	327.5789 Ops/s	349.4489 Ops/s	$\textbf{\color{#d91a1a}-6.26\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.2283ms	0.5407ms	1.8495 KOps/s	1.8819 KOps/s	$\color{#d91a1a}-1.72\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7233ms	0.5151ms	1.9414 KOps/s	1.9755 KOps/s	$\color{#d91a1a}-1.73\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.2718ms	3.1414ms	318.3320 Ops/s	334.1695 Ops/s	$\color{#d91a1a}-4.74\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8173ms	0.6718ms	1.4885 KOps/s	1.5096 KOps/s	$\color{#d91a1a}-1.40\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7615ms	0.6484ms	1.5423 KOps/s	1.5593 KOps/s	$\color{#d91a1a}-1.09\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1372s	7.6478ms	130.7574 Ops/s	138.7468 Ops/s	$\textbf{\color{#d91a1a}-5.76\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	17.1664ms	14.7230ms	67.9209 Ops/s	67.4706 Ops/s	$\color{#35bf28}+0.67\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.1441ms	1.0313ms	969.6228 Ops/s	934.2267 Ops/s	$\color{#35bf28}+3.79\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1145s	7.0019ms	142.8187 Ops/s	106.4061 Ops/s	$\textbf{\color{#35bf28}+34.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1273s	16.7221ms	59.8011 Ops/s	67.9551 Ops/s	$\textbf{\color{#d91a1a}-12.00\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	1.0801ms	0.9952ms	1.0048 KOps/s	937.7059 Ops/s	$\textbf{\color{#35bf28}+7.16\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1168s	7.3440ms	136.1649 Ops/s	134.5591 Ops/s	$\color{#35bf28}+1.19\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	17.2623ms	14.8083ms	67.5297 Ops/s	66.4834 Ops/s	$\color{#35bf28}+1.57\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	2.6411ms	1.4237ms	702.4132 Ops/s	719.5333 Ops/s	$\color{#d91a1a}-2.38\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	54.3489ms	53.2906ms	18.7651 Ops/s	18.0633 Ops/s	$\color{#35bf28}+3.88\%$
test_sync	40.9002ms	30.1839ms	33.1303 Ops/s	35.5799 Ops/s	$\textbf{\color{#d91a1a}-6.89\%}$
test_async	43.7192ms	28.7981ms	34.7245 Ops/s	37.2473 Ops/s	$\textbf{\color{#d91a1a}-6.77\%}$
test_simple	0.4130s	0.3487s	2.8681 Ops/s	2.8978 Ops/s	$\color{#d91a1a}-1.02\%$
test_transformed	0.5597s	0.4952s	2.0195 Ops/s	2.0684 Ops/s	$\color{#d91a1a}-2.36\%$
test_serial	1.2905s	1.2274s	0.8147 Ops/s	0.8202 Ops/s	$\color{#d91a1a}-0.66\%$
test_parallel	1.0616s	0.9987s	1.0013 Ops/s	1.0095 Ops/s	$\color{#d91a1a}-0.81\%$
test_step_mdp_speed[True-True-True-True-True]	0.1615ms	22.0870μs	45.2756 KOps/s	45.6377 KOps/s	$\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-True-True-True-False]	42.3590μs	13.1931μs	75.7971 KOps/s	75.8922 KOps/s	$\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-True-True-False-True]	32.3800μs	12.6357μs	79.1408 KOps/s	78.1261 KOps/s	$\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-True-True-False-False]	35.0950μs	7.6467μs	130.7755 KOps/s	129.7103 KOps/s	$\color{#35bf28}+0.82\%$
test_step_mdp_speed[True-True-False-True-True]	60.2020μs	23.4060μs	42.7240 KOps/s	42.8477 KOps/s	$\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-True-False-True-False]	43.7120μs	14.5251μs	68.8461 KOps/s	68.7048 KOps/s	$\color{#35bf28}+0.21\%$
test_step_mdp_speed[True-True-False-False-True]	62.0670μs	14.0207μs	71.3233 KOps/s	70.8044 KOps/s	$\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-True-False-False-False]	41.5680μs	8.8605μs	112.8605 KOps/s	111.3717 KOps/s	$\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-False-True-True-True]	84.5880μs	24.5412μs	40.7478 KOps/s	40.3262 KOps/s	$\color{#35bf28}+1.05\%$
test_step_mdp_speed[True-False-True-True-False]	58.3690μs	15.9409μs	62.7317 KOps/s	63.1120 KOps/s	$\color{#d91a1a}-0.60\%$
test_step_mdp_speed[True-False-True-False-True]	67.9470μs	13.9562μs	71.6529 KOps/s	70.1240 KOps/s	$\color{#35bf28}+2.18\%$
test_step_mdp_speed[True-False-True-False-False]	54.8630μs	8.8986μs	112.3766 KOps/s	111.9519 KOps/s	$\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-False-False-True-True]	69.6200μs	25.8802μs	38.6396 KOps/s	38.7600 KOps/s	$\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-False-False-True-False]	60.5830μs	17.3372μs	57.6793 KOps/s	58.7172 KOps/s	$\color{#d91a1a}-1.77\%$
test_step_mdp_speed[True-False-False-False-True]	77.9340μs	15.2111μs	65.7414 KOps/s	65.8228 KOps/s	$\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-False-False-False-False]	37.8610μs	10.1978μs	98.0607 KOps/s	98.4287 KOps/s	$\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-True-True-True-True]	62.6370μs	24.7197μs	40.4535 KOps/s	40.6872 KOps/s	$\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-True-True-True-False]	61.1140μs	15.9759μs	62.5944 KOps/s	63.3915 KOps/s	$\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-True-True-False-True]	81.5600μs	16.2286μs	61.6195 KOps/s	61.4861 KOps/s	$\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-True-True-False-False]	38.2610μs	10.2494μs	97.5666 KOps/s	98.8304 KOps/s	$\color{#d91a1a}-1.28\%$
test_step_mdp_speed[False-True-False-True-True]	50.4950μs	26.5099μs	37.7218 KOps/s	38.1974 KOps/s	$\color{#d91a1a}-1.25\%$
test_step_mdp_speed[False-True-False-True-False]	85.3390μs	17.2853μs	57.8525 KOps/s	58.4288 KOps/s	$\color{#d91a1a}-0.99\%$
test_step_mdp_speed[False-True-False-False-True]	44.6430μs	17.6726μs	56.5846 KOps/s	56.6607 KOps/s	$\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-True-False-False-False]	41.1270μs	11.5635μs	86.4793 KOps/s	87.6510 KOps/s	$\color{#d91a1a}-1.34\%$
test_step_mdp_speed[False-False-True-True-True]	69.9410μs	27.3627μs	36.5461 KOps/s	36.7102 KOps/s	$\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-False-True-True-False]	44.7840μs	18.7020μs	53.4702 KOps/s	54.1544 KOps/s	$\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-False-True-False-True]	43.6220μs	17.6940μs	56.5162 KOps/s	57.2646 KOps/s	$\color{#d91a1a}-1.31\%$
test_step_mdp_speed[False-False-True-False-False]	35.6060μs	11.6593μs	85.7684 KOps/s	87.3743 KOps/s	$\color{#d91a1a}-1.84\%$
test_step_mdp_speed[False-False-False-True-True]	60.1620μs	28.6839μs	34.8627 KOps/s	35.3839 KOps/s	$\color{#d91a1a}-1.47\%$
test_step_mdp_speed[False-False-False-True-False]	64.3220μs	19.8187μs	50.4573 KOps/s	51.4295 KOps/s	$\color{#d91a1a}-1.89\%$
test_step_mdp_speed[False-False-False-False-True]	76.8730μs	18.7257μs	53.4026 KOps/s	53.7347 KOps/s	$\color{#d91a1a}-0.62\%$
test_step_mdp_speed[False-False-False-False-False]	42.5200μs	12.6593μs	78.9930 KOps/s	79.6508 KOps/s	$\color{#d91a1a}-0.83\%$
test_values[generalized_advantage_estimate-True-True]	10.4256ms	9.4290ms	106.0560 Ops/s	107.8991 Ops/s	$\color{#d91a1a}-1.71\%$
test_values[vec_generalized_advantage_estimate-True-True]	36.9142ms	33.8620ms	29.5317 Ops/s	28.1802 Ops/s	$\color{#35bf28}+4.80\%$
test_values[td0_return_estimate-False-False]	0.2307ms	0.1741ms	5.7455 KOps/s	5.5499 KOps/s	$\color{#35bf28}+3.52\%$
test_values[td1_return_estimate-False-False]	30.2335ms	23.1795ms	43.1416 Ops/s	43.0435 Ops/s	$\color{#35bf28}+0.23\%$
test_values[vec_td1_return_estimate-False-False]	35.4062ms	33.8798ms	29.5162 Ops/s	28.0275 Ops/s	$\textbf{\color{#35bf28}+5.31\%}$
test_values[td_lambda_return_estimate-True-False]	35.9587ms	33.4995ms	29.8512 Ops/s	29.6812 Ops/s	$\color{#35bf28}+0.57\%$
test_values[vec_td_lambda_return_estimate-True-False]	39.0295ms	34.2691ms	29.1808 Ops/s	28.1194 Ops/s	$\color{#35bf28}+3.77\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	10.3094ms	8.0445ms	124.3086 Ops/s	122.0121 Ops/s	$\color{#35bf28}+1.88\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.3787ms	1.9596ms	510.3113 Ops/s	511.4659 Ops/s	$\color{#d91a1a}-0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5219ms	0.3537ms	2.8276 KOps/s	2.8214 KOps/s	$\color{#35bf28}+0.22\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	40.5009ms	39.5251ms	25.3004 Ops/s	21.7989 Ops/s	$\textbf{\color{#35bf28}+16.06\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	4.6823ms	3.1575ms	316.7056 Ops/s	330.9819 Ops/s	$\color{#d91a1a}-4.31\%$
test_dqn_speed	7.4214ms	1.3878ms	720.5771 Ops/s	736.3747 Ops/s	$\color{#d91a1a}-2.15\%$
test_ddpg_speed	3.2232ms	2.7603ms	362.2771 Ops/s	370.1300 Ops/s	$\color{#d91a1a}-2.12\%$
test_sac_speed	9.9532ms	8.3794ms	119.3405 Ops/s	112.8927 Ops/s	$\textbf{\color{#35bf28}+5.71\%}$
test_redq_speed	14.4512ms	13.4713ms	74.2321 Ops/s	75.8377 Ops/s	$\color{#d91a1a}-2.12\%$
test_redq_deprec_speed	97.8147ms	14.8107ms	67.5187 Ops/s	76.7659 Ops/s	$\textbf{\color{#d91a1a}-12.05\%}$
test_td3_speed	8.7824ms	8.2575ms	121.1016 Ops/s	121.4695 Ops/s	$\color{#d91a1a}-0.30\%$
test_cql_speed	38.0220ms	36.4904ms	27.4045 Ops/s	27.2986 Ops/s	$\color{#35bf28}+0.39\%$
test_a2c_speed	8.1000ms	7.4198ms	134.7737 Ops/s	134.8758 Ops/s	$\color{#d91a1a}-0.08\%$
test_ppo_speed	9.1600ms	7.7091ms	129.7170 Ops/s	129.4913 Ops/s	$\color{#35bf28}+0.17\%$
test_reinforce_speed	7.1727ms	6.5920ms	151.6979 Ops/s	151.4866 Ops/s	$\color{#35bf28}+0.14\%$
test_iql_speed	33.8263ms	32.6830ms	30.5969 Ops/s	30.5811 Ops/s	$\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.5546ms	2.2996ms	434.8654 Ops/s	422.2930 Ops/s	$\color{#35bf28}+2.98\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.2265ms	0.5063ms	1.9752 KOps/s	1.9282 KOps/s	$\color{#35bf28}+2.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7496ms	0.4787ms	2.0890 KOps/s	2.0636 KOps/s	$\color{#35bf28}+1.23\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.4061ms	2.3181ms	431.3952 Ops/s	407.9908 Ops/s	$\textbf{\color{#35bf28}+5.74\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.1101ms	0.4992ms	2.0032 KOps/s	1.9550 KOps/s	$\color{#35bf28}+2.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6756ms	0.4753ms	2.1041 KOps/s	2.0755 KOps/s	$\color{#35bf28}+1.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.3180ms	1.2145ms	823.4120 Ops/s	799.6965 Ops/s	$\color{#35bf28}+2.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.3443ms	1.1444ms	873.7840 Ops/s	855.8672 Ops/s	$\color{#35bf28}+2.09\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.5539ms	2.3825ms	419.7297 Ops/s	396.7025 Ops/s	$\textbf{\color{#35bf28}+5.80\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1128ms	0.6199ms	1.6132 KOps/s	1.5746 KOps/s	$\color{#35bf28}+2.45\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7776ms	0.5934ms	1.6853 KOps/s	1.6625 KOps/s	$\color{#35bf28}+1.37\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.6219ms	2.2931ms	436.0963 Ops/s	424.6391 Ops/s	$\color{#35bf28}+2.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0391ms	0.5057ms	1.9776 KOps/s	1.9206 KOps/s	$\color{#35bf28}+2.96\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6300ms	0.4795ms	2.0854 KOps/s	2.0055 KOps/s	$\color{#35bf28}+3.98\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.6041ms	2.2925ms	436.1981 Ops/s	411.8752 Ops/s	$\textbf{\color{#35bf28}+5.91\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6595ms	0.5019ms	1.9924 KOps/s	1.9598 KOps/s	$\color{#35bf28}+1.66\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.6623ms	0.4802ms	2.0823 KOps/s	2.0664 KOps/s	$\color{#35bf28}+0.77\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.7939ms	2.3998ms	416.6977 Ops/s	416.6106 Ops/s	$\color{#35bf28}+0.02\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1637ms	0.6239ms	1.6028 KOps/s	1.5660 KOps/s	$\color{#35bf28}+2.35\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7428ms	0.5918ms	1.6896 KOps/s	1.6398 KOps/s	$\color{#35bf28}+3.04\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1148s	7.9929ms	125.1110 Ops/s	117.9332 Ops/s	$\textbf{\color{#35bf28}+6.09\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	14.4695ms	12.2056ms	81.9296 Ops/s	82.0594 Ops/s	$\color{#d91a1a}-0.16\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.6069ms	1.0733ms	931.7247 Ops/s	918.2486 Ops/s	$\color{#35bf28}+1.47\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1050s	5.7339ms	174.4002 Ops/s	168.7127 Ops/s	$\color{#35bf28}+3.37\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	14.4824ms	12.2556ms	81.5954 Ops/s	81.8487 Ops/s	$\color{#d91a1a}-0.31\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.0071ms	1.1308ms	884.2996 Ops/s	912.2367 Ops/s	$\color{#d91a1a}-3.06\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1082s	8.3316ms	120.0249 Ops/s	114.5555 Ops/s	$\color{#35bf28}+4.77\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	14.9929ms	12.5586ms	79.6266 Ops/s	72.0937 Ops/s	$\textbf{\color{#35bf28}+10.45\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	4.4112ms	1.4558ms	686.9084 Ops/s	663.3291 Ops/s	$\color{#35bf28}+3.55\%$