[Quality] Fix flaky test

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] Documentation #3192
• [Feature] SAC Trainer #3191
• [Feature] PPO Trainer Updates #3190
• [Feature] Trainer Algorithms - Configuration System #3189
• [Feature] Trainer Infrastructure - Timing and Utilities #3188
• [CI] LLM tests integration #3216
• -> [Quality] Fix flaky test #3211

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3211

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 8 New Failures, 5 Pending, 2 Unrelated Failures

As of commit 952b7e7 with merge base 963fdd4 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 139.
• Generate documentation / build-docs (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 96364d03f872aabeb1d65fa63f30b3bea6c6701220899488700dae16f89f0ac0 /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 7cac6347acc93136d6aca8b464bb6003374c2714cb02a1c4c112f10434c1a443 /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t a0d930791becd5e8b007a7294e04f4949d90b62f488b19dd0836c10c8591bb9f /exec failed with exit code 1
• Libs Tests on Linux / unittests-sklearn (3.10, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 031da6374ddc043faa8a3e24db23fc6ad061cdd3a1a8c424c511b7e222c7f376 /exec failed with exit code 2
• Lint / python-source-and-configs / linux-job (gh)
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 09feab083cb7d6100d44b91ea01d3935d71909ab08349eacaabbde9096a224fd /exec failed with exit code 1
• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh)
  Process completed with exit code 2.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Linux / tests-olddeps (3.9, 11.6) / linux-job (gh) (trunk failure)
  test/test_collector.py::TestPolicyFactory::test_weight_update
• Unit-tests on Linux / tests-optdeps (3.11, 12.8) / linux-job (gh) (trunk failure)
  test/test_collector.py::TestCollectorGeneric::test_update_weights[True-True]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}20$. Worsened: $\large\color{#d91a1a}15$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	85.4804μs	83.2978μs	12.0051 KOps/s	11.6622 KOps/s	$\color{#35bf28}+2.94\%$
test_tensor_to_bytestream_speed[torch.save]	0.1436ms	0.1433ms	6.9788 KOps/s	6.8860 KOps/s	$\color{#35bf28}+1.35\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1141s	0.1139s	8.7821 Ops/s	8.6899 Ops/s	$\color{#35bf28}+1.06\%$
test_tensor_to_bytestream_speed[numpy]	2.8640μs	2.8551μs	350.2456 KOps/s	365.2068 KOps/s	$\color{#d91a1a}-4.10\%$
test_tensor_to_bytestream_speed[safetensors]	43.1163μs	42.4935μs	23.5330 KOps/s	22.1231 KOps/s	$\textbf{\color{#35bf28}+6.37\%}$
test_simple	0.5443s	0.5385s	1.8569 Ops/s	1.7815 Ops/s	$\color{#35bf28}+4.23\%$
test_transformed	1.2074s	1.1175s	0.8948 Ops/s	0.9004 Ops/s	$\color{#d91a1a}-0.61\%$
test_serial	1.6448s	1.6332s	0.6123 Ops/s	0.6077 Ops/s	$\color{#35bf28}+0.75\%$
test_parallel	1.1706s	1.0838s	0.9227 Ops/s	0.9347 Ops/s	$\color{#d91a1a}-1.29\%$
test_step_mdp_speed[True-True-True-True-True]	0.2117ms	43.0609μs	23.2229 KOps/s	22.8771 KOps/s	$\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-True-True-True-False]	54.6430μs	24.6192μs	40.6188 KOps/s	40.0319 KOps/s	$\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-True-True-False-True]	63.1840μs	24.1515μs	41.4053 KOps/s	40.2106 KOps/s	$\color{#35bf28}+2.97\%$
test_step_mdp_speed[True-True-True-False-False]	38.4530μs	13.5535μs	73.7818 KOps/s	73.4947 KOps/s	$\color{#35bf28}+0.39\%$
test_step_mdp_speed[True-True-False-True-True]	0.1250ms	46.4244μs	21.5404 KOps/s	21.2412 KOps/s	$\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-True-False-True-False]	60.0640μs	26.7508μs	37.3821 KOps/s	36.8236 KOps/s	$\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-True-False-False-True]	60.1030μs	26.7758μs	37.3472 KOps/s	36.8040 KOps/s	$\color{#35bf28}+1.48\%$
test_step_mdp_speed[True-True-False-False-False]	41.1820μs	16.0938μs	62.1357 KOps/s	63.0461 KOps/s	$\color{#d91a1a}-1.44\%$
test_step_mdp_speed[True-False-True-True-True]	83.3850μs	48.5524μs	20.5963 KOps/s	19.6229 KOps/s	$\color{#35bf28}+4.96\%$
test_step_mdp_speed[True-False-True-True-False]	0.1098ms	29.4117μs	34.0000 KOps/s	32.9863 KOps/s	$\color{#35bf28}+3.07\%$
test_step_mdp_speed[True-False-True-False-True]	60.1530μs	26.9380μs	37.1223 KOps/s	35.9974 KOps/s	$\color{#35bf28}+3.12\%$
test_step_mdp_speed[True-False-True-False-False]	52.4840μs	15.9715μs	62.6117 KOps/s	61.8816 KOps/s	$\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-False-False-True-True]	85.4250μs	51.0105μs	19.6038 KOps/s	19.1489 KOps/s	$\color{#35bf28}+2.38\%$
test_step_mdp_speed[True-False-False-True-False]	64.5340μs	32.2197μs	31.0369 KOps/s	31.6065 KOps/s	$\color{#d91a1a}-1.80\%$
test_step_mdp_speed[True-False-False-False-True]	62.3140μs	29.4881μs	33.9120 KOps/s	33.8604 KOps/s	$\color{#35bf28}+0.15\%$
test_step_mdp_speed[True-False-False-False-False]	46.0820μs	18.5764μs	53.8318 KOps/s	53.7932 KOps/s	$\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-True-True-True-True]	83.7750μs	48.9688μs	20.4212 KOps/s	20.0971 KOps/s	$\color{#35bf28}+1.61\%$
test_step_mdp_speed[False-True-True-True-False]	61.3240μs	29.3232μs	34.1027 KOps/s	33.1264 KOps/s	$\color{#35bf28}+2.95\%$
test_step_mdp_speed[False-True-True-False-True]	2.4577ms	30.7573μs	32.5126 KOps/s	31.6706 KOps/s	$\color{#35bf28}+2.66\%$
test_step_mdp_speed[False-True-True-False-False]	44.1620μs	18.0692μs	55.3429 KOps/s	55.1566 KOps/s	$\color{#35bf28}+0.34\%$
test_step_mdp_speed[False-True-False-True-True]	0.1062ms	51.1922μs	19.5342 KOps/s	18.7826 KOps/s	$\color{#35bf28}+4.00\%$
test_step_mdp_speed[False-True-False-True-False]	63.3540μs	32.2386μs	31.0188 KOps/s	30.5907 KOps/s	$\color{#35bf28}+1.40\%$
test_step_mdp_speed[False-True-False-False-True]	73.0940μs	33.0143μs	30.2899 KOps/s	29.3728 KOps/s	$\color{#35bf28}+3.12\%$
test_step_mdp_speed[False-True-False-False-False]	51.0730μs	20.2087μs	49.4835 KOps/s	48.6804 KOps/s	$\color{#35bf28}+1.65\%$
test_step_mdp_speed[False-False-True-True-True]	90.6650μs	54.6648μs	18.2933 KOps/s	18.0330 KOps/s	$\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-False-True-True-False]	73.2640μs	34.7696μs	28.7607 KOps/s	28.7822 KOps/s	$\color{#d91a1a}-0.07\%$
test_step_mdp_speed[False-False-True-False-True]	63.4840μs	32.7775μs	30.5087 KOps/s	29.8535 KOps/s	$\color{#35bf28}+2.19\%$
test_step_mdp_speed[False-False-True-False-False]	87.1050μs	20.3201μs	49.2125 KOps/s	50.4584 KOps/s	$\color{#d91a1a}-2.47\%$
test_step_mdp_speed[False-False-False-True-True]	96.3750μs	55.2780μs	18.0904 KOps/s	17.3508 KOps/s	$\color{#35bf28}+4.26\%$
test_step_mdp_speed[False-False-False-True-False]	82.3950μs	37.2132μs	26.8722 KOps/s	26.7888 KOps/s	$\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-False-False-True]	76.4540μs	35.3163μs	28.3156 KOps/s	27.6439 KOps/s	$\color{#35bf28}+2.43\%$
test_step_mdp_speed[False-False-False-False-False]	55.5230μs	22.6372μs	44.1751 KOps/s	43.9306 KOps/s	$\color{#35bf28}+0.56\%$
test_values[generalized_advantage_estimate-True-True]	10.4515ms	10.1174ms	98.8395 Ops/s	100.3478 Ops/s	$\color{#d91a1a}-1.50\%$
test_values[vec_generalized_advantage_estimate-True-True]	14.5016ms	11.0591ms	90.4236 Ops/s	57.5516 Ops/s	$\textbf{\color{#35bf28}+57.12\%}$
test_values[td0_return_estimate-False-False]	0.2394ms	0.1316ms	7.5980 KOps/s	7.7157 KOps/s	$\color{#d91a1a}-1.53\%$
test_values[td1_return_estimate-False-False]	29.3825ms	27.4174ms	36.4732 Ops/s	36.5386 Ops/s	$\color{#d91a1a}-0.18\%$
test_values[vec_td1_return_estimate-False-False]	11.4683ms	11.1196ms	89.9312 Ops/s	57.2669 Ops/s	$\textbf{\color{#35bf28}+57.04\%}$
test_values[td_lambda_return_estimate-True-False]	41.5614ms	40.6813ms	24.5813 Ops/s	24.4825 Ops/s	$\color{#35bf28}+0.40\%$
test_values[vec_td_lambda_return_estimate-True-False]	12.9133ms	11.1537ms	89.6566 Ops/s	57.3202 Ops/s	$\textbf{\color{#35bf28}+56.41\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.9179ms	8.6631ms	115.4324 Ops/s	117.7317 Ops/s	$\color{#d91a1a}-1.95\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.7202ms	1.4638ms	683.1653 Ops/s	658.1917 Ops/s	$\color{#35bf28}+3.79\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.8520ms	0.4192ms	2.3856 KOps/s	2.3682 KOps/s	$\color{#35bf28}+0.73\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	29.6831ms	28.9752ms	34.5123 Ops/s	28.8413 Ops/s	$\textbf{\color{#35bf28}+19.66\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.8109ms	1.6965ms	589.4445 Ops/s	582.7143 Ops/s	$\color{#35bf28}+1.15\%$
test_dqn_speed[False-None]	1.7651ms	1.4055ms	711.5073 Ops/s	701.5318 Ops/s	$\color{#35bf28}+1.42\%$
test_dqn_speed[False-backward]	2.0746ms	1.9105ms	523.4223 Ops/s	522.9646 Ops/s	$\color{#35bf28}+0.09\%$
test_dqn_speed[True-None]	0.6605ms	0.5091ms	1.9641 KOps/s	1.9843 KOps/s	$\color{#d91a1a}-1.02\%$
test_dqn_speed[True-backward]	0.9843ms	0.9505ms	1.0521 KOps/s	937.1316 Ops/s	$\textbf{\color{#35bf28}+12.27\%}$
test_dqn_speed[reduce-overhead-None]	0.8578ms	0.5077ms	1.9697 KOps/s	1.9255 KOps/s	$\color{#35bf28}+2.30\%$
test_dqn_speed[reduce-overhead-backward]	0.9700ms	0.9423ms	1.0613 KOps/s	956.9946 Ops/s	$\textbf{\color{#35bf28}+10.90\%}$
test_ddpg_speed[False-None]	3.1813ms	2.8435ms	351.6748 Ops/s	351.3104 Ops/s	$\color{#35bf28}+0.10\%$
test_ddpg_speed[False-backward]	4.3280ms	4.0579ms	246.4299 Ops/s	247.0413 Ops/s	$\color{#d91a1a}-0.25\%$
test_ddpg_speed[True-None]	1.7435ms	1.3481ms	741.7681 Ops/s	717.7321 Ops/s	$\color{#35bf28}+3.35\%$
test_ddpg_speed[True-backward]	2.4384ms	2.3379ms	427.7412 Ops/s	351.8686 Ops/s	$\textbf{\color{#35bf28}+21.56\%}$
test_ddpg_speed[reduce-overhead-None]	1.7345ms	1.3387ms	746.9938 Ops/s	723.0267 Ops/s	$\color{#35bf28}+3.31\%$
test_ddpg_speed[reduce-overhead-backward]	2.4335ms	2.3304ms	429.1054 Ops/s	352.9781 Ops/s	$\textbf{\color{#35bf28}+21.57\%}$
test_sac_speed[False-None]	8.6205ms	7.8095ms	128.0495 Ops/s	125.2131 Ops/s	$\color{#35bf28}+2.27\%$
test_sac_speed[False-backward]	11.4523ms	11.0673ms	90.3563 Ops/s	89.1763 Ops/s	$\color{#35bf28}+1.32\%$
test_sac_speed[True-None]	2.4808ms	2.0941ms	477.5303 Ops/s	473.5303 Ops/s	$\color{#35bf28}+0.84\%$
test_sac_speed[True-backward]	4.2055ms	4.0134ms	249.1683 Ops/s	217.0857 Ops/s	$\textbf{\color{#35bf28}+14.78\%}$
test_sac_speed[reduce-overhead-None]	2.2100ms	2.0878ms	478.9677 Ops/s	472.6396 Ops/s	$\color{#35bf28}+1.34\%$
test_sac_speed[reduce-overhead-backward]	4.2270ms	4.0674ms	245.8585 Ops/s	232.0335 Ops/s	$\textbf{\color{#35bf28}+5.96\%}$
test_redq_speed[False-None]	10.9505ms	10.2989ms	97.0980 Ops/s	95.1398 Ops/s	$\color{#35bf28}+2.06\%$
test_redq_speed[False-backward]	19.1102ms	17.9101ms	55.8344 Ops/s	54.3788 Ops/s	$\color{#35bf28}+2.68\%$
test_redq_speed[True-None]	4.6776ms	4.3982ms	227.3675 Ops/s	223.5037 Ops/s	$\color{#35bf28}+1.73\%$
test_redq_speed[True-backward]	10.3787ms	9.8498ms	101.5253 Ops/s	99.5723 Ops/s	$\color{#35bf28}+1.96\%$
test_redq_speed[reduce-overhead-None]	4.7817ms	4.3845ms	228.0755 Ops/s	231.9482 Ops/s	$\color{#d91a1a}-1.67\%$
test_redq_speed[reduce-overhead-backward]	10.2224ms	9.9530ms	100.4720 Ops/s	102.7333 Ops/s	$\color{#d91a1a}-2.20\%$
test_redq_deprec_speed[False-None]	11.6648ms	10.8761ms	91.9450 Ops/s	92.9222 Ops/s	$\color{#d91a1a}-1.05\%$
test_redq_deprec_speed[False-backward]	16.2396ms	15.7094ms	63.6561 Ops/s	64.5043 Ops/s	$\color{#d91a1a}-1.31\%$
test_redq_deprec_speed[True-None]	3.9220ms	3.6296ms	275.5153 Ops/s	276.9988 Ops/s	$\color{#d91a1a}-0.54\%$
test_redq_deprec_speed[True-backward]	7.8113ms	7.5299ms	132.8035 Ops/s	123.2352 Ops/s	$\textbf{\color{#35bf28}+7.76\%}$
test_redq_deprec_speed[reduce-overhead-None]	3.9604ms	3.5914ms	278.4459 Ops/s	283.4700 Ops/s	$\color{#d91a1a}-1.77\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.8801ms	7.5960ms	131.6480 Ops/s	131.8910 Ops/s	$\color{#d91a1a}-0.18\%$
test_td3_speed[False-None]	8.1178ms	7.9002ms	126.5794 Ops/s	127.5906 Ops/s	$\color{#d91a1a}-0.79\%$
test_td3_speed[False-backward]	11.2878ms	10.7838ms	92.7316 Ops/s	93.5989 Ops/s	$\color{#d91a1a}-0.93\%$
test_td3_speed[True-None]	1.9779ms	1.8232ms	548.4838 Ops/s	550.4572 Ops/s	$\color{#d91a1a}-0.36\%$
test_td3_speed[True-backward]	3.7213ms	3.6028ms	277.5599 Ops/s	262.3105 Ops/s	$\textbf{\color{#35bf28}+5.81\%}$
test_td3_speed[reduce-overhead-None]	1.8130ms	1.7803ms	561.6896 Ops/s	566.9099 Ops/s	$\color{#d91a1a}-0.92\%$
test_td3_speed[reduce-overhead-backward]	3.7882ms	3.6905ms	270.9664 Ops/s	243.1891 Ops/s	$\textbf{\color{#35bf28}+11.42\%}$
test_cql_speed[False-None]	26.5511ms	25.7861ms	38.7806 Ops/s	38.7389 Ops/s	$\color{#35bf28}+0.11\%$
test_cql_speed[False-backward]	39.5475ms	35.7323ms	27.9859 Ops/s	27.9115 Ops/s	$\color{#35bf28}+0.27\%$
test_cql_speed[True-None]	15.5311ms	12.7873ms	78.2027 Ops/s	80.9479 Ops/s	$\color{#d91a1a}-3.39\%$
test_cql_speed[True-backward]	19.0119ms	18.6127ms	53.7268 Ops/s	54.6940 Ops/s	$\color{#d91a1a}-1.77\%$
test_cql_speed[reduce-overhead-None]	12.9595ms	12.5514ms	79.6721 Ops/s	80.1081 Ops/s	$\color{#d91a1a}-0.54\%$
test_cql_speed[reduce-overhead-backward]	19.0228ms	18.5652ms	53.8642 Ops/s	54.5932 Ops/s	$\color{#d91a1a}-1.34\%$
test_a2c_speed[False-None]	5.6790ms	5.3201ms	187.9660 Ops/s	187.4227 Ops/s	$\color{#35bf28}+0.29\%$
test_a2c_speed[False-backward]	12.0589ms	11.7900ms	84.8174 Ops/s	83.6760 Ops/s	$\color{#35bf28}+1.36\%$
test_a2c_speed[True-None]	3.9539ms	3.7428ms	267.1762 Ops/s	266.8450 Ops/s	$\color{#35bf28}+0.12\%$
test_a2c_speed[True-backward]	8.8488ms	8.5191ms	117.3837 Ops/s	115.7966 Ops/s	$\color{#35bf28}+1.37\%$
test_a2c_speed[reduce-overhead-None]	3.9036ms	3.7265ms	268.3465 Ops/s	273.3838 Ops/s	$\color{#d91a1a}-1.84\%$
test_a2c_speed[reduce-overhead-backward]	8.9726ms	8.7681ms	114.0495 Ops/s	110.2228 Ops/s	$\color{#35bf28}+3.47\%$
test_ppo_speed[False-None]	6.3436ms	5.9121ms	169.1448 Ops/s	173.2359 Ops/s	$\color{#d91a1a}-2.36\%$
test_ppo_speed[False-backward]	13.1913ms	12.6261ms	79.2012 Ops/s	80.2591 Ops/s	$\color{#d91a1a}-1.32\%$
test_ppo_speed[True-None]	3.8028ms	3.6466ms	274.2244 Ops/s	272.2143 Ops/s	$\color{#35bf28}+0.74\%$
test_ppo_speed[True-backward]	8.7271ms	8.4606ms	118.1954 Ops/s	119.4549 Ops/s	$\color{#d91a1a}-1.05\%$
test_ppo_speed[reduce-overhead-None]	3.7973ms	3.6272ms	275.6963 Ops/s	296.6159 Ops/s	$\textbf{\color{#d91a1a}-7.05\%}$
test_ppo_speed[reduce-overhead-backward]	9.0422ms	8.7569ms	114.1955 Ops/s	119.2535 Ops/s	$\color{#d91a1a}-4.24\%$
test_reinforce_speed[False-None]	4.8251ms	4.5482ms	219.8654 Ops/s	233.8528 Ops/s	$\textbf{\color{#d91a1a}-5.98\%}$
test_reinforce_speed[False-backward]	7.6312ms	7.3493ms	136.0682 Ops/s	137.7603 Ops/s	$\color{#d91a1a}-1.23\%$
test_reinforce_speed[True-None]	3.0709ms	2.8717ms	348.2241 Ops/s	370.6964 Ops/s	$\textbf{\color{#d91a1a}-6.06\%}$
test_reinforce_speed[True-backward]	7.9598ms	7.6886ms	130.0632 Ops/s	127.7442 Ops/s	$\color{#35bf28}+1.82\%$
test_reinforce_speed[reduce-overhead-None]	3.2096ms	2.8638ms	349.1880 Ops/s	354.0800 Ops/s	$\color{#d91a1a}-1.38\%$
test_reinforce_speed[reduce-overhead-backward]	8.1181ms	7.8923ms	126.7063 Ops/s	126.5864 Ops/s	$\color{#35bf28}+0.09\%$
test_iql_speed[False-None]	26.1002ms	20.2293ms	49.4333 Ops/s	49.9179 Ops/s	$\color{#d91a1a}-0.97\%$
test_iql_speed[False-backward]	31.1785ms	30.4557ms	32.8346 Ops/s	33.0050 Ops/s	$\color{#d91a1a}-0.52\%$
test_iql_speed[True-None]	8.8790ms	8.5124ms	117.4755 Ops/s	117.3440 Ops/s	$\color{#35bf28}+0.11\%$
test_iql_speed[True-backward]	18.1683ms	16.9424ms	59.0235 Ops/s	59.6199 Ops/s	$\color{#d91a1a}-1.00\%$
test_iql_speed[reduce-overhead-None]	8.9109ms	8.5395ms	117.1025 Ops/s	114.7375 Ops/s	$\color{#35bf28}+2.06\%$
test_iql_speed[reduce-overhead-backward]	17.5484ms	17.2473ms	57.9801 Ops/s	59.0904 Ops/s	$\color{#d91a1a}-1.88\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.3939ms	5.8203ms	171.8133 Ops/s	169.5396 Ops/s	$\color{#35bf28}+1.34\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6774ms	0.3326ms	3.0066 KOps/s	3.2102 KOps/s	$\textbf{\color{#d91a1a}-6.34\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6067ms	0.3571ms	2.8002 KOps/s	3.4106 KOps/s	$\textbf{\color{#d91a1a}-17.90\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.8384ms	5.5527ms	180.0921 Ops/s	178.7035 Ops/s	$\color{#35bf28}+0.78\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.3757ms	0.3959ms	2.5256 KOps/s	3.3168 KOps/s	$\textbf{\color{#d91a1a}-23.85\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5762ms	0.3562ms	2.8075 KOps/s	3.5649 KOps/s	$\textbf{\color{#d91a1a}-21.25\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.6742ms	1.4594ms	685.2264 Ops/s	810.3007 Ops/s	$\textbf{\color{#d91a1a}-15.44\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.6046ms	1.3754ms	727.0812 Ops/s	867.6869 Ops/s	$\textbf{\color{#d91a1a}-16.20\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	9.7430ms	5.8756ms	170.1941 Ops/s	173.0801 Ops/s	$\color{#d91a1a}-1.67\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.8812ms	0.5335ms	1.8745 KOps/s	2.2777 KOps/s	$\textbf{\color{#d91a1a}-17.70\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6989ms	0.4528ms	2.2086 KOps/s	2.3412 KOps/s	$\textbf{\color{#d91a1a}-5.66\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.7041ms	5.5545ms	180.0351 Ops/s	178.7968 Ops/s	$\color{#35bf28}+0.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.7160ms	0.3511ms	2.8479 KOps/s	814.5161 Ops/s	$\textbf{\color{#35bf28}+249.64\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.4885ms	0.3068ms	3.2599 KOps/s	3.8724 KOps/s	$\textbf{\color{#d91a1a}-15.82\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.8180ms	5.5172ms	181.2514 Ops/s	175.6993 Ops/s	$\color{#35bf28}+3.16\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.5156ms	0.2798ms	3.5743 KOps/s	3.7249 KOps/s	$\color{#d91a1a}-4.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.4844ms	0.2519ms	3.9703 KOps/s	3.9751 KOps/s	$\color{#d91a1a}-0.12\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.0886ms	5.6758ms	176.1877 Ops/s	169.8840 Ops/s	$\color{#35bf28}+3.71\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9083ms	0.4356ms	2.2958 KOps/s	2.3157 KOps/s	$\color{#d91a1a}-0.86\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7341ms	0.5094ms	1.9633 KOps/s	2.4482 KOps/s	$\textbf{\color{#d91a1a}-19.81\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.5275s	15.4370ms	64.7793 Ops/s	196.8328 Ops/s	$\textbf{\color{#d91a1a}-67.09\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	9.3767ms	2.0025ms	499.3739 Ops/s	442.1335 Ops/s	$\textbf{\color{#35bf28}+12.95\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	7.2183ms	1.1594ms	862.4827 Ops/s	796.1238 Ops/s	$\textbf{\color{#35bf28}+8.34\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	6.5624ms	5.0325ms	198.7083 Ops/s	55.8295 Ops/s	$\textbf{\color{#35bf28}+255.92\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	9.7276ms	2.0202ms	495.0018 Ops/s	515.0467 Ops/s	$\color{#d91a1a}-3.89\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	5.8670ms	1.2043ms	830.3297 Ops/s	867.6778 Ops/s	$\color{#d91a1a}-4.30\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.5153s	15.4371ms	64.7791 Ops/s	186.5260 Ops/s	$\textbf{\color{#d91a1a}-65.27\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	3.9598ms	1.8900ms	529.0944 Ops/s	451.5786 Ops/s	$\textbf{\color{#35bf28}+17.17\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.1118ms	0.9758ms	1.0248 KOps/s	729.9722 Ops/s	$\textbf{\color{#35bf28}+40.39\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	34.6493ms	32.6400ms	30.6372 Ops/s	30.3260 Ops/s	$\color{#35bf28}+1.03\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	18.9737ms	17.2761ms	57.8835 Ops/s	56.4763 Ops/s	$\color{#35bf28}+2.49\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	35.7249ms	33.4986ms	29.8520 Ops/s	29.2865 Ops/s	$\color{#35bf28}+1.93\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.1031ms	17.4120ms	57.4316 Ops/s	56.0204 Ops/s	$\color{#35bf28}+2.52\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	38.0573ms	35.2981ms	28.3301 Ops/s	28.3502 Ops/s	$\color{#d91a1a}-0.07\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.7478ms	18.8663ms	53.0045 Ops/s	52.5878 Ops/s	$\color{#35bf28}+0.79\%$