Merge branch 'master' into feat/sac-jax

.github/pull_request_template.md

@@ -11,22 +11,23 @@
 ## Checklist:
 <!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
 <!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
-- [ ] I've read the [CONTRIBUTION](https://github.com/vwxyzjn/cleanrl/blob/master/CONTRIBUTING.md) guide (**required**).
+- [ ] I've read the [CONTRIBUTION](https://docs.cleanrl.dev/contribution/) guide (**required**).
 - [ ] I have ensured `pre-commit run --all-files` passes (**required**).
-- [ ] I have updated the documentation and previewed the changes via `mkdocs serve`.
 - [ ] I have updated the tests accordingly (if applicable).
-
-If you are adding new algorithm variants or your change could result in performance difference, you may need to (re-)run tracked experiments. See https://github.com/vwxyzjn/cleanrl/pull/137 as an example PR. 
-- [ ] I have contacted [vwxyzjn](https://github.com/vwxyzjn) to obtain access to the [openrlbenchmark W&B team](https://wandb.ai/openrlbenchmark) (**required**).
-- [ ] I have tracked applicable experiments in [openrlbenchmark/cleanrl](https://wandb.ai/openrlbenchmark/cleanrl) with `--capture-video` flag toggled on (**required**).
-- [ ] I have added additional documentation and previewed the changes via `mkdocs serve`.
+- [ ] I have updated the documentation and previewed the changes via `mkdocs serve`.
     - [ ] I have explained note-worthy implementation details.
     - [ ] I have explained the logged metrics.
-    - [ ] I have added links to the original paper and related papers (if applicable).
-    - [ ] I have added links to the PR related to the algorithm variant.
-    - [ ] I have created a table comparing my results against those from reputable sources (i.e., the original paper or other reference implementation).
-    - [ ] I have added the learning curves (in PNG format).
-    - [ ] I have added links to the tracked experiments.
-    - [ ] I have updated the overview sections at the [docs](https://docs.cleanrl.dev/rl-algorithms/overview/) and the [repo](https://github.com/vwxyzjn/cleanrl#overview)
-- [ ] I have updated the tests accordingly (if applicable).
+    - [ ] I have added links to the original paper and related papers.
+
+If you need to run benchmark experiments for a performance-impacting changes:
+
+- [ ] I have contacted @vwxyzjn to obtain access to the [openrlbenchmark W&B team](https://wandb.ai/openrlbenchmark).
+- [ ] I have used the [benchmark utility](/get-started/benchmark-utility/) to submit the tracked experiments to the [openrlbenchmark/cleanrl](https://wandb.ai/openrlbenchmark/cleanrl) W&B project, optionally with `--capture-video`.
+- [ ] I have performed RLops with `python -m openrlbenchmark.rlops`.
+    - For new feature or bug fix:
+        - [ ] I have used the RLops utility to understand the performance impact of the changes and confirmed there is no regression.
+    - For new algorithm:
+        - [ ] I have created a table comparing my results against those from reputable sources (i.e., the original paper or other reference implementation).
+    - [ ] I have added the learning curves generated by the `python -m openrlbenchmark.rlops` utility to the documentation.
+    - [ ] I have added links to the tracked experiments in W&B, generated by `python -m openrlbenchmark.rlops ....your_args... --report`,  to the documentation.
 

.github/workflows/pre-commit.yml

@@ -2,9 +2,11 @@ name: pre-commit
 
 on:
   push:
-    branches: [ master ]
+    paths-ignore:
+      - 'docs/blog/*' # dummy ignore
   pull_request:
-    branches: [ master ]
+    paths-ignore:
+      - 'docs/blog/*' # dummy ignore
 jobs:
   build:
     runs-on: ubuntu-latest