feat: reinforcer initial commit

.gitignore

@@ -1,156 +1,29 @@
-# log and data files
-*.model
-*.pkl
-*.pt
-.DS_Store
-.hydra
-.bash_history.local
+# Adding to .gitignore helps reduce the size of your working_dir
 
-# Byte-compiled / optimized / DLL files
+.git
+*.out
+*.log
+*.tar
+*.tar.gz
+.venv
+venv
 __pycache__/
-*.py[cod]
-*$py.class
-**.pyc
-
-# C extensions
-*.so
-
-# Distribution / packaging
-.idea
-.Python
-wandb
+_build/
 build/
-develop-eggs/
+apidocs/
 dist/
-downloads/
-eggs/
-.eggs/
-lib/
-lib64/
-#parts/
-sdist/
-var/
-wheels/
-pip-wheel-metadata/
-share/python-wheels/
 *.egg-info/
-.installed.cfg
-*.egg
-MANIFEST
+*.vscode/
 
-# PyInstaller
-#  Usually these files are written by a python script from a template
-#  before PyInstaller builds the exe, so as to inject date/other infos into it.
-*.manifest
-*.spec
-
-# Installer logs
-pip-log.txt
-pip-delete-this-directory.txt
-
-# Unit test / coverage reports
-htmlcov/
-.tox/
-.nox/
+# Test
 .coverage
-.coverage.*
-.cache
-nosetests.xml
-coverage.xml
-*.cover
-.hypothesis/
-.pytest_cache/
-
-# Translations
-*.mo
-*.pot
-
-# Django stuff:
-*.log
-local_settings.py
-db.sqlite3
-
-# Flask stuff:
-instance/
-.webassets-cache
-
-# Scrapy stuff:
-.scrapy
-
-# Sphinx documentation
-docs/build
-
-# PyBuilder
-target/
-
-# Jupyter Notebook
-.ipynb_checkpoints
-
-# Override Jupyter in Github Language states for more accurate estimate of repo code.
-# Reference: https://github.com/github/linguist/blob/master/docs/overrides.md#generated-code
-*.ipynb linguist-generated
-
-# IPython
-profile_default/
-ipython_config.py
-
-# pyenv
-.python-version
-
-# pipenv
-#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
-#   However, in case of collaboration, if having platform-specific dependencies or dependencies
-#   having no cross-platform support, pipenv may install dependencies that don’t work, or not
-#   install all needed dependencies.
-#Pipfile.lock
-
-# celery beat schedule file
-celerybeat-schedule
-
-# SageMath parsed files
-*.sage.py
-
-# Environments
-.env
-.venv
-venv/
-env.bak/
-venv.bak/
-
-# VSCode project settins
-.vscode/
-
-# Spyder project settings
-.spyderproject
-.spyproject
-
-# Rope project settings
-.ropeproject
-
-# mkdocs documentation
-/site
-/docs/html
-/docs/docs_zh/zh
-
-# mypy
-.mypy_cache/
-.dmypy.json
-dmypy.json
-
-# Pyre type checker
-.pyre/
-
-# Emacs backup files
-*~
-
-*.tar.gz
-
-# Test data.
-tests/.data
-tests/data
-
-# outputs folder
-wandb
 
-# Checkpoints, config files and temporary files created in tutorials.
-.hydra/
+# Cache
+uv_cache/
+hf_home/
+*logs/
+datasets/
+docker/
+wandb/
+checkpoints/
+results/

CONTRIBUTING.md

@@ -0,0 +1,141 @@
+# Contributing To Nemo-Reinforcer
+
+Thanks for your interest in contributing to Nemo-Reinforcer!
+
+## Setting Up
+
+### Development Environment
+
+1. **Build and run the Docker container**:
+```bash
+docker buildx build -t nemo-reinforcer -f Dockerfile .
+# Run the container with your local nemo-reinforcer directory mounted
+docker run -it --gpus all -v /path/to/nemo-reinforcer:/workspace/nemo-reinforcer nemo-reinforcer
+```
+
+2. **Install the package in development mode**:
+```bash
+cd /workspace/nemo-reinforcer
+pip install -e .
+```
+
+## Making Changes
+
+### Workflow: Clone and Branch (No Fork Required)
+
+#### Before You Start: Install pre-commit
+
+From the [`nemo-reinforcer` root directory](.), run:
+```bash
+python3 -m pip install pre-commit
+pre-commit install
+```
+
+Pre-commit checks (using `ruff`) will help ensure your code follows our formatting and style guidelines.
+
+We follow a direct clone and branch workflow for now:
+
+1. Clone the repository directly:
+   ```bash
+   git clone https://github.com/NVIDIA/nemo__placeholder
+   cd nemo-reinforcer
+   ```
+
+2. Create a new branch for your changes:
+   ```bash
+   git checkout -b your-feature-name
+   ```
+
+3. Make your changes and commit them:
+   ```bash
+   git add .
+   git commit --signoff -m "Your descriptive commit message"
+   ```
+
+We require signing commits with `--signoff` (or `-s` for short). See [Signing Your Work](#signing-your-work) for details.
+
+4. Push your branch to the repository:
+   ```bash
+   git push origin feature/your-feature-name
+   ```
+
+5. Create a pull request from your branch to the `main` branch.
+
+### Design Documentation Requirement
+
+**Important**: All new key features (ex: enabling a new parallelization technique, enabling a new RL algorithm) must include documentation update (either a new doc or updating an existing one). This document update should:
+
+- Explain the motivation and purpose of the feature
+- Outline the technical approach and architecture
+- Provide clear usage examples and instructions for users
+- Document internal implementation details where appropriate
+
+This ensures that all significant changes are well-thought-out and properly documented for future reference. Comprehensive documentation serves two critical purposes:
+
+1. **User Adoption**: Helps users understand how to effectively use the library's features in their projects
+2. **Developer Extensibility**: Enables developers to understand the internal architecture and implementation details, making it easier to modify, extend, or adapt the code for their specific use cases
+
+Quality documentation is essential for both the usability of Nemo-Reinforcer and its ability to be customized by the community.
+
+## Code Quality
+
+- Follow the existing code style and conventions
+- Write tests for new features
+- Update documentation to reflect your changes
+- Ensure all tests pass before submitting a PR
+- Do not add arbitrary defaults for configs, be as explicit as possible.
+
+
+## Signing Your Work
+
+* We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
+
+  * Any contribution which contains commits that are not Signed-Off will not be accepted.
+
+* To sign off on a commit you simply use the `--signoff` (or `-s`) option when committing your changes:
+  ```bash
+  $ git commit -s -m "Add cool feature."
+  ```
+  This will append the following to your commit message:
+  ```
+  Signed-off-by: Your Name <your@email.com>
+  ```
+
+* Full text of the DCO:
+
+  ```
+  Developer Certificate of Origin
+  Version 1.1
+
+  Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
+
+  Everyone is permitted to copy and distribute verbatim copies of this
+  license document, but changing it is not allowed.
+
+
+  Developer's Certificate of Origin 1.1
+
+  By making a contribution to this project, I certify that:
+
+  (a) The contribution was created in whole or in part by me and I
+      have the right to submit it under the open source license
+      indicated in the file; or
+
+  (b) The contribution is based upon previous work that, to the best
+      of my knowledge, is covered under an appropriate open source
+      license and I have the right under that license to submit that
+      work with modifications, whether created in whole or in part
+      by me, under the same open source license (unless I am
+      permitted to submit under a different license), as indicated
+      in the file; or
+
+  (c) The contribution was provided directly to me by some other
+      person who certified (a), (b) or (c) and I have not modified
+      it.
+
+  (d) I understand and agree that this project and the contribution
+      are public and that a record of the contribution (including all
+      personal information I submit with it, including my sign-off) is
+      maintained indefinitely and may be redistributed consistent with
+      this project or the open source license(s) involved.
+  ```