chore(customizer): update RL image

[anubhutivyas]

Summary by CodeRabbit

• New Features

  • Added support for sandboxed Gym and episode broker actors in RL runtime environments.
  • Gym environment virtual environments can now be created on first use or pre-fetched through configuration.

• Documentation

  • Updated RL container guidance to explain prefetching behavior, sandboxed actors, runtime setup, and storage requirements.

• Tests

  • Expanded smoke-test coverage for the new sandboxed Gym and episode broker actors.

[coderabbitai]

📝 Walkthrough

Walkthrough

The RL image now pins a new NeMo-RL commit, prefetches and validates sandbox actor venvs, and disables Gym environment prefetching by default. Documentation and GPU smoke coverage reflect the seven-actor configuration and runtime environment installation.

Changes

RL image prefetch behavior

Layer / File(s)	Summary
Sandbox actor venv prefetching docker-bake.hcl, docker/rl/Dockerfile.nmp-rl-base, docker/rl/README.md, tests/smoke_gpu/test_rl_training.py	The build uses the new NeMo-RL commit. It prefetches and validates SandboxedGymActor and SandboxEpisodeBrokerActor. Smoke coverage checks their imports.
Opt-in Gym environment prefetching docker/rl/Dockerfile.nmp-rl-base, docker/rl/README.md	NEMO_GYM_PREFETCH_CONFIGS defaults to empty. Gym environments install at first use unless configuration is supplied. /opt/uv_cache also supports prefetched Gym environments.

Possibly related PRs

• NVIDIA-NeMo/nemo-platform#901: Updates the related NeMo-RL Docker prefetch configuration and smoke tests.

Suggested reviewers: ironcommit, gabwow, crookedstorm

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately identifies the primary change as an update to the RL image.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch anubhutiv/update-rl-image

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

docker/rl/Dockerfile.nmp-rl-base (1)

334-354: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Provide a writable runtime uv cache.

Line 340 installs built-in environments at runtime by default. The published base runs as UID 1000, but /opt/uv_cache remains read-only. A direct nmp-rl-base consumer can fail when uv must cache dependencies for a runtime environment install.

Set a writable runtime cache in this image, or do not present this stage as directly runnable for Gym workloads.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docker/rl/Dockerfile.nmp-rl-base` around lines 334 - 354, Update the NeMo-Gym
environment setup around NEMO_GYM_PREFETCH_CONFIGS and /opt/gym_venvs to provide
a writable /opt/uv_cache for the published UID 1000 runtime; ensure ownership
and permissions allow uv dependency caching during first-use environment
installation.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/smoke_gpu/test_rl_training.py`:
- Around line 69-89: Add “tenacity” to the import requirement lists for NemoGym,
SandboxedGymActor, and SandboxEpisodeBrokerActor in the environment dependency
assertions. Preserve all existing entries, including nemo_gym, opensandbox, and
each actor-specific module.

---

Outside diff comments:
In `@docker/rl/Dockerfile.nmp-rl-base`:
- Around line 334-354: Update the NeMo-Gym environment setup around
NEMO_GYM_PREFETCH_CONFIGS and /opt/gym_venvs to provide a writable /opt/uv_cache
for the published UID 1000 runtime; ensure ownership and permissions allow uv
dependency caching during first-use environment installation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 79ddd3c0-06bf-4227-9de2-a3b50783110b

📥 Commits

Reviewing files that changed from the base of the PR and between 47a57c3 and d130ae8.

📒 Files selected for processing (4)

• docker-bake.hcl
• docker/rl/Dockerfile.nmp-rl-base
• docker/rl/README.md
• tests/smoke_gpu/test_rl_training.py

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	29340/37347	78.6%	63.2%
Integration Tests	17288/36065	47.9%	20.6%

[svvarom]

LGTM - we have builds hanging rn - https://github.com/NVIDIA-NeMo/Platform-Deploy/actions/runs/30647335761. This will help fix it