feat: start nemo gym and other environments with cached venvs

[terrykong]

This avoids issues like seeing (raylet) Installing ... which can lead to failures in high node count settings

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Issues

List issues that this PR closes (syntax):
closes #1925

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

Summary by CodeRabbit

• Bug Fixes
  • Enhanced environment variable propagation in distributed setups.
  • Added automatic virtual environment initialization on compute nodes when using uv-based Python environments.

[coderabbitai]

📝 Walkthrough

Walkthrough

Modified environment creation logic in utils.py to conditionally create local virtual environments on actor nodes when using the "uv" Python executable, with proper environment variable propagation to the runtime environment.

Changes

Cohort / File(s)	Summary
Virtual Environment Setup nemo_rl/environments/utils.py	Added import for create_local_venv_on_each_node utility. Implemented conditional venv creation when actor Python executable starts with "uv"; introduced extra_env_vars dict to propagate VIRTUAL_ENV and UV_PROJECT_ENVIRONMENT paths. Updated py_executable resolution to use conditionally-set actor_py_exec and merged environment variables from os.environ with extra_env_vars.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Results For Major Changes	❓ Inconclusive	Unable to verify PR description and test documentation status due to missing git repository context and file access.	Provide access to git repository with PR details and actual file changes to assess whether testing documentation requirements are met.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding functionality to start environments with cached virtual environments (venvs) to avoid runtime installation.
Linked Issues check	✅ Passed	The PR implementation aligns with issue #1925 objectives: it adds venv caching logic to prevent runtime installation messages and improve environment startup reliability.
Out of Scope Changes check	✅ Passed	All changes in nemo_rl/environments/utils.py are directly related to implementing cached venv functionality for environment initialization, with no extraneous modifications.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch tk/start-environment-gym

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@nemo_rl/environments/utils.py`:
- Around line 116-119: The environment variables VIRTUAL_ENV and
UV_PROJECT_ENVIRONMENT are being set to actor_py_exec (the Python executable
path returned by create_local_venv_on_each_node) instead of the venv root;
compute the venv root from actor_py_exec (e.g., venv_root =
os.path.dirname(os.path.dirname(actor_py_exec))) and set both VIRTUAL_ENV and
UV_PROJECT_ENVIRONMENT to venv_root in the extra_env_vars mapping, or
alternatively change create_local_venv_on_each_node to return both
(executable_path, venv_root) and use the returned venv_root when populating
extra_env_vars.

🧹 Nitpick comments (2)

nemo_rl/environments/utils.py (2)

111-115: Minor: dotted FQN used as venv_name creates unusually-named directories.

actor_class_fqn (e.g. "nemo_rl.environments.nemo_gym.NemoGym") is passed as venv_name. This will create directories with dots in the name. Not a bug, but consider sanitizing (e.g. replacing . with _) or using just the class name for readability.

---

122-123: Propagating the entire driver os.environ to workers may leak sensitive variables.

{**dict(os.environ), **extra_env_vars} copies every environment variable from the driver process to worker runtime environments. This could unintentionally propagate secrets, tokens, or driver-specific config. Consider allowlisting only the variables that workers actually need, or at minimum filtering out known sensitive prefixes.