From f021f0e0c1ef2555766beafd8b12771e38e652c8 Mon Sep 17 00:00:00 2001 From: Ahmad Kiswani Date: Mon, 1 Dec 2025 17:50:39 -0800 Subject: [PATCH 1/7] docs: Moved configuration system under about Signed-off-by: Ahmad Kiswani --- docs/about/concepts/index.md | 7 ------- docs/about/{concepts => }/configuration-system.md | 0 docs/index.md | 1 + 3 files changed, 1 insertion(+), 7 deletions(-) rename docs/about/{concepts => }/configuration-system.md (100%) diff --git a/docs/about/concepts/index.md b/docs/about/concepts/index.md index 097c36eb2..088ae11c0 100644 --- a/docs/about/concepts/index.md +++ b/docs/about/concepts/index.md @@ -26,12 +26,6 @@ Each explainer below covers one foundational idea and links to deeper material. Understand how Models, Resources, and Agents remain decoupled yet coordinated as independent HTTP services, including which endpoints each abstraction exposes. ::: -:::{grid-item-card} {octicon}`gear;1.5em;sd-mr-1` Configuration System -:link: configuration-management -:link-type: ref -Learn how NeMo Gym's three-tier configuration system (YAML → env.yaml → CLI) enables secure secrets management and flexible multi-environment deployments. -::: - :::{grid-item-card} {octicon}`check-circle;1.5em;sd-mr-1` Task Verification :link: task-verification :link-type: ref @@ -53,7 +47,6 @@ Essential vocabulary for agent training, RL workflows, and NeMo Gym. This glossa :maxdepth: 1 Core Abstractions -Configuration System Task Verification Key Terminology ``` diff --git a/docs/about/concepts/configuration-system.md b/docs/about/configuration-system.md similarity index 100% rename from docs/about/concepts/configuration-system.md rename to docs/about/configuration-system.md diff --git a/docs/index.md b/docs/index.md index c5766c014..97e7d9be7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -114,6 +114,7 @@ Home about/index.md Concepts Ecosystem +Configuration System ``` ```{toctree} From 700514d24096fc1700c65671f011d970e655a24d Mon Sep 17 00:00:00 2001 From: "L.B" Date: Wed, 3 Dec 2025 14:53:19 -0500 Subject: [PATCH 2/7] config pages (#428) Signed-off-by: L.B --- docs/about/concepts/configuration.md | 60 ++++++++ docs/about/concepts/index.md | 7 + docs/get-started/setup-installation.md | 3 +- docs/index.md | 13 +- docs/reference/configuration.md | 202 +++++++++++++++++++++++++ docs/reference/index.md | 9 ++ docs/troubleshooting/configuration.md | 103 +++++++++++++ docs/troubleshooting/index.md | 11 ++ 8 files changed, 405 insertions(+), 3 deletions(-) create mode 100644 docs/about/concepts/configuration.md create mode 100644 docs/reference/configuration.md create mode 100644 docs/reference/index.md create mode 100644 docs/troubleshooting/configuration.md create mode 100644 docs/troubleshooting/index.md diff --git a/docs/about/concepts/configuration.md b/docs/about/concepts/configuration.md new file mode 100644 index 000000000..c4c72620b --- /dev/null +++ b/docs/about/concepts/configuration.md @@ -0,0 +1,60 @@ +(configuration-concepts)= + +# Configuration System + +NeMo Gym uses YAML configuration files to define which servers to run. Understanding the config structure helps you avoid common confusion when setting up training environments. + +## The Three-Level Pattern + +Every server definition follows this pattern: + +```yaml +server_id: # What YOU call it (your choice) + server_type: # What KIND of server (matches a folder) + implementation: # Which CODE to run (matches a subfolder) + entrypoint: ... +``` + +**Why three levels?** Each level serves a different purpose: + +| Level | You Control | Must Match | +|-------|-------------|------------| +| **Server ID** | ✅ Yes - name it anything | Nothing - it's your identifier | +| **Server Type** | ❌ Pick from 3 options | `responses_api_models`, `resources_servers`, or `responses_api_agents` | +| **Implementation** | ❌ Pick existing implementation | A folder inside that server type | + +### Understanding the Naming Pattern + +In many examples, you'll see the same name appear twice: + +```yaml +example_simple_weather: # ← Server ID + resources_servers: + example_simple_weather: # ← Implementation +``` + +These serve different purposes: + +- **Server ID** (`example_simple_weather` on line 1): Your chosen identifier for this server instance. Used in API requests and when other servers reference it. You could name it `my_weather` or `weather_prod` instead. + +- **Implementation** (`example_simple_weather` on line 3): Must match the folder `resources_servers/example_simple_weather/`. This tells NeMo Gym which code to run. + +Examples often use matching names for simplicity, but the two values are independent choices. + +## Policy Model Variables + +NeMo Gym provides three standard placeholders for "the model being trained": + +- `policy_base_url` - Model API endpoint +- `policy_api_key` - Authentication key +- `policy_model_name` - Model identifier + +These let you reference the training model consistently across configs without hardcoding values. Define them once in `env.yaml`, then use `${variable_name}` syntax anywhere you need them. + +--- + +:::{seealso} +- {doc}`/reference/configuration` for complete syntax and field specifications +- {doc}`/troubleshooting/configuration` for common errors +::: + diff --git a/docs/about/concepts/index.md b/docs/about/concepts/index.md index 088ae11c0..964d4beaf 100644 --- a/docs/about/concepts/index.md +++ b/docs/about/concepts/index.md @@ -38,6 +38,12 @@ Explore how resource servers score agent outputs with `verify()` implementations Essential vocabulary for agent training, RL workflows, and NeMo Gym. This glossary defines terms you'll encounter throughout the tutorials and documentation. ::: +:::{grid-item-card} {octicon}`gear;1.5em;sd-mr-1` Configuration System +:link: configuration-concepts +:link-type: ref +Understand the three-level config pattern and why server IDs and implementations are independent choices. +::: + :::: --- @@ -49,4 +55,5 @@ Essential vocabulary for agent training, RL workflows, and NeMo Gym. This glossa Core Abstractions Task Verification Key Terminology +Configuration System ``` diff --git a/docs/get-started/setup-installation.md b/docs/get-started/setup-installation.md index 41a554935..c457ef07a 100644 --- a/docs/get-started/setup-installation.md +++ b/docs/get-started/setup-installation.md @@ -123,6 +123,8 @@ Replace `sk-your-actual-openai-api-key-here` with your real OpenAI API key. This - The model must support function calling (most GPT-4 models do) - Refer to [OpenAI's models documentation](https://platform.openai.com/docs/models) 🔗 for available models +Refer to {doc}`/reference/configuration` for additional `env.yaml` options. + **Why GPT-4.1?** We use GPT-4.1 for getting started because it provides low latency (no reasoning step) and reliable function calling support out-of-the-box, letting you focus on learning NeMo Gym without configuration complexity. **Can I use my own model?** Yes! NeMo Gym works with any OpenAI-compatible inference server that supports function calling: @@ -130,7 +132,6 @@ Replace `sk-your-actual-openai-api-key-here` with your real OpenAI API key. This - **Other providers**: Any inference server that implements the OpenAI API specification Simply update `policy_base_url`, `policy_api_key`, and `policy_model_name` in your `env.yaml` to point to your chosen endpoint. - ::: :::{dropdown} Optional: Validate your API key before proceeding diff --git a/docs/index.md b/docs/index.md index 97e7d9be7..129212785 100644 --- a/docs/index.md +++ b/docs/index.md @@ -114,7 +114,6 @@ Home about/index.md Concepts Ecosystem -Configuration System ``` ```{toctree} @@ -142,8 +141,18 @@ how-to-faq.md ```{toctree} -:caption: Development +:caption: Reference :hidden: +:maxdepth: 1 +Configuration apidocs/index.rst ``` + +```{toctree} +:caption: Troubleshooting +:hidden: +:maxdepth: 1 + +troubleshooting/configuration.md +``` diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md new file mode 100644 index 000000000..165fe3943 --- /dev/null +++ b/docs/reference/configuration.md @@ -0,0 +1,202 @@ +(configuration-reference)= + +# Configuration Reference + +Complete syntax and options for NeMo Gym configuration. + +:::{seealso} +{doc}`/about/concepts/configuration` for understanding the three-level config pattern. +::: + +## File Locations + +| File | Location | Version Control | +|------|----------|-----------------| +| Server configs | `//configs/*.yaml` | ✅ Committed | +| env.yaml | Repository root (`./env.yaml`) | ❌ Gitignored (user creates) | + +**Example paths**: + +```text +resources_servers/example_simple_weather/configs/simple_weather.yaml +responses_api_models/openai_model/configs/openai_model.yaml +responses_api_agents/simple_agent/configs/simple_agent.yaml +./env.yaml ← you create this +``` + +--- + +## Config File Structure + +Each config file defines one or more server instances using three-level nesting: + +```yaml +server_id: # Level 1: Your chosen name (used in API requests) + server_type: # Level 2: resources_servers | responses_api_models | responses_api_agents + implementation: # Level 3: Must match a folder inside the server type directory + entrypoint: app.py # Required: Python file to run + # Additional fields vary by server type... +``` + +**Example**: + +```yaml +my_agent: # Server ID (you choose this) + responses_api_agents: # Server type + simple_agent: # Implementation (must match folder: responses_api_agents/simple_agent/) + entrypoint: app.py + # ... +``` + +:::{tip} +The server ID and implementation name are independent. Use descriptive server IDs that reflect your use case (for example, `math_tutor_agent`), while the implementation name must match an existing folder. +::: + +### Server Types + +| Type | Purpose | Required Fields | +|------|---------|-----------------| +| `responses_api_models` | LLM inference endpoints | `entrypoint` | +| `resources_servers` | Tools and verification logic | `entrypoint`, `domain` | +| `responses_api_agents` | Orchestration between models and resources | `entrypoint`, `resources_server`, `model_server` | + +### Resources Server Fields + +```yaml +my_resource: + resources_servers: + my_implementation: + entrypoint: app.py + domain: math # Required (see domain values below) + verified: false # Optional: marks server as production-ready + description: "Short description" # Optional +``` + +**Domain values**: `math`, `coding`, `agent`, `knowledge`, `instruction_following`, `long_context`, `safety`, `games`, `e2e`, `other` + +### Agent Server Fields + +Agent servers reference other servers using the `type` and `name` pattern: + +```yaml +my_agent: + responses_api_agents: + simple_agent: + entrypoint: app.py + resources_server: + type: resources_servers + name: my_resource # Must match a loaded server ID + model_server: + type: responses_api_models + name: policy_model # Must match a loaded server ID + datasets: # Optional: define associated datasets + - name: example + type: example # example | train | validation + jsonl_fpath: path/to/data.jsonl +``` + +### Model Server Fields + +```yaml +policy_model: + responses_api_models: + openai_model: + entrypoint: app.py + openai_base_url: ${policy_base_url} + openai_api_key: ${policy_api_key} + openai_model: ${policy_model_name} +``` + +--- + +## env.yaml Options + +Store secrets and environment-specific values that should not be committed to version control. + +```yaml +# Required for OpenAI-compatible models +policy_base_url: https://api.openai.com/v1 +policy_api_key: sk-your-api-key +policy_model_name: gpt-4o-2024-11-20 + +# Optional: store config paths for reuse +my_config_paths: + - responses_api_models/openai_model/configs/openai_model.yaml + - resources_servers/example_simple_weather/configs/simple_weather.yaml + +# Optional: validation behavior +error_on_almost_servers: true # Default: true (exit on invalid configs) +``` + +--- + +## Command Line Overrides + +Override any configuration value at runtime using Hydra syntax. Command line arguments have the highest priority. + +```bash +# Load config files +ng_run "+config_paths=[config1.yaml,config2.yaml]" + +# Override nested values (use dot notation after server ID) +ng_run "+config_paths=[config.yaml]" \ + +my_server.resources_servers.my_impl.domain=coding + +# Override model selection +ng_run "+config_paths=[config.yaml]" \ + +policy_model_name=gpt-4o-mini + +# Use paths stored in env.yaml +ng_run '+config_paths=${my_config_paths}' + +# Disable strict validation +ng_run "+config_paths=[config.yaml]" +error_on_almost_servers=false +``` + +--- + +## Policy Model Variables + +Standard placeholders resolved from `env.yaml` for consistent model references across configurations. + +| Variable | Description | Example | +|----------|-------------|---------| +| `policy_base_url` | Model API endpoint | `https://api.openai.com/v1` | +| `policy_api_key` | Authentication key | `sk-...` | +| `policy_model_name` | Model identifier | `gpt-4o-2024-11-20` | + +Reference in config files using `${variable_name}`: + +```yaml +openai_base_url: ${policy_base_url} +openai_api_key: ${policy_api_key} +openai_model: ${policy_model_name} +``` + +--- + +## Dataset Configuration + +Define datasets associated with servers for training data collection. + +```yaml +datasets: + - name: my_dataset + type: example # example | train | validation + jsonl_fpath: path/to/data.jsonl + num_repeats: 1 # Optional: repeat dataset (default: 1) + license: "Apache 2.0" # Required for train/validation types +``` + +**License values**: `Apache 2.0`, `MIT`, `Creative Commons Attribution 4.0 International`, `Creative Commons Attribution-ShareAlike 4.0 International`, `TBD` + +**Dataset types**: +- `example`: For testing and development +- `train`: Requires `license` and GitLab identifier +- `validation`: Requires `license` and GitLab identifier + +--- + +:::{seealso} +{doc}`/troubleshooting/configuration` for common configuration errors and solutions. +::: diff --git a/docs/reference/index.md b/docs/reference/index.md new file mode 100644 index 000000000..bb9ce4ae3 --- /dev/null +++ b/docs/reference/index.md @@ -0,0 +1,9 @@ +--- +orphan: true +--- + +(reference)= + +# Reference + +Quick lookup for configuration syntax, options, and specifications. diff --git a/docs/troubleshooting/configuration.md b/docs/troubleshooting/configuration.md new file mode 100644 index 000000000..a443b784c --- /dev/null +++ b/docs/troubleshooting/configuration.md @@ -0,0 +1,103 @@ +(troubleshooting-configuration)= + +# Configuration Errors + +These errors appear when running `ng_run` or `ng_collect_rollouts`. NeMo Gym validates configuration at startup before launching servers. + +:::{seealso} +{doc}`/reference/configuration` for complete configuration syntax and options. +::: + +--- + +## Startup Errors + +Errors that prevent servers from starting. + +### Missing Mandatory Value + +``` +omegaconf.errors.MissingMandatoryValue: Missing mandatory value: policy_api_key +``` + +**When**: A required variable (usually in `env.yaml`) is not defined. + +**Fix**: Add the missing value to `env.yaml`: + +```yaml +policy_api_key: sk-your-api-key +``` + +Or pass via command line: + +```bash +ng_run "+config_paths=[config.yaml]" +policy_api_key=sk-your-api-key +``` + +### Server Reference Not Found + +``` +AssertionError: Could not find ResourcesServerRef(type='resources_servers', name='typo_weather') +in the list of available servers: [ResourcesServerRef(...), ...] +``` + +**When**: A server config references another server that doesn't exist or isn't loaded. + +**Common causes**: +- Typo in the server name +- Referenced server's config file not included in `+config_paths` +- Server defined in a different config file that wasn't loaded + +**Fix**: +1. Check server name spelling in your config +2. Ensure all required config files are in `+config_paths`: + +```bash +ng_run "+config_paths=[model.yaml,resource.yaml,agent.yaml]" +``` + +--- + +## Validation Errors + +Errors where config structure is correct but values are invalid. + +### Almost-Servers Detected + +``` +Configuration Warnings: Almost-Servers Detected + + Almost-Server Detected: 'example_simple_agent' + This server configuration failed validation: + +- ResourcesServerInstanceConfig -> resources_servers -> example_server -> domain: + Input should be 'math', 'coding', 'agent', 'knowledge', 'instruction_following', + 'long_context', 'safety', 'games', 'e2e' or 'other' +``` + +**When**: Config has the right structure (server type, entrypoint) but contains invalid field values. + +**Common causes**: +- Invalid `domain` value for resources servers +- Invalid `license` value in dataset configs +- Missing required fields for the server type + +**Fix**: Check the validation error path (for example, `resources_servers -> example_server -> domain`) and update the field with a valid value. Refer to {ref}`configuration-reference` for valid field values. + +#### Bypass Strict Validation + +To continue with invalid configs (invalid servers will be skipped): + +```yaml +# In env.yaml +error_on_almost_servers: false +``` + +```bash +# Or via command line +ng_run "+config_paths=[config.yaml]" +error_on_almost_servers=false +``` + +:::{warning} +Bypassing validation means invalid servers won't start. Use this only for debugging, not production. +::: diff --git a/docs/troubleshooting/index.md b/docs/troubleshooting/index.md new file mode 100644 index 000000000..8253e9ea0 --- /dev/null +++ b/docs/troubleshooting/index.md @@ -0,0 +1,11 @@ +--- +orphan: true +--- + +(troubleshooting)= + +# Troubleshooting + +Solutions for common errors and issues. + + From d056e379511477cf1e2f7627b3c8d197a2bf922b Mon Sep 17 00:00:00 2001 From: Chris Wing Date: Wed, 3 Dec 2025 17:54:27 -0800 Subject: [PATCH 3/7] Further improvements to configuration documentation (#429) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Improves the configuration documentation by restructuring both the concepts and reference pages for clarity and reducing redundancy. ## Changes ### Concepts Page (`docs/about/concepts/configuration.md`) - Add "How Servers Connect" section explaining Agent-Model-Resources relationship - Add "Config File Locations" section with directory tree visualization - Rename "Three-Level Pattern" to "Server Block Structure" - Improve inline YAML comments for clarity - Remove "Policy Model Variables" section (moved to reference page) - Reframe intro to be benefit-oriented ### Reference Page (`docs/reference/configuration.md`) - Reorganize into: Server Configuration → Local Configuration → Command Line Usage → Troubleshooting - Add detailed inline comments for all server type YAML examples - Move Dataset Configuration under Agent Server as subsection (where it's used) - Add links to API docs for Domain enum and DatasetConfig.license - Merge Policy Model Variables into env.yaml section (reduce redundancy) - Rename "Environment Configuration" to "Local Configuration" (avoid confusion with training environment) ## Why - Concepts page now focuses on mental model; reference page has implementation details - Reduced duplication between pages - Clearer structure for users looking up specific config fields - Better inline documentation in YAML examples Addresses https://github.com/NVIDIA-NeMo/Gym/issues/394 --------- Signed-off-by: Chris Wing --- docs/about/concepts/configuration.md | 61 ++++---- docs/reference/configuration.md | 202 ++++++++++++--------------- 2 files changed, 120 insertions(+), 143 deletions(-) diff --git a/docs/about/concepts/configuration.md b/docs/about/concepts/configuration.md index c4c72620b..46f3c4086 100644 --- a/docs/about/concepts/configuration.md +++ b/docs/about/concepts/configuration.md @@ -2,30 +2,47 @@ # Configuration System -NeMo Gym uses YAML configuration files to define which servers to run. Understanding the config structure helps you avoid common confusion when setting up training environments. +NeMo Gym uses YAML configuration files to define [Model, Resources, and Agent servers](./core-abstractions). Each server gets its own configuration block, providing modular control over the entire training environment. -## The Three-Level Pattern +## How Servers Connect -Every server definition follows this pattern: +A training environment typically includes all three server types working together. The Agent server config specifies which Model and Resources servers to use by referencing their server IDs. This wiring is what ties each training environment together — the Agent knows which Model to call and which Resources to use. -```yaml -server_id: # What YOU call it (your choice) - server_type: # What KIND of server (matches a folder) - implementation: # Which CODE to run (matches a subfolder) - entrypoint: ... +## Config File Locations + +Each server type has a dedicated directory with its implementations and their configs: + +```text +# Model Server Config +responses_api_models/ + └── openai_model/ + └── configs/openai_model.yaml + +# Resources Server Config +resources_servers/ + └── example_simple_weather/ + └── configs/simple_weather.yaml + +# Agent Server Config +responses_api_agents/ + └── simple_agent/ + └── configs/simple_agent.yaml ``` -**Why three levels?** Each level serves a different purpose: +## Server Block Structure -| Level | You Control | Must Match | -|-------|-------------|------------| -| **Server ID** | ✅ Yes - name it anything | Nothing - it's your identifier | -| **Server Type** | ❌ Pick from 3 options | `responses_api_models`, `resources_servers`, or `responses_api_agents` | -| **Implementation** | ❌ Pick existing implementation | A folder inside that server type | +Each config file defines a server using this structure: +```yaml +server_id: # Your unique name for this server + server_type: # responses_api_models | resources_servers | responses_api_agents + implementation: # Directory name inside the server type directory + entrypoint: app.py # Python file to run + # ... additional fields vary by server type +``` -### Understanding the Naming Pattern +Different server types have additional required fields (e.g., `domain` for resources servers, `resources_server` and `model_server` for agents). See {doc}`/reference/configuration` for complete field specifications. -In many examples, you'll see the same name appear twice: +In many config files in NeMo Gym, you'll see the same name used for both server ID and implementation: ```yaml example_simple_weather: # ← Server ID @@ -41,20 +58,10 @@ These serve different purposes: Examples often use matching names for simplicity, but the two values are independent choices. -## Policy Model Variables - -NeMo Gym provides three standard placeholders for "the model being trained": - -- `policy_base_url` - Model API endpoint -- `policy_api_key` - Authentication key -- `policy_model_name` - Model identifier - -These let you reference the training model consistently across configs without hardcoding values. Define them once in `env.yaml`, then use `${variable_name}` syntax anywhere you need them. - --- :::{seealso} - {doc}`/reference/configuration` for complete syntax and field specifications -- {doc}`/troubleshooting/configuration` for common errors +- {doc}`/troubleshooting/configuration` for troubleshooting configuration related errors ::: diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 165fe3943..b47b45eaf 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -2,10 +2,10 @@ # Configuration Reference -Complete syntax and options for NeMo Gym configuration. +Complete syntax and field specifications for NeMo Gym configuration files. :::{seealso} -{doc}`/about/concepts/configuration` for understanding the three-level config pattern. +{doc}`/about/concepts/configuration` for understanding how configs are structured and how servers connect. ::: ## File Locations @@ -15,106 +15,110 @@ Complete syntax and options for NeMo Gym configuration. | Server configs | `//configs/*.yaml` | ✅ Committed | | env.yaml | Repository root (`./env.yaml`) | ❌ Gitignored (user creates) | -**Example paths**: - -```text -resources_servers/example_simple_weather/configs/simple_weather.yaml -responses_api_models/openai_model/configs/openai_model.yaml -responses_api_agents/simple_agent/configs/simple_agent.yaml -./env.yaml ← you create this -``` - --- -## Config File Structure +## Server Configuration -Each config file defines one or more server instances using three-level nesting: +All servers share this structure: ```yaml -server_id: # Level 1: Your chosen name (used in API requests) - server_type: # Level 2: resources_servers | responses_api_models | responses_api_agents - implementation: # Level 3: Must match a folder inside the server type directory - entrypoint: app.py # Required: Python file to run - # Additional fields vary by server type... +server_id: # Your unique name for this server + server_type: # responses_api_models | resources_servers | responses_api_agents + implementation: # Directory name inside the server type directory + entrypoint: app.py # Python file to run + # ... additional fields vary by server type ``` -**Example**: +### Model Server Fields ```yaml -my_agent: # Server ID (you choose this) - responses_api_agents: # Server type - simple_agent: # Implementation (must match folder: responses_api_agents/simple_agent/) - entrypoint: app.py - # ... +policy_model: # Server ID (use "policy_model" — agent configs expect this name) + responses_api_models: # Server type (must be "responses_api_models" for model servers) + openai_model: # Implementation (use "openai_model", "vllm_model", or "azure_openai_model") + entrypoint: app.py # Python file to run + openai_base_url: ${policy_base_url} # API endpoint URL + openai_api_key: ${policy_api_key} # Authentication key + openai_model: ${policy_model_name} # Model identifier ``` :::{tip} -The server ID and implementation name are independent. Use descriptive server IDs that reflect your use case (for example, `math_tutor_agent`), while the implementation name must match an existing folder. +Keep the server ID as `policy_model` — agent configs reference this name by default. The `${policy_base_url}`, `${policy_api_key}`, and `${policy_model_name}` placeholders should be defined in `env.yaml` at the repository root, allowing you to change model settings in one place. ::: -### Server Types - -| Type | Purpose | Required Fields | -|------|---------|-----------------| -| `responses_api_models` | LLM inference endpoints | `entrypoint` | -| `resources_servers` | Tools and verification logic | `entrypoint`, `domain` | -| `responses_api_agents` | Orchestration between models and resources | `entrypoint`, `resources_server`, `model_server` | - ### Resources Server Fields ```yaml -my_resource: - resources_servers: - my_implementation: - entrypoint: app.py - domain: math # Required (see domain values below) - verified: false # Optional: marks server as production-ready - description: "Short description" # Optional +my_resource: # Server ID (your choice — agents reference this name) + resources_servers: # Server type (must be "resources_servers" for resources servers) + example_simple_weather: # Implementation (must match a directory in resources_servers/) + entrypoint: app.py # Python file to run + domain: agent # Server category (see values below) + verified: false # Passed reward profiling and training checks (default: false) + description: "Short description" # Server description + value: "What this improves" # Training value provided ``` -**Domain values**: `math`, `coding`, `agent`, `knowledge`, `instruction_following`, `long_context`, `safety`, `games`, `e2e`, `other` +**Domain values:** `math`, `coding`, `agent`, `knowledge`, `instruction_following`, `long_context`, `safety`, `games`, `e2e`, `other` (see {py:class}`~nemo_gym.config_types.Domain`) ### Agent Server Fields -Agent servers reference other servers using the `type` and `name` pattern: +Agent servers must include both a `resources_server` and `model_server` block to specify which servers to use. ```yaml -my_agent: - responses_api_agents: - simple_agent: - entrypoint: app.py - resources_server: - type: resources_servers - name: my_resource # Must match a loaded server ID - model_server: - type: responses_api_models - name: policy_model # Must match a loaded server ID - datasets: # Optional: define associated datasets - - name: example - type: example # example | train | validation - jsonl_fpath: path/to/data.jsonl +my_agent: # Server ID (your choice — used in API requests) + responses_api_agents: # Server type (must be "responses_api_agents" for agent servers) + simple_agent: # Implementation (must match a directory in responses_api_agents/) + entrypoint: app.py # Python file to run + resources_server: # Specifies which resources server to use + type: resources_servers # Always "resources_servers" + name: my_resource # Server ID of the resources server + model_server: # Specifies which model server to use + type: responses_api_models # Always "responses_api_models" + name: policy_model # Server ID of the model server + datasets: # Optional: define for training workflows + - name: train # Dataset identifier + type: train # example | train | validation + jsonl_fpath: path/to/data.jsonl # Path to data file + license: Apache 2.0 # Required for train/validation ``` -### Model Server Fields +#### Dataset Configuration + +Define datasets associated with agent servers for training and evaluation. ```yaml -policy_model: - responses_api_models: - openai_model: - entrypoint: app.py - openai_base_url: ${policy_base_url} - openai_api_key: ${policy_api_key} - openai_model: ${policy_model_name} +datasets: + - name: my_dataset + type: train + jsonl_fpath: path/to/data.jsonl + license: Apache 2.0 + num_repeats: 1 ``` +| Field | Required | Description | +|-------|----------|-------------| +| `name` | Yes | Dataset identifier | +| `type` | Yes | `example`, `train`, or `validation` | +| `jsonl_fpath` | Yes | Path to data file | +| `license` | For train/validation | License identifier (see values below) | +| `num_repeats` | No | Repeat dataset n times (default: `1`) | + +**Dataset types:** +- `example` — For testing and development +- `train` — Training data (requires `license`) +- `validation` — Evaluation data (requires `license`) + +**License values:** `Apache 2.0`, `MIT`, `Creative Commons Attribution 4.0 International`, `Creative Commons Attribution-ShareAlike 4.0 International`, `TBD` (see {py:attr}`~nemo_gym.config_types.DatasetConfig.license`) + --- -## env.yaml Options +## Local Configuration (env.yaml) -Store secrets and environment-specific values that should not be committed to version control. +Store secrets and local settings at the repository root. This file is gitignored. ```yaml -# Required for OpenAI-compatible models +# Policy model (required for most setups) +# Reference these variables in server configs using `${variable_name}` syntax (e.g., `${policy_base_url}`) policy_base_url: https://api.openai.com/v1 policy_api_key: sk-your-api-key policy_model_name: gpt-4o-2024-11-20 @@ -125,77 +129,43 @@ my_config_paths: - resources_servers/example_simple_weather/configs/simple_weather.yaml # Optional: validation behavior -error_on_almost_servers: true # Default: true (exit on invalid configs) +error_on_almost_servers: true # Exit on invalid configs (default: true) ``` --- -## Command Line Overrides +## Command Line Usage -Override any configuration value at runtime using Hydra syntax. Command line arguments have the highest priority. +Run servers using `ng_run` with Hydra syntax for configuration. + +### Loading Configs ```bash -# Load config files +# Load one or more config files ng_run "+config_paths=[config1.yaml,config2.yaml]" +# Use paths stored in env.yaml +ng_run '+config_paths=${my_config_paths}' +``` + +### Overriding Values + +```bash # Override nested values (use dot notation after server ID) ng_run "+config_paths=[config.yaml]" \ +my_server.resources_servers.my_impl.domain=coding -# Override model selection +# Override policy model ng_run "+config_paths=[config.yaml]" \ +policy_model_name=gpt-4o-mini -# Use paths stored in env.yaml -ng_run '+config_paths=${my_config_paths}' - # Disable strict validation ng_run "+config_paths=[config.yaml]" +error_on_almost_servers=false ``` --- -## Policy Model Variables - -Standard placeholders resolved from `env.yaml` for consistent model references across configurations. - -| Variable | Description | Example | -|----------|-------------|---------| -| `policy_base_url` | Model API endpoint | `https://api.openai.com/v1` | -| `policy_api_key` | Authentication key | `sk-...` | -| `policy_model_name` | Model identifier | `gpt-4o-2024-11-20` | - -Reference in config files using `${variable_name}`: - -```yaml -openai_base_url: ${policy_base_url} -openai_api_key: ${policy_api_key} -openai_model: ${policy_model_name} -``` - ---- - -## Dataset Configuration - -Define datasets associated with servers for training data collection. - -```yaml -datasets: - - name: my_dataset - type: example # example | train | validation - jsonl_fpath: path/to/data.jsonl - num_repeats: 1 # Optional: repeat dataset (default: 1) - license: "Apache 2.0" # Required for train/validation types -``` - -**License values**: `Apache 2.0`, `MIT`, `Creative Commons Attribution 4.0 International`, `Creative Commons Attribution-ShareAlike 4.0 International`, `TBD` - -**Dataset types**: -- `example`: For testing and development -- `train`: Requires `license` and GitLab identifier -- `validation`: Requires `license` and GitLab identifier - ---- +## Troubleshooting :::{seealso} {doc}`/troubleshooting/configuration` for common configuration errors and solutions. From 391fa15ba25377a5925f9e9d6e53b1c8a70da789 Mon Sep 17 00:00:00 2001 From: Ahmad Kiswani Date: Thu, 4 Dec 2025 09:08:30 -0800 Subject: [PATCH 4/7] Minor fixes minor fixes and imprvoments after feedback from @heatherlxd in https://github.com/NVIDIA-NeMo/Gym/pull/429 Signed-off-by: Ahmad Kiswani --- docs/about/concepts/configuration.md | 4 ++-- docs/reference/configuration.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/about/concepts/configuration.md b/docs/about/concepts/configuration.md index 46f3c4086..216691bbf 100644 --- a/docs/about/concepts/configuration.md +++ b/docs/about/concepts/configuration.md @@ -6,7 +6,7 @@ NeMo Gym uses YAML configuration files to define [Model, Resources, and Agent se ## How Servers Connect -A training environment typically includes all three server types working together. The Agent server config specifies which Model and Resources servers to use by referencing their server IDs. This wiring is what ties each training environment together — the Agent knows which Model to call and which Resources to use. +A training environment typically includes all three server types working together. The Agent server config specifies which Model and Resources servers to use by referencing their server IDs. These references connect each training environment together — the Agent knows which Model to call and which Resources to use. ## Config File Locations @@ -42,7 +42,7 @@ server_id: # Your unique name for this server Different server types have additional required fields (e.g., `domain` for resources servers, `resources_server` and `model_server` for agents). See {doc}`/reference/configuration` for complete field specifications. -In many config files in NeMo Gym, you'll see the same name used for both server ID and implementation: +Config files in NeMo Gym often use the same name for both server ID and implementation: ```yaml example_simple_weather: # ← Server ID diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index b47b45eaf..f48155c72 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -136,7 +136,7 @@ error_on_almost_servers: true # Exit on invalid configs (default: true) ## Command Line Usage -Run servers using `ng_run` with Hydra syntax for configuration. +Run servers using `ng_run`, NeMo Gym uses [Hydra](https://hydra.cc/) for configuration management." ### Loading Configs @@ -145,7 +145,7 @@ Run servers using `ng_run` with Hydra syntax for configuration. ng_run "+config_paths=[config1.yaml,config2.yaml]" # Use paths stored in env.yaml -ng_run '+config_paths=${my_config_paths}' +ng_run "+config_paths=${my_config_paths}" ``` ### Overriding Values From d611bf4751b613261122272f2ebe5af97115b0d8 Mon Sep 17 00:00:00 2001 From: Brian Yu Date: Wed, 10 Dec 2025 10:06:17 -0800 Subject: [PATCH 5/7] fix ref Signed-off-by: Brian Yu --- docs/about/concepts/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/about/concepts/configuration.md b/docs/about/concepts/configuration.md index 216691bbf..85ce55b32 100644 --- a/docs/about/concepts/configuration.md +++ b/docs/about/concepts/configuration.md @@ -2,7 +2,7 @@ # Configuration System -NeMo Gym uses YAML configuration files to define [Model, Resources, and Agent servers](./core-abstractions). Each server gets its own configuration block, providing modular control over the entire training environment. +NeMo Gym uses YAML configuration files to define [Model, Resources, and Agent servers](./core-components.md). Each server gets its own configuration block, providing modular control over the entire training environment. ## How Servers Connect From 7bc6a465419c745fb3a67a974ae0089d6b9fba50 Mon Sep 17 00:00:00 2001 From: Brian Yu Date: Wed, 10 Dec 2025 10:06:54 -0800 Subject: [PATCH 6/7] remove duplicate file Signed-off-by: Brian Yu --- docs/about/concepts/index.md | 1 - docs/about/configuration-system.md | 210 ----------------------------- 2 files changed, 211 deletions(-) delete mode 100644 docs/about/configuration-system.md diff --git a/docs/about/concepts/index.md b/docs/about/concepts/index.md index 6b2993b83..9e371b49d 100644 --- a/docs/about/concepts/index.md +++ b/docs/about/concepts/index.md @@ -53,7 +53,6 @@ Understand the three-level config pattern and why server IDs and implementations :maxdepth: 1 Core Components -Configuration System Task Verification Key Terminology Configuration System diff --git a/docs/about/configuration-system.md b/docs/about/configuration-system.md deleted file mode 100644 index a37816441..000000000 --- a/docs/about/configuration-system.md +++ /dev/null @@ -1,210 +0,0 @@ -(configuration-management)= - -# Configuration Management - -NeMo Gym uses a powerful configuration system with three sources that are resolved in this order: - -``` -Server YAML Config Files < env.yaml < Command Line Arguments - (lowest priority) (highest priority) -``` - -This allows for: -- Base configuration in YAML files (shared settings) -- Secrets and environment-specific values in `env.yaml` -- Runtime overrides via command line arguments - -::::{tab-set} - -:::{tab-item} 1. Server YAML Config Files - -These base configurations define server structures and default values, with later files overriding earlier ones. - -Example: Multi-Server Configuration -```bash -# Define which config files to load -config_paths="responses_api_models/openai_model/configs/openai_model.yaml,\ -resources_servers/example_simple_weather/configs/simple_weather.yaml,\ -responses_api_agents/simple_agent/configs/simple_agent.yaml" - -ng_run "+config_paths=[${config_paths}]" -``` - -Every config file defines **server instances** with this hierarchy: - -```yaml -# Server ID - unique name used in requests and references -simple_weather_simple_agent: - # Server type - must be one of: responses_api_models, resources_servers, responses_api_agents - # These match the 3 top-level folders in NeMo-Gym - responses_api_agents: - # Implementation type - must match a folder name inside responses_api_agents/ - simple_agent: - # Entrypoint - Python file to run (relative to implementation folder) - entrypoint: app.py - # Server-specific configuration (varies by implementation) - resources_server: - type: resources_servers # What type of server to reference - name: simple_weather # Which specific server instance - model_server: - type: responses_api_models - name: policy_model # References the model server -``` - -::: -:::{tab-item} 2. env.yaml - -Your `env.yaml` file contains **secrets and environment-specific values** that should never be committed to version control. - -### Basic env.yaml - -```yaml -# API credentials (never commit these!) -policy_base_url: https://api.openai.com/v1 -policy_api_key: sk-your-actual-api-key-here -policy_model_name: gpt-4o-2024-11-20 -``` - -### Advanced env.yaml with Config Paths - -```yaml -# Store complex config paths for convenience -simple_weather_config_paths: - - responses_api_models/openai_model/configs/openai_model.yaml - - resources_servers/example_simple_weather/configs/simple_weather.yaml - -# Different environments -dev_model_name: gpt-4o-mini -prod_model_name: gpt-4o-2024-11-20 - -# Custom server settings -custom_host: 0.0.0.0 -custom_port: 8080 -``` - -**Usage with stored config paths**: -```bash -ng_run '+config_paths=${simple_weather_config_paths}' -``` - -::: - -:::{tab-item} 3. Command Line Arguments - -**Runtime overrides** using Hydra syntax for maximum flexibility. These command line arguments have the highest priority and can override any settings from config.yaml or env.yaml files. - -Basic Overrides -```bash -# Override a specific model -ng_run "+config_paths=[config.yaml]" \ - +policy_model.responses_api_models.openai_model.openai_model=gpt-4o-mini - -# Point agent to different resource server -ng_run "+config_paths=[config.yaml]" \ - +simple_agent.responses_api_agents.simple_agent.resources_server.name=different_weather -``` - -Advanced Overrides -```bash -# Multiple overrides for testing -ng_run "+config_paths=[${config_paths}]" \ - +policy_model_name=gpt-4o-mini \ - +simple_weather.resources_servers.simple_weather.host=localhost \ - +simple_weather.resources_servers.simple_weather.port=9090 -``` - -::: -:::: - -## Special Policy Model Variables - -NeMo Gym provides standard placeholders for the {term}`Policy Model` being trained: - -```yaml -# These variables are available in any config file -policy_base_url: https://api.openai.com/v1 # Model API endpoint -policy_api_key: sk-your-key # Authentication -policy_model_name: gpt-4o-2024-11-20 # Model identifier -``` - -**Why these exist**: When training agents, you need consistent references to "the model being trained" across different resource servers and agents. - -**Usage in config files**: -```yaml -policy_model: - responses_api_models: - openai_model: - openai_base_url: ${policy_base_url} # Resolves from env.yaml - openai_api_key: ${policy_api_key} # Resolves from env.yaml - openai_model: ${policy_model_name} # Resolves from env.yaml -``` - - -## Troubleshooting - -NeMo Gym validates your configuration and provides helpful error messages. - -:::{dropdown} "Missing mandatory value" -``` -omegaconf.errors.MissingMandatoryValue: Missing mandatory value: policy_api_key -``` -**Fix**: Add the missing value to `env.yaml` or command line. -::: - -:::{dropdown} "Could not find X in the list of available servers" -``` -AssertionError: Could not find type='resources_servers' name='typo_weather' -in the list of available servers: [simple_weather, math_with_judge, ...] -``` -**Fix**: Check your server name spelling and ensure the config is loaded. -::: - -:::{dropdown} "Almost-Servers Detected" -Example: -```bash -═══════════════════════════════════════════════════ -Configuration Warnings: Almost-Servers Detected -═══════════════════════════════════════════════════ - - Almost-Server Detected: 'example_simple_agent' - This server configuration failed validation: - -- ResourcesServerInstanceConfig -> resources_servers -> example_server -> domain: Input should be 'math', 'coding', 'agent', 'knowledge', 'instruction_following', 'long_context', 'safety', 'games', 'e2e' or 'other' - - This server will NOT be started. -``` -**What this means**: Your server configuration has the correct structure (entrypoint, server type, etc.) but contains invalid values that prevent it from starting. - -**Common causes**: -- Invalid `license` enum values in datasets (must be one of the allowed options). - - see the `license` field in `DatasetConfig` in `config_types.py`. -- Missing or invalid `domain` field for resources servers (math, coding, agent, knowledge, etc.) - - see the `Domain` class in `config_types.py`. -- Malformed server references (wrong type or name) - -**Fix**: Update the configuration based on the validation errors shown. The warning will detail exactly which fields are problematic. - -#### Strict Validation Mode - -By default, invalid servers will throw an error. You can bypass strict validation and just show a warning: - -**In env.yaml:** -```yaml -error_on_almost_servers: false # Will not error on invalid config -``` - -**Via command line:** -```bash -ng_run "+config_paths=[config.yaml]" +error_on_almost_servers=false -``` - -**Default behavior** (`error_on_almost_servers=true`): -- All configuration issues are detected and warnings are printed -- NeMo Gym exits with an error, preventing servers from starting with invalid configs - -**When disabled** (`error_on_almost_servers=false`): -- All configuration issues are still detected and warnings are printed -- NeMo Gym continues execution despite the invalid configurations -- Invalid servers are skipped, and valid servers will attempt to start - -::: From f7f8ab1efe74eb82314c5b2f82125f27d4fc1552 Mon Sep 17 00:00:00 2001 From: Brian Yu Date: Wed, 10 Dec 2025 10:11:48 -0800 Subject: [PATCH 7/7] reorder Signed-off-by: Brian Yu --- docs/about/concepts/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/about/concepts/index.md b/docs/about/concepts/index.md index 9e371b49d..75f67af47 100644 --- a/docs/about/concepts/index.md +++ b/docs/about/concepts/index.md @@ -53,7 +53,7 @@ Understand the three-level config pattern and why server IDs and implementations :maxdepth: 1 Core Components +Configuration System Task Verification Key Terminology -Configuration System ```