docs: add agent harness reference and selection guide. - #2390
Closed
ffrujeri wants to merge 1 commit into
Closed
Conversation
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Contributor
Author
|
Closed and superseded by #2311 |
ananthsub
added a commit
to ananthsub/Gym
that referenced
this pull request
Aug 6, 2026
Terminology: say "Agent server" consistently for the NeMo Gym component, in prose, headings, and table headers. Correct how a wrapper is structured. The harness runs from the Agent server's responses(); run() is the episode orchestrator that seeds the session, calls responses(), and verifies. Every agent server follows this split (simple_agent, opencode_agent, claude_code_agent, codex_agent, pi_agent). Drop the built-in vs external framing. The agent servers that wrap a third-party harness are also in this repo, so that split was not a real distinction. Group by who implements the agent instead: a Gym-native implementation, or a third-party harness. Rename "Agent Frameworks" to "External Frameworks", since that table holds both agent frameworks (LangGraph, Stirrup) and environment frameworks (Aviary, Verifiers). Use "in" rather than "against" for the harness-to-environment relation. "Against" frames the agent as the sole driver, which does not hold if the environment orchestrates agents for multi-agent workloads. Add the material from the closed PR NVIDIA-NeMo#2390 that this page lacked: a selection guide, the resources_server/model_server and sandbox config shapes, the per-request policy endpoint override, the skills caveat, and the commands to start servers and collect rollouts. Rebased onto current main, which changes three facts documented earlier: - mini_swe_agent and mini_swe_agent_2 accept policy_base_url and policy_api_key on /run (NVIDIA-NeMo#2166). Other agent servers allow extra keys, so they accept the fields and ignore them; the page says so. - kilocode_agent now routes through a Gym model server (NVIDIA-NeMo#2319), so all six CLI agent servers take a model_server ref. - harbor_agent can execute trials in any NeMo Gym sandbox provider (NVIDIA-NeMo#2296), not only Singularity. Verified with `make docs-check` (fern check, 0 errors). Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
ananthsub
added a commit
to ananthsub/Gym
that referenced
this pull request
Sep 1, 2026
Terminology: say "Agent server" consistently for the NeMo Gym component, in prose, headings, and table headers. Correct how a wrapper is structured. The harness runs from the Agent server's responses(); run() is the episode orchestrator that seeds the session, calls responses(), and verifies. Every agent server follows this split (simple_agent, opencode_agent, claude_code_agent, codex_agent, pi_agent). Drop the built-in vs external framing. The agent servers that wrap a third-party harness are also in this repo, so that split was not a real distinction. Group by who implements the agent instead: a Gym-native implementation, or a third-party harness. Rename "Agent Frameworks" to "External Frameworks", since that table holds both agent frameworks (LangGraph, Stirrup) and environment frameworks (Aviary, Verifiers). Use "in" rather than "against" for the harness-to-environment relation. "Against" frames the agent as the sole driver, which does not hold if the environment orchestrates agents for multi-agent workloads. Add the material from the closed PR NVIDIA-NeMo#2390 that this page lacked: a selection guide, the resources_server/model_server and sandbox config shapes, the per-request policy endpoint override, the skills caveat, and the commands to start servers and collect rollouts. Rebased onto current main, which changes three facts documented earlier: - mini_swe_agent and mini_swe_agent_2 accept policy_base_url and policy_api_key on /run (NVIDIA-NeMo#2166). Other agent servers allow extra keys, so they accept the fields and ignore them; the page says so. - kilocode_agent now routes through a Gym model server (NVIDIA-NeMo#2319), so all six CLI agent servers take a model_server ref. - harbor_agent can execute trials in any NeMo Gym sandbox provider (NVIDIA-NeMo#2296), not only Singularity. Verified with `make docs-check` (fern check, 0 errors). Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #2372
Testing
cd fern && npm run check