hermes_agent: repin to upstream hermes and fix the wrapper for its API by j-nolan · Pull Request #2052 · NVIDIA-NeMo/Gym

j-nolan · 2026-07-16T13:18:12Z

Repins gym's hermes_agent from the cmunley1 fork to upstream NousResearch/hermes-agent (569b912, today's tip), drops the fork, and updates the wrapper for that build's AIAgent API. Raises gym's openai cap so the pin co-resolves.

Why

The current pin is cmunley1/hermes-agent@26bb847a, which is upstream be43bee1 plus a single one-line openai relax. Pointing at upstream directly removes the fork as a maintenance point. Upstream's AIAgent API has changed since April, so the wrapper needs updating to match.

Changes

hermes_agent/requirements.txt: pin NousResearch/hermes-agent@569b912 (was cmunley1/hermes agent@26bb847a).
pyproject.toml: openai cap raised <=2.7.2 to <=2.24.0, because upstream hermes pins openai==2.24.0. This is a gym-wide ceiling change; it was smoke-tested against the gym unit suite (1145 passed; the only failures are the unrelated opensandbox tests from a missing optional dep).
hermes_agent/app.py, updated for the 569b912 AIAgent API:
- use_streaming and persist_session constructor kwargs were removed upstream, so dropped (non-streaming and no persistence are now the default paths).
- insert_reasoning=True is now reasoning_config={"enabled": True}.
- temperature is now request_overrides={"temperature": ...}.
- The build streams on the wire even with no stream consumers, and gym's model server is non-streaming only (it rejects stream=true with HTTP 422). Fixed by setting agent._disable_streaming=True (the successor to use_streaming=False) plus forcing stream=False in the existing _build_api_kwargs patch.

Validation

The signature smoke test passes, but it only catches signature-level breaks. The streaming and grading issues above were only surfaced by a real run, so the acceptance gate is an actual Terminal Bench pass, not the unit test. Validated end to end at 62/89 resolved on Terminal Bench 2.1 with hermes 569b912 driving Claude Opus (verifier working, no constructor or stream errors).

copy-pr-bot · 2026-07-16T13:18:16Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

cmunley1 · 2026-07-16T20:11:10Z

the pin is there to support training, i think it should stay until we implement support for blackbox agent training. right now, training will break if you remove the pin

j-nolan · 2026-07-20T12:32:30Z

the pin is there to support training, i think it should stay until we implement support for blackbox agent training. right now, training will break if you remove the pin

@cmunley1 got it, thanks for the follow up. Do you have a rough idea of when the blackbox agent training will land? Currently, we are working on a fork of gym just to work around this, it's OK for now but hoping we can converge again one day

Glorf · 2026-07-21T08:05:14Z

When it comes to ETA for training, I guess weeks? @cmunley1 maybe @j-nolan could cherry-pick your changes into new hermes so that we can unblock him and work on the blackbox agents training in the meantime?

cmunley1 · 2026-07-21T08:15:04Z

cherry picking would be fine, it is primarly these changes plus i had to relax an openai version pin in the past which probably has changed too. https://github.com/NousResearch/hermes-agent/tree/nemo-gym-changes

cmunley1 · 2026-07-21T08:15:44Z

yeah, probably a few weeks for training as some other work is blocking

j-nolan · 2026-07-21T14:17:39Z

Thanks @Glorf and @cmunley1 ! I had a quick look at upstreaming the change to hermes. It seems a bit more involved than I hoped: hermes changed quite a bit since the original pin, especially agent_loop.py, and I'm not familiar with the RL bits that depend on it. If you're OK, I think we'll keep our own fork for now. Happy to tackle this later if our fork because unmanageable before the blackbox work is done. OK with you?

elisam0 · 2026-08-26T14:38:38Z

/claude review

cmunley1 · 2026-09-03T02:40:03Z

/ok to test 94c33a2

cmunley1 · 2026-09-03T02:54:23Z

/ok to test 862918c

cmunley1 · 2026-09-03T03:28:29Z

+        agent._disable_streaming = True
        _original_build_api_kwargs = agent._build_api_kwargs

        def _patched_build_api_kwargs(api_messages):


Hermes v2026.8.31 may call this method with tools_for_api=..., but the wrapper only accepts api_messages. This will raise TypeError: unexpected keyword argument 'tools_for_api' when that path is used. Should we preserve the upstream signature and forward tools_for_api to _original_build_api_kwargs?

Makes sense, thank you for finding it.

hermes-agent's conversation loop calls agent._build_api_kwargs(api_messages, tools_for_api=tools_for_api) whenever a call uses a scoped toolset (not the agent's full default set) — a live, common path, not an edge case. Since the patch replaces the method entirely with a function that only accepted api_messages, that call would raise: TypeError: unexpected keyword argument 'tools_for_api'. Confirmed both ways empirically: the old wrapper fails with exactly that error, the fixed one forwards the kwarg cleanly. Signed-off-by: Riccardo Roveri <rroveri@nvidia.com>

…r-branch

hermes-agent's own setup.py refuses to build as a wheel unless HERMES_NIX_BUILD=1 is set (it expects installation via its shell installer, Docker image, or Nix, not a plain pip/uv build). Scoped to exactly one server: dir_path.name == 'hermes_agent' is false for every other server, so this is a no-op — literally the same generated command as before — everywhere else. No cross-contamination between sibling server builds either, since each runs as its own subprocess with its own env dict copied fresh from the parent process. Verified live: ran the actual generated install command via run_command() against a fresh clone, no manual env var set — hermes-agent installed and imported successfully (previously failed with 'Building wheels or sdists for hermes-agent is not supported'). Signed-off-by: Riccardo Roveri <rroveri@nvidia.com>

rroverin · 2026-09-03T10:03:55Z

@cmunley1 Also fixed the Unit tests / Server suite CI failure (shard 4) — hermes-agent's build refuses to produce a wheel without HERMES_NIX_BUILD=1, so I added it to hermes_agent's own venv setup, scoped to just that server.

pyproject.toml intentionally pins openai==2.44.0 twice (once in [project].dependencies, once in override-dependencies, to force uv's resolver past hermes-agent's own openai==2.24.0 hard-pin). The grep that extracts OPENAI_PIN matched both occurrences, producing a two-line string that broke `pip install` with an "Invalid requirement" error and failed the prewarm-harness SLURM job (verified against gitlab-master.nvidia.com/interactive-agents/slurm-evaluations jobs 424346546 and 424441611). Pipe the extraction through `sort -u` so the duplicate matches collapse to the single distinct pin. Signed-off-by: Pascal Bérard <pberard@nvidia.com>

The model server forwards to engines that accept fields OpenAI does not define, and callers send them: vLLM's chat_template_kwargs drives a model's thinking mode, and clients label tool results with the tool's name. Forbidding extras rejects the request outright, so the caller loses the field and the conversation with it. Requests are inbound from callers Gym does not control, so carrying what is not modelled is the right default for a proxy; being strict about what Gym itself sends is unaffected. Message contents are still validated, so a malformed role or a broken tool call is still rejected. The strictness arrived with the openai pin in NVIDIA-NeMo#2456, whose issue asked only to support newer vllm versions. It was not the point of that change, and its effect is to reject the extension field vllm itself needs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

NVIDIA-NeMo#3061 added guards that the tool-call name normalization must not loosen the request model. Relaxing the schema loosens it deliberately, so those assertions now describe behaviour the model no longer has. Invert the two that assert rejection, and add one that pins down what did not change: a tool call missing its required function is still rejected, so carrying unknown keys has not turned off structural validation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

I, James Nolan <jnolan@nvidia.com>, hereby add my Signed-off-by to this commit: 382ce83 I, James Nolan <jnolan@nvidia.com>, hereby add my Signed-off-by to this commit: a24a3d4 Signed-off-by: James Nolan <jnolan@nvidia.com>

cmunley1 · 2026-09-04T17:43:31Z

+    # Inbound requests carry vendor extensions this schema does not enumerate, and the engines
+    # they are forwarded to understand them. Rejecting the request loses the caller's field and
+    # the whole conversation with it; forwarding what we do not model is the proxy's job.
+    model_config = ConfigDict(extra="allow")


do we need this? I think we did forbid on purpose cc @bxyu-nvidia

cmunley1 · 2026-09-04T17:43:40Z

+    # Absent from the OpenAI type but sent by real clients and accepted by real servers.
+    # Without it the request schema rejects the whole conversation, so an agent that calls a
+    # tool cannot receive the result.
+    name: NotRequired[str]


cmunley1 · 2026-09-04T17:48:56Z

+        env_setup_cmd = (
+            f"{uv_venv_cmd}{prefix_cmd} && source {venv_activate_fpath} && "
+            f"{extra_env_exports}{install_cmd}{prefix_cmd}"
+        )


couldnt we just set this envvar in app.py setup_webserver or something. would like to avoid core change when possible

on further thought i see how this might need a core change if
HERMES_NIX_BUILD=1 gym env start --environment hermes_math ...
is not sufficient... hmmm.. cc @bxyu-nvidia @ananthsub

j-nolan requested a review from a team as a code owner July 16, 2026 13:18

j-nolan marked this pull request as draft July 16, 2026 13:18

nemo-automation-bot Bot added the community-request Issue reported or requested by someone from the community label Jul 16, 2026

j-nolan marked this pull request as ready for review July 16, 2026 13:29

j-nolan force-pushed the jnolan/hermes-latest-pin branch 2 times, most recently from ea6a874 to ceaa05a Compare July 16, 2026 13:36

svcnvidia-nemo-ci added the waiting-on-customer Waiting on the original author to respond label Jul 16, 2026

github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Jul 17, 2026

github-actions Bot mentioned this pull request Jul 17, 2026

[Tracker] Pull request handoff SLA #2067

Open

anwithk requested a review from Glorf July 17, 2026 22:38

github-actions Bot removed the sla:triage-overdue Review assignment is over the one-business-day SLA label Jul 17, 2026

svcnvidia-nemo-ci removed the waiting-on-customer Waiting on the original author to respond label Jul 20, 2026

github-actions Bot added the sla:review-overdue Review response is over the one-business-day SLA label Jul 20, 2026

j-nolan force-pushed the jnolan/hermes-latest-pin branch 2 times, most recently from a8d243c to fa768a2 Compare July 21, 2026 05:58

svcnvidia-nemo-ci added the waiting-on-customer Waiting on the original author to respond label Jul 21, 2026

svcnvidia-nemo-ci added waiting-on-maintainers Waiting on maintainers to respond and removed waiting-on-customer Waiting on the original author to respond labels Jul 21, 2026

elisam0 force-pushed the jnolan/hermes-latest-pin branch 4 times, most recently from 6a7e7ac to 965b778 Compare August 26, 2026 13:13

copy-pr-bot Bot temporarily deployed to public September 3, 2026 02:40 Inactive

copy-pr-bot Bot temporarily deployed to public September 3, 2026 02:42 Inactive

Merge branch 'main' into jnolan/hermes-latest-pin

862918c

copy-pr-bot Bot temporarily deployed to public September 3, 2026 02:54 Inactive

copy-pr-bot Bot temporarily deployed to public September 3, 2026 02:57 Inactive

cmunley1 reviewed Sep 3, 2026

View reviewed changes

svcnvidia-nemo-ci added the waiting-on-customer Waiting on the original author to respond label Sep 3, 2026

rroverin added 3 commits September 3, 2026 11:29

Merge remote-tracking branch 'jnolan/jnolan/hermes-latest-pin' into p…

2db30e8

…r-branch

nv-pberard and others added 5 commits September 3, 2026 19:42

Merge remote-tracking branch 'upstream/main' into HEAD

f8b3f37

DCO Remediation Commit for James Nolan <jnolan@nvidia.com>

5a55e1e

I, James Nolan <jnolan@nvidia.com>, hereby add my Signed-off-by to this commit: 382ce83 I, James Nolan <jnolan@nvidia.com>, hereby add my Signed-off-by to this commit: a24a3d4 Signed-off-by: James Nolan <jnolan@nvidia.com>

cmunley1 approved these changes Sep 4, 2026

View reviewed changes

cmunley1 reviewed Sep 4, 2026

View reviewed changes

cmunley1 requested changes Sep 4, 2026

View reviewed changes

svcnvidia-nemo-ci added waiting-on-maintainers Waiting on maintainers to respond and removed waiting-on-customer Waiting on the original author to respond labels Sep 4, 2026

github-actions Bot mentioned this pull request Sep 8, 2026

[Tracker] Pull request handoff SLA #3158

Open

svcnvidia-nemo-ci added waiting-on-customer Waiting on the original author to respond and removed waiting-on-maintainers Waiting on maintainers to respond labels Sep 9, 2026

Uh oh!

Conversation

j-nolan commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

Changes

Validation

Uh oh!

copy-pr-bot Bot commented Jul 16, 2026

Uh oh!

cmunley1 commented Jul 16, 2026

Uh oh!

j-nolan commented Jul 20, 2026

Uh oh!

Glorf commented Jul 21, 2026

Uh oh!

cmunley1 commented Jul 21, 2026

Uh oh!

cmunley1 commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

j-nolan commented Jul 21, 2026

Uh oh!

elisam0 commented Aug 26, 2026

Uh oh!

cmunley1 commented Sep 3, 2026

Uh oh!

cmunley1 commented Sep 3, 2026

Uh oh!

cmunley1 Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

rroverin Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

rroverin commented Sep 3, 2026

Uh oh!

cmunley1 Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

cmunley1 Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

cmunley1 Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

cmunley1 Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

j-nolan commented Jul 16, 2026 •

edited

Loading

cmunley1 commented Jul 21, 2026 •

edited

Loading