feat: Add anthropic messages api support as a responses_api_model. - #1546
Closed
ffrujeri wants to merge 16 commits into
Closed
feat: Add anthropic messages api support as a responses_api_model.#1546ffrujeri wants to merge 16 commits into
ffrujeri wants to merge 16 commits into
Conversation
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
cmunley1
reviewed
Jun 9, 2026
cmunley1
reviewed
Jun 9, 2026
cmunley1
left a comment
Contributor
There was a problem hiding this comment.
did you test with claude_code_agent and would you mind updating that agent instructions to use this if so?
cmunley1
reviewed
Jun 9, 2026
cmunley1
reviewed
Jun 9, 2026
cmunley1
reviewed
Jun 9, 2026
cmunley1
reviewed
Jun 9, 2026
cmunley1
reviewed
Jun 9, 2026
cmunley1
reviewed
Jun 9, 2026
cmunley1
reviewed
Jun 9, 2026
…eason mapping, Opus 4.7/4.8 sampling restrictions, and README example. Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Contributor
Author
So, this PR only implements the ResponsesAPI <--> Anthropic Messages. I think what you mean by this question is the other direction of translation that would allow Anthropic Messages <--> ResponsesAPI could be handled in a different Issue and PR. |
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
This was referenced Jun 16, 2026
ffrujeri
added a commit
that referenced
this pull request
Jun 24, 2026
…se Gym… (#1627) # What does this PR do? [Design note](#1620 (comment)) Makes **every NeMo Gym model server speak the Anthropic Messages API** by adding a `POST /v1/messages` route to the base `SimpleResponsesAPIModel`, with a default implementation that maps Anthropic Messages ↔ Responses around the server's own `responses()`. External harnesses that require an Anthropic endpoint — notably `claude_code_agent` (the Claude Code CLI only talks `/v1/messages`) — can now run against **any** Gym backend (vLLM, OpenAI, an inference provider), not just Anthropic's API. The Messages ↔ Responses mapping lives in one shared, transport-free, SDK-free converter, `nemo_gym/anthropic_converter.py`, used in both directions and shared with the egress `anthropic_model` provider (#1146/#1546). See `DESIGN-NOTE.md` for rationale. # Why Gym standardizes on the Responses API, but real blackbox agents speak other protocols. Rather than a bespoke per-agent bridge or a separate proxy server, the Messages spoke becomes a base-class capability: implement `responses()` once and get `/v1/messages` for free. This is the model-proxy layer the CustomAgent EPIC (#1042) assumes — its sidecar "path-preserving forwards" `/v1/messages` to these routes. # Key design points - **Route on the base class, not a separate proxy.** Every model server inherits `/v1/messages`; no extra process or network hop. (An earlier `anthropic_messages_proxy` + `SimpleMessagesAPIModel` iteration was removed in favor of this.) - **Delegates to the server's own `responses()`** — reuses whatever backend the server has; a small `inspect`-based dispatch handles both existing `responses()` signatures. - **Synthesized SSE.** Backends are non-streaming via Responses; the converter fabricates the Anthropic SSE event stream from the complete response (the Claude CLI requires streaming). - **Shared bidirectional converter**, transport-free and **no `anthropic` SDK** (local shapes over Gym's aiohttp). Fail-loud (`NotImplementedError`) on unsupported content. - **Eval-first.** Token-ids live only at the Responses hop and cannot ride the Anthropic SSE response; an RL path would side-channel them, not thread them through Messages. # Issues Closes #1620 (*"Add a default `/v1/messages` (Anthropic Messages) route to the base Gym model server"*). Closes #1566 Relationship to adjacent work: - **#1620** — this PR is its implementation: the `/v1/messages` route + default Messages ↔ Responses mapping on `SimpleResponsesAPIModel`, the shared converter, and the `claude_code_agent` end-to-end run satisfy its acceptance criteria. - **#1146 / #1546** (egress `anthropic_model`) — inverse direction; shares the same converter. - **#1286** (`responses_converter`) — sibling Chat-Completions ↔ Responses converter; same pattern. - **EPIC #1042** (CustomAgent) — its sidecar model proxy "path-preserving forwards" `/v1/messages` to the routes this PR adds. # Usage Every model server now answers Anthropic Messages directly — this is config-independent and is the core of the change. Against any running Gym model server: ```bash curl -s http://127.0.0.1:<model_server_port>/v1/messages -H 'content-type: application/json' -d '{ "model": "x", "max_tokens": 64, "messages": [{"role": "user", "content": "hi"}] }' ``` `claude_code_agent` reaches any Gym model server by setting its `model_server` ref — the harness resolves `ANTHROPIC_BASE_URL` from it and the CLI appends `/v1/messages` (see `responses_api_agents/claude_code_agent/app.py:_resolve_base_url`). The end-to-end run in **Testing** below used this path against a local vLLM. The verified `reasoning_gym` Claude Code config keeps its direct `anthropic_base_url` default; targeting a Gym model server is an opt-in `model_server` ref. # Additional information - **Changed:** `nemo_gym/base_responses_api_model.py` (new `/v1/messages` route + default `messages()`); **added** `nemo_gym/anthropic_converter.py` (shared bidirectional converter); `responses_api_agents/claude_code_agent/README.md` (documents the model-server path). No change to the verified `reasoning_gym_claude_code_agent.yaml` — targeting a Gym model server is an opt-in `model_server` ref on the agent. - **Converter surface:** `responses_to_anthropic` / `anthropic_to_responses` (egress, shared with #1546); `anthropic_request_to_responses` / `responses_to_anthropic_response` / `anthropic_response_to_sse` (ingress). Supports text, base64 images, function tools, tool_use/tool_result, thinking, stop-reason. Anthropic-API egress *policy* stays in the `anthropic_model` server, not the converter. # Testing - `tests/unit_tests/test_anthropic_converter.py` (ingress + round-trip) and `…_egress.py` (egress) — **100%** converter coverage. - `tests/unit_tests/test_responses_api_model_messages.py` — base-class `/v1/messages` for both `responses()` signatures + SSE framing. - Full unit suite green (284); `openai_model` tests pass unchanged. - **Live e2e:** `claude_code_agent → policy_model (vLLM) /v1/messages` on reasoning_gym — `finished_naturally=1`, 1 turn, ~5.5k tokens, 0 failures (reward reflects the small model's answer, not the plumbing). - Verified: `ruff check`, `ruff format --check`, `pre-commit`. # Open questions - Versioning/regression suite for the converters (OpenAI/Anthropic bump frequently). - Audit/align the existing Chat-Completions ↔ Responses tests. - Confirm fail-loud vs drop policy for items absent in a target schema (e.g. computer-use). --------- Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com> Signed-off-by: Lin Jia <linj@nvidia.com> Co-authored-by: Lin Jia <linj@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve add/add conflicts in anthropic_converter and its tests by keeping main (#1627) as the source of truth.
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Contributor
|
🌿 Preview your docs: https://nvidia-preview-ffrujeri-claude-model-support.docs.buildwithfern.com/nemo/gym Here are the markdown pages you've updated: |
ritaneves
pushed a commit
that referenced
this pull request
Jun 25, 2026
…se Gym… (#1627) # What does this PR do? [Design note](#1620 (comment)) Makes **every NeMo Gym model server speak the Anthropic Messages API** by adding a `POST /v1/messages` route to the base `SimpleResponsesAPIModel`, with a default implementation that maps Anthropic Messages ↔ Responses around the server's own `responses()`. External harnesses that require an Anthropic endpoint — notably `claude_code_agent` (the Claude Code CLI only talks `/v1/messages`) — can now run against **any** Gym backend (vLLM, OpenAI, an inference provider), not just Anthropic's API. The Messages ↔ Responses mapping lives in one shared, transport-free, SDK-free converter, `nemo_gym/anthropic_converter.py`, used in both directions and shared with the egress `anthropic_model` provider (#1146/#1546). See `DESIGN-NOTE.md` for rationale. # Why Gym standardizes on the Responses API, but real blackbox agents speak other protocols. Rather than a bespoke per-agent bridge or a separate proxy server, the Messages spoke becomes a base-class capability: implement `responses()` once and get `/v1/messages` for free. This is the model-proxy layer the CustomAgent EPIC (#1042) assumes — its sidecar "path-preserving forwards" `/v1/messages` to these routes. # Key design points - **Route on the base class, not a separate proxy.** Every model server inherits `/v1/messages`; no extra process or network hop. (An earlier `anthropic_messages_proxy` + `SimpleMessagesAPIModel` iteration was removed in favor of this.) - **Delegates to the server's own `responses()`** — reuses whatever backend the server has; a small `inspect`-based dispatch handles both existing `responses()` signatures. - **Synthesized SSE.** Backends are non-streaming via Responses; the converter fabricates the Anthropic SSE event stream from the complete response (the Claude CLI requires streaming). - **Shared bidirectional converter**, transport-free and **no `anthropic` SDK** (local shapes over Gym's aiohttp). Fail-loud (`NotImplementedError`) on unsupported content. - **Eval-first.** Token-ids live only at the Responses hop and cannot ride the Anthropic SSE response; an RL path would side-channel them, not thread them through Messages. # Issues Closes #1620 (*"Add a default `/v1/messages` (Anthropic Messages) route to the base Gym model server"*). Closes #1566 Relationship to adjacent work: - **#1620** — this PR is its implementation: the `/v1/messages` route + default Messages ↔ Responses mapping on `SimpleResponsesAPIModel`, the shared converter, and the `claude_code_agent` end-to-end run satisfy its acceptance criteria. - **#1146 / #1546** (egress `anthropic_model`) — inverse direction; shares the same converter. - **#1286** (`responses_converter`) — sibling Chat-Completions ↔ Responses converter; same pattern. - **EPIC #1042** (CustomAgent) — its sidecar model proxy "path-preserving forwards" `/v1/messages` to the routes this PR adds. # Usage Every model server now answers Anthropic Messages directly — this is config-independent and is the core of the change. Against any running Gym model server: ```bash curl -s http://127.0.0.1:<model_server_port>/v1/messages -H 'content-type: application/json' -d '{ "model": "x", "max_tokens": 64, "messages": [{"role": "user", "content": "hi"}] }' ``` `claude_code_agent` reaches any Gym model server by setting its `model_server` ref — the harness resolves `ANTHROPIC_BASE_URL` from it and the CLI appends `/v1/messages` (see `responses_api_agents/claude_code_agent/app.py:_resolve_base_url`). The end-to-end run in **Testing** below used this path against a local vLLM. The verified `reasoning_gym` Claude Code config keeps its direct `anthropic_base_url` default; targeting a Gym model server is an opt-in `model_server` ref. # Additional information - **Changed:** `nemo_gym/base_responses_api_model.py` (new `/v1/messages` route + default `messages()`); **added** `nemo_gym/anthropic_converter.py` (shared bidirectional converter); `responses_api_agents/claude_code_agent/README.md` (documents the model-server path). No change to the verified `reasoning_gym_claude_code_agent.yaml` — targeting a Gym model server is an opt-in `model_server` ref on the agent. - **Converter surface:** `responses_to_anthropic` / `anthropic_to_responses` (egress, shared with #1546); `anthropic_request_to_responses` / `responses_to_anthropic_response` / `anthropic_response_to_sse` (ingress). Supports text, base64 images, function tools, tool_use/tool_result, thinking, stop-reason. Anthropic-API egress *policy* stays in the `anthropic_model` server, not the converter. # Testing - `tests/unit_tests/test_anthropic_converter.py` (ingress + round-trip) and `…_egress.py` (egress) — **100%** converter coverage. - `tests/unit_tests/test_responses_api_model_messages.py` — base-class `/v1/messages` for both `responses()` signatures + SSE framing. - Full unit suite green (284); `openai_model` tests pass unchanged. - **Live e2e:** `claude_code_agent → policy_model (vLLM) /v1/messages` on reasoning_gym — `finished_naturally=1`, 1 turn, ~5.5k tokens, 0 failures (reward reflects the small model's answer, not the plumbing). - Verified: `ruff check`, `ruff format --check`, `pre-commit`. # Open questions - Versioning/regression suite for the converters (OpenAI/Anthropic bump frequently). - Audit/align the existing Chat-Completions ↔ Responses tests. - Confirm fail-loud vs drop policy for items absent in a target schema (e.g. computer-use). --------- Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com> Signed-off-by: Lin Jia <linj@nvidia.com> Co-authored-by: Lin Jia <linj@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Rita Fernandes Neves <rfernandesne@nvidia.com>
ananthsub
previously approved these changes
Jun 25, 2026
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Design Note
Adds a native
anthropic_modelResponses API model server that translates NeMo Gym requests to Anthropic/v1/messagesand maps Anthropic responses back to NeMo Gym Responses API objects.The Anthropic ↔ Responses mapping lives in a shared, transport-free module
nemo_gym/anthropic_converter.py(AnthropicConverter) — following the precedent ofnemo_gym/responses_converter.py. The converter is bidirectional: this server uses the egress direction (Responses → Anthropic), and the same module exposes the inverse direction (Anthropic → Responses) plus Anthropic SSE synthesis for a future Anthropic Messages ingress proxy (e.g. letting the Claude Code CLI run against any Gym backend). The hard mapping rules (tool_use/tool_result, thinking, stop_reason, images) are written once and guarded by round-trip tests, instead of being duplicated in opposite directions.Issues
Closes #1146
Usage
anthropic_modelwith an Anthropic base URL, API key, model name, and max token limit:Once the servers are up, direct model-server smoke test:
Minimal end-to-end smoke test with the example single-tool-call environment:
Collect one rollout through the simple agent:
Additional Information
nemo_gym/anthropic_converter.py. It is bidirectional: egress methods (responses_to_anthropic,anthropic_to_responses) back this server; ingress methods (anthropic_request_to_responses,responses_to_anthropic_response,anthropic_response_to_sse) back a future Anthropic Messages proxy. Egress behavior is unchanged — the existinganthropic_modeltests pass byte-for-byte.nemo_gym.server_utils.request()instead of the Anthropic Python SDK.extra_bodyfor less common Anthropic pass-through fields.input_imagedata URLs to Anthropicimageblocks forimage/jpeg,image/png,image/gif, andimage/webp; remote image URLs and invalid base64 are rejected with clear 400 errors.incomplete_details:max_tokensandmodel_context_window_exceededbecomemax_output_tokens;refusalbecomescontent_filter.temperature,top_p, ortop_kare explicitly configured.tests/unit_tests/test_anthropic_converter.pyfor the ingress direction (anthropic_request_to_responses,responses_to_anthropic_response, SSE synthesis), shared-helper branches, and round-trip property tests that guard the two directions against drift.nemo_gym/anthropic_converter.pyis at 100% line coverage.uv run ruff check nemo_gym/anthropic_converter.py responses_api_models/anthropic_modeluv run ruff format --check nemo_gym/anthropic_converter.py responses_api_models/anthropic_modeluv run pytest responses_api_models/anthropic_model/tests/test_app.py tests/unit_tests/test_anthropic_converter.py -q(51 passed)uv run pytest tests/unit_tests/ -q(267 passed — no regressions from moving the converter into core)