feat: add api_mode=responses for generic OpenAI Responses API (#33600)

[ousiaresearch]

What does this PR do?

Adds api_mode='responses' for custom providers using the OpenAI Responses API (/v1/responses). Generic (non-Codex) dispatch.

Fixes #33600

Type of Change

• ✨ New feature

Changes Made

• hermes_cli/runtime_provider.py: Added "responses" to _VALID_API_MODES
• agent/chat_completion_helpers.py: Added _normalize_responses_to_chat() + responses api_mode branch in _call()

How to Test

1. Configure custom provider with api_mode: responses in config.yaml
2. Use hermes model to select custom provider
3. Send message — verify responses work with text and tool calls

Checklist

• I have tested these changes locally

[alt-glitch]

Competes with #33604, #33637, #33765 — all address api_mode=responses for #33600. Also bundled in #33884. This PR adds responses as a first-class api_mode rather than aliasing to codex_responses.

[ousiaresearch]

Design notes — first-class api_mode vs alias normalization

This PR adds responses as a first-class api_mode value, with a dedicated normalization layer in agent/chat_completion_helpers.py that converts OpenAI Responses API output into the standard chat.completions shape the agent loop expects.

Why first-class vs alias (#33604, #33765):

Competing PRs alias responses → codex_responses:

```python
if normalized == "responses":
normalized = "codex_responses"
```

This is a problem because it conflates two distinct protocols:

• codex_responses: Tightly coupled to OpenAI Codex-specific response shapes (used for OpenAI-codex provider)
• responses: Generic OpenAI Responses API — works with ANY provider that supports it (OpenRouter, custom endpoints, LiteLLM, etc.)

By normalizing to codex_responses, the competing PRs lose the ability to distinguish between these at the routing layer. Downstream provider selection in provider_responses.py uses api_mode to decide which code path to take — collapsing them means a custom endpoint using the Responses API gets routed to the Codex-specific handler.

Our approach:

1. responses is its own recognized api_mode value
2. Normalization happens at the response-parsing layer (mapping Responses API output → chat.completions shape), NOT at the config validation layer
3. Provider routing stays clean — api_mode=responses maps to the generic Responses handler, not the Codex-specific one
4. Works with any base_url, not just OpenAI endpoints

[teknium1]

Thanks for isolating generic Responses API support from the Codex-specific path. The capability remains absent on current main, but this patch needs a broader salvage.

Problems

• Current AIAgent only admits the modes listed in agent/agent_init.py:452; an explicit responses mode falls through to chat_completions at agent/agent_init.py:483, so the added dispatch branch would not run after salvage.
• The new responses.create() call needs Responses-format kwargs. Current agent/chat_completion_helpers.py:818-882 produces input and converted tools only for codex_responses; the default path from line 884 constructs Chat Completions messages kwargs.
• The patch has no tests for custom resolution, payload conversion, text, or function-call continuation.

Suggested changes

• Wire responses through mode admission, transport/payload conversion, centralized dispatch, and the relevant retry/fallback paths; add end-to-end-style unit coverage around a custom provider.
• Update the documented API-mode enumerations, including website/docs/developer-guide/model-provider-plugin.md:94.

Automated hermes-sweeper review.

[teknium1]

responses.create() requires Responses-format input and tool definitions, but this patch leaves build_api_kwargs() on the Chat Completions messages path for api_mode="responses". Wire this mode through the existing Responses transport/payload conversion before dispatching here.