feat(api-server): expose provider models in /v1/models and stream reasoning content by syncoe6368 · Pull Request #24946 · NousResearch/hermes-agent

syncoe6368 · 2026-05-13T10:24:06Z

Summary

Two improvements to gateway/platforms/api_server.py for better compatibility with OpenAI-compatible frontends (Open WebUI, LibreChat, ChatBox, etc.), plus a setup skill.

1. Expose configured provider models in `GET /v1/models`

Problem: /v1/models only returned a single hermes-agent model. Frontends connecting to the API server had no way to present a model selector.

Fix: Added _get_exposed_models() which reads all models from config.yaml providers "models" dicts and "default_model" fields and advertises them alongside the primary model name. Each provider model sets "parent" to the primary model name so frontends can group them correctly.

2. Stream reasoning/thinking content via `delta.reasoning_content`

Problem: Reasoning models (GLM, DeepSeek, Qwen, Kimi) emit thinking tokens through reasoning_callback, but the API server never wired this callback — all thinking content was silently dropped.

Fix:

Added _on_reasoning() callback that tags reasoning as ("__reasoning__", text) in the stream queue
Wired reasoning_callback through _create_agent() and _run_agent()
Updated _emit() in _write_sse_chat_completion() to emit delta.reasoning_content
Updated _dispatch() in _write_sse_responses() for the Responses API path

3. Setup skill: `skills/devops/hermes-gateway-openwebui/SKILL.md`

Complete guide for deploying Hermes gateway + Open WebUI:

Hermes Agent installation (Linux, macOS, WSL2, Windows)
Provider configuration (GLM, OpenRouter, NVIDIA NIM)
OpenAI-compatible API server setup on port 8642
Open WebUI connection on port 8080
Patch application procedure
WSL2-specific networking guidance
Troubleshooting reference

Testing

Verified with local Hermes gateway + Open WebUI:

/v1/models returns all 7 configured models from 3 providers
Streaming chat completions with GLM models show both reasoning_content and content chunks
Open WebUI renders thinking blocks correctly

Changes

gateway/platforms/api_server.py — 119 insertions, 26 deletions (features 1 & 2)
skills/devops/hermes-gateway-openwebui/SKILL.md — new setup skill (388 lines)

…soning content Two improvements for better compatibility with OpenAI-compatible frontends (Open WebUI, LibreChat, ChatBox, etc.): 1. Expose configured provider models in GET /v1/models Previously, /v1/models only returned a single 'hermes-agent' entry. Frontends connecting to the API server had no way to present a model selector to users. Now, all models declared in config.yaml providers' 'models' dicts and 'default_model' fields are advertised alongside the primary model name. Each provider model's 'parent' is set to the primary model name so frontends can group them correctly. 2. Stream reasoning/thinking content via delta.reasoning_content Reasoning models (GLM, DeepSeek, Qwen, Kimi, etc.) emit thinking tokens through the agent's reasoning_callback, but the API server never wired this callback — thinking content was silently dropped. Now, reasoning deltas are forwarded as '__reasoning__' tagged tuples in the stream queue and emitted as 'delta.reasoning_content' chunks in both /v1/chat/completions and /v1/responses SSE streams. This is the standard OpenAI field that frontends like Open WebUI use to render collapsible thinking/reasoning blocks. Changes: - Add _get_exposed_models() to collect models from config providers - Add _on_reasoning() callback for chat completions and responses streams - Wire reasoning_callback through _create_agent() and _run_agent() - Update _emit() in _write_sse_chat_completion() for reasoning chunks - Update _dispatch() in _write_sse_responses() for reasoning chunks

…nto feat/api-server-expose-provider-models-and-reasoning

Complete setup guide for deploying Hermes Agent gateway with Open WebUI: - Install Hermes Agent (Linux, macOS, WSL2, Windows) - Configure providers (GLM, OpenRouter, NVIDIA NIM) - Set up OpenAI-compatible API server on port 8642 - Connect Open WebUI on port 8080 - Apply patches for model exposure and reasoning content - WSL2-specific networking guidance - Troubleshooting reference Works as a self-contained skill that can be installed via: hermes skills install skills/devops/hermes-gateway-openwebui

alt-glitch · 2026-05-13T10:35:37Z

Re-submission of closed #23068 (same author, same title). Related: #13401 (reasoning streaming), #23638 (inline think unification), #21655 (Responses API reasoning gap).

teknium1

Thanks for addressing two real API-server gaps. Current main still lacks API-server reasoning callback wiring, but this branch needs integration with the newer routing contract.

Problems

gateway/platforms/api_server.py:924-929 stringifies every providers.*.models value. Current config supports dictionary-valued model metadata (hermes_cli/config.py:4785-4810), so this can advertise invalid IDs.
The advertised provider IDs are not connected to request routing. Current main's supported path exposes model_routes aliases (gateway/platforms/api_server.py:1421-1453) and resolves them before agent creation (gateway/platforms/api_server.py:2194-2197).
gateway/platforms/api_server.py:1859-1860 sends Responses reasoning via visible output_text handling, mixing thinking into final response text.
skills/devops/hermes-gateway-openwebui/SKILL.md:256 directs users to overwrite an installed core file from a feature branch; that can replace unrelated upstream fixes.

Suggested changes

Integrate model discovery with model_routes, add chat/Responses reasoning-stream tests, and replace the overwrite instructions with updates to website/docs/user-guide/messaging/open-webui.md.

Automated hermes-sweeper review.

teknium1 · 2026-07-13T04:08:50Z

+                provider_models = entry.get("models")
+                if isinstance(provider_models, dict):
+                    for _alias, model_id in provider_models.items():
+                        mid = str(model_id)


providers.*.models may be a mapping from model ID to metadata dict; current config normalization preserves that shape. Converting each value with str() can advertise "{}" or a metadata representation as the model ID. Normalize from the mapping key/metadata shape and connect the result to the current model_routes routing source.

teknium1 · 2026-07-13T04:08:50Z

                    elif tag == "__tool_completed__":
                        await _emit_tool_completed(payload)
+                    elif tag == "__reasoning__":
+                        await _emit_text_delta(payload)


This sends reasoning through _emit_text_delta, which is the normal visible response.output_text.delta path and contributes to final output. Use a distinct Responses reasoning representation, or keep this path out of the reasoning feature, so hidden thinking is not presented as assistant answer text.

teknium1 · 2026-07-13T04:08:50Z

+    cd "$SOURCE_REPO"
+    if git show-ref --verify --quiet "refs/heads/$BRANCH"; then
+        echo "Extracting patched file from branch '$BRANCH'..."
+        git show "$BRANCH:gateway/platforms/api_server.py" > "$GATEWAY_FILE"


Do not ship instructions that overwrite an installed core module from a feature branch. Reapplying this after upgrades can erase unrelated fixes; document supported released configuration instead after the feature is integrated.

GottZ

This was generated by AI during triage.

Summary

Three PRs address the same API-server gaps: #23068 and #26667 contain the same model-discovery and reasoning-streaming patch, while #24946 carries that core diff plus an Open WebUI setup skill. No recorded Verify verdict exists for any of the three; the available contributor review finds the gaps valid but identifies routing, model-metadata, Responses API, and installation-safety defects in #24946.

Related pull requests

#23068 [closed] duplicate — (+119/-26) — superseded: Adds provider-model enumeration and wires reasoning callbacks into Chat Completions and Responses streaming, but stringifies metadata-valued model entries, does not connect advertised IDs to model_routes, and emits Responses reasoning as visible output text. It remains relevant as the closed original implementation superseded by #24946.
#24946 related — (+507/-26) — keep open for revision: Reuses #23068's API-server patch and adds a 388-line Open WebUI deployment skill. The contributor keep_open review confirms the underlying gaps remain, but requires integration with model_routes, metadata-safe model discovery, protocol-correct Responses reasoning events, tests, and removal of instructions that overwrite an installed core file from a feature branch before merge.
#26667 [closed] duplicate — (+119/-26) — duplicate: Repeats the same model-listing and reasoning-callback implementation, including the same routing, metadata, and Responses-output defects. It remains relevant as the closed resubmission explicitly identified by a contributor as a duplicate of #24946 and replacement attempt for #23068.

Duplicates

#23068 and #26667 are effectively the same core change; #24946 contains that same implementation plus the Open WebUI skill.

Suggested consolidation

Merge #24946 only after the contributor's keep_open findings are resolved: use model_routes as the advertised and resolved model contract, handle dictionary-valued model metadata correctly, emit Responses reasoning through protocol-appropriate reasoning events rather than output_text, add focused tests, and remove the unsafe installed-file overwrite guidance. Keep #23068 and #26667 closed as superseded/duplicate implementations.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    subgraph Dup23068 ["PRs duplicating each other"]
        P23068["PR #23068 (closed)"]
        P24946["PR #24946 (open)"]
        P26667["PR #26667 (closed)"]
    end
    class P23068 closed
    class P24946 open
    class P26667 closed
    class P24946 target
    click P23068 "https://github.com/NousResearch/hermes-agent/pull/23068"
    click P24946 "https://github.com/NousResearch/hermes-agent/pull/24946"
    click P26667 "https://github.com/NousResearch/hermes-agent/pull/26667"

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 3 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 48 kB of PR diffs, 7 kB of issue/PR text, 2 kB of discussion (3 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

syncoe6368 added 4 commits May 10, 2026 15:41

Merge branch 'main' of https://github.com/NousResearch/hermes-agent i…

152efdb

…nto feat/api-server-expose-provider-models-and-reasoning

Merge branch 'main' of https://github.com/NousResearch/hermes-agent i…

28a2537

…nto feat/api-server-expose-provider-models-and-reasoning

alt-glitch added type/feature New feature or request comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have labels May 13, 2026

This was referenced May 16, 2026

feat(api-server): expose provider models in /v1/models and stream reasoning content #26667

Closed

feat(api): expose configured provider models #30311

Closed

fix(api-server): wire reasoning_callback and emit reasoning_content in SSE stream (#30449) #30509

Open

This was referenced May 30, 2026

[codex] Expose chat completions reasoning SSE #35083

Draft

feat: advertise configured governed API server models #36110

Open

teknium1 reviewed Jul 13, 2026

View reviewed changes

GottZ reviewed Jul 30, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat(api-server): expose provider models in /v1/models and stream reasoning content - #24946

feat(api-server): expose provider models in /v1/models and stream reasoning content#24946
syncoe6368 wants to merge 4 commits into
NousResearch:mainfrom
syncoe6368:feat/api-server-model-listing-and-reasoning-v2

syncoe6368 commented May 13, 2026 •

edited

Loading

Uh oh!

alt-glitch commented May 13, 2026

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jul 13, 2026

Uh oh!

teknium1 Jul 13, 2026

Uh oh!

teknium1 Jul 13, 2026

Uh oh!

GottZ left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

syncoe6368 commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

1. Expose configured provider models in GET /v1/models

2. Stream reasoning/thinking content via delta.reasoning_content

3. Setup skill: skills/devops/hermes-gateway-openwebui/SKILL.md

Testing

Changes

Uh oh!

alt-glitch commented May 13, 2026

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

GottZ left a comment

Choose a reason for hiding this comment

Summary

Related pull requests

Duplicates

Suggested consolidation

Complex graph

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

syncoe6368 commented May 13, 2026 •

edited

Loading

1. Expose configured provider models in `GET /v1/models`

2. Stream reasoning/thinking content via `delta.reasoning_content`

3. Setup skill: `skills/devops/hermes-gateway-openwebui/SKILL.md`