Skip to content

fix(responses): HERMES_DISABLE_REASONING_INCLUDE env-var escape hatch - #25189

Open
nnnet wants to merge 15 commits into
NousResearch:mainfrom
nnnet:fix/bug-2-encrypted-content-env-escape
Open

fix(responses): HERMES_DISABLE_REASONING_INCLUDE env-var escape hatch#25189
nnnet wants to merge 15 commits into
NousResearch:mainfrom
nnnet:fix/bug-2-encrypted-content-env-escape

Conversation

@nnnet

@nnnet nnnet commented May 13, 2026

Copy link
Copy Markdown
Contributor

Summary

agent/transports/codex.py and agent/auxiliary_client.py unconditionally set

kwargs["include"] = ["reasoning.encrypted_content"]

on every Responses-API call whenever reasoning_enabled is true (the default). The parameter is only valid for reasoning-capable models (o1 / o3 / o4-mini / gpt-5*). For every other OpenAI model the API returns:

HTTP 400: Encrypted content is not supported with this model.
param='include', code='model_not_found'

Users running Hermes against api.openai.com with a non-reasoning model (e.g. gpt-4o-mini for cost reasons) currently have no way to opt out — every request fails.

Add an env-var escape hatch: HERMES_DISABLE_REASONING_INCLUDE=true suppresses both include and reasoning in the outgoing Responses-API kwargs. Default behaviour is preserved when the env var is unset / falsy.

Repro (end-to-end on the fork's Docker setup)

  1. ~/.hermes/.env: OPENAI_API_KEY=sk-...
  2. ~/.hermes/config.yaml:
    model:
      default: gpt-4o-mini
      provider: auto
      base_url: https://api.openai.com/v1
  3. docker compose restart gateway
  4. curl -s -X POST http://127.0.0.1:8642/v1/responses -H "Authorization: Bearer $API_SERVER_KEY" -H "Content-Type: application/json" -d '{"model":"hermes-agent","input":"Say OK","conversation":"repro","stream":false}'

Observed:

HTTP 400: Encrypted content is not supported with this model.
provider=openrouter base_url=https://api.openai.com/v1 model=gpt-4o-mini

Expected: 200 OK with a normal reply.

Root cause

agent/transports/codex.py:109,124 and agent/auxiliary_client.py:690 set include=["reasoning.encrypted_content"] whenever reasoning_enabled is True (the default). There is no model-family check before the include is added, so non-reasoning models hit the OpenAI 400.

Fix

Add _reasoning_include_disabled() in agent/transports/codex.py:

def _reasoning_include_disabled() -> bool:
    return (os.getenv("HERMES_DISABLE_REASONING_INCLUDE", "") or "").strip().lower() in {
        "1", "true", "yes", "on",
    }

Gate both call sites (codex transport build_kwargs, auxiliary client codex-responses path) on not _reasoning_include_disabled(). When the flag is on, both kwargs["include"] and kwargs["reasoning"] are skipped — equivalent to running without the reasoning add-ons.

Why an env var, not auto-detection of the reasoning family?

Roughly 30 tests across tests/agent/test_auxiliary_client.py, tests/agent/transports/test_codex_transport.py, tests/run_agent/test_provider_parity.py, and friends prescriptively assert include == ["reasoning.encrypted_content"] whenever a reasoning code path runs. Adding agent.model_metadata.is_reasoning_model() and switching the transports to consult it would require updating every one of those expectations — a much larger refactor and a riskier PR.

The env var is opt-in and 100% backward-compatible, which lets users unblock themselves today without us having to coordinate a test-rewrite. A follow-up PR can introduce automatic detection in agent/model_metadata.py and deprecate the env var once the rewrite lands.

Testing

End-to-end verified on the fork's Docker setup:

Test 1 — sanity (default reasoning model, env unset)

  • Config: model.default: gpt-5-nano (reasoning-capable), HERMES_DISABLE_REASONING_INCLUDE unset.
  • POST /v1/responses → 200, "OK" returned, no regression. include=["reasoning.encrypted_content"] is still sent.

Test 2.1 — repro of the bug (non-reasoning model, env unset)

  • Config: model.default: gpt-4o-mini (non-reasoning), HERMES_DISABLE_REASONING_INCLUDE unset.
  • POST /v1/responses
    HTTP 400: Encrypted content is not supported with this model.
    provider=openrouter base_url=https://api.openai.com/v1 model=gpt-4o-mini
    
  • Confirms the bug is present with the unmodified code path.

Test 2.2 — fix verified (non-reasoning model, env set)

  • Same config as 2.1, but HERMES_DISABLE_REASONING_INCLUDE=true exported into the gateway container.
  • Verified the env reached the container: docker exec hermes printenv HERMES_DISABLE_REASONING_INCLUDEtrue.
  • POST /v1/responses
    {"status": "completed", "output": [{"role": "assistant", "content": [{"text": "PASS"}]}]}
    
  • Container logs contain zero HTTP 400 / Encrypted content entries during the test.
  • Fix verified.

Plus the in-process unit-style check from the original branch: _reasoning_include_disabled() returns False by default and True for any of {"1", "true", "yes", "on"}. With the env set, ResponsesApiTransport.build_kwargs("gpt-4o-mini", ...) returns kwargs that contain no include and no reasoning block.

Breaking changes

None. Default behaviour is unchanged. The env var is opt-in.

Follow-up

A separate PR can introduce agent.model_metadata.is_reasoning_model(model_id) -> bool and switch the transports to consult it automatically, at which point the env var becomes a forced override (e.g. for users behind proxies that munge requests). I'd like to land this escape hatch first because it unblocks anyone hitting the bug today without touching the prescriptive tests.


PR by Claude Code on behalf of @nnnet. Tracks internal bug tracker entry BUG-2.

nnnet added 15 commits May 13, 2026 20:00
Two related issues in the gateway voice-input path:

1. After ``_enrich_message_with_transcription`` the gateway also sent a
   hardcoded English notice via ``_stt_adapter.send()``. TTS would read
   it aloud while the LLM produced its own (correctly localized) reply
   from the enriched prompt — two messages, wrong language, double UX.

2. The enrichment templates themselves talked extensively about STT
   setup and pretended that a separate help message had been sent:

       [The user sent a voice message but I can't listen to it right now
        — no STT provider is configured. A direct message has already
        been sent to the user with setup instructions. You have a skill
        called hermes-agent-setup that can help users configure ...]

   That text becomes part of the persisted conversation history. Once
   it appears in any past turn, the LLM keeps the topic alive: every
   subsequent voice message — even one transcribed successfully —
   provokes a reply about "configuring STT" instead of an answer to
   what the user actually said. Observed in production with
   ``gpt-5-nano``: after a single STT failure the assistant volunteered
   Whisper / Vosk setup instructions on every voice turn for the rest
   of the session.

Fix:

- Drop the standalone hardcoded send (lines 6802-6829).
- Rewrite the four enrichment templates to be minimal and neutral:
  success becomes a plain quoted line; every failure branch becomes a
  single bracketed neutral marker. No mentions of providers, setup,
  skills, or "a message has been sent".
- Move the operator-facing failure cause to a ``logger.info`` line so
  it stays diagnosable in container logs without leaking into the
  LLM-visible prompt.
Adds ``.github/workflows/sync.yml`` that fast-forwards the fork's
``main`` branch from ``NousResearch/hermes-agent:main`` every hour and
on manual ``workflow_dispatch``.

Uses GitHub's ``/repos/{owner}/{repo}/merge-upstream`` REST endpoint,
which performs a fast-forward when there are no conflicts and reports
``merge_type=none`` (with an "up-to-date" message) when the fork is
already in sync. Real merge conflicts surface as ``merge_type=none``
with a different message and fail the workflow — the run log shows the
full API response for triage.

No personal access token (PAT) is required: the default
``GITHUB_TOKEN`` issued to the workflow has ``contents: write`` on
this fork (set explicitly in the workflow's ``permissions:`` block)
and the ``merge-upstream`` endpoint is designed for exactly this case.

The workflow is gated by ``if: github.repository != 'NousResearch/hermes-agent'``
so it is a no-op if this file ever lands in the canonical upstream.
The default TTS voice (``en-US-AriaNeural``, English-only) silently
mangles non-English replies. Edge-tts has no language auto-detection:
it tries to read Russian/Chinese/Japanese/Korean/Arabic text using
English phonemes, drops characters it cannot map, and produces a clip
that matches no language well. Users see a long, on-topic text reply
and hear a short, garbled, partly-English audio clip — the two
disagree, and the audio sometimes contains only the *Latin* fragments
of the text expanded according to English orthographic rules.

Concrete production case (gpt-5-nano, Russian voice mode):

  Text reply:  "Анекдот: Почему программисты путают Хэллоуин и
                Рождество? Потому что Oct 31 = Dec 25."
  Audio:       "October 31st equals December 25th."

Fix this on two fronts:

1. **Multilingual default**: switch ``DEFAULT_EDGE_VOICE`` to
   ``en-US-AvaMultilingualNeural``. This single voice handles ~50
   languages with mid-sentence switching; it is the safest out-of-box
   choice and immediately resolves the symptom above for users who
   never touch the TTS config. ``hermes setup`` now writes the
   multilingual voice into new ``~/.hermes/config.yaml`` files.

2. **Per-language voice routing**: a new optional
   ``tts.<provider>.voice_by_language`` config block routes each TTS
   call to a voice that matches the detected script of the text. For
   users who want native single-language voices (e.g.
   ``ru-RU-SvetlanaNeural`` for Russian, ``zh-CN-XiaoxiaoNeural`` for
   Chinese), this is opt-in and 100% backward compatible.

   Detection is a tiny Unicode-block heuristic — no new pip
   dependency. It identifies the five non-Latin scripts where an
   English voice produces total garbage (ru/zh/ja/ko/ar). Anything
   else (English/German/Spanish/French/Polish/...) falls back to the
   default voice, which is now multilingual.

3. **Environment-variable overrides**: every module-level default in
   ``tools/tts_tool.py`` (provider names, model ids, voice ids, base
   URLs, per-provider max-text-length caps, language-detection
   threshold, Gemini sample rate / channel layout, etc.) is now read
   through ``get_env_value()`` with ``HERMES_TTS_*`` env vars. Hard
   defaults remain as the second argument so the module still works
   without any env-var configuration.

Wired into ``_generate_edge_tts`` only; other cloud providers
(OpenAI/Gemini/ElevenLabs/Mistral) already use multilingual models and
do not need per-call voice swapping. Local providers
(piper/neutts/kittentts) tie voice and language at the model level
and are out of scope for this change.
The Docker image and the base pip install intentionally exclude the
heavy ``[voice]`` extra (faster-whisper pulls in ctranslate2 +
onnxruntime), and ``LAZY_DEPS["stt.faster_whisper"]`` is declared in
``tools/lazy_deps.py`` for exactly this scenario.  However, no call site
references that key, so a user who sets ``stt.provider: local`` always
hits "STT provider 'local' configured but unavailable" — the lazy-install
path is wired up everywhere except the one place it's needed.

Mirror the pattern used by ``gateway/platforms/{telegram,slack,...}.py``
and other ``tools/`` modules: when the user has explicitly opted into
local STT and the wheel isn't present, call
``lazy_deps.ensure("stt.faster_whisper", prompt=False)``, re-check the
import, then continue.

Auto-detect (no explicit ``stt.provider``) deliberately does NOT
lazy-install — that would impose a multi-MB native-wheel download on
users who never asked for local STT.  Only the explicit ``provider:
local`` path triggers it.
When ``model.base_url`` is set to a native vendor endpoint (e.g.
``https://api.openai.com/v1``) and ``model.provider`` is left at
``auto``, ``hermes_cli.auth.resolve_provider()`` falls through to its
``"openrouter"`` default — which is correct for credential resolution
but produces the misleading user-facing label "(openrouter)" in the
context-compression warning even though the request never touches
OpenRouter.

Translate the displayed ``_main_provider`` label from the live
``self.base_url`` when it points at a known vendor host. Routing is
unchanged — this is cosmetic only, but actively confuses users trying
to debug provider issues when the label disagrees with the URL.

Uses ``base_url_host_matches`` so subdomain-spoof URLs like
``api.openai.com.attacker.com`` are not mislabelled as openai.
When ``hermes_cli.auth.resolve_provider()`` returns ``"openrouter"``
(its catch-all for ``OPENAI_API_KEY``-only configs) but the live
``model.base_url`` is set to a native vendor host such as
``https://api.openai.com/v1``, requests bypass the aggregator and go
straight to the vendor — which rejects the aggregator-style
``openai/gpt-4o-mini`` slug with:

    HTTP 400: The requested model 'openai/gpt-4o-mini' does not exist

The existing ``run_agent.HermesAgent`` constructor only calls
``normalize_model_for_provider()`` when the provider is **not** an
aggregator (see L1320), so the prefix survives.

Extend that branch: when the provider is an aggregator AND the base_url
matches a known native vendor host, strip the matching ``<vendor>/``
prefix from ``self.model`` before any kwargs are built. Uses
``base_url_host_matches`` so subdomain-spoof URLs cannot bypass the
guard. Only the prefix is touched — the rest of the model id flows
through unmodified.
The Responses API rejects ``include: ["reasoning.encrypted_content"]``
on non-reasoning models with:

    HTTP 400: Encrypted content is not supported with this model.
    param='include', code='model_not_found'

Users targeting a non-reasoning OpenAI model (e.g. ``gpt-4o-mini`` via
``base_url: https://api.openai.com/v1``) currently have no way to opt
out other than switching to a reasoning model.

Add ``HERMES_DISABLE_REASONING_INCLUDE=true`` (~/.hermes/.env) that
suppresses both the ``include`` parameter and the ``reasoning`` block
from the outgoing Responses-API kwargs.  Default behavior — sending
``include=["reasoning.encrypted_content"]`` on every Responses-API call
— is preserved when the env var is unset or falsy.

Touches two call sites:

- ``agent/transports/codex.py`` — main agent's Responses API transport.
- ``agent/auxiliary_client.py:690`` — Codex auxiliary client path.

A long-term fix is automatic detection of the reasoning family in
``agent/model_metadata.py``, but the existing tests pin the current
unconditional behavior, so the env-var escape hatch is the smallest
change that addresses the user-visible breakage without revising
~30 pinned test expectations.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API P2 Medium — degraded but workaround exists labels May 13, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Resubmission of #25102 (closed without merge).

Competing PRs for the same root cause (#23450):

This PR adds an env-var escape hatch (HERMES_DISABLE_REASONING_INCLUDE) instead of auto-detecting model capability.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for documenting a concrete Responses-API failure and identifying both main and auxiliary construction paths.

Problems

  • The proposed HERMES_DISABLE_REASONING_INCLUDE is a new user-facing behavioral environment variable, but repository policy requires non-secret behavior to live in config.yaml (AGENTS.md:102-105).
  • The transport-level gate would also suppress xAI's encrypted-reasoning include. Current main deliberately requires that include for xAI cross-turn replay (agent/transports/codex.py:265-281; tests/agent/transports/test_codex_transport.py:275-290; b4afc6546).
  • No tests accompany the request-construction changes, and the PR bundles unrelated CI, gateway, TTS, STT, and model-routing work.

Suggested changes

  • Split unrelated commits, re-scope the control through config.yaml, preserve xAI replay, and add direct-OpenAI, auxiliary, and xAI regression tests.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/openai OpenAI / Codex Responses API sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants