Skip to content

feat(codex): add agent.text_verbosity config for GPT-5+ response length control - #59711

Closed
qxxaa wants to merge 2 commits into
NousResearch:mainfrom
qxxaa:feat/codex-text-verbosity
Closed

feat(codex): add agent.text_verbosity config for GPT-5+ response length control#59711
qxxaa wants to merge 2 commits into
NousResearch:mainfrom
qxxaa:feat/codex-text-verbosity

Conversation

@qxxaa

@qxxaa qxxaa commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Problem

GPT-5+ models are verbose by default. In agent workflows, this has real
cost: every output token is billed, and longer responses increase latency
on subsequent turns by inflating the conversation history that gets
re-sent as input tokens. Over a multi-turn agent session, verbose output
compounds - each turn's excess output becomes the next turn's excess input.

The OpenAI Responses API accepts a text.verbosity parameter ("low",
"medium", "high") that controls output length for GPT-5+ models. There
is currently no way to configure this in Hermes.

I initially explored using agent.request_overrides in config.yaml to
inject the parameter without any code changes. While the SDK accepts text
as a first-class parameter on responses.create(), request_overrides is
an internal runtime dict populated only by fast mode and custom provider
logic - it is never read from config.yaml. Settings placed there are
silently ignored.

Measured impact: setting text.verbosity: "low" on GPT-5.5 reduced output
from 585 words to 369 words (37% reduction) on identical prompts. In a
typical 20-turn agent session, that reduction compounds across every turn -
shorter assistant responses mean fewer input tokens on every subsequent API
call, reducing both cost and time-to-first-token latency.

Fix

New config key agent.text_verbosity:

agent:
  text_verbosity: "low"  # "low" | "medium" | "high" | "" (default, no injection)

Default is empty string - zero behaviour change for anyone who doesn't
set it. No text parameter is sent, and the model uses its own default
verbosity.

When set, injects text: {"verbosity": "<value>"} into the Responses API
payload. Scoped to GPT-5+ models on the codex_responses transport only.

Design decisions

Model guard: Parses gpt-X.Y from the model name, extracts major
version, injects only when >= 5. Strips vendor prefixes (openai/gpt-5.5
-> gpt-5.5) before matching. Non-GPT models (Claude, Grok, Gemini) are
silently skipped - no errors, no injection.

Dict merge: Merges verbosity into any existing text dict rather than
replacing it. If request_overrides sets text.format, both format and
verbosity coexist in the final payload.

Transport scope: Only the codex_responses transport injects it. Chat
completions and Anthropic message transports are untouched - the parameter
is Responses API specific.

No validation: Consistent with reasoning_effort handling. Invalid
values pass through to the API, which returns a clear error. No client-side
allowlist.

Files changed (13)

File Change
hermes_cli/config.py Default "" in agent defaults
agent/agent_init.py Constructor param + agent attribute
run_agent.py Pass-through
cli.py Load from config
hermes_cli/cli_agent_setup_mixin.py Pass to agent init
hermes_cli/cli_commands_mixin.py Pass to agent init (background tasks)
gateway/run.py Loader + 3 load sites + 2 init sites + 1 hot-reload
gateway/platforms/api_server.py Pass to agent init
tui_gateway/server.py Loader + 2 injection sites
agent/chat_completion_helpers.py Pass to codex transport
agent/transports/codex.py GPT-5+ guard + kwargs injection
tests/agent/transports/test_codex_transport.py 19 tests
website/docs/user-guide/configuration.md Documentation

Regression risk

Low. Default is empty string (no injection). The if text_verbosity and isinstance(text_verbosity, str) guard ensures no injection happens unless
explicitly configured. Existing behaviour is preserved for all users who
don't set the config key.

Tests

19 tests covering:

  • Model guard: inject on 6 GPT-5+ variants, skip on 5 non-GPT-5 models
  • Bare gpt- prefix (no version): skip
  • Empty verbosity: no injection (2 tests)
  • Merge preservation: text.format + text.verbosity coexist
  • Vendor-prefixed models: inject on 3 prefixed GPT-5+, skip on prefixed GPT-4
scripts/run_tests.sh tests/agent/transports/test_codex_transport.py -q

79/79 passed (60 existing + 19 new). Zero regressions.

qxxaa added 2 commits July 6, 2026 15:18
…th control

Add a new config key `agent.text_verbosity` that injects the
`text.verbosity` parameter into OpenAI Responses API payloads for
GPT-5+ models. Valid values: "low", "medium", "high", or empty
string (default, no injection).

The OpenAI Responses API accepts text.verbosity to control output
length. Setting "low" produces noticeably shorter responses - measured
37% reduction in a controlled test (585 to 369 words on identical
prompts).

Implementation:
- Config default in hermes_cli/config.py (empty = no change)
- Plumbed through all agent init paths: CLI, gateway, API server, TUI
- Injected in codex_responses transport only (chat_completions and
  anthropic_messages transports are untouched)
- GPT-5+ model guard: parses major version from model name, injects
  only when >= 5. Strips vendor prefixes (openai/gpt-5.5) before
  matching. Non-GPT models silently skipped.
- Merges into existing text dict (preserves text.format from
  request_overrides)
- Hot-reload supported in gateway

19 tests cover: model guard (inject on GPT-5+, skip on GPT-4/Claude/
Grok/Gemini), empty default, merge preservation, vendor-prefixed
model names.
APIServerAdapter does not inherit GatewayRunner.__init__, so
self._text_verbosity was never set. Call GatewayRunner._load_text_verbosity()
directly, matching the existing pattern for reasoning_config.
@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API area/config Config system, migrations, profiles P3 Low — cosmetic, nice to have labels Jul 6, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Implements the OPEN feature request #20203 (expose the OpenAI Responses API text.verbosity as an agent.text_verbosity config key). Related to that issue — keeping it open as the feature spec/discussion thread.

@qxxaa

qxxaa commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favour of #29574, which takes a cleaner approach by loading text_verbosity inside init_agent() directly rather than threading it through every caller. Much smaller surface area for the same result.

Two things worth noting from this PR that #29574 could pick up:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants