feat(codex): add agent.text_verbosity config for GPT-5+ response length control - #59711
Closed
qxxaa wants to merge 2 commits into
Closed
feat(codex): add agent.text_verbosity config for GPT-5+ response length control#59711qxxaa wants to merge 2 commits into
agent.text_verbosity config for GPT-5+ response length control#59711qxxaa wants to merge 2 commits into
Conversation
…th control Add a new config key `agent.text_verbosity` that injects the `text.verbosity` parameter into OpenAI Responses API payloads for GPT-5+ models. Valid values: "low", "medium", "high", or empty string (default, no injection). The OpenAI Responses API accepts text.verbosity to control output length. Setting "low" produces noticeably shorter responses - measured 37% reduction in a controlled test (585 to 369 words on identical prompts). Implementation: - Config default in hermes_cli/config.py (empty = no change) - Plumbed through all agent init paths: CLI, gateway, API server, TUI - Injected in codex_responses transport only (chat_completions and anthropic_messages transports are untouched) - GPT-5+ model guard: parses major version from model name, injects only when >= 5. Strips vendor prefixes (openai/gpt-5.5) before matching. Non-GPT models silently skipped. - Merges into existing text dict (preserves text.format from request_overrides) - Hot-reload supported in gateway 19 tests cover: model guard (inject on GPT-5+, skip on GPT-4/Claude/ Grok/Gemini), empty default, merge preservation, vendor-prefixed model names.
APIServerAdapter does not inherit GatewayRunner.__init__, so self._text_verbosity was never set. Call GatewayRunner._load_text_verbosity() directly, matching the existing pattern for reasoning_config.
Collaborator
Implements the OPEN feature request #20203 (expose the OpenAI Responses API |
Contributor
Author
|
Closing in favour of #29574, which takes a cleaner approach by loading Two things worth noting from this PR that #29574 could pick up:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
GPT-5+ models are verbose by default. In agent workflows, this has real
cost: every output token is billed, and longer responses increase latency
on subsequent turns by inflating the conversation history that gets
re-sent as input tokens. Over a multi-turn agent session, verbose output
compounds - each turn's excess output becomes the next turn's excess input.
The OpenAI Responses API accepts a
text.verbosityparameter ("low","medium","high") that controls output length for GPT-5+ models. Thereis currently no way to configure this in Hermes.
I initially explored using
agent.request_overridesin config.yaml toinject the parameter without any code changes. While the SDK accepts
textas a first-class parameter on
responses.create(),request_overridesisan internal runtime dict populated only by fast mode and custom provider
logic - it is never read from config.yaml. Settings placed there are
silently ignored.
Measured impact: setting
text.verbosity: "low"on GPT-5.5 reduced outputfrom 585 words to 369 words (37% reduction) on identical prompts. In a
typical 20-turn agent session, that reduction compounds across every turn -
shorter assistant responses mean fewer input tokens on every subsequent API
call, reducing both cost and time-to-first-token latency.
Fix
New config key
agent.text_verbosity:Default is empty string - zero behaviour change for anyone who doesn't
set it. No
textparameter is sent, and the model uses its own defaultverbosity.
When set, injects
text: {"verbosity": "<value>"}into the Responses APIpayload. Scoped to GPT-5+ models on the
codex_responsestransport only.Design decisions
Model guard: Parses
gpt-X.Yfrom the model name, extracts majorversion, injects only when >= 5. Strips vendor prefixes (
openai/gpt-5.5->
gpt-5.5) before matching. Non-GPT models (Claude, Grok, Gemini) aresilently skipped - no errors, no injection.
Dict merge: Merges
verbosityinto any existingtextdict rather thanreplacing it. If
request_overridessetstext.format, bothformatandverbositycoexist in the final payload.Transport scope: Only the
codex_responsestransport injects it. Chatcompletions and Anthropic message transports are untouched - the parameter
is Responses API specific.
No validation: Consistent with
reasoning_efforthandling. Invalidvalues pass through to the API, which returns a clear error. No client-side
allowlist.
Files changed (13)
hermes_cli/config.py""in agent defaultsagent/agent_init.pyrun_agent.pycli.pyhermes_cli/cli_agent_setup_mixin.pyhermes_cli/cli_commands_mixin.pygateway/run.pygateway/platforms/api_server.pytui_gateway/server.pyagent/chat_completion_helpers.pyagent/transports/codex.pytests/agent/transports/test_codex_transport.pywebsite/docs/user-guide/configuration.mdRegression risk
Low. Default is empty string (no injection). The
if text_verbosity and isinstance(text_verbosity, str)guard ensures no injection happens unlessexplicitly configured. Existing behaviour is preserved for all users who
don't set the config key.
Tests
19 tests covering:
gpt-prefix (no version): skiptext.format+text.verbositycoexist79/79 passed (60 existing + 19 new). Zero regressions.