Skip to content

feat(agent): add OpenAI Ultra multi-agent mode - #62156

Open
buddhaholic420 wants to merge 3 commits into
NousResearch:mainfrom
buddhaholic420:feat/openai-ultra-multi-agent
Open

feat(agent): add OpenAI Ultra multi-agent mode#62156
buddhaholic420 wants to merge 3 commits into
NousResearch:mainfrom
buddhaholic420:feat/openai-ultra-multi-agent

Conversation

@buddhaholic420

@buddhaholic420 buddhaholic420 commented Jul 10, 2026

Copy link
Copy Markdown

What changed

Adds an explicit OpenAI Ultra mode behind /reasoning ultra without sending the unsupported literal reasoning.effort: ultra value.

  • Direct api.openai.com Responses calls enable the Multi-agent beta for gpt-5.6, gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna.
  • The direct route sends ?beta=true, the responses_multi_agent=v1 header, multi_agent.enabled: true, a default concurrency of three, and reasoning.effort: max. Positive caller concurrency is preserved; no undocumented upper bound is imposed.
  • Unsupported Multi-agent fields are removed or rejected: literal ultra, reasoning.summary, and max_tool_calls never reach the wire, and unknown multi_agent override keys fail closed.
  • Multi-agent output is preserved in exact order for continuation, including multi_agent_call, multi_agent_call_output, agent_message, reasoning, messages, and agent-attributed function calls. Hosted actions never become Hermes client tools, and only /root text is rendered as the user-facing answer.
  • Codex app-server scans the paginated live model/list catalog once per subprocess, caches all advertised reasoning capabilities, and stops repeated cursors. Ultra still fails closed unless the selected model advertises it.
  • App-server turns send the explicitly selected model and effort every turn so Codex's sticky turn overrides cannot leave a later non-Ultra turn running in Ultra mode.
  • The normal ChatGPT Codex HTTP route, custom OpenAI-compatible endpoints, unsupported models, and non-Responses providers fail closed with actionable guidance.
  • CLI/gateway completion, localized validation help, configuration docs, and Codex app-server docs expose the mode.

Why

OpenAI documents Responses Multi-agent as similar to Ultra mode in Codex, but ultra is not a valid Responses reasoning effort. The public API requires its beta query/header/body contract, while Codex app-server exposes Ultra as a turn-level capability advertised by model/list.

This intentionally does not implement the generic max command-surface work in #61638 or provider projection in #61834. The only overlapping command/localization edits add the separate ultra value.

Usage

Direct OpenAI Responses API:

/reasoning ultra

ChatGPT/Codex subscription route:

/codex-runtime app_server
/reasoning ultra

The app-server path fails closed when the per-model capability catalog does not advertise ultra.

Verification

Tested on Windows 10 with Python 3.11.15, OpenAI Python SDK 2.24.0, and Codex CLI 0.144.0.

  • git diff --check: passed
  • ruff check on all changed Python files: passed
  • Final touched behavior matrix: 680 passed, 1 skipped
  • Late-review regressions cover sticky Ultra reset, one-scan catalog caching, repeated-cursor termination, mapping-backed SDK items, cyclic provider values, and Windows-safe SessionDB cleanup
  • Docusaurus production build: passed; only existing repository link/translation warnings were emitted
  • OpenAI SDK in-memory HTTP transport verified /responses?beta=true, the beta header/body, strongest-effort projection, absence of unsupported fields, and serialization of all hosted replay item types
  • Deterministic interleaved-stream tests verified subagent text is retained for replay but excluded from the root answer
  • Live Codex app-server smoke: gpt-5.6-sol advertised ultra, accepted the Ultra turn, returned ULTRA_OK, and the subprocess exited cleanly
  • Public artifact plus TruffleHog scan of the exact PR patch: passed with zero findings

Safety and compatibility

Ultra is opt-in. Existing reasoning levels remain unchanged. App-server turns now carry the selected model and non-empty effort explicitly to prevent sticky cross-turn overrides. The public beta route is limited to the four documented public model identifiers on api.openai.com; custom endpoints and unsupported providers/models remain fail closed.

Closes #61634

Copilot AI review requested due to automatic review settings July 10, 2026 15:36
@buddhaholic420
buddhaholic420 force-pushed the feat/openai-ultra-multi-agent branch from d133ff4 to 508cf77 Compare July 10, 2026 15:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in “Ultra” reasoning mode (/reasoning ultra) that enables OpenAI’s hosted Multi-agent Responses beta (or Codex app-server Ultra turns) without ever sending the unsupported literal reasoning.effort: "ultra" on the wire. The change updates transport request-shaping, app-server capability validation, CLI/gateway surfaces, docs, locales, and adds regression tests.

Changes:

  • Implement Ultra request shaping for direct api.openai.com Responses calls (Multi-agent beta header/body, strongest supported effort projection, hosted tool budget defaults, and fail-closed behavior).
  • Add Codex app-server Ultra capability validation via live model/list, and forward model + effort: ultra only when advertised.
  • Expose ultra in CLI/gateway command surfaces, completions, docs/locales, and add test coverage for both routes and response normalization.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
agent/chat_completion_helpers.py Adds fail-closed guard when Ultra is selected on unsupported API modes.
agent/codex_responses_adapter.py Preserves hosted multi-agent output items and ensures they are not dispatched as Hermes tools.
agent/codex_runtime.py Forwards model + reasoning effort into Codex app-server turns.
agent/transports/codex.py Implements Ultra → OpenAI Multi-agent beta request shaping (headers/body/effort projection).
agent/transports/codex_app_server_session.py Adds live model/list capability checks and forwards model/effort for Ultra turns.
gateway/run.py Updates gateway config reasoning-effort docstring to include ultra.
gateway/slash_commands.py Accepts /reasoning ultra in gateway command parsing.
hermes_cli/cli_commands_mixin.py Updates CLI /reasoning help/validation strings to include ultra.
hermes_cli/commands.py Adds ultra to /reasoning subcommand completion list.
hermes_constants.py Extends VALID_REASONING_EFFORTS / parsing to accept ultra.
locales/af.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/de.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/en.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/es.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/fr.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/ga.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/hu.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/it.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/ja.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/ko.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/pt.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/ru.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/tr.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/uk.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/zh-hant.yaml Updates localized /reasoning unknown-arg help to include ultra.
locales/zh.yaml Updates localized /reasoning unknown-arg help to include ultra.
tests/agent/test_codex_app_server_persist.py Ensures app-server persistence path forwards model + ultra reasoning effort.
tests/agent/test_codex_responses_adapter.py Adds coverage for preserving hosted multi-agent telemetry items and tool continuation.
tests/agent/transports/test_codex_app_server_session.py Verifies Ultra is forwarded only when live catalog advertises it; otherwise fail-closed.
tests/agent/transports/test_codex_transport.py Adds Ultra request-shaping tests (beta headers, effort projection, overrides, fail-closed).
tests/cli/test_reasoning_command.py Updates CLI parsing tests to accept ultra.
tests/gateway/test_reasoning_command.py Adds gateway /reasoning ultra acceptance test (session-scoped).
tests/hermes_cli/test_commands.py Adds CLI completion test ensuring ultra is suggested.
tests/run_agent/test_run_agent_codex_responses.py Adds agent-level _build_api_kwargs tests for Ultra request shaping and fail-closed behavior.
tests/test_hermes_constants.py Updates constants documentation guard test to include ultra.
website/docs/user-guide/configuration.md Documents /reasoning ultra behavior and supported routes in user configuration docs.
website/docs/user-guide/features/codex-app-server-runtime.md Documents Ultra behavior on the Codex app-server runtime.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread agent/codex_responses_adapter.py Outdated
Comment thread website/docs/user-guide/features/codex-app-server-runtime.md Outdated
Comment thread website/docs/user-guide/configuration.md Outdated
@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery comp/cli CLI entry point, hermes_cli/, setup wizard provider/openai OpenAI / Codex Responses API P3 Low — cosmetic, nice to have labels Jul 10, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough transport, replay, and capability-gating coverage. One state-reset path remains blocking.

Problems

  • agent/codex_runtime.py:390-401 maps {enabled: false} to reasoning_effort=None. agent/transports/codex_app_server_session.py:516-528 then omits turn/start.effort. Since /reasoning none is represented as {enabled: false} (gateway/slash_commands.py:2734-2735), an Ultra → none transition on the same app-server thread leaves no explicit replacement for the sticky Ultra override. The new regression only covers Ultra → high (tests/agent/transports/test_codex_app_server_session.py:302-348).

Suggested changes

  • Send an explicit supported disabled/default replacement (or reset the relevant app-server state) for disabled reasoning, and add an Ultra → none regression asserting the second turn/start cannot retain Ultra.

Automated hermes-sweeper review.

Comment thread agent/codex_runtime.py
reasoning_effort = None
if (
isinstance(reasoning_config, dict)
and reasoning_config.get("enabled") is not False

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/reasoning none produces {enabled: false}, so this guard leaves reasoning_effort unset. CodexAppServerSession.run_turn() then omits turn/start.effort, allowing a reused app-server thread to retain the prior sticky Ultra override. Please explicitly clear or replace the effort for the disabled case and add an Ultra → none regression.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: GPT-5.6 reasoning support is incomplete: gateway/CLI block max; Ultra needs Multi-agent integration

5 participants