fix(xai): OAuth Responses native web_search, incomplete guard, grok-composer 262k context

[XVVH]

Summary

Fixes grok-composer-2.5-fast (and other xAI OAuth / codex_responses) sessions that failed web-research turns or showed a 131k context cap.

Supersedes the earlier context-only PR #40046 (closed while we finished root-causing search). This is one consolidated change: metadata + transport + adapter.

Problems

1. Context: grok-composer-2.5-fast is absent from GET /v1/models on OAuth; Hermes fell through to the generic grok → 131072 substring rule. Live /v1/responses enforces ~262144 total (input+output). Users hit premature compression and a misleading context UI.

2. Incomplete loop: When Hermes declared web_search as a client function, xAI routed to server-side search without a matching {"type":"web_search"} built-in. Streams often ended reasoning-only → finish_reason="incomplete" → three continuation retries → Codex response remained incomplete after 3 continuation attempts.

3. Declaration: xAI returns HTTP 400 on duplicate tool names if both client web_search and native built-in are sent. Fix: for is_xai_responses, drop the client function and inject native {"type":"web_search"} (same path for composer and grok-4.x).

Changes

Area	What
agent/model_metadata.py	"grok-composer": 262144
agent/transports/codex.py	Native web_search inject + drop client web_search for xAI Responses
agent/codex_responses_adapter.py	Ignore in-flight server-side *_call items (e.g. web_search_call) when judging incomplete; preflight allowlist for built-in tool types

Tests

uv run pytest tests/agent/test_codex_responses_adapter.py \
  tests/agent/transports/test_codex_transport.py::TestCodexBuildKwargs \
  tests/run_agent/test_codex_xai_oauth_recovery.py -q

118 passed on this branch.

Trade-off (explicit)

On xAI Responses, web_search uses Grok's native search, not Hermes Tavily/citation plumbing. Results appear in the assistant message rather than as a client-observed tool result. Scoped to is_xai_responses only; other providers unchanged.

Verification

• Exercised on SuperGrok OAuth: previously failing composer search turns complete with sourced answers.
• Post-merge smoke on a production gateway (Telegram): composer web search OK.

Related

• Root-cause write-up aligned with issues around xAI server-side tools and Codex incomplete handling.
• Replaces closed fix(xai): resolve grok-composer-2.5-fast context for OAuth #40046 (context metadata only).

[liuhao1024]

Verification review — Reviewed the full diff (7 files, +310/-2) for xAI OAuth Responses native web_search support.

What I verified:

1. ✅ _RESPONSES_BUILTIN_TOOL_TYPES declaration — the set correctly lists all provider-executed built-in tool types (web_search, web_search_preview, file_search, code_interpreter, image_generation, computer_use_preview, local_shell). The preflight validator now passes these through verbatim instead of rejecting with "unsupported type".
2. ✅ _SERVER_SIDE_TOOL_CALL_TYPES guard — server-side *_call items with status="in_progress" no longer flip has_incomplete_items. The guard correctly scopes to server-side types only; an in_progress message item (genuine model output still streaming) still marks the turn incomplete. This prevents the 3-fruitless-retry burn observed with grok-composer-2.5-fast.
3. ✅ Client-side web_search filtering — when is_xai_responses is true, the client-side web_search function tool is removed and replaced with {"type": "web_search"} native declaration. The comment correctly notes this routes search to Grok's native engine instead of Hermes's configured web provider (Tavily/etc.), and results bypass tool-trace/citation plumbing.
4. ✅ grok-composer context window — 262144 tokens added to _MODEL_CONTEXT_WINDOWS with correct substring matching. Test updated.
5. ✅ Test coverage — 4 new test functions covering: in_progress server-side tool calls ignored, in_progress messages still incomplete, native web_search passes preflight, unknown tool types still rejected.

Clean implementation. The inline comments thoroughly document the exact failure mode (incomplete status not reconciled by xAI's /v1/responses surface) and the architectural tradeoff (native search bypasses Hermes tool-trace).

[liuhao1024]

✅ Code Review — Clean

Reviewed the full diff across 5 files (codex_responses_adapter.py, model_metadata.py, codex.py, and 3 test files).

What I verified:

• Server-side tool call exemption: _SERVER_SIDE_TOOL_CALL_TYPES correctly scopes the incomplete-status exemption to provider-executed *_call types only — genuine message items with in_progress status still correctly mark the turn incomplete (tested in test_normalize_codex_response_in_progress_message_still_incomplete)
• Native web_search injection: The xAI path correctly drops client-side web_search function tools before appending {"type": "web_search"} built-in, preventing HTTP 400 "Duplicate tool names" (tested in test_xai_drops_clientside_web_search_to_avoid_duplicate)
• Non-xAI scoping: The injection is gated on is_xai_responses — Codex/GitHub paths keep client-side web_search untouched (tested in test_non_xai_path_does_not_inject_native_web_search)
• Preflight validation: _RESPONSES_BUILTIN_TOOL_TYPES allows built-in tool types through preflight without forcing them through function-tool validation (tested in test_preflight_passes_native_web_search_tool_through + test_preflight_still_rejects_unknown_tool_type)
• Lambda capture correctness: In build_kwargs, the filtered list comprehension and append happen before any async dispatch, so no late-binding risk

The _SERVER_SIDE_TOOL_CALL_TYPES set includes mcp_call — this is safe because MCP tool calls from xAI's server side would also have the stale in_progress status issue. Client-side MCP function calls use type="function" and are not affected.

No issues found. LGTM.

[teknium1]

Merged via #48108 — your commit was cherry-picked onto current main with your authorship preserved, plus two review-driven corrections:

1. Native web_search injection is now a 1:1 swap for an already-present client web_search, not an additive grant. The original appended {"type":"web_search"} on every is_xai_responses turn with any tools, which force-enabled Grok server-side search even when the user never enabled the web toolset (silently bypassing the configured web provider + tool-trace/citation plumbing). Now gated on a client web_search actually being present, with a no-web-toolset guard test.
2. grok-composer context reconciled to 200000 (xAI's published usable window for Composer 2.5; 262144 is the /v1/responses input+output budget). This also avoided a duplicate-key conflict with feat(xai): surface Composer 2.5 in the xAI OAuth model picker #47908 which already merged 200000.

The incomplete-guard (server-side *_call items at in_progress no longer flipping the incomplete verdict) and the preflight built-in-tool allowlist — the real heart of the fix — went in as-is. Solid root-causing on the stalled-search-loop bug. Thanks! #48108