Managed OpenAI Inference + Native Web Search

[noanflaherty]

Summary

Enable OpenAI as a first-class managed inference provider and route inference-provider-native web search through OpenAI's native Responses web search tool. Remove Anthropic-only assumptions in macOS settings so users can choose OpenAI in managed mode.

Self-review result

PASS after 3 rounds of review+remediation (4 fix PRs addressing integration gaps)

PRs merged into feature branch

• [managed] Enable OpenAI managed-proxy fallback routing in provider bootstrap #26211: [managed] Enable OpenAI managed-proxy fallback routing in provider bootstrap
• [openai] Route inference-provider-native web search to Responses native web_search tool #26212: [openai] Route inference-provider-native web search to Responses native web_search tool
• [macOS] Allow managed inference provider selection beyond Anthropic #26218: [macOS] Allow managed inference provider selection beyond Anthropic
• [macOS] Allow provider-native web search when managed inference provider supports it #26230: [macOS] Allow provider-native web search when managed inference provider supports it

Fix PRs

• fix: emit server_tool_start/complete events for OpenAI native web search #26240: fix: emit server_tool_start/complete events for OpenAI native web search
• fix: render web_search_call output items in LLM context diagnostics #26243: fix: render web_search_call output items in LLM context diagnostics
• fix: add server_tool_use content blocks and tests for OpenAI native web search #26246: fix: add server_tool_use content blocks and tests for OpenAI native web search
• fix: diagnostics polish — web_search_call tests, tool count fix, loop consistency #26250: fix: diagnostics polish — web_search_call tests, tool count fix, loop consistency

Part of plan: managed-openai-native-web-search.md

---

[noanflaherty]

Review (Worktree + Targeted Validation)

Pulled this PR into a dedicated worktree and reviewed all touched files end-to-end (backend + macOS settings + tests).

Findings

1) [P1] Persisting unpaired server_tool_use from OpenAI native search can inject synthetic failures after provider switch

• In assistant/src/providers/openai/responses-provider.ts (content assembly around lines 301-312), we now persist server_tool_use blocks for OpenAI web search calls.
• Those blocks are persisted without a paired web_search_tool_result.
• Anthropic’s request repair path explicitly treats this as orphaned tool state and injects synthetic web_search_tool_result_error entries (assistant/src/providers/anthropic/client.ts around lines 346-399).

Why this matters:
A conversation that used OpenAI native web search can be silently rewritten to include synthetic “search unavailable” failures when the user later switches provider to Anthropic in the same thread.

Suggested fix:

• Either avoid persisting OpenAI server_tool_use blocks in history (keep event-stream + raw diagnostics only),
• Or persist a paired result block so cross-provider handoff does not synthesize failure artifacts.

---

2) [P2] server_tool_complete is always emitted as success for OpenAI web search

• In assistant/src/providers/openai/responses-provider.ts (lines ~285-291), server_tool_complete is emitted with isError: false for every tracked web-search call on response.completed.
• This happens regardless of response/call status.

Why this matters:
Tool telemetry/UI can report successful completion even when the search call is interrupted/failed/incomplete.

Suggested fix:

• Derive completion state from web-search call status (stream status events or final response output item status),
• Emit isError: true when status is non-success,
• Add a regression test for non-completed web search status.

---

If helpful, I can also add a concrete regression test scenario for the OpenAI->Anthropic provider-switch path described in Finding #1.

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 6 additional findings in Devin Review.

[devin-ai-integration]

🚩 OpenAI managed proxy requires companion platform-side route support

The new proxy path /v1/runtime-proxy/openai at assistant/src/providers/managed-proxy/constants.ts:32 requires the companion vellum-assistant-platform repo to have a corresponding proxy route that forwards OpenAI Responses API requests. The AGENTS.md mentions checking the sibling repo for compatibility when making HTTP API or container changes. The OpenAI SDK will construct URLs like {baseURL}/responses for the Responses API stream endpoint, so the platform proxy needs to handle this path correctly.

---

Was this helpful? React with 👍 or 👎 to provide feedback.