Skip to content

Add proxy support for OpenAI Responses API endpoint#2

Merged
ErwinsExpertise merged 3 commits intomainfrom
copilot/add-proxy-support-responses-api
Nov 4, 2025
Merged

Add proxy support for OpenAI Responses API endpoint#2
ErwinsExpertise merged 3 commits intomainfrom
copilot/add-proxy-support-responses-api

Conversation

Copy link

Copilot AI commented Nov 4, 2025

vLLM added support for the Responses API (/v1/responses) in PR #20504. This endpoint provides enhanced agentic capabilities but was not proxied by KubeAI.

Changes

  • Route registration (internal/openaiserver/handler.go): Added /openai/v1/responses endpoint handler
  • Request parsing (internal/apiutils/request.go): Added /v1/responses path case, reusing ChatCompletionRequest structure (matches vLLM's implementation)
  • Test coverage (internal/modelproxy/handler_test.go): Added TestHandlerResponsesAPI to verify routing, model lookup, and request proxying

The Responses API follows the same request format as Chat Completions, so existing proxy logic (model routing, load balancing, retries, adapter handling) applies without modification.

// Request to /openai/v1/responses now properly routes through KubeAI
handle("/openai/v1/responses", http.StripPrefix("/openai", modelProxy))
Original prompt

This section details on the original issue you should resolve

<issue_title>Responses API Support</issue_title>
<issue_description>vLLM recently added support for the new Responses API, which provides more flexible and agentic capabilities compared to the older endpoints. To take advantage of this improvement, KubeAI should add proxy support for the Responses API endpoint. This would allow users to access the new functionality seamlessly through KubeAI, enabling more advanced and interactive workflows. Adding this support ensures that users can leverage the enhanced features of vLLM’s Responses API.
vllm-project/vllm#20504

Relevant section in KubeAI router:
https://github.com/ErwinsExpertise/kubeai/blob/main/internal/openaiserver/handler.go#L38</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@ErwinsExpertise ErwinsExpertise marked this pull request as ready for review November 4, 2025 02:13
Copilot AI review requested due to automatic review settings November 4, 2025 02:14
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI and others added 2 commits November 4, 2025 02:15
Co-authored-by: ErwinsExpertise <44278151+ErwinsExpertise@users.noreply.github.com>
Co-authored-by: ErwinsExpertise <44278151+ErwinsExpertise@users.noreply.github.com>
Copilot AI changed the title [WIP] Add proxy support for Responses API in KubeAI Add proxy support for OpenAI Responses API endpoint Nov 4, 2025
Copilot AI requested a review from ErwinsExpertise November 4, 2025 02:21
@ErwinsExpertise ErwinsExpertise merged commit 8f6a17e into main Nov 4, 2025
@ErwinsExpertise ErwinsExpertise deleted the copilot/add-proxy-support-responses-api branch November 4, 2025 02:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Responses API Support

3 participants