feat: Add support for Responses API in OpenAI Compatible Provider while keeping the Chat Completions compatibility (with Azure Portal style base url support) #7355

Lagyu · 2025-08-23T16:04:42Z

Related GitHub Issue
Closes: #6862

Roo Code Task Context (Optional)
N/A

Description
This PR adds full Responses API support to the OpenAI Compatible provider while preserving all existing Chat Completions behavior. It addresses the “400 Unsupported parameter: 'messages'” error when users target the Responses endpoints (Azure and OpenAI) by building the correct payload and selecting the appropriate endpoint automatically.

Note that the manual toggle of the API style (auto or override with responses or chat completions) from settings was discussed in the original issue #6862 and I once implemented it, but I do not think that feature is required and removed. (Settings UI should be kept clean)

Key technical points:

API flavor selection (Auto/Responses/Chat)
- Auto-detect from base URL path:
  - Path includes or ends with “/responses” → Responses API
  - Path includes “/chat/completions” → Chat Completions
Responses API payload builder
- Convert input to a single string transcript (Developer: , User: ) per OpenAI Responses format (matches existing OpenAI Native handler)
- Azure naming: send max_output_tokens for Responses flavor when includeMaxTokens is enabled
Azure Responses (v1 preview)
- For Azure base URLs pointing to Responses, normalize SDK configuration to:
  - baseURL: https://{resource}.openai.azure.com/openai/v1
  - apiVersion: preview (azure only accept this in responses API flavor as far as I tested.)
- Portal-style URL is automatically converted into the documented v1 style:
  - Portal: https://{res}.openai.azure.com/openai/responses?api-version=2025-04-01-preview
    - I do not know why, but Azure Portal provides this style.
  - v1 style: https://{res}.openai.azure.com/openai/v1/responses?api-version=preview
    - Documented here and Azure OpenAI server only accepts this style
- Model parameter is the Azure deployment name; stream usage preserved where applicable
Backwards compatibility
- Chat Completions unchanged and remains the default for non-Responses URLs
- Existing settings (rate limit, temperature, includeMaxTokens, etc.) remain intact

Files Modified

Tests: payloads, URL patterns, Azure normalization, downgrade behavior on unsupported params
- src/api/providers/tests/openai.spec.ts
Types/settings: manual flavor selection option
- packages/types/src/provider-settings.ts

Test Procedure
Unit Tests (Vitest)

From workspace root, run:
- cd src; npx vitest run api/providers/tests/openai.spec.ts
Coverage:
- Auto-detect flavor
- Responses payload shape (string input transcript)
- Reasoning effort mapping (Responses vs Chat)
- Verbosity handling (include text.verbosity; retry on 400 unknown)
- Azure Responses normalization for both portal and v1 URLs
- Azure naming (max_output_tokens for Responses; max_completion_tokens for Chat)

Pre-Submission Checklist

Issue Linked: 400 Unsupported parameter: 'messages'. error on OpenAI Compatible Azure deployment of GPT-5 #6862
Scope: Focused on Responses API support + backward compatible Chat Completions behavior
Self-Review: Codepaths audited for regression and settings compatibility
Testing: Added/updated unit tests for payloads, URL detection, Azure normalization, downgrade behavior
Documentation Impact: Notes below; the new UI toggle should be mentioned

Screenshots / Videos
N/A (provider change; unit tests cover logic)

Documentation Updates
N/A

Additional Notes

Using Responses API flavor for GPT-5 is preferable to enable reuse of internal reasoning context and features like minimal reasoning
Chat Completions paths remain untouched to serve existing deployments seamlessly

Get in Touch
Discord: lagyu

Important

Adds Responses API support to OpenAI Compatible Provider with automatic endpoint selection and Azure-specific configurations.

Behavior:
- Adds support for Responses API in OpenAiHandler, auto-selecting endpoint based on URL path.
- Formats payload for Responses API, converting input to string transcript.
- Handles Azure-specific configurations, normalizing URLs and setting apiVersion to preview.
Functions:
- _resolveApiFlavor(), _formatResponsesInput(), _normalizeAzureResponsesBaseUrlAndVersion() added to openai.ts for API selection and payload formatting.
- _yieldResponsesResult() and _extractResponsesText() handle Responses API output.
Tests:
- Updated openai.spec.ts to test Responses API support, including URL normalization and payload verification.
- Tests for auto-detection of API flavor and Azure-specific behavior.

^{This description was created by}^{for f05544b. You can customize this summary. It will automatically update as commits are pushed.}

roomote

Thank you for your contribution! I've reviewed the changes and this is a solid implementation of Responses API support. The auto-detection logic is clever and the backward compatibility is well maintained. I have some suggestions for improvement, particularly around streaming support and type safety.

roomote · 2025-08-23T16:09:33Z

src/api/providers/openai.ts

+				payload.max_output_tokens = this.options.modelMaxTokens || modelInfo.maxTokens
+			}
+
+			// NOTE: Streaming for Responses API isn't covered by current tests.


The comment indicates streaming isn't covered by tests and defaults to non-streaming. Since streaming is a critical feature for user experience, could we consider implementing streaming support for the Responses API? If not feasible now, should we at least document this limitation more prominently?

I implemented the streaming based on the openai-native provider code.

roomote · 2025-08-23T16:09:33Z

src/api/providers/openai.ts

+			// NOTE: Streaming for Responses API isn't covered by current tests.
+			// We call non-streaming for now to preserve stable behavior.
+			try {
+				const response: any = await (this.client as any).responses.create(payload)


Is this intentional that we're casting to any here? The OpenAI client might not have a responses property. Should we add validation or a more graceful fallback?

I removed casting to any as much as I can.

roomote · 2025-08-23T16:09:33Z

src/api/providers/openai.ts

+		}
+	}
+
+	private _toResponsesInput(anthropicMessages: Anthropic.Messages.MessageParam[]): Array<{


This helper method _toResponsesInput appears to be unused. Is this intentional or leftover from development? If it's for future use, could we add a comment explaining its purpose?

I fixed to use it, allowing the image input with responses API.

roomote · 2025-08-23T16:09:33Z

src/api/providers/openai.ts

+				this._isAzureOpenAiResponses(baseURL)
+
+			// Always use 'preview' for Azure Responses API calls (per user requirement)
+			const azureVersion = isResponsesFlavor


The API version is hardcoded to "preview" for Azure Responses. Could this break when Azure releases a stable version? Should we make this configurable or at least add a comment explaining why "preview" is always used?

As of now, only preview version is available and I cannot do anything for it.

roomote · 2025-08-23T16:09:33Z

src/api/providers/openai.ts

+
+			// Verbosity: include via text.verbosity (Responses API expectation per openai-native handler)
+			if (this.options.verbosity || verbosity) {
+				;(payload as any).text = { verbosity: this.options.verbosity || verbosity }


Multiple uses of as any bypass TypeScript's type checking. Could we define proper types for the Responses API to improve type safety?

I removed casting to any as much as I can.

roomote · 2025-08-23T16:09:33Z

src/api/providers/openai.ts

+
+	// --- Responses helpers ---
+
+	private _resolveApiFlavor(baseUrl: string): "responses" | "chat" {


The auto-detection logic is clever but not immediately obvious. Could we add JSDoc comments explaining the URL pattern matching?

roomote · 2025-08-23T16:09:33Z

src/api/providers/__tests__/openai.spec.ts

+
+// -- Added Responses API tests (TDD) --
+
+describe("OpenAI Compatible - Responses API", () => {


Great test coverage for the Responses API! Consider adding a test for streaming support once it's implemented. Also, it might be helpful to test the error case when the OpenAI client doesn't support the Responses API.

Added tests for streaming support.

Lagyu · 2025-08-25T04:40:20Z

I found my implementation is not working with requests with Images.
Making fix for it and review comments from roomate.

… handling; tests - Add previous_response_id retry path on 400 “Previous response … not found” - Non-streaming and streaming: drop previous_response_id and retry once; clear continuity state - Code: [src/api/providers/openai.ts](src/api/providers/openai.ts:238), [src/api/providers/openai.ts](src/api/providers/openai.ts:291), guard [OpenAiHandler._isPreviousResponseNotFoundError()](src/api/providers/openai.ts:934) - Support GPT‑5-style reasoning summary and minimal effort on Responses API - Default enable summary: "auto" unless explicitly disabled in settings - Include reasoning: { effort: "minimal" | "low" | "medium" | "high", summary?: "auto" } - Code: constructor default [OpenAiHandler](src/api/providers/openai.ts:38), payload assembly [createMessage](src/api/providers/openai.ts:193) - Improve Responses streaming event coverage - Handle response.content_part.added (emit text) - Handle response.audio_transcript.delta (emit text as transcript) - Preserve response.id via stream callback for continuity - Code: [handleResponsesStream](src/api/transform/responses-stream.ts:91), [src/api/transform/responses-stream.ts](src/api/transform/responses-stream.ts:47), responseId callback [src/api/transform/responses-stream.ts](src/api/transform/responses-stream.ts:19) and usage in [openai.ts](src/api/providers/openai.ts:283) - Maintain conversation continuity for Responses API - Store lastResponseId on both streaming and non-streaming paths; pass previous_response_id unless suppressed - Code: stream wiring [src/api/providers/openai.ts](src/api/providers/openai.ts:283), non-streaming capture [src/api/providers/openai.ts](src/api/providers/openai.ts:889) - Update and extend tests - Add tests for 400 previous_response_id retry (streaming and non-streaming) - Add tests for content_part and audio_transcript events - Add tests for reasoning minimal + summary auto, and summary disabling - Adjust expectation to allow summary in reasoning payload - Tests: [src/api/providers/__tests__/openai.spec.ts](src/api/providers/__tests__/openai.spec.ts:1663), [src/api/providers/__tests__/openai.spec.ts](src/api/providers/__tests__/openai.spec.ts:1170) - Minor: default enableGpt5ReasoningSummary to true in compatible provider for Responses flows

Lagyu · 2025-08-25T07:38:01Z

Now I have fixed the problems.
I manually tested on Azure OpenAI GPT-5 + Responses API and works fine.

I think we need to refactor to unify the implementation of the OpenAI Native provider and the OpenAI Compatitive provider, but that should be handled in another request and now I mostly duplicated the code from the OpenAI Native` provider.

Lagyu · 2025-08-25T07:50:06Z

src/package.json

 		"node-ipc": "^12.0.0",
 		"ollama": "^0.5.17",
-		"openai": "^5.0.0",
+		"openai": "^5.15.0",


Updaetd to "openai": "^5.15.0" because verbosity parameter is added for Responses API request.

daniel-lxs · 2025-08-27T19:40:49Z

Thanks for the implementation! The Responses API support is working well, but before merging I think there are a few areas that could use some cleanup.

One thing I noticed is the error retry logic – the checks for _isPreviousResponseNotFoundError, _isVerbosityUnsupportedError, and _isInputTextInvalidError are repeated in multiple places (both in streaming and non-streaming paths). That duplication makes the code harder to maintain and could introduce bugs down the line.

Another area is type safety. There’s a lot of as unknown as casting, especially around _toResponsesInput and the client responses object. It feels like the type definitions could be tightened up so those casts aren’t necessary.

Lastly, createMessage has gotten pretty large (around 474 lines). It might make sense to pull the Responses API logic into separate methods to keep things easier to follow.

The functionality itself looks solid, I just think a bit of refactoring here would make the implementation easier to maintain in the long run.

…reateWithRetries; dedupe checks for previous_response_id, verbosity, and Azure input_text invalid in streaming and non-streaming paths

…ible-responses-api

…atible-responses-api # Conflicts: # pnpm-lock.yaml # src/api/providers/openai.ts # src/package.json

…gate from createMessage - Move Responses API logic to private _handleResponsesFlavor - Preserve streaming, retries, conversation continuity, reasoning/verbosity, and usage - All existing tests pass

Lagyu · 2025-09-05T08:50:59Z

@daniel-lxs Thank you for your feedback, and sorry for late reply.
I’ve addressed all three areas:

De-duplicated error/retry logic. Centralized the Responses error handling into a single _responsesCreateWithRetries path and removed the repeated checks for previous_response_id not found, verbosity unsupported, and Azure input_text invalid across both streaming and non-streaming flows.
Improved type safety. Reduced unnecessary as unknown as casts (especially around _toResponsesInput) and tightened the types where we interact with the client.responses object.
Split out createMessage. Extracted the Responses-specific logic into a private helper (_handleResponsesFlavor), so createMessage now delegates and is easier to follow. All existing tests pass.

Please take another look and let me know if you want any further changes.

…nuity (previous_response_id/store), temp/verbosity gating, and image support (input_image/output_text)

…nscript for text-only, array for multimodal; retry-on-verbosity; continuity handling

Feature/integrate OpenAI recent update

…am fails before id (store: true default)

Lagyu · 2025-09-05T12:11:38Z

I found conversation history in the session lost after retrying, so integrated upstream (openai-native.ts) changes from #7067.
Retry got better, and image handling should also be improved.

Tamrac-web · 2025-09-16T07:53:09Z

My friend, you are the true hero.

thomasmhofmann · 2025-09-17T10:55:15Z

Is there any chance to get this merged soonish? I have been using my own build based on the PR and it works fine for me.

daniel-lxs · 2025-09-17T14:24:19Z

I'm not too sure about this PR. It would be best to have 2 clear paths in the code when the Responses API is enabled or disabled. Maybe we can base it on the OpenAI native provider, which was migrated to the Responses API recently.

The idea would be to have a checkbox to enable the Responses API in the provider, in which case the path with the Response API logic would be used.

I'm closing this PR for now. I'll try to scope the issue. Feel free to continue the discussion.

Lagyu · 2025-09-18T01:00:38Z

Thank you for the review and discussion.

I also felt this implementation became more complex than necessary.

I think we should:

Unify shared logic with openai-native.ts, so that common pieces can live in one place.
Extract a provider-agnostic Responses API layer (shared with openai-native.ts), to clearly separate ResponsesAPI specific logic from provider wiring.

This should reduce duplication and make the codebase more maintainable in future.

Note: For those urgently need responses API with Azure OpenAI implementation now, you can clone, install and build this branch with pnpm vsix command.

Lagyu added 2 commits August 23, 2025 23:52

feat: Add support for responses API in Azure Compatible Provider.

b0673a9

feat: remove api flavor override from setting ui.

f05544b

Lagyu requested review from cte, jr and mrubens as code owners August 23, 2025 16:04

github-project-automation bot added this to Roo Code Roadmap and Roo Code Roadmap Aug 23, 2025

github-project-automation bot moved this to New in Roo Code Roadmap Aug 23, 2025

github-project-automation bot moved this to Triage in Roo Code Roadmap Aug 23, 2025

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. enhancement New feature or request labels Aug 23, 2025

Lagyu mentioned this pull request Aug 23, 2025

400 Unsupported parameter: 'messages'. error on OpenAI Compatible Azure deployment of GPT-5 #6862

Closed

roomote bot reviewed Aug 23, 2025

View reviewed changes

hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Aug 23, 2025

daniel-lxs moved this from Triage to PR [Needs Prelim Review] in Roo Code Roadmap Aug 23, 2025

hannesrudolph added PR - Needs Preliminary Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Aug 23, 2025

dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Aug 25, 2025

Lagyu added 3 commits August 25, 2025 16:12

chore: update openai package, to use the verbosity parameter.

08570ea

chore: update pnpm-lock.yaml

bc3661c

docs: add JSDoc describing the auto url detection logic

825c502

Lagyu commented Aug 25, 2025

View reviewed changes

fix: omit the conversation in responses api.

cd51254

Lagyu force-pushed the feature/openai-compatible-responses-api branch from c2aebf9 to cd51254 Compare August 27, 2025 03:13

chore: remove unnecessary type cast

bf49d77

daniel-lxs moved this from PR [Needs Prelim Review] to PR [Changes Requested] in Roo Code Roadmap Aug 27, 2025

hannesrudolph added PR - Changes Requested and removed PR - Needs Preliminary Review labels Aug 27, 2025

Lagyu added 5 commits August 29, 2025 18:03

refactor(openai): centralize Responses error handling via _responsesC…

48d1a61

…reateWithRetries; dedupe checks for previous_response_id, verbosity, and Azure input_text invalid in streaming and non-streaming paths

Merge remote-tracking branch 'origin/main' into feature/openai-compat…

aecb6b2

…ible-responses-api

chore: remove unnecessary type cast

eb25c45

Merge remote-tracking branch 'upstream/main' into feature/openai-comp…

6f449a2

…atible-responses-api # Conflicts: # pnpm-lock.yaml # src/api/providers/openai.ts # src/package.json

refactor(openai): extract Responses API handling into helper and dele…

1144bf9

…gate from createMessage - Move Responses API logic to private _handleResponsesFlavor - Preserve streaming, retries, conversation continuity, reasoning/verbosity, and usage - All existing tests pass

Lagyu and others added 4 commits September 5, 2025 18:28

fix(openai): Responses API parity with native structured input, conti…

43eaa3c

…nuity (previous_response_id/store), temp/verbosity gating, and image support (input_image/output_text)

test(openai): align Responses API payload shape with tests string tra…

0126f3a

…nscript for text-only, array for multimodal; retry-on-verbosity; continuity handling

Merge pull request #1 from Lagyu/feature/integrate-openai-recent-update

26ed0f5

Feature/integrate OpenAI recent update

test(openai): add regression for Responses continuity when prior stre…

848a0ed

…am fails before id (store: true default)

daniel-lxs closed this Sep 17, 2025

github-project-automation bot moved this from New to Done in Roo Code Roadmap Sep 17, 2025

github-project-automation bot moved this from PR [Changes Requested] to Done in Roo Code Roadmap Sep 17, 2025


		// --- Responses helpers ---

		private _resolveApiFlavor(baseUrl: string): "responses" \| "chat" {


		// -- Added Responses API tests (TDD) --

		describe("OpenAI Compatible - Responses API", () => {

feat: Add support for Responses API in OpenAI Compatible Provider while keeping the Chat Completions compatibility (with Azure Portal style base url support) #7355

feat: Add support for Responses API in OpenAI Compatible Provider while keeping the Chat Completions compatibility (with Azure Portal style base url support) #7355

Uh oh!

Conversation

Lagyu commented Aug 23, 2025 • edited by ellipsis-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

roomote bot left a comment

Choose a reason for hiding this comment

Uh oh!

roomote bot Aug 23, 2025

Choose a reason for hiding this comment

Uh oh!

Lagyu Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

roomote bot Aug 23, 2025

Choose a reason for hiding this comment

Uh oh!

Lagyu Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

roomote bot Aug 23, 2025

Choose a reason for hiding this comment

Uh oh!

Lagyu Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

roomote bot Aug 23, 2025

Choose a reason for hiding this comment

Uh oh!

Lagyu Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

roomote bot Aug 23, 2025

Choose a reason for hiding this comment

Uh oh!

Lagyu Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

roomote bot Aug 23, 2025

Choose a reason for hiding this comment

Uh oh!

roomote bot Aug 23, 2025

Choose a reason for hiding this comment

Uh oh!

Lagyu Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

Lagyu commented Aug 25, 2025

Uh oh!

Lagyu commented Aug 25, 2025

Uh oh!

Lagyu Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

daniel-lxs commented Aug 27, 2025

Uh oh!

Lagyu commented Sep 5, 2025

Uh oh!

Lagyu commented Sep 5, 2025

Uh oh!

Tamrac-web commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thomasmhofmann commented Sep 17, 2025

Uh oh!

daniel-lxs commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Lagyu commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

Lagyu commented Aug 23, 2025 •

edited by ellipsis-dev bot

Loading

Tamrac-web commented Sep 16, 2025 •

edited

Loading

daniel-lxs commented Sep 17, 2025 •

edited

Loading

Lagyu commented Sep 18, 2025 •

edited

Loading