Skip to content

fix(provider): route Azure GPT models through Chat Completions #3818

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kojiwakayama merged 1 commit into from
Aug 17, 2026
Merged

fix(provider): route Azure GPT models through Chat Completions #3818

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions docs/api-reference/veryfront/provider.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Clear all registered model providers and reset lazy built-ins (for testing).
| Name | Description | Source |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `DEFAULT_VERYFRONT_CLOUD_MODEL_ID` | Default Veryfront Cloud model ID used when no model is configured. Update this when the current default is deprecated - otherwise the default path silently breaks for users who have not set an explicit model. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L48) |
| `VERYFRONT_CLOUD_CHAT_MODELS` | Shared Veryfront Cloud chat models value. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L234) |
| `VERYFRONT_CLOUD_CHAT_MODELS` | Shared Veryfront Cloud chat models value. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L244) |
| `VERYFRONT_CLOUD_MODEL_PREFIX` | Shared Veryfront Cloud model prefix value. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L50) |

### Functions
Expand All @@ -75,26 +75,26 @@ Clear all registered model providers and reset lazy built-ins (for testing).
| ---------------------------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `clearModelProviders` | Clear all registered model providers and reset lazy built-ins (for testing). | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/model-registry.ts#L402) |
| `ensureModelReady` | Eagerly verify that the resolved model's runtime is available. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/model-registry.ts#L389) |
| `findVeryfrontCloudModel` | Find Veryfront Cloud model. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L266) |
| `findVeryfrontCloudModelByModelId` | Find Veryfront Cloud model by model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L278) |
| `findVeryfrontCloudModel` | Find Veryfront Cloud model. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L276) |
| `findVeryfrontCloudModelByModelId` | Find Veryfront Cloud model by model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L288) |
| `getCurrentVeryfrontCloudContext` | | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/context.ts#L31) |
| `getRegisteredModelProviders` | Get provider names available in the current scope. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/model-registry.ts#L377) |
| `getVeryfrontCloudBootstrap` | | [source](https://github.com/veryfront/veryfront-code/blob/main/src/platform/cloud/resolver.ts#L121) |
| `getVeryfrontCloudProviderFromModelId` | Return Veryfront Cloud provider from model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L286) |
| `groupVeryfrontCloudModelsByProvider` | Group Veryfront Cloud models by provider. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L477) |
| `getVeryfrontCloudProviderFromModelId` | Return Veryfront Cloud provider from model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L296) |
| `groupVeryfrontCloudModelsByProvider` | Group Veryfront Cloud models by provider. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L487) |
| `hasModelProvider` | Check whether a model provider is available in the current scope. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/model-registry.ts#L363) |
| `markCurrentVeryfrontCloudBillingGroupUsed` | | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/context.ts#L35) |
| `normalizeVeryfrontCloudModelId` | Normalizes Veryfront Cloud model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L271) |
| `normalizeVeryfrontCloudModelId` | Normalizes Veryfront Cloud model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L281) |
| `registerModelProvider` | Register a custom model provider factory for the active project scope or application bootstrap. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/model-registry.ts#L119) |
| `resolveModel` | Resolve a "provider/model" string to a framework-compatible model runtime. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/model-registry.ts#L304) |
| `resolveVeryfrontCloudGatewayModelId` | Resolves Veryfront Cloud gateway model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L337) |
| `resolveVeryfrontCloudModelId` | Resolves Veryfront Cloud model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L311) |
| `resolveVeryfrontCloudModelThinking` | Resolves Veryfront Cloud model thinking. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L363) |
| `resolveVeryfrontCloudReasoningOption` | Resolves provider-neutral runtime reasoning for a Veryfront Cloud model. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L383) |
| `resolveVeryfrontCloudThinkingProviderOptions` | Options accepted by resolve Veryfront Cloud thinking provider. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L417) |
| `resolveVeryfrontCloudGatewayModelId` | Resolves Veryfront Cloud gateway model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L347) |
| `resolveVeryfrontCloudModelId` | Resolves Veryfront Cloud model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L321) |
| `resolveVeryfrontCloudModelThinking` | Resolves Veryfront Cloud model thinking. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L373) |
| `resolveVeryfrontCloudReasoningOption` | Resolves provider-neutral runtime reasoning for a Veryfront Cloud model. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L393) |
| `resolveVeryfrontCloudThinkingProviderOptions` | Options accepted by resolve Veryfront Cloud thinking provider. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L427) |
| `runWithVeryfrontCloudContext` | Context for run with Veryfront Cloud. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/context.ts#L16) |
| `runWithVeryfrontCloudContextAsync` | Run with Veryfront Cloud context async. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/context.ts#L24) |
| `tryGetVeryfrontCloudProviderFromModelId` | Try to get Veryfront Cloud provider from model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L300) |
| `tryGetVeryfrontCloudProviderFromModelId` | Try to get Veryfront Cloud provider from model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L310) |

### Types

Expand All @@ -113,7 +113,7 @@ Clear all registered model providers and reset lazy built-ins (for testing).

| Name | Description | Source |
| ------------------------------------ | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `resolveHostedVeryfrontCloudModelId` | Resolves hosted Veryfront Cloud model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L492) |
| `resolveHostedVeryfrontCloudModelId` | Resolves hosted Veryfront Cloud model ID. | [source](https://github.com/veryfront/veryfront-code/blob/main/src/provider/veryfront-cloud/model-catalog.ts#L502) |

## Deep imports

Expand Down
44 changes: 44 additions & 0 deletions extensions/ext-llm-openai/src/openai-provider.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1008,6 +1008,50 @@ describe("openai-provider", () => {
assertEquals(chatResult.content, [{ type: "text", text: "Chat response" }]);
});

it("rejects hosted search when Chat Completions is explicitly configured", async () => {
let fetchCalled = false;
const provider = new OpenAIProvider();
const runtime = provider.createModel("gpt-5.4", {
credential: "test-openai-key",
baseURL: "https://example.openai.test/v1",
openAITransport: "chat-completions",
fetch: () => {
fetchCalled = true;
return Promise.resolve(new Response("{}", { status: 200 }));
},
});

await assertRejects(
() =>
runtime.doGenerate({
prompt: [{ role: "user", content: [{ type: "text", text: "Research Veryfront." }] }],
tools: [{
type: "provider",
name: "web_search",
id: "openai.web_search",
args: {},
}],
}),
TypeError,
"OpenAI hosted tools require the Responses API",
);
await assertRejects(
() =>
runtime.doStream({
prompt: [{ role: "user", content: [{ type: "text", text: "Research Veryfront." }] }],
tools: [{
type: "provider",
name: "web_search",
id: "openai.web_search",
args: {},
}],
}),
TypeError,
"OpenAI hosted tools require the Responses API",
);
assertEquals(fetchCalled, false);
});

it("keeps OpenAI-compatible provider identity separate from display labels", async () => {
const encoder = new TextEncoder();
let requestedUrl = "";
Expand Down
47 changes: 45 additions & 2 deletions extensions/ext-llm-openai/src/openai-provider.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,17 @@ function getLLMOpenAIProviderName(config: LLMProviderConfig): string {
return normalizeOpenAIProviderName(config.providerName) ?? "openai";
}

type OpenAIModelTransport = "auto" | "chat-completions" | "responses";

function getOpenAIModelTransport(config: LLMProviderConfig): OpenAIModelTransport {
const transport = config.openAITransport;
if (transport === undefined) return "auto";
if (transport === "chat-completions" || transport === "responses") {
return transport;
}
throw new TypeError('OpenAI transport must be "chat-completions" or "responses"');
}

type OpenAICompatibleProviderKind = "openai" | "mistral" | "moonshotai";

type OpenAIResponseContext = {
Expand Down Expand Up @@ -1218,6 +1229,30 @@ function requestUsesOpenAIHostedTool(optionsForRuntime: OpenAICompatibleLanguage
) !== undefined;
}

function createOpenAIChatCompletionsOnlyRuntime(
chatRuntime: ModelRuntime<OpenAICompatibleLanguageOptions, RuntimeAssistantContentPart>,
): ModelRuntime<OpenAICompatibleLanguageOptions, RuntimeAssistantContentPart> {
function assertHostedToolsSupported(options: OpenAICompatibleLanguageOptions): void {
if (requestUsesOpenAIHostedTool(options)) {
throw new TypeError(
"OpenAI hosted tools require the Responses API and are unavailable with Chat Completions",
);
}
}

return {
...chatRuntime,
async doGenerate(optionsForRuntime: OpenAICompatibleLanguageOptions) {
assertHostedToolsSupported(optionsForRuntime);
return await chatRuntime.doGenerate(optionsForRuntime);
},
async doStream(optionsForRuntime: OpenAICompatibleLanguageOptions) {
assertHostedToolsSupported(optionsForRuntime);
return await chatRuntime.doStream(optionsForRuntime);
},
};
}

function createOpenAIAdaptiveModelRuntime(
chatRuntime: ModelRuntime<OpenAICompatibleLanguageOptions, RuntimeAssistantContentPart>,
responsesRuntime: ModelRuntime<OpenAICompatibleLanguageOptions, RuntimeAssistantContentPart>,
Expand Down Expand Up @@ -1305,12 +1340,20 @@ export class OpenAIProvider implements LLMProvider {
runtimeConfig,
modelId,
);
if (isOpenAIReasoningModel(modelId, providerName)) {
const chatRuntime = createOpenAIModelRuntime(runtimeConfig, modelId);
const transport = getOpenAIModelTransport(config);
if (
transport === "responses" ||
(transport === "auto" && isOpenAIReasoningModel(modelId, providerName))
) {
return responsesRuntime;
}
if (transport === "chat-completions") {
return createOpenAIChatCompletionsOnlyRuntime(chatRuntime);
}

return createOpenAIAdaptiveModelRuntime(
createOpenAIModelRuntime(runtimeConfig, modelId),
chatRuntime,
responsesRuntime,
);
}
Expand Down
18 changes: 18 additions & 0 deletions src/provider/veryfront-cloud/model-catalog.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import {
resolveVeryfrontCloudGatewayModelId,
resolveVeryfrontCloudModelId,
resolveVeryfrontCloudModelThinking,
resolveVeryfrontCloudOpenAITransport,
resolveVeryfrontCloudReasoningOption,
resolveVeryfrontCloudThinkingProviderOptions,
tryGetVeryfrontCloudProviderFromModelId,
Expand Down Expand Up @@ -198,6 +199,23 @@ describe("provider/veryfront-cloud/model-catalog", () => {
}
});

it("resolves model-specific OpenAI transport overrides", () => {
for (
const modelId of [
"openai/gpt-5.4",
"veryfront-cloud/openai/gpt-5.4",
"openai/gpt-5.5",
"veryfront-cloud/openai/gpt-5.5",
]
) {
assertEquals(resolveVeryfrontCloudOpenAITransport(modelId), "chat-completions");
}

assertEquals(resolveVeryfrontCloudOpenAITransport("openai/gpt-5.2"), undefined);
assertEquals(resolveVeryfrontCloudOpenAITransport("openai/gpt-5.4-mini"), undefined);
assertEquals(resolveVeryfrontCloudOpenAITransport("openai/gpt-5.4-nano"), undefined);
});

it("rejects non-positive and non-safe thinking budgets", () => {
const invalidBudgets = [
0,
Expand Down
10 changes: 10 additions & 0 deletions src/provider/veryfront-cloud/model-catalog.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ export function normalizeVeryfrontCloudProviderAlias(

type VeryfrontCloudModelTransportCapabilities = {
readonly anthropicThinkingMode?: "adaptive";
readonly openAITransport?: "chat-completions" | "responses";
};

/**
Expand All @@ -85,6 +86,8 @@ const VERYFRONT_CLOUD_MODEL_TRANSPORT_CAPABILITIES = new Map<
>([
["anthropic/claude-opus-4-7", Object.freeze({ anthropicThinkingMode: "adaptive" })],
["anthropic/claude-opus-4-8", Object.freeze({ anthropicThinkingMode: "adaptive" })],
["openai/gpt-5.4", Object.freeze({ openAITransport: "chat-completions" })],
["openai/gpt-5.5", Object.freeze({ openAITransport: "chat-completions" })],
]);

function getVeryfrontCloudModelTransportCapabilities(
Expand All @@ -95,6 +98,13 @@ function getVeryfrontCloudModelTransportCapabilities(
);
}

/** Resolves a model-specific OpenAI transport override for Veryfront Cloud. */
export function resolveVeryfrontCloudOpenAITransport(
modelId: string,
): "chat-completions" | "responses" | undefined {
return getVeryfrontCloudModelTransportCapabilities(modelId)?.openAITransport;
}

/** Returns true if the given model ID is a Mistral model in the catalog. */
export function isSupportedMistralModelId(modelId: string): boolean {
return VERYFRONT_CLOUD_CHAT_MODELS.some(
Expand Down
2 changes: 2 additions & 0 deletions src/provider/veryfront-cloud/openai.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ const openLLMProvider = new OpenAIProvider();
interface VeryfrontCloudOpenAIConfig {
apiToken: string;
baseURL: string;
openAITransport?: "chat-completions" | "responses";
fetch: typeof globalThis.fetch;
}

Expand All @@ -18,6 +19,7 @@ export function createVeryfrontCloudOpenAIModel(
baseURL: config.baseURL,
name: "veryfront-cloud",
providerName: "veryfront-cloud",
openAITransport: config.openAITransport,
fetch: config.fetch,
});
}
Expand Down
Loading