feat: configs added for providers#5
Closed
Pratham-Mishra04 wants to merge 1 commit into03-22-fix_bifrost_context_fixesfrom
Closed
feat: configs added for providers#5Pratham-Mishra04 wants to merge 1 commit into03-22-fix_bifrost_context_fixesfrom
Pratham-Mishra04 wants to merge 1 commit into03-22-fix_bifrost_context_fixesfrom
Conversation
Collaborator
Author
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
This was referenced Mar 25, 2025
thiscantbeserious
added a commit
to thiscantbeserious/bifrost
that referenced
this pull request
Apr 17, 2026
CodeRabbit round 2:
10. /models query-param fallback now checks for existing client_version
before injecting. Preserves caller-supplied params like foo=bar and
picks up caller's own client_version when present. Uses simple
string split (no net/url allocation) since paths are well-formed.
11. chatGPTOAuthWebSocketHeaders now routes through
mergeHeadersCaseInsensitive so caller-supplied headers with
different casing (OPENAI-BETA, Openai-Beta, etc.) can't cause
duplicate-case headers reaching the wire — OAuth values and the
Authorization bearer always win deterministically.
Greptile findings:
P1 maximhq#1 Non-streaming Responses() path: return a clear BifrostOperationError
pointing callers to ResponsesStream when chatgpt_oauth is on.
The ChatGPT backend only accepts stream=true (matching the
openai-oauth reference proxy behavior), so non-streaming cannot
work via Bifrost without server-side SSE aggregation which is
out of scope for this PR. In practice Codex always streams.
P1 maximhq#2 /models response format: OpenAIListModelsResponse now has a
UnmarshalJSON that accepts BOTH the standard OpenAI shape
({"data":[{id,...}]}) and the ChatGPT backend shape
({"models":[{slug}]}). ChatGPT entries are projected into
OpenAIModel with ID=slug, Object="model", OwnedBy="chatgpt-oauth".
P2 maximhq#3 client_version doc comment: corrected — the fallback only
applies when the inbound path reaches chatGPTOAuthPath without
a caller-supplied client_version (previously the doc implied
forwarding logic that didn't exist; it now does).
P2 maximhq#4 store doc comment: corrected from "sets store to false if not
already present" to "forces store to false" to match the code
and explanation.
P2 maximhq#5 Double JWT decode per Responses/ResponsesStream: fixed by
passing raw networkConfig.ExtraHeaders to chatGPTOAuthApplyRequest
instead of the already-merged effectiveExtraHeaders(key). The
merge now happens exactly once per call.
Test coverage: 100% on every function in chatgpt_oauth.go. Added tests
for query-param preservation, models dual-shape parsing, and the
streaming-only sentinel error.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
22 tasks
thiscantbeserious
added a commit
to thiscantbeserious/bifrost
that referenced
this pull request
Apr 24, 2026
CodeRabbit round 2:
10. /models query-param fallback now checks for existing client_version
before injecting. Preserves caller-supplied params like foo=bar and
picks up caller's own client_version when present. Uses simple
string split (no net/url allocation) since paths are well-formed.
11. chatGPTOAuthWebSocketHeaders now routes through
mergeHeadersCaseInsensitive so caller-supplied headers with
different casing (OPENAI-BETA, Openai-Beta, etc.) can't cause
duplicate-case headers reaching the wire — OAuth values and the
Authorization bearer always win deterministically.
Greptile findings:
P1 maximhq#1 Non-streaming Responses() path: return a clear BifrostOperationError
pointing callers to ResponsesStream when chatgpt_oauth is on.
The ChatGPT backend only accepts stream=true (matching the
openai-oauth reference proxy behavior), so non-streaming cannot
work via Bifrost without server-side SSE aggregation which is
out of scope for this PR. In practice Codex always streams.
P1 maximhq#2 /models response format: OpenAIListModelsResponse now has a
UnmarshalJSON that accepts BOTH the standard OpenAI shape
({"data":[{id,...}]}) and the ChatGPT backend shape
({"models":[{slug}]}). ChatGPT entries are projected into
OpenAIModel with ID=slug, Object="model", OwnedBy="chatgpt-oauth".
P2 maximhq#3 client_version doc comment: corrected — the fallback only
applies when the inbound path reaches chatGPTOAuthPath without
a caller-supplied client_version (previously the doc implied
forwarding logic that didn't exist; it now does).
P2 maximhq#4 store doc comment: corrected from "sets store to false if not
already present" to "forces store to false" to match the code
and explanation.
P2 maximhq#5 Double JWT decode per Responses/ResponsesStream: fixed by
passing raw networkConfig.ExtraHeaders to chatGPTOAuthApplyRequest
instead of the already-merged effectiveExtraHeaders(key). The
merge now happens exactly once per call.
Test coverage: 100% on every function in chatgpt_oauth.go. Added tests
for query-param preservation, models dual-shape parsing, and the
streaming-only sentinel error.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ryan-orphic
added a commit
to Orphic-AI/bifrost
that referenced
this pull request
May 4, 2026
…output tool
Bedrock Converse rejects toolConfig.toolChoice.tool on Meta Llama variants
with HTTP 400 ("This model doesn't support the toolConfig.toolChoice.tool
field. Remove toolConfig.toolChoice.tool and try again."). The synthetic
bf_so_* tool path used for structured output unconditionally pinned the
synthetic tool via toolChoice.tool, which broke every with_structured_output
caller against Llama 4 Maverick / Scout / Llama 3.x on Bedrock.
This patch adds an IsLlamaModel helper next to the existing IsNovaModel /
IsAnthropicModel family and gates the forced-tool emission off it in four
places (covering both API surfaces and both the synthetic and explicit
tool_choice paths on each):
1. core/providers/bedrock/utils.go: ChatCompletions synthetic-tool injection
(response_format=json_schema -> bf_so_* tool path).
2. core/providers/bedrock/responses.go: OpenAI Responses API mirror of maximhq#1
(text.format=json_schema -> bf_so_* tool path).
3. core/providers/bedrock/utils.go: convertToolConfigFromFiltered's explicit
tool_choice path (defense-in-depth for ChatCompletions callers using
bind_tools(tool_choice={"type": "function", "function": {"name": "X"}})
directly).
4. core/providers/bedrock/responses.go: Responses API mirror of maximhq#3 — the
explicit tool_choice path runs convertResponsesToolChoice and writes
BedrockToolChoice{Tool: ...} straight onto the request, which trips the
same Llama 400 without the gate.
With one synthetic tool bound and Bedrock's default "auto" behavior, omitting
tool_choice yields the same outcome on Llama because there's exactly one tool
the model can call. This mirrors the gate langchain-aws ships
(supports_tool_choice_values for llama4 / llama3-1 / llama3-3 sets only
"auto", causing with_structured_output to emit no tool_choice at all).
Tests added (mirroring PR maximhq#3184's TestBedrockAnthropicChatStructuredOutputUsesSyntheticTool
naming and structure):
- TestBedrockLlamaChatStructuredOutputOmitsForcedToolChoice — synthetic tool
present, ToolChoice nil for Llama on the ChatCompletions path.
- TestBedrockNonLlamaChatStructuredOutputForcesToolChoice — regression guard:
Nova / Anthropic still get the forced tool_choice (gate doesn't over-fire).
- TestToBedrockResponsesRequest_LlamaStructuredOutputOmitsForcedToolChoice —
same negative+positive on the Responses API synthetic-tool path.
- TestBedrockLlamaConvertToolConfigOmitsForcedToolChoice — defense-in-depth
coverage for ChatCompletions explicit tool_choice struct callers.
- TestToBedrockResponsesRequest_LlamaConvertResponsesToolChoiceOmitsForcedToolChoice
— defense-in-depth coverage for Responses API explicit tool_choice
(function-typed) struct callers — the gap surfaced in code review.
- TestToBedrockResponsesRequest_NonLlamaConvertResponsesToolChoiceForcesToolChoice
— Anthropic regression guard for maximhq#5 so the Responses-side Llama gate
doesn't over-fire.
Refs:
- https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ToolChoice.html
- https://github.com/langchain-ai/langchain-aws (bedrock_converse.py
supports_tool_choice_values family-detection lines 678-711).
- PR maximhq#3184 (Anthropic structured-output fallback to synthetic tool path) —
immediate predecessor; this PR completes the per-family gating story.
akshaydeo
pushed a commit
that referenced
this pull request
May 4, 2026
…output tool (#3196) Bedrock Converse rejects toolConfig.toolChoice.tool on Meta Llama variants with HTTP 400 ("This model doesn't support the toolConfig.toolChoice.tool field. Remove toolConfig.toolChoice.tool and try again."). The synthetic bf_so_* tool path used for structured output unconditionally pinned the synthetic tool via toolChoice.tool, which broke every with_structured_output caller against Llama 4 Maverick / Scout / Llama 3.x on Bedrock. This patch adds an IsLlamaModel helper next to the existing IsNovaModel / IsAnthropicModel family and gates the forced-tool emission off it in four places (covering both API surfaces and both the synthetic and explicit tool_choice paths on each): 1. core/providers/bedrock/utils.go: ChatCompletions synthetic-tool injection (response_format=json_schema -> bf_so_* tool path). 2. core/providers/bedrock/responses.go: OpenAI Responses API mirror of #1 (text.format=json_schema -> bf_so_* tool path). 3. core/providers/bedrock/utils.go: convertToolConfigFromFiltered's explicit tool_choice path (defense-in-depth for ChatCompletions callers using bind_tools(tool_choice={"type": "function", "function": {"name": "X"}}) directly). 4. core/providers/bedrock/responses.go: Responses API mirror of #3 — the explicit tool_choice path runs convertResponsesToolChoice and writes BedrockToolChoice{Tool: ...} straight onto the request, which trips the same Llama 400 without the gate. With one synthetic tool bound and Bedrock's default "auto" behavior, omitting tool_choice yields the same outcome on Llama because there's exactly one tool the model can call. This mirrors the gate langchain-aws ships (supports_tool_choice_values for llama4 / llama3-1 / llama3-3 sets only "auto", causing with_structured_output to emit no tool_choice at all). Tests added (mirroring PR #3184's TestBedrockAnthropicChatStructuredOutputUsesSyntheticTool naming and structure): - TestBedrockLlamaChatStructuredOutputOmitsForcedToolChoice — synthetic tool present, ToolChoice nil for Llama on the ChatCompletions path. - TestBedrockNonLlamaChatStructuredOutputForcesToolChoice — regression guard: Nova / Anthropic still get the forced tool_choice (gate doesn't over-fire). - TestToBedrockResponsesRequest_LlamaStructuredOutputOmitsForcedToolChoice — same negative+positive on the Responses API synthetic-tool path. - TestBedrockLlamaConvertToolConfigOmitsForcedToolChoice — defense-in-depth coverage for ChatCompletions explicit tool_choice struct callers. - TestToBedrockResponsesRequest_LlamaConvertResponsesToolChoiceOmitsForcedToolChoice — defense-in-depth coverage for Responses API explicit tool_choice (function-typed) struct callers — the gap surfaced in code review. - TestToBedrockResponsesRequest_NonLlamaConvertResponsesToolChoiceForcesToolChoice — Anthropic regression guard for #5 so the Responses-side Llama gate doesn't over-fire. Refs: - https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ToolChoice.html - https://github.com/langchain-ai/langchain-aws (bedrock_converse.py supports_tool_choice_values family-detection lines 678-711). - PR #3184 (Anthropic structured-output fallback to synthetic tool path) — immediate predecessor; this PR completes the per-family gating story.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.