fix(flare-proxy): substitute route.upstream_model into the outgoing request - #263
Merged
Merged
Conversation
…equest messages_to_chat() copies the incoming Anthropic model string as-is; proxy_request never overrode it with the resolved route's upstream_model, so upstream providers received e.g. claude-sonnet-4-5-20250601 verbatim and rejected it (OpenRouter: "claude-sonnet-4-5-20250601 is not a valid model ID"). Verified live against OpenRouter and NVIDIA NIM: before the fix the raw Anthropic model string reached the upstream API, after the fix the correct provider-native model (e.g. openrouter/auto -> resolved to openai/gpt-5.6-sol) is called and a valid Anthropic-shaped SSE stream comes back.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe proxy forwarding path now overwrites the translated request’s model with the provider route’s configured upstream model before forwarding. ChangesProvider model forwarding
Estimated code review effort: 2 (Simple) | ~5 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
3 tasks
4 tasks
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes agentflare item #211.
crates/flare-proxy/src/providers.rsdefinesModelRoute.upstream_model(e.g.openrouter/auto,meta/llama-3.1-405b-instruct) butforward.rs::proxy_requestnever read it.shape_xlat::messages_to_chatcopies the incoming Anthropicmodelstring as-is into the translated OpenAI-shaped request, so upstream providers received the raw Anthropic model id (e.g.claude-sonnet-4-5-20250601) verbatim and rejected it.Fix: after translation, override
openai_req["model"]withroute.upstream_model.Verification (live, manual — no mock-server test harness exists yet for
forward.rs's HTTP layer, a pre-existing gap)POST /proxy/v1/messageswithmodel: claude-sonnet-4-5-20250601(routes to OpenRouter) → OpenRouter rejected with"claude-sonnet-4-5-20250601 is not a valid model ID".openrouter/auto, resolved by OpenRouter toopenai/gpt-5.6-sol) is called, valid Anthropic-shaped SSE stream returned.cargo test -p flare-proxy: 16/16 pass (pre-existing unit tests forshape_xlat/heuristic/think; none previously coveredforward.rs's request-building path, which is why this shipped unnoticed).Follow-up (not fixed here, filed separately)
providers.rs's NVIDIA NIMdefault_free()config hardcodesmeta/llama-3.1-405b-instruct, which no longer exists in NVIDIA's current model catalog (confirmed via live/v1/modelscall — onlymeta/llama-3.1-70b-instruct/8b/meta/llama-3.3-70b-instructexist now). Separate stale-config issue.Summary by CodeRabbit