Skip to content

fix(flare-proxy): substitute route.upstream_model into the outgoing request - #263

Merged
getappz merged 1 commit into
masterfrom
fix/flare-proxy-upstream-model-211
Jul 19, 2026
Merged

fix(flare-proxy): substitute route.upstream_model into the outgoing request#263
getappz merged 1 commit into
masterfrom
fix/flare-proxy-upstream-model-211

Conversation

@getappz

@getappz getappz commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes agentflare item #211.

crates/flare-proxy/src/providers.rs defines ModelRoute.upstream_model (e.g. openrouter/auto, meta/llama-3.1-405b-instruct) but forward.rs::proxy_request never read it. shape_xlat::messages_to_chat copies the incoming Anthropic model string as-is into the translated OpenAI-shaped request, so upstream providers received the raw Anthropic model id (e.g. claude-sonnet-4-5-20250601) verbatim and rejected it.

Fix: after translation, override openai_req["model"] with route.upstream_model.

Verification (live, manual — no mock-server test harness exists yet for forward.rs's HTTP layer, a pre-existing gap)

  • Before fix: POST /proxy/v1/messages with model: claude-sonnet-4-5-20250601 (routes to OpenRouter) → OpenRouter rejected with "claude-sonnet-4-5-20250601 is not a valid model ID".
  • After fix: same request → correct provider-native model (openrouter/auto, resolved by OpenRouter to openai/gpt-5.6-sol) is called, valid Anthropic-shaped SSE stream returned.
  • cargo test -p flare-proxy: 16/16 pass (pre-existing unit tests for shape_xlat/heuristic/think; none previously covered forward.rs's request-building path, which is why this shipped unnoticed).

Follow-up (not fixed here, filed separately)

providers.rs's NVIDIA NIM default_free() config hardcodes meta/llama-3.1-405b-instruct, which no longer exists in NVIDIA's current model catalog (confirmed via live /v1/models call — only meta/llama-3.1-70b-instruct/8b/meta/llama-3.3-70b-instruct exist now). Separate stale-config issue.

Summary by CodeRabbit

  • Bug Fixes
    • Provider-routed requests now use the configured upstream model, improving compatibility with upstream APIs.
    • Prevents requests from being rejected or ignored when the incoming model name isn’t recognized.

…equest

messages_to_chat() copies the incoming Anthropic model string as-is;
proxy_request never overrode it with the resolved route's upstream_model,
so upstream providers received e.g. claude-sonnet-4-5-20250601 verbatim
and rejected it (OpenRouter: "claude-sonnet-4-5-20250601 is not a valid
model ID"). Verified live against OpenRouter and NVIDIA NIM: before the
fix the raw Anthropic model string reached the upstream API, after the
fix the correct provider-native model (e.g. openrouter/auto -> resolved
to openai/gpt-5.6-sol) is called and a valid Anthropic-shaped SSE stream
comes back.
@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 80e14d6c-33e7-40dc-b71a-e3e07db39495

📥 Commits

Reviewing files that changed from the base of the PR and between d9cdcdd and c5cba64.

📒 Files selected for processing (1)
  • crates/flare-proxy/src/forward.rs

📝 Walkthrough

Walkthrough

The proxy forwarding path now overwrites the translated request’s model with the provider route’s configured upstream model before forwarding.

Changes

Provider model forwarding

Layer / File(s) Summary
Override translated model
crates/flare-proxy/src/forward.rs
Makes the translated request mutable and replaces its model field with route.upstream_model, with an inline comment documenting the translation behavior.

Estimated code review effort: 2 (Simple) | ~5 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: using route.upstream_model in outgoing proxy requests.
Description check ✅ Passed It covers the summary, verification, and follow-up notes, with only the template’s exact Test plan and Notes for reviewers sections omitted.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/flare-proxy-upstream-model-211

Comment @coderabbitai help to get the list of available commands.

@getappz
getappz merged commit bba89a1 into master Jul 19, 2026
18 checks passed
@getappz
getappz deleted the fix/flare-proxy-upstream-model-211 branch July 19, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant