Conversation
mssteuer
force-pushed
the
fix/api-server-model-override
branch
from
May 4, 2026 23:08
6842163 to
dafec84
Compare
…paths When API consumers (e.g. CCC task dispatcher) pass a 'model' field in their request body, the API server now correctly uses that model instead of falling back to the gateway's default. Previously, the model field was extracted from the request body but never passed through to _create_agent(). This caused all API-spawned sessions to use the gateway default (claude-opus-4-6) regardless of what model was requested — silently burning expensive Opus tokens on mechanical coding work that should route to cheaper models. Changes: - _create_agent() now accepts model_override parameter - When override contains 'openrouter/' prefix: resolves OpenRouter credentials from providers config - When override contains 'litellm-' or 'litellm/' prefix: routes to LiteLLM proxy - Chat completions (streaming + non-streaming) passes body.model as model_override - Responses API (streaming + non-streaming) passes body.model as model_override - Spawn/run endpoint passes body.model as model_override - Added regression tests covering all override paths Fixes the issue where CCC sends model='openrouter/openai/gpt-5.5' but agents were created with claude-opus-4-6, causing ~24% of weekly Claude quota to be wasted on mechanical tasks.
mssteuer
force-pushed
the
fix/api-server-model-override
branch
from
May 4, 2026 23:14
dafec84 to
41a633d
Compare
alt-glitch
added
type/bug
Contributor
Author
|
Superseded by #22825 — same fix, rebased fresh on current main with updated tests. This branch was 685 commits behind and the conflicts aren't worth resolving when the replacement PR is already green. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When API consumers (e.g. CCC task dispatcher, external UIs) pass a
modelfield in their request body, the API server adapter now correctly routes to that model instead of silently falling back to the gateway configured default.Problem
The
modelfield was extracted from request bodies but never passed through to_create_agent(). All API-spawned sessions used the gateway default — in our caseclaude-opus-4-6— regardless of what was requested. This silently burned expensive Opus tokens on mechanical work that should route to cheaper/different models.Impact observed: CCC task dispatcher was correctly requesting
openrouter/openai/gpt-5.5for coding agents, but every session was recorded asclaude-opus-4-6. This consumed ~257K Opus tokens/day unnecessarily.Changes
_create_agent()accepts a newmodel_overrideparameteropenrouter/,litellm-), provider credentials and base URL are resolved from configTests
7 new tests in
tests/gateway/test_api_server_model_override.py:All 166 existing API server tests continue to pass.
Related Issues
Closes #16216 — Honor per-request model override from API server adapter
Related: #15789, #14974 (delegate_task model overrides — different scope but same symptom)
Related: #12467 (model field override behavior)
CI Notes
The Contributor Attribution and Docs Site checks are pre-existing failures unrelated to this change (unmapped contributor emails from other PRs, and an ASCII box linter threshold).