fix: fast mode pricing for anthropic - #4267
Merged
Merged
Conversation
Contributor
📝 WalkthroughWalkthroughCaptures Anthropic response speed, threads it through conversion and passthrough paths, persists fast-mode pricing columns, extends pricing models with fast-mode fields, and applies flat fast-token rates during cost calculation with unit tests. ChangesAnthropic Fast-Mode Implementation
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Comment |
|
tejas ghatte seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Collaborator
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
TejasGhatte
marked this pull request as ready for review
June 10, 2026 19:58
TejasGhatte
force-pushed
the
06-10-fix_fast_mode_pricing_for_anthropic
branch
from
June 10, 2026 19:59
546ce22 to
a8dbd79
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/providers/anthropic/chat.go`:
- Around line 1038-1041: The Anthropic stream serializer
(ToAnthropicChatStreamResponse) never sets streamResp.Usage.Speed, so streaming
clients lose usage.speed; update ToAnthropicChatStreamResponse to copy bifrost
chunk Speed into streamResp.Usage.Speed whenever the incoming Bifrost
response/chunk has Speed (same logic used in the non-stream path that sets
anthropicResp.Usage.Speed from bifrostResp.Speed). Locate
ToAnthropicChatStreamResponse and add assignment(s) to set
streamResp.Usage.Speed = bifrostChunk.Speed (or equivalent field) on the
final/appropriate chunk(s) so the streamed Anthropic response includes
usage.speed.
In `@core/schemas/responses.go`:
- Line 132: The WithDefaults() implementation is not copying the new Speed
*string field, so any provider-populated Speed gets dropped during
normalization; update the WithDefaults() method (the function named WithDefaults
on the response struct that currently copies SearchResults, PromptCacheKey,
etc.) to also copy the Speed pointer into the returned struct (i.e., set
dst.Speed = src.Speed or otherwise preserve the pointer) so that downstream
callers like BifrostResponsesStreamResponse.WithDefaults() retain the fast-mode
billing value.
In `@framework/modelcatalog/datasheet/cost.go`:
- Around line 762-766: The bug is that fast-mode pricing (tier.isFast == true /
pricing.InputCostPerTokenFast) is being applied to cached-token fallback rates;
modify the cache-rate lookup so it never queries helpers with tier.isFast set —
i.e., when calling the cache helpers from computeTextCost (and the similar block
around the other occurrence), ensure you clear or override tier.isFast to false
before passing the tier to tieredInputRate/tieredOutputRate or any cache-rate
helper so cached tokens fall back to the normal tiered rates rather than
*_per_token_fast.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 53cf960c-4dd5-46b0-bf4e-e924eac82229
📒 Files selected for processing (14)
core/providers/anthropic/chat.gocore/providers/anthropic/passthrough_usage.gocore/providers/anthropic/responses.gocore/providers/anthropic/types.gocore/schemas/chatcompletions.gocore/schemas/passthrough.gocore/schemas/responses.goframework/configstore/migrations.goframework/configstore/rdb.goframework/configstore/tables/modelpricing.goframework/modelcatalog/datasheet/cost.goframework/modelcatalog/datasheet/cost_test.goframework/modelcatalog/datasheet/overrides.goframework/modelcatalog/datasheet/types.go
Contributor
Confidence Score: 3/5Three correctness bugs remain unfixed: the beta header fires for any non-nil speed (including 'standard'), the streaming Responses path never sets Speed so fast requests bill at standard rates, and the cache-token fallback applies the fast rate when no explicit cache rate is configured. All three bugs affect billing correctness on live request paths. The beta-header bug could cause unwanted routing for callers explicitly opting out of fast mode. The streaming Speed omission means fast-mode billing is completely bypassed on the streaming Responses API. The cache-token fallback means cache reads/writes on fast requests silently apply the fast rate instead of the standard cache rate when no explicit cache cost is set. core/providers/anthropic/utils.go, core/providers/anthropic/responses.go, and framework/modelcatalog/datasheet/cost.go all need fixes before merge. Important Files Changed
|
TejasGhatte
force-pushed
the
06-10-fix_fast_mode_pricing_for_anthropic
branch
from
June 11, 2026 07:07
a8dbd79 to
d1887c5
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
core/schemas/responses.go (1)
132-132:⚠️ Potential issue | 🔴 Critical | ⚡ Quick win
Speedfield is not copied inWithDefaults()— fast-mode billing will be silently dropped.The new
Speedfield must be copied in theWithDefaults()method (lines 191–326) alongside other optional fields likePromptCacheKey,PromptCacheRetention, andSafetyIdentifier. Without this copy, whenWithDefaults()is called during response normalization (e.g., fromBifrostResponsesStreamResponse.WithDefaults()at line 2692), theSpeedvalue populated by the provider converter will be lost, and the fast-mode billing logic inframework/modelcatalog/datasheet/cost.go:195will never see it.🩹 Proposed fix
Add the copy after line 223 in the
WithDefaults()method:result.PromptCacheRetention = resp.PromptCacheRetention result.SafetyIdentifier = resp.SafetyIdentifier +result.Speed = resp.Speed result.MaxToolCalls = resp.MaxToolCalls🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/schemas/responses.go` at line 132, The Speed field is not being preserved by WithDefaults(), so add logic in the WithDefaults() method to copy the Speed pointer into the returned struct alongside the other optional fields (e.g., PromptCacheKey, PromptCacheRetention, SafetyIdentifier) so provider-populated Speed isn't dropped; this ensures BifrostResponsesStreamResponse.WithDefaults() will retain Speed for downstream billing logic that inspects Speed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@framework/modelcatalog/datasheet/cost_test.go`:
- Line 229: Remove or correct the misleading inline computation comment that
claims the input cost equals 0.0019; update it to either remove the leftover "(?
recompute)" comment or replace it with the correct computation/result
(300*0.00001 + 1500*0.0000005 + 200*0.00000625 = 0.005) so it no longer
contradicts the existing expected value (0.03) asserted later in the test;
locate the comment in cost_test.go near the test asserting the 0.03 expected
value and edit that comment accordingly.
---
Duplicate comments:
In `@core/schemas/responses.go`:
- Line 132: The Speed field is not being preserved by WithDefaults(), so add
logic in the WithDefaults() method to copy the Speed pointer into the returned
struct alongside the other optional fields (e.g., PromptCacheKey,
PromptCacheRetention, SafetyIdentifier) so provider-populated Speed isn't
dropped; this ensures BifrostResponsesStreamResponse.WithDefaults() will retain
Speed for downstream billing logic that inspects Speed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: f6fb7c87-c860-4511-b855-b46a3cc9a82b
📒 Files selected for processing (15)
core/providers/anthropic/chat.gocore/providers/anthropic/passthrough_usage.gocore/providers/anthropic/responses.gocore/providers/anthropic/types.gocore/providers/anthropic/utils.gocore/schemas/chatcompletions.gocore/schemas/passthrough.gocore/schemas/responses.goframework/configstore/migrations.goframework/configstore/rdb.goframework/configstore/tables/modelpricing.goframework/modelcatalog/datasheet/cost.goframework/modelcatalog/datasheet/cost_test.goframework/modelcatalog/datasheet/overrides.goframework/modelcatalog/datasheet/types.go
TejasGhatte
force-pushed
the
06-10-fix_fast_mode_pricing_for_anthropic
branch
from
June 11, 2026 07:42
d1887c5 to
fc3120d
Compare
TejasGhatte
changed the base branch from
graphite-base/4267
to
06-11-fix_vertex_batch_api_tests
June 11, 2026 07:42
This was referenced Jun 11, 2026
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/providers/anthropic/utils.go (1)
1082-1086:⚠️ Potential issue | 🟠 Major | ⚡ Quick winIncorrect beta header injection for non-fast speed values.
The beta header
AnthropicFastModeBetaHeader(fast-mode-2026-02-01) should only be injected whenreq.Speedis explicitly"fast", not for any non-nil value such as"standard". The removed check*req.Speed == "fast"was correct.Evidence:
- Line 1079 comment: "Check for fast mode" (not "check for speed field")
SupportsFastModedoc (line 796-799): specifically aboutspeed:"fast"requests; other models reject with 400stripUnsupportedAnthropicFields(line 211): gatesSpeedonfeatures.FastMode, tying it to fast mode specifically- Header name: "FastModeBetaHeader" indicates the beta feature is fast mode, not the speed parameter in general
If
req.Speed == "standard", sending the fast-mode beta header is incorrect—standard mode doesn't require the beta preview header.🔧 Restore the value check
- if req.Speed != nil { + if req.Speed != nil && *req.Speed == "fast" { if (!hasProvider || features.FastMode) && SupportsFastMode(req.Model) { headers = appendUniqueHeader(headers, AnthropicFastModeBetaHeader) } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/providers/anthropic/utils.go` around lines 1082 - 1086, The header injection currently adds AnthropicFastModeBetaHeader whenever req.Speed is non-nil; change it to only add the header when req.Speed is explicitly "fast". In the block that checks req.Speed (the code using SupportsFastMode, appendUniqueHeader and AnthropicFastModeBetaHeader), restore the removed value check so it verifies *req.Speed == "fast" (and still respects features.FastMode and SupportsFastMode(req.Model)) before calling appendUniqueHeader; this aligns with SupportsFastMode and stripUnsupportedAnthropicFields behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@core/providers/anthropic/utils.go`:
- Around line 1082-1086: The header injection currently adds
AnthropicFastModeBetaHeader whenever req.Speed is non-nil; change it to only add
the header when req.Speed is explicitly "fast". In the block that checks
req.Speed (the code using SupportsFastMode, appendUniqueHeader and
AnthropicFastModeBetaHeader), restore the removed value check so it verifies
*req.Speed == "fast" (and still respects features.FastMode and
SupportsFastMode(req.Model)) before calling appendUniqueHeader; this aligns with
SupportsFastMode and stripUnsupportedAnthropicFields behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e6e5f06d-f386-4cbd-9250-e208c1709286
📒 Files selected for processing (15)
core/providers/anthropic/chat.gocore/providers/anthropic/passthrough_usage.gocore/providers/anthropic/responses.gocore/providers/anthropic/types.gocore/providers/anthropic/utils.gocore/schemas/chatcompletions.gocore/schemas/passthrough.gocore/schemas/responses.goframework/configstore/migrations.goframework/configstore/rdb.goframework/configstore/tables/modelpricing.goframework/modelcatalog/datasheet/cost.goframework/modelcatalog/datasheet/cost_test.goframework/modelcatalog/datasheet/overrides.goframework/modelcatalog/datasheet/types.go
Pratham-Mishra04
changed the base branch from
06-11-fix_vertex_batch_api_tests
to
graphite-base/4267
June 11, 2026 07:51
Pratham-Mishra04
force-pushed
the
06-10-fix_fast_mode_pricing_for_anthropic
branch
from
June 11, 2026 07:53
fc3120d to
c22e11f
Compare
Pratham-Mishra04
force-pushed
the
graphite-base/4267
branch
from
June 11, 2026 07:53
1e8f240 to
61869d8
Compare
Pratham-Mishra04
force-pushed
the
06-10-fix_fast_mode_pricing_for_anthropic
branch
from
June 11, 2026 07:53
c22e11f to
45ac0fb
Compare
TejasGhatte
force-pushed
the
06-10-fix_fast_mode_pricing_for_anthropic
branch
from
June 11, 2026 08:05
45ac0fb to
2f34a9f
Compare
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (2)
core/providers/anthropic/chat.go (1)
1450-1459:⚠️ Potential issue | 🟠 Major | ⚡ Quick winStreaming path still missing
speedforwarding (per prior review).
ToAnthropicChatStreamResponse()buildsstreamResp.Usagebut never copiesbifrostResp.Speedinto it, so Anthropic-compatible streaming clients loseusage.speedeven when the final Bifrost chunk carries it. This breaks parity with the non-streaming path fixed in this PR.Suggested fix
if bifrostResp.Usage != nil { if streamResp.Type == "" { streamResp.Type = "message_delta" } streamResp.Usage = &AnthropicUsage{ InputTokens: bifrostResp.Usage.PromptTokens, OutputTokens: bifrostResp.Usage.CompletionTokens, } + if bifrostResp.Speed != nil { + streamResp.Usage.Speed = bifrostResp.Speed + } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/providers/anthropic/chat.go` around lines 1450 - 1459, ToAnthropicChatStreamResponse currently sets streamResp.Usage from bifrostResp.Usage but never forwards bifrostResp.Speed, so update the function (ToAnthropicChatStreamResponse) to copy bifrostResp.Speed into the Anthropic usage struct: after constructing streamResp.Usage (the AnthropicUsage instance), set its Speed field from bifrostResp.Speed (or set streamResp.Usage.Speed = bifrostResp.Speed) so streaming responses include usage.speed just like the non-streaming path.framework/modelcatalog/datasheet/cost.go (1)
755-759:⚠️ Potential issue | 🟠 Major | ⚡ Quick winPrevent fast-mode leakage into cached-token fallback rates.
Fast-mode precedence here makes cached-token fallback incorrect: when cache-specific rates are unset, Line 923 and Line 936 fall back via
tieredInputRate(..., tier), sotier.isFast=truecauses cached tokens to be billed at fast non-cached rates.💡 Suggested fix
func tieredCacheReadInputTokenRate(pricing *configstoreTables.TableModelPricing, totalTokens int, tier serviceTier) float64 { + standardTier := tier + standardTier.isFast = false if tier.isFlex && pricing.CacheReadInputTokenCostFlex != nil { return *pricing.CacheReadInputTokenCostFlex } @@ if pricing.CacheReadInputTokenCost != nil { return *pricing.CacheReadInputTokenCost } - return tieredInputRate(pricing, totalTokens, tier) + return tieredInputRate(pricing, totalTokens, standardTier) } @@ func tieredCacheCreationInputTokenRate(pricing *configstoreTables.TableModelPricing, totalTokens int, tier serviceTier) float64 { + standardTier := tier + standardTier.isFast = false if totalTokens > TokenTierAbove200K && pricing.CacheCreationInputTokenCostAbove200kTokens != nil { return *pricing.CacheCreationInputTokenCostAbove200kTokens } if pricing.CacheCreationInputTokenCost != nil { return *pricing.CacheCreationInputTokenCost } - return tieredInputRate(pricing, totalTokens, tier) + return tieredInputRate(pricing, totalTokens, standardTier) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@framework/modelcatalog/datasheet/cost.go` around lines 755 - 759, The fast-mode check (tier.isFast && pricing.InputCostPerTokenFast != nil) is leaking into cached-token fallbacks; adjust the fast-rate precedence so it only applies for normal (non-cached) rate calculation. Add a boolean flag to tieredInputRate (or an equivalent parameter) to indicate a cached-token fallback and change the fast-mode branch to: if tier.isFast && pricing.InputCostPerTokenFast != nil && !isCachedFallback { return *pricing.InputCostPerTokenFast }, then update callers that compute cached rates (the sites that invoke tieredInputRate for cached-token fallbacks) to pass isCachedFallback=true so cached-token logic never returns the fast non-cached rate.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@core/providers/anthropic/chat.go`:
- Around line 1450-1459: ToAnthropicChatStreamResponse currently sets
streamResp.Usage from bifrostResp.Usage but never forwards bifrostResp.Speed, so
update the function (ToAnthropicChatStreamResponse) to copy bifrostResp.Speed
into the Anthropic usage struct: after constructing streamResp.Usage (the
AnthropicUsage instance), set its Speed field from bifrostResp.Speed (or set
streamResp.Usage.Speed = bifrostResp.Speed) so streaming responses include
usage.speed just like the non-streaming path.
In `@framework/modelcatalog/datasheet/cost.go`:
- Around line 755-759: The fast-mode check (tier.isFast &&
pricing.InputCostPerTokenFast != nil) is leaking into cached-token fallbacks;
adjust the fast-rate precedence so it only applies for normal (non-cached) rate
calculation. Add a boolean flag to tieredInputRate (or an equivalent parameter)
to indicate a cached-token fallback and change the fast-mode branch to: if
tier.isFast && pricing.InputCostPerTokenFast != nil && !isCachedFallback {
return *pricing.InputCostPerTokenFast }, then update callers that compute cached
rates (the sites that invoke tieredInputRate for cached-token fallbacks) to pass
isCachedFallback=true so cached-token logic never returns the fast non-cached
rate.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9dcd7855-bdb2-4ef1-b477-0790abe13111
📒 Files selected for processing (17)
core/providers/anthropic/chat.gocore/providers/anthropic/passthrough_usage.gocore/providers/anthropic/responses.gocore/providers/anthropic/types.gocore/providers/anthropic/utils.gocore/schemas/chatcompletions.gocore/schemas/passthrough.gocore/schemas/responses.goframework/configstore/migrations.goframework/configstore/rdb.goframework/configstore/tables/modelpricing.goframework/modelcatalog/datasheet/cost.goframework/modelcatalog/datasheet/cost_test.goframework/modelcatalog/datasheet/overrides.goframework/modelcatalog/datasheet/types.goui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsxui/lib/types/governance.ts
Contributor
Merge activity
|
2 tasks
akshaydeo
pushed a commit
that referenced
this pull request
Jun 12, 2026
## Summary Adds support for Anthropic's fast mode (research preview), where `usage.speed` is returned as `"fast"` or `"standard"` to indicate which speed was actually served. This field drives fast-mode billing, which uses a flat per-token rate across the full context window rather than the tiered 128k/200k/272k rates. ## Changes - Added `Speed *string` field to `AnthropicUsage`, `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage` to carry the `usage.speed` value from Anthropic responses through the full response pipeline. - Propagated `Speed` through all relevant conversion paths: chat, responses, passthrough (including streaming). - Added `InputCostPerTokenFast` and `OutputCostPerTokenFast` pricing columns to `TableModelPricing` and the `Options`/`Entry` types, with a corresponding DB migration (`add_fast_mode_pricing_columns`). - Extended `tieredInputRate` and `tieredOutputRate` to check `tier.isFast` first, applying the flat fast-mode rate and bypassing all token-count tiers when fast mode was served and the columns are populated. - Added `isFastSpeed` helper and wired `tier.isFast` into all `extractCostInput` call sites (chat, responses, responses stream, passthrough). - Fast-mode pricing falls back gracefully to standard rates when the fast columns are not configured (e.g. non-Opus models). - Cache tokens on fast requests continue to bill at standard cache rates; only non-cached input and output tokens use the fast rate. - Added `pricingSyncUpdateColumns` entries so fast-mode pricing syncs correctly from the catalog. - Added unit tests covering fast vs. standard rate selection, flat-rate behavior above the 200k tier, fallback when unconfigured, cache token billing at standard rates, and the `isFastSpeed` helper. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/providers/anthropic/... go test ./framework/modelcatalog/datasheet/... go test ./... ``` To validate end-to-end: 1. Configure a model with `input_cost_per_token_fast` and `output_cost_per_token_fast` (e.g. Anthropic Opus 4.x). 2. Send a request with fast mode enabled. 3. Confirm the response includes `"speed": "fast"` and that the computed cost reflects the fast-mode flat rate rather than the standard or tiered rate. 4. Send a request without fast mode and confirm `"speed": "standard"` and standard rates are applied. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. No new auth, secrets, or PII handling introduced. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Speed indicators now propagate across chat, responses, and passthrough usage, exposing whether replies used "fast" or "standard" mode. * Fast-mode headers are added whenever a speed is present for supported models. * Pricing override fields for fast-mode token rates added to the UI. * **Database Schema** * Added columns to store separate input/output fast-token pricing rates. * **Tests** * Added unit tests validating fast-mode pricing behavior and speed parsing. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This was referenced Jun 13, 2026
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for Anthropic's fast mode (research preview), where
usage.speedis returned as"fast"or"standard"to indicate which speed was actually served. This field drives fast-mode billing, which uses a flat per-token rate across the full context window rather than the tiered 128k/200k/272k rates.Changes
Speed *stringfield toAnthropicUsage,BifrostChatResponse,BifrostResponsesResponse, andBifrostPassthroughUsageto carry theusage.speedvalue from Anthropic responses through the full response pipeline.Speedthrough all relevant conversion paths: chat, responses, passthrough (including streaming).InputCostPerTokenFastandOutputCostPerTokenFastpricing columns toTableModelPricingand theOptions/Entrytypes, with a corresponding DB migration (add_fast_mode_pricing_columns).tieredInputRateandtieredOutputRateto checktier.isFastfirst, applying the flat fast-mode rate and bypassing all token-count tiers when fast mode was served and the columns are populated.isFastSpeedhelper and wiredtier.isFastinto allextractCostInputcall sites (chat, responses, responses stream, passthrough).pricingSyncUpdateColumnsentries so fast-mode pricing syncs correctly from the catalog.isFastSpeedhelper.Type of change
Affected areas
How to test
To validate end-to-end:
input_cost_per_token_fastandoutput_cost_per_token_fast(e.g. Anthropic Opus 4.x)."speed": "fast"and that the computed cost reflects the fast-mode flat rate rather than the standard or tiered rate."speed": "standard"and standard rates are applied.Breaking changes
Related issues
Security considerations
None. No new auth, secrets, or PII handling introduced.
Checklist
docs/contributing/README.mdand followed the guidelinesSummary by CodeRabbit
New Features
Database Schema
Tests