fix: inference geo cost on anthropic - #5072
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 11 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (17)
📝 WalkthroughWalkthroughAdds Anthropic inference_geo and web-search billing support: provider conversions forward inference_geo and web-search counts into Bifrost schemas and streaming outputs, pricing storage gains an inference_geo multiplier column, cost calculation applies it to token costs, and UI pricing overrides expose the new field. ChangesAnthropic Inference Geo and Web Search Billing
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant AnthropicStreamHandler
participant UsageAccumulator
participant Client
AnthropicStreamHandler->>UsageAccumulator: usageToProcess (WebSearchRequests, Speed, InferenceGeo)
UsageAccumulator-->>AnthropicStreamHandler: NumSearchQueries, servedSpeed, servedInferenceGeo
AnthropicStreamHandler->>Client: final chunk with response.Speed and response.InferenceGeo
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
This stack of pull requests is managed by Graphite. Learn more about stacking. |
Confidence Score: 4/5This is close, but the compaction billing path should be fixed before merging.
framework/modelcatalog/datasheet/cost.go and the compaction response schema Important Files Changed
Reviews (3): Last reviewed commit: "fix: inference geo cost on anthropic" | Re-trigger Greptile |
|
|
There was a problem hiding this comment.
🧹 Nitpick comments (1)
core/providers/anthropic/anthropic.go (1)
886-889: 📐 Maintainability & Code Quality | 🔵 TrivialWeb search accumulation duplicated across two sites; minor pointer-idiom inconsistency.
The max-tracking-and-set logic for
NumSearchQuerieshere (lines 966-973) is structurally identical toaccumulateAnthropicResponsesUsage(lines 624-643) but operates onBifrostLLMUsageinstead ofResponsesResponseUsage. Also,&nis used at line 971 instead ofschemas.Ptr(n), which is used consistently in the sibling accumulator function in this same file.Not blocking — behavior is correct in both places — but consider extracting a small shared helper to avoid future divergence, and align on
schemas.Ptr()for consistency.Also applies to: 965-980, 1142-1148
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/providers/anthropic/anthropic.go` around lines 886 - 889, The `NumSearchQueries` max-tracking and assignment logic is duplicated between `accumulateAnthropicResponsesUsage` and the `BifrostLLMUsage` accumulator in `anthropic.go`, and the latter also uses `&n` instead of the file’s consistent `schemas.Ptr()` pattern. Extract the shared web-search accumulation into a small helper that both paths call, keep the max-of-values behavior identical for both usage types, and update the pointer assignment in the `BifrostLLMUsage` path to use `schemas.Ptr()` so the two accumulator sites stay aligned.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@core/providers/anthropic/anthropic.go`:
- Around line 886-889: The `NumSearchQueries` max-tracking and assignment logic
is duplicated between `accumulateAnthropicResponsesUsage` and the
`BifrostLLMUsage` accumulator in `anthropic.go`, and the latter also uses `&n`
instead of the file’s consistent `schemas.Ptr()` pattern. Extract the shared
web-search accumulation into a small helper that both paths call, keep the
max-of-values behavior identical for both usage types, and update the pointer
assignment in the `BifrostLLMUsage` path to use `schemas.Ptr()` so the two
accumulator sites stay aligned.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 9ff0ab71-3796-492c-85cf-41f43044f743
📒 Files selected for processing (17)
core/providers/anthropic/anthropic.gocore/providers/anthropic/chat.gocore/providers/anthropic/passthrough_usage.gocore/providers/anthropic/responses.gocore/providers/anthropic/utils_test.gocore/schemas/chatcompletions.gocore/schemas/passthrough.gocore/schemas/responses.goframework/configstore/migrations.goframework/configstore/rdb.goframework/configstore/tables/modelpricing.goframework/modelcatalog/datasheet/cost.goframework/modelcatalog/datasheet/cost_test.goframework/modelcatalog/datasheet/overrides.goframework/modelcatalog/datasheet/types.goui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsxui/lib/types/governance.ts
7b0bd1e to
aa19027
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
core/providers/anthropic/anthropic.go (1)
970-971: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMinor: use
schemas.Ptrfor pointer consistency.
accumulateAnthropicResponsesUsage(Line 633/640) usesschemas.Ptr(n)for the same field, but here the address operator is used. Aligning both keeps the pattern consistent within this change.- if n := usageToProcess.ServerToolUse.WebSearchRequests; usage.CompletionTokensDetails.NumSearchQueries == nil || n > *usage.CompletionTokensDetails.NumSearchQueries { - usage.CompletionTokensDetails.NumSearchQueries = &n - } + if n := usageToProcess.ServerToolUse.WebSearchRequests; usage.CompletionTokensDetails.NumSearchQueries == nil || n > *usage.CompletionTokensDetails.NumSearchQueries { + usage.CompletionTokensDetails.NumSearchQueries = schemas.Ptr(n) + }Based on learnings, prefer
bifrost.Ptr()/schemas.Ptr()over the address operator&even when&is syntactically valid, applied consistently across code paths.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/providers/anthropic/anthropic.go` around lines 970 - 971, The NumSearchQueries assignment in accumulateAnthropicResponsesUsage is using the address operator directly instead of the pointer helper used elsewhere in this change. Update the WebSearchRequests handling in the Anthropic usage accumulation path to use schemas.Ptr(n) when setting CompletionTokensDetails.NumSearchQueries, matching the existing pointer style used in the same function and keeping pointer creation consistent across code paths.Source: Learnings
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@core/providers/anthropic/anthropic.go`:
- Around line 970-971: The NumSearchQueries assignment in
accumulateAnthropicResponsesUsage is using the address operator directly instead
of the pointer helper used elsewhere in this change. Update the
WebSearchRequests handling in the Anthropic usage accumulation path to use
schemas.Ptr(n) when setting CompletionTokensDetails.NumSearchQueries, matching
the existing pointer style used in the same function and keeping pointer
creation consistent across code paths.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 5a9f7e52-a220-4800-9fc9-774e3cdfbf77
📒 Files selected for processing (17)
core/providers/anthropic/anthropic.gocore/providers/anthropic/chat.gocore/providers/anthropic/passthrough_usage.gocore/providers/anthropic/responses.gocore/providers/anthropic/utils_test.gocore/schemas/chatcompletions.gocore/schemas/passthrough.gocore/schemas/responses.goframework/configstore/migrations.goframework/configstore/rdb.goframework/configstore/tables/modelpricing.goframework/modelcatalog/datasheet/cost.goframework/modelcatalog/datasheet/cost_test.goframework/modelcatalog/datasheet/overrides.goframework/modelcatalog/datasheet/types.goui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsxui/lib/types/governance.ts
🚧 Files skipped from review as they are similar to previous changes (15)
- ui/lib/types/governance.ts
- core/schemas/responses.go
- core/schemas/chatcompletions.go
- framework/modelcatalog/datasheet/overrides.go
- framework/configstore/tables/modelpricing.go
- core/providers/anthropic/responses.go
- framework/configstore/rdb.go
- core/schemas/passthrough.go
- framework/modelcatalog/datasheet/types.go
- core/providers/anthropic/chat.go
- core/providers/anthropic/passthrough_usage.go
- ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx
- framework/configstore/migrations.go
- framework/modelcatalog/datasheet/cost.go
- framework/modelcatalog/datasheet/cost_test.go
Merge activity
|
The base branch was changed.
aa19027 to
c1a6a91
Compare
## Summary Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. ## Changes - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... # UI cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. ## Breaking changes - [ ] Yes - [x] No ## Security considerations No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds end-to-end support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves a request with `inference_geo: "us"`, a 1.1× multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field from the Anthropic API response through all converter paths (chat, responses, streaming, passthrough) and wires it into the cost engine so US-residency requests bill correctly. It also fixes a related streaming bug where web search request counts were lost on the terminal chunk of streamed Responses requests, causing web search to go unbilled. ## Changes - **`inference_geo` field propagation**: Added `InferenceGeo *string` to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage` schemas. All Anthropic converters (`ToBifrostChatResponse`, `ToBifrostResponsesResponse`, `ToBifrostResponsesStream`, passthrough usage) now forward `inference_geo` from the provider's usage block. - **Streaming terminal chunk fix**: In both `HandleAnthropicChatCompletionStreaming` and `HandleAnthropicResponsesStream`, `servedSpeed` and `servedInferenceGeo` are captured across events and re-applied to the terminal chunk. The terminal `response.completed` chunk is built fresh with no served modifiers, so without this re-apply, fast-mode and data-residency billing silently fell back to standard rates. - **Web search count accumulation fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into both the response usage accumulator and the mirrored billed usage. Previously the per-event count was overwritten by the terminal chunk's fresh usage, causing web search to go unbilled on streamed Responses requests. - **Cost engine**: `serviceTier` gains an `inferenceGeoUS bool` field. `tierFromResponse` accepts a third `inferenceGeo *string` argument and sets `inferenceGeoUS` when the value is `"us"` (case-insensitive). `computeTextCost` applies `InferenceGeoUSMultiplier` to all token/cache costs when `inferenceGeoUS` is true; the flat per-search fee is intentionally excluded from the multiplier. - **Database migration**: Adds `inference_geo_us_multiplier` column to `TableModelPricing` via a new migration step. The column is included in pricing sync, override patching, and the entry↔table conversion helpers. - **UI**: Exposes `inference_geo_us_multiplier` as a configurable field in the custom pricing override sheet. - **Tests**: New unit tests cover the accumulator web-search fix, the chat and responses converters forwarding `inference_geo` and web search counts, the cost engine multiplier logic (including the safe no-op when the column is unpopulated), `tierFromResponse` for `inferenceGeoUS`, and an end-to-end streaming test (`TestResponsesStream_TerminalChunkCarriesServedModifiers`) that replays `message_start → message_delta → message_stop` and asserts the terminal billed chunk carries `speed`, `inference_geo`, web search count, and cache-creation tokens. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... # UI cd ui pnpm i pnpm build ``` To validate end-to-end: send a chat or responses request to an Anthropic model with data residency enabled. Confirm the response carries `inference_geo: "us"` and that the billed cost reflects the 1.1× multiplier on token/cache costs with the per-search fee unchanged. For streaming, confirm the terminal chunk carries both `speed` and `inference_geo` after `message_stop`. The `inference_geo_us_multiplier` column must be populated in the model pricing datasheet for the multiplier to apply; if the column is `null`, billing is a safe no-op (standard rates). ## Breaking changes - [x] No `tierFromResponse` gains a third parameter but is an internal function. All call sites have been updated. ## Related issues ## Security considerations No auth, secrets, or PII implications. The multiplier is a pricing configuration value stored in the existing model pricing table with the same access controls as other pricing columns. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. - [ ] Yes - [x] No No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. - [ ] Yes - [x] No No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. - [ ] Yes - [x] No No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. - [ ] Yes - [x] No No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. - [ ] Yes - [x] No No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. - [ ] Yes - [x] No No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. - [ ] Yes - [x] No No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds end-to-end support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves a request with `inference_geo: "us"`, a 1.1× multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field from the Anthropic API response through all converter paths (chat, responses, streaming, passthrough) and wires it into the cost engine so US-residency requests bill correctly. It also fixes a related streaming bug where web search request counts were lost on the terminal chunk of streamed Responses requests, causing web search to go unbilled. ## Changes - **`inference_geo` field propagation**: Added `InferenceGeo *string` to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage` schemas. All Anthropic converters (`ToBifrostChatResponse`, `ToBifrostResponsesResponse`, `ToBifrostResponsesStream`, passthrough usage) now forward `inference_geo` from the provider's usage block. - **Streaming terminal chunk fix**: In both `HandleAnthropicChatCompletionStreaming` and `HandleAnthropicResponsesStream`, `servedSpeed` and `servedInferenceGeo` are captured across events and re-applied to the terminal chunk. The terminal `response.completed` chunk is built fresh with no served modifiers, so without this re-apply, fast-mode and data-residency billing silently fell back to standard rates. - **Web search count accumulation fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into both the response usage accumulator and the mirrored billed usage. Previously the per-event count was overwritten by the terminal chunk's fresh usage, causing web search to go unbilled on streamed Responses requests. - **Cost engine**: `serviceTier` gains an `inferenceGeoUS bool` field. `tierFromResponse` accepts a third `inferenceGeo *string` argument and sets `inferenceGeoUS` when the value is `"us"` (case-insensitive). `computeTextCost` applies `InferenceGeoUSMultiplier` to all token/cache costs when `inferenceGeoUS` is true; the flat per-search fee is intentionally excluded from the multiplier. - **Database migration**: Adds `inference_geo_us_multiplier` column to `TableModelPricing` via a new migration step. The column is included in pricing sync, override patching, and the entry↔table conversion helpers. - **UI**: Exposes `inference_geo_us_multiplier` as a configurable field in the custom pricing override sheet. - **Tests**: New unit tests cover the accumulator web-search fix, the chat and responses converters forwarding `inference_geo` and web search counts, the cost engine multiplier logic (including the safe no-op when the column is unpopulated), `tierFromResponse` for `inferenceGeoUS`, and an end-to-end streaming test (`TestResponsesStream_TerminalChunkCarriesServedModifiers`) that replays `message_start → message_delta → message_stop` and asserts the terminal billed chunk carries `speed`, `inference_geo`, web search count, and cache-creation tokens. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... # UI cd ui pnpm i pnpm build ``` To validate end-to-end: send a chat or responses request to an Anthropic model with data residency enabled. Confirm the response carries `inference_geo: "us"` and that the billed cost reflects the 1.1× multiplier on token/cache costs with the per-search fee unchanged. For streaming, confirm the terminal chunk carries both `speed` and `inference_geo` after `message_stop`. The `inference_geo_us_multiplier` column must be populated in the model pricing datasheet for the multiplier to apply; if the column is `null`, billing is a safe no-op (standard rates). ## Breaking changes - [x] No `tierFromResponse` gains a third parameter but is an internal function. All call sites have been updated. ## Related issues ## Security considerations No auth, secrets, or PII implications. The multiplier is a pricing configuration value stored in the existing model pricing table with the same access controls as other pricing columns. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. - [ ] Yes - [x] No No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds end-to-end support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves a request with `inference_geo: "us"`, a 1.1× multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field from the Anthropic API response through all converter paths (chat, responses, streaming, passthrough) and wires it into the cost engine so US-residency requests bill correctly. It also fixes a related streaming bug where web search request counts were lost on the terminal chunk of streamed Responses requests, causing web search to go unbilled. ## Changes - **`inference_geo` field propagation**: Added `InferenceGeo *string` to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage` schemas. All Anthropic converters (`ToBifrostChatResponse`, `ToBifrostResponsesResponse`, `ToBifrostResponsesStream`, passthrough usage) now forward `inference_geo` from the provider's usage block. - **Streaming terminal chunk fix**: In both `HandleAnthropicChatCompletionStreaming` and `HandleAnthropicResponsesStream`, `servedSpeed` and `servedInferenceGeo` are captured across events and re-applied to the terminal chunk. The terminal `response.completed` chunk is built fresh with no served modifiers, so without this re-apply, fast-mode and data-residency billing silently fell back to standard rates. - **Web search count accumulation fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into both the response usage accumulator and the mirrored billed usage. Previously the per-event count was overwritten by the terminal chunk's fresh usage, causing web search to go unbilled on streamed Responses requests. - **Cost engine**: `serviceTier` gains an `inferenceGeoUS bool` field. `tierFromResponse` accepts a third `inferenceGeo *string` argument and sets `inferenceGeoUS` when the value is `"us"` (case-insensitive). `computeTextCost` applies `InferenceGeoUSMultiplier` to all token/cache costs when `inferenceGeoUS` is true; the flat per-search fee is intentionally excluded from the multiplier. - **Database migration**: Adds `inference_geo_us_multiplier` column to `TableModelPricing` via a new migration step. The column is included in pricing sync, override patching, and the entry↔table conversion helpers. - **UI**: Exposes `inference_geo_us_multiplier` as a configurable field in the custom pricing override sheet. - **Tests**: New unit tests cover the accumulator web-search fix, the chat and responses converters forwarding `inference_geo` and web search counts, the cost engine multiplier logic (including the safe no-op when the column is unpopulated), `tierFromResponse` for `inferenceGeoUS`, and an end-to-end streaming test (`TestResponsesStream_TerminalChunkCarriesServedModifiers`) that replays `message_start → message_delta → message_stop` and asserts the terminal billed chunk carries `speed`, `inference_geo`, web search count, and cache-creation tokens. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... # UI cd ui pnpm i pnpm build ``` To validate end-to-end: send a chat or responses request to an Anthropic model with data residency enabled. Confirm the response carries `inference_geo: "us"` and that the billed cost reflects the 1.1× multiplier on token/cache costs with the per-search fee unchanged. For streaming, confirm the terminal chunk carries both `speed` and `inference_geo` after `message_stop`. The `inference_geo_us_multiplier` column must be populated in the model pricing datasheet for the multiplier to apply; if the column is `null`, billing is a safe no-op (standard rates). ## Breaking changes - [x] No `tierFromResponse` gains a third parameter but is an internal function. All call sites have been updated. ## Related issues ## Security considerations No auth, secrets, or PII implications. The multiplier is a pricing configuration value stored in the existing model pricing table with the same access controls as other pricing columns. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds support for Anthropic's data-residency billing feature (`inference_geo`). When Anthropic serves inference in the US (`inference_geo: "us"`), a 1.1x multiplier is applied to all token and cache costs. This PR propagates the `inference_geo` field through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field. A secondary fix ensures Anthropic server-tool web search request counts (`ServerToolUse.WebSearchRequests`) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage. - **`inference_geo` propagation**: `InferenceGeo *string` added to `BifrostChatResponse`, `BifrostResponsesResponse`, and `BifrostPassthroughUsage`. The field is forwarded from `AnthropicUsage` in all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (`message_delta`), and passthrough. - **Data-residency multiplier in cost computation**: `serviceTier` gains an `inferenceGeoUS bool` flag. `tierFromResponse` now accepts and evaluates `inferenceGeo`. `computeTextCost` applies `InferenceGeoUSMultiplier` to token/cache costs only — the flat per-search fee is intentionally excluded. - **Database migration**: `migrationAddInferenceGeoMultiplierColumn` adds `inference_geo_us_multiplier` to `TableModelPricing`. The column is included in the pricing sync update list and mapped through `convertEntryToTablePricing` / `convertTablePricingToEntry`. - **Web search billing fix**: `accumulateAnthropicResponsesUsage` now carries `ServerToolUse.WebSearchRequests` into the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count into `CompletionTokensDetails.NumSearchQueries`. - **UI**: `inference_geo_us_multiplier` added to the custom pricing override sheet and `PricingOverridePatch` TypeScript type. - **Tests**: New unit tests cover the accumulator web-search fix, chat/responses converter forwarding of `InferenceGeo` and web search counts, multiplier application (including the no-multiplier-column safe no-op), and `tierFromResponse` geo detection (case-insensitive). - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include `inference_geo: "us"` and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged. - [ ] Yes - [x] No No auth, secrets, or PII implications. The `inference_geo` value is sourced from Anthropic's API response and used only for billing computation. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable

Summary
Adds support for Anthropic's data-residency billing feature (
inference_geo). When Anthropic serves inference in the US (inference_geo: "us"), a 1.1x multiplier is applied to all token and cache costs. This PR propagates theinference_geofield through the full response pipeline (chat, responses, streaming, passthrough) and wires it into the cost computation layer, including a new database column and UI override field.A secondary fix ensures Anthropic server-tool web search request counts (
ServerToolUse.WebSearchRequests) are correctly forwarded and billed in both streaming and non-streaming paths, where previously the count could be lost when the terminal chunk overwrote the accumulated usage.Changes
inference_geopropagation:InferenceGeo *stringadded toBifrostChatResponse,BifrostResponsesResponse, andBifrostPassthroughUsage. The field is forwarded fromAnthropicUsagein all conversion paths: non-streaming chat, non-streaming responses, streaming chat (captured across events and set on the final chunk), streaming responses (message_delta), and passthrough.serviceTiergains aninferenceGeoUS boolflag.tierFromResponsenow accepts and evaluatesinferenceGeo.computeTextCostappliesInferenceGeoUSMultiplierto token/cache costs only — the flat per-search fee is intentionally excluded.migrationAddInferenceGeoMultiplierColumnaddsinference_geo_us_multipliertoTableModelPricing. The column is included in the pricing sync update list and mapped throughconvertEntryToTablePricing/convertTablePricingToEntry.accumulateAnthropicResponsesUsagenow carriesServerToolUse.WebSearchRequestsinto the accumulator so the count survives the terminal-chunk overwrite on streamed Responses requests. The non-streaming chat converter also forwards the count intoCompletionTokensDetails.NumSearchQueries.inference_geo_us_multiplieradded to the custom pricing override sheet andPricingOverridePatchTypeScript type.InferenceGeoand web search counts, multiplier application (including the no-multiplier-column safe no-op), andtierFromResponsegeo detection (case-insensitive).Type of change
Affected areas
How to test
To validate end-to-end: send a request to an Anthropic model with data residency enabled. The response should include
inference_geo: "us"and the computed cost should reflect the 1.1x multiplier on token/cache costs, with the per-search fee unchanged.Breaking changes
Security considerations
No auth, secrets, or PII implications. The
inference_geovalue is sourced from Anthropic's API response and used only for billing computation.Checklist
docs/contributing/README.mdand followed the guidelines