fix: cost in usd ticks for xai usage by TejasGhatte · Pull Request #5950 · maximhq/bifrost

TejasGhatte · 2026-08-07T11:36:13Z

Summary

xAI's grok-imagine image generation API returns a cost_in_usd_ticks field in its usage object instead of token counts. Without this field on ImageUsage, the value was silently dropped during unmarshalling, causing the response to surface an empty "usage":{}.

Fixes #5498

Changes

Added CostInUsdTicks *int64 to ImageUsage with omitempty so it is only serialized when present, leaving existing provider responses (OpenAI, Gemini, etc.) unaffected.
Extended DeepCopy to allocate a new pointer for CostInUsdTicks, preserving the no-shared-pointers contract relied on by cost calculation logic.
Added tests covering round-trip marshal/unmarshal of cost_in_usd_ticks, omission of the field when absent, and pointer independence after DeepCopy.

Type of change

Affected areas

How to test

go test ./core/schemas/...

Expected: all three new tests pass — TestImageUsage_CostInUsdTicksRoundTrip, TestImageUsage_CostInUsdTicksOmittedWhenAbsent, and TestImageUsage_DeepCopyCostInUsdTicks.

Breaking changes

Yes
No

Security considerations

No security implications. The new field is a cost/billing value returned by xAI and is passed through as-is.

Checklist

I read docs/contributing/README.md and followed the guidelines
I added/updated tests where appropriate
I updated documentation where needed
I verified builds succeed (Go and UI)
I verified the CI pipeline passes locally if applicable

coderabbitai · 2026-08-07T11:36:25Z

📝 Walkthrough

Summary by CodeRabbit

New Features
- Added support for capturing provider-reported image and language-model usage costs.
- Automatically converts xAI usage values into USD when applicable.
- Preserves cost information across supported response formats and usage conversions.
- Uses provider-reported image costs for billing, with pricing-table fallback when unavailable.
Bug Fixes
- Improved handling of missing, invalid, or already-normalized cost values.
- Ensured usage data remains accurate when copied or converted.

Walkthrough

xAI usage responses now preserve USD tick costs and normalize them into provider costs for chat, responses, and image generation. Image billing uses the reported provider cost when available and falls back to datasheet pricing otherwise.

Changes

Provider cost normalization

Layer / File(s)	Summary
Define and normalize provider costs `core/schemas/chatcompletions.go`, `core/schemas/images.go`, `core/schemas/responses.go`	Usage types store xAI USD ticks and derive `BifrostCost` values when no cost already exists. Image deep copies now copy cost pointers independently.
Preserve costs across usage conversions `core/schemas/mux.go`, `core/schemas/providercost_test.go`	Chat and response usage conversions retain USD tick values. Tests cover normalization, precedence, invalid values, serialization, copying, and conversions.
Normalize costs in xAI responses `core/providers/xai/xai.go`	Chat, response, and image handlers return delegated errors explicitly and normalize provider costs on successful responses.
Use reported image costs for billing `framework/modelcatalog/datasheet/cost.go`, `framework/modelcatalog/datasheet/cost_test.go`	Image billing uses a positive provider-reported cost and falls back to datasheet pricing when no reported cost exists.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant xAIProvider
  participant OpenAIHandler
  participant BifrostUsage
  participant ModelCatalog
  xAIProvider->>OpenAIHandler: send delegated request
  OpenAIHandler-->>xAIProvider: return response and error
  xAIProvider->>BifrostUsage: normalize provider cost
  xAIProvider-->>ModelCatalog: return usage with cost
  ModelCatalog-->>ModelCatalog: use reported image cost or datasheet price

Suggested reviewers: akshaydeo, pratham-mishra04

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The description links issue `#5498`, but the objective identifies that issue as a separate Responses streaming feature.	Link the issue that tracks xAI cost tick handling, or correct the issue reference and description.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check	✅ Passed	The description covers the purpose, changes, testing, affected areas, breaking changes, security, and checklist items.
Out of Scope Changes check	✅ Passed	The code changes consistently support xAI cost propagation, provider-cost normalization, billing, conversions, and related tests.
Title check	✅ Passed	The title clearly identifies the xAI usage cost tick fix and matches the primary change.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch 08-07-fix_cost_in_usd_ticks_for_xai_usage

_{Comment @coderabbitai help to get the list of available commands.}

CLAassistant · 2026-08-07T11:36:25Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

tejas ghatte seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

TejasGhatte · 2026-08-07T11:37:08Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

akshaydeo · 2026-08-10T15:13:21Z

Merge activity

Aug 10, 3:13 PM UTC: A user started a stack merge that includes this pull request via Graphite.
Aug 10, 3:16 PM UTC: Graphite couldn't merge this pull request because a downstack PR video requests info in logs ui #5946 failed to merge.
Aug 10, 3:19 PM UTC: A user started a stack merge that includes this pull request via Graphite.
Aug 10, 3:24 PM UTC: Graphite couldn't merge this pull request because a downstack PR fix: bedrock files handling in inference #5947 failed to merge.
Aug 10, 4:36 PM UTC: A user started a stack merge that includes this pull request via Graphite.
Aug 10, 4:41 PM UTC: @akshaydeo merged this pull request with Graphite.

The base branch was changed.

## Summary xAI's `grok-imagine` image generation API returns a `cost_in_usd_ticks` field in its usage object instead of token counts. Without this field on `ImageUsage`, the value was silently dropped during unmarshalling, causing the response to surface an empty `"usage":{}`. Fixes #5498 ## Changes - Added `CostInUsdTicks *int64` to `ImageUsage` with `omitempty` so it is only serialized when present, leaving existing provider responses (OpenAI, Gemini, etc.) unaffected. - Extended `DeepCopy` to allocate a new pointer for `CostInUsdTicks`, preserving the no-shared-pointers contract relied on by cost calculation logic. - Added tests covering round-trip marshal/unmarshal of `cost_in_usd_ticks`, omission of the field when absent, and pointer independence after `DeepCopy`. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/schemas/... ``` Expected: all three new tests pass — `TestImageUsage_CostInUsdTicksRoundTrip`, `TestImageUsage_CostInUsdTicksOmittedWhenAbsent`, and `TestImageUsage_DeepCopyCostInUsdTicks`. ## Breaking changes - [ ] Yes - [x] No ## Security considerations No security implications. The new field is a cost/billing value returned by xAI and is passed through as-is. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable

@jeremym-tanium

## ✨ Features - **MCP Per-User OAuth** - MCP clients can hold per-user OAuth credentials and per-user headers, configurable from `config.json` as well as the UI, with a documented shared vs per-identity token lookup contract and VK/Users filters on the OAuth Grants and MCP Auth Sessions sidebars - **Token Exchange IDP Credentials** - New `use_idp_credentials` on `token_exchange` reuses SSO login app credentials for providers that require it, such as Microsoft Entra ID; `client_id` becomes optional when it is set (#6068, #6069) - **Bedrock VPC Endpoints** - AWS Bedrock keys can target VPC endpoints (#6064) - **Per-Request Flat-Fee Pricing** - New `cost_per_request` field flows through datasheet sync, the cost engine, custom overrides and the UI override form (#6079) - **Pricing Overrides in the Model Catalog** - `/api/models/details` exposes resolved pricing overrides, and catalog rows resolve overrides server-side (#6055, #6056) - **MCP Tool Discovery Persistence** - Discovered MCP tools persist and resync uniformly across all client types through a hash-gated core callback, surviving restarts and propagating across a cluster - **W3C Trace ID Propagation** - Requests carry a W3C trace ID on the context (#5945) - **Cancellable Log Cost Recalculation** - Log cost recalculation tasks can be cancelled from the backend (#5801) - **Separate OTEL Metrics Pipeline** - The OTEL collector supports a metrics tab independent of traces, plus separate headers for traces and metrics (#5939, #5940) - **Roots-Only Log Filter** - New `roots_only` filter collapses fallback chains into their root entry with child aggregates (#5737) - **MCP Log Redaction and Plugin Logs** - MCP tool logs carry redaction mappings and plugin logs (#5744, #5746) - **User Agent and App Attribution in Logs** - Logs and MCP tool logs record user agent, app, source, decision, app key and device ID - **S3 Log Export Metadata** - Additional metadata is written alongside S3 log exports (#6070) - **Matview Maintenance Off Switch** - `matview_refresh_interval` accepts `"off"` to disable logstore matview maintenance entirely (thanks [@jeremym-tanium](https://github.com/jeremym-tanium)!) (#5693) - **Video Request Info in Logs UI** - Video requests surface their details in the logs UI (#5946) - **Shell Rewriter Hook** - The UI handler exposes a `ShellRewriter` hook for pre-hydration HTML rewriting (#5807) - **Auth Skip Path** - Adds a context path letting trusted internal callers bypass auth resolution ## 🐞 Fixed - **Path Normalization Auth Bypass** - Fixed a path normalization flaw that allowed auth to be bypassed (#5763) - **Minimal Reasoning Effort on GPT-5 Models** - `reasoning_effort: "minimal"` is preserved for GPT-5-family OpenAI models instead of being downgraded to `low` (thanks [@jitokim](https://github.com/jitokim)!) (#6046) - **Gemini Truncated Response Finish Reason** - Truncated Gemini responses report `MAX_TOKENS` instead of `OTHER` (thanks [@AdityaPainuli](https://github.com/AdityaPainuli)!) (#5979) - **Null Tool-Call Function Name on Streaming** - Streaming continuation deltas no longer materialize an absent tool-call function name as `null` (thanks [@AdityaPainuli](https://github.com/AdityaPainuli)!) (#5966) - **Bedrock Document Uploads** - Fixed Bedrock file handling in inference so office and PDF documents sent as OpenAI `type: "file"` are accepted (#5947) - **xAI Usage Cost** - Fixed USD cost ticks for xAI usage (#5950) - **Anthropic Encrypted Reasoning** - Added an Anthropic error branch when stripping encrypted reasoning content - **MCP Reconnect and Lock Ordering** - Broke a lock-order inversion in `ConnectionCheckerManager`, rebuilt ephemeral clients across the whole connect+init retry, preserved last-known tool maps across close-first reconnects, bound connect attempts to entry identity, deduped background reconnects and gated SSE `OnConnectionLost` on connection identity - **MCP OAuth Session Correctness** - Restricted `Reauthorize` to shared OAuth clients, rejected inactive tokens in `ValidateToken`, made the OAuth flow claim atomic against concurrent reauth, stopped dropping stored scopes on decode failure, and closed a verify-headers double-submit race that also dropped TLS, timeout and per-user-header fields - **Session Stickiness Reconciliation** - `needs_session_stickiness` is pinned across `config.json` reconciliation, so an unrelated file edit can no longer silently revert a client to per-call - **Credential Cache Cancellation** - `headerCredentialCache.Fill` and `userTokenCache.Fill` propagate context so a cancelled request unblocks instead of waiting on an unrelated leader; LRU entries carry a version so a rejected stale `Get` cannot evict a concurrently-updated value - **Governance List-Models Call** - Budgets and rate limits no longer trigger a list-models call (#6051) - **Realtime Response Create Input** - Guarded `response.create` input (#6050) - **HTTP Server Timeouts** - Configured bounded `http.Server` timeouts and a request-body limit - **MCP Client State Badges** - State badges render with spaces instead of underscores, and the state filter bucket was renamed from `disconnected` to `unstable` - **Entra OBO Scope** - `offline_access` is combined with `<audience>/.default` for Entra OBO instead of replacing it (#6078) ## 🔧 Maintenance - **Governance Route Families** - Editions can override governance route families (#5839) - **Dependency Upgrades** - Dependabot updates across all modules, plus module path fixes (#6040, #5864) - **Documentation** - config.schema.json doc fixes and Datadog env var reference fixes in the helm chart docs (#5938, #6019) ## 🗄️ Database Migrations **configstore:** - **add_mcp_client_pending_oauth_config_json_column** - Adds `pending_oauth_config_json` to `config_mcp_clients`. Reversible: drops the added column. - **merge_oauth_token_tables** - Consolidates `oauth_tokens` and `oauth_user_tokens` into `mcp_oauth_tokens`. **Non-reversible**: rollback deliberately leaves `mcp_oauth_tokens` in place, because every OAuth read and write targets it from this migration onward and dropping it would destroy any token created or refreshed since, forcing every holder to re-authorize. - **create_mcp_oauth_flows_table** - Creates `mcp_oauth_flows` to track in-flight OAuth flows. Reversible: drops the new table. - **drop_oauth_config_pkce_columns** - Drops CSRF state, PKCE verifier and `expires_at` from the OAuth config table now that they live on `mcp_oauth_flows`. **Non-reversible**: forward-only, the dropped values were per-flow ephemeral and re-adding empty columns would restore nothing. - **drop_oauth_config_token_id_column** - Drops `token_id`. **Non-reversible**: forward-only, it was a pure FK shortcut now reachable via `(oauth_config_id, auth_mode)`. - **add_mcp_admin_auth_mode_indexes** - Adds admin partial unique indexes on `mcp_oauth_tokens` and `mcp_per_user_header_credentials`. Reversible: drops both indexes. - **add_mcp_client_token_exchange_json_column** - Adds `token_exchange_json` to `config_mcp_clients`. Reversible: drops the added column. - **add_needs_session_stickiness_column** - Adds `needs_session_stickiness` to `config_mcp_clients`. Reversible: drops the added column. - **add_bedrock_endpoints_columns** - Adds Bedrock VPC endpoint columns to the keys table. Reversible: drops the added columns. - **add_cost_per_request_pricing_column** - Adds `cost_per_request` to model pricing. Reversible: drops the added column. **logstore:** - **logs_add_guardrail_debug_column** - Adds `guardrail_debug` to logs. Reversible: drops the added column. - **mcp_tool_logs_add_redaction_mapping_column** - Adds the redaction mapping column to MCP tool logs. **Non-reversible**: rollback is a no-op because dropping the column would permanently destroy reveal data for already-redacted MCP logs. - **logs_add_user_agent_column** - Adds user agent and app columns, their indexes, and a `UserAgentMapping` table. Reversible: drops the indexes and the mapping table. - **mcp_tool_logs_add_user_agent_column** - Adds user agent and app columns plus indexes to MCP tool logs. Reversible: drops both indexes and the `app` column. - **mcp_tool_logs_add_endpoint_columns** - Adds `source`, `decision`, `app_key` and `device_id` to MCP tool logs. Reversible: drops all four columns. - **mcp_tool_logs_add_plugin_logs_column** - Adds `plugin_logs` to MCP tool logs. Reversible: drops the added column. - **logs_recreate_matviews_with_user_agent_column** and **logs_recreate_matviews_with_app_column** - Recreate the log materialized views to include the new columns. Rollback is a no-op because `ensureMatViews` recreates them on next startup. <Warning> **High-throughput deployments: run the logstore migrations during a low-activity window.** Every logstore migration above alters `logs` or `mcp_tool_logs`, the two highest-insert tables in Bifrost, and several also build indexes on them. On a busy instance the index builds hold locks that block concurrent log inserts for the duration of the build, and the matview recreations rebuild against the full table. Schedule the upgrade for a low-traffic period, or expect elevated log-write latency and possible request-path backpressure while the migrations run. </Warning> <Warning> `merge_oauth_token_tables`, `drop_oauth_config_pkce_columns` and `drop_oauth_config_token_id_column` transform or remove existing OAuth state and cannot be rolled back. Take a database backup before upgrading, and do not roll the binary back past this release once the migration has run. </Warning> ## 🐙 Closed GitHub Issues - [#123](#123) - Files API Support - [#5472](#5472) - [Bug]: Bedrock rejects office/PDF document uploads via OpenAI `type:"file"` - "The PDF specified was not valid" - [#5900](#5900) - [Bug]: Streaming continuation chunks materialize omitted tool-call metadata as null - [#5978](#5978) - [Bug]: Gemini egress reports truncated responses as FinishReason OTHER, IncompleteDetails switch matches a string that never occurs - [#6044](#6044) - [Bug]: normalizeOpenAIReasoningEffort maps 'minimal' to 'low' for ALL OpenAI models, even ones that natively support 'minimal'

## Summary xAI's `grok-imagine` image generation API returns a `cost_in_usd_ticks` field in its usage object instead of token counts. Without this field on `ImageUsage`, the value was silently dropped during unmarshalling, causing the response to surface an empty `"usage":{}`. Fixes #5498 ## Changes - Added `CostInUsdTicks *int64` to `ImageUsage` with `omitempty` so it is only serialized when present, leaving existing provider responses (OpenAI, Gemini, etc.) unaffected. - Extended `DeepCopy` to allocate a new pointer for `CostInUsdTicks`, preserving the no-shared-pointers contract relied on by cost calculation logic. - Added tests covering round-trip marshal/unmarshal of `cost_in_usd_ticks`, omission of the field when absent, and pointer independence after `DeepCopy`. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/schemas/... ``` Expected: all three new tests pass — `TestImageUsage_CostInUsdTicksRoundTrip`, `TestImageUsage_CostInUsdTicksOmittedWhenAbsent`, and `TestImageUsage_DeepCopyCostInUsdTicks`. ## Breaking changes - [ ] Yes - [x] No ## Security considerations No security implications. The new field is a cost/billing value returned by xAI and is passed through as-is. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable

TejasGhatte mentioned this pull request Aug 7, 2026

tests: harness tests for bedrock documents #5949

Merged

17 tasks

TejasGhatte marked this pull request as ready for review August 7, 2026 11:36

This was referenced Aug 7, 2026

fix update passthrough docs #5906

Merged

video requests info in logs ui #5946

Merged

TejasGhatte mentioned this pull request Aug 7, 2026

fix: bedrock files handling in inference #5947

Merged

18 tasks

coderabbitai Bot requested a review from Pratham-Mishra04 August 7, 2026 11:37

coderabbitai Bot previously approved these changes Aug 7, 2026

View reviewed changes

coderabbitai Bot mentioned this pull request Aug 7, 2026

[fix]: xai - image generation usage no longer drops cost_in_usd_ticks #5951

Closed

22 tasks

TejasGhatte force-pushed the 08-07-fix_cost_in_usd_ticks_for_xai_usage branch from b14dad5 to 1565566 Compare August 7, 2026 13:35

coderabbitai Bot requested a review from akshaydeo August 7, 2026 13:36

TejasGhatte mentioned this pull request Aug 7, 2026

fix: add anthropic error branch on stripping on encrypted content #5960

Merged

18 tasks

tejas ghatte added 3 commits August 10, 2026 20:55

fix: bedrock files handling in inference

29c18f3

tests: harness tests for bedrock documents

19a9973

fix: cost in usd ticks for xai usage

a4bd159

TejasGhatte force-pushed the 08-07-fix_cost_in_usd_ticks_for_xai_usage branch from 1565566 to a4bd159 Compare August 10, 2026 15:27

TejasGhatte force-pushed the 08-07-tests_harness_tests_for_bedrock_documents branch from f9b058d to 19a9973 Compare August 10, 2026 15:27

akshaydeo changed the base branch from 08-07-tests_harness_tests_for_bedrock_documents to graphite-base/5950 August 10, 2026 16:40

akshaydeo changed the base branch from graphite-base/5950 to dev August 10, 2026 16:40

akshaydeo merged commit 03ea5b6 into dev Aug 10, 2026
9 of 10 checks passed

akshaydeo deleted the 08-07-fix_cost_in_usd_ticks_for_xai_usage branch August 10, 2026 16:41

This was referenced Aug 11, 2026

[Bug]: normalizeOpenAIReasoningEffort maps 'minimal' → 'low' for ALL OpenAI models, even ones that natively support 'minimal' #6044

Closed

feat: add cost_per_request flat-fee pricing field across DB, cost engine, overrides, and docs #6079

Merged

akshaydeo mentioned this pull request Aug 13, 2026

v2.0.0-prerelease3 #6127

Merged

Navigation Menu

Uh oh!

fix: cost in usd ticks for xai usage - #5950

fix: cost in usd ticks for xai usage#5950
akshaydeo merged 3 commits into
devfrom
08-07-fix_cost_in_usd_ticks_for_xai_usage

TejasGhatte commented Aug 7, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 7, 2026 •

edited

Loading

Summary by CodeRabbit

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

CLAassistant commented Aug 7, 2026

Uh oh!

TejasGhatte commented Aug 7, 2026 •

edited

Loading

Uh oh!

akshaydeo commented Aug 10, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

TejasGhatte commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Type of change

Affected areas

How to test

Breaking changes

Security considerations

Checklist

Uh oh!

coderabbitai Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

CLAassistant commented Aug 7, 2026

Uh oh!

TejasGhatte commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

akshaydeo commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

TejasGhatte commented Aug 7, 2026 •

edited

Loading

coderabbitai Bot commented Aug 7, 2026 •

edited

Loading

TejasGhatte commented Aug 7, 2026 •

edited

Loading

akshaydeo commented Aug 10, 2026 •

edited

Loading