feat: use provider-reported cost (usage.cost) for accurate cost tracking

[henrydeclety]

Summary

Today goose computes session cost as tokens × catalog_price, looking the price up from the canonical (models.dev) catalog by (provider, model). For any model id that isn't in that catalog — custom or self-hosted ids, or anything served through a gateway — the lookup misses and the cost tracker shows "Pricing data unavailable for {model}", even though the gateway knows the exact cost.

This change lets goose use a provider-reported cost when one is present: if the provider/gateway includes a cost field (USD) in its usage object — exactly as OpenRouter already does (usage.cost), and as an Anthropic-compatible gateway can — goose uses that number. It reflects the model/route actually served (including failover and cache discounts), which a static catalog keyed by model id structurally can't.

It's fully backward-compatible and provider-agnostic:

• Add an optional cost: Option<f64> to Usage (#[serde(default, skip_serializing_if = ...)]), parsed from usage.cost in both the Anthropic (providers/formats/anthropic.rs) and OpenAI-compatible (formats/openai.rs) get_usage paths. On merge it takes the latest non-None value (the total carried on the final message_delta), not a sum.
• Agent::accumulate_cost prefers the provider-reported cost when present and falls back to the existing catalog computation otherwise.
• Desktop CostTracker: a provider-reported cost now also wins the tooltip, so a catalog miss no longer reads "Pricing data unavailable" when the real cost is known. No generated-type changes — the value flows through the existing TokenState.accumulatedCost.

Testing

• Added unit tests in token_usage.rs: cost defaults to None and is omitted from JSON when absent; deserializes from a usage that includes cost; and the merge takes the latest non-None cost rather than summing (so message_start + final message_delta yields the total).
• Manually traced the data flow: get_usage → ProviderUsage → accumulate_cost → session.accumulated_cost → TokenState.accumulatedCost → CostTracker.
• Transparency: I don't have a Rust/Node toolchain on my dev box, so I couldn't run cargo test / pnpm locally — relying on CI to validate the build/tests. The change is small and mechanical; happy to iterate on any CI feedback.

Related Issues

Relates to gateway/custom-endpoint cost tracking (e.g. routing goose through an Anthropic-compatible proxy or OpenRouter-style gateway, where the cost is known per-request but the model id isn't in the public catalog).

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0cc38207c4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Preserve provider cost when merging Anthropic deltas

When an Anthropic-compatible streaming response has the normal message_start usage first and then reports usage.cost on a later message_delta, this parsed cost is dropped before it reaches accumulate_cost: the EVENT_MESSAGE_DELTA path rebuilds merged_usage with Usage::new(...) and never carries delta_usage.cost (see the merge around lines 877-888). As a result, the new provider-reported cost support does not work for the streaming Anthropic-compatible gateway case described here unless there was no prior message_start usage.

Useful? React with 👍 / 👎.

[henrydeclety]

Thanks for the reviews. Addressed in 3b942bb:

• rustfmt: formatted the new test (the one Diff in token_usage.rs the format check flagged).
• Codex P2 (cost dropped on Anthropic delta merge): good catch — the EVENT_MESSAGE_DELTA path rebuilt usage via Usage::new(...) and lost delta_usage.cost when a message_start usage preceded it. Now carried through with .with_cost(delta_usage.cost.or(existing.cost)), so provider-reported cost survives the streaming merge.

[kimnamu]

Thanks for this, Henry — provider-reported cost is the right fix for gateway/custom-id routing, and gating on usage.cost (USD) keeps it fully backward-compatible. I'm not a maintainer, just another Bedrock/provider user who read this carefully and ran the branch locally. Two things I checked specifically, both look correct:

1. Streaming merge no longer doubles or drops the cost (the codex P2 you already fixed in 3b942bb). I confirmed the fix works end-to-end at the SSE layer, not just at the Usage::add unit level. Feeding a real message_start (input only, no cost) + final message_delta (output + cost: 0.0123) through response_to_streaming_message yields:

cost=Some(0.0123) input=Some(100) output=Some(50)

i.e. the single delta total — not doubled (0.0246) and not lost (None). To prove the test actually catches the bug, I reverted just the .with_cost(delta_usage.cost.or(existing_usage.usage.cost)) line and re-ran:

assertion `left == right` failed: cost should be the delta total, not doubled/lost

So the streaming path is correct on 3b942bb.

2. No double-counting across the session accumulation either. accumulate_cost runs once per response (via update_session_metrics) with the final merged usage, and the provider branch existing + provider_cost is symmetric with the catalog branch existing + chunk_cost — both add one response's total exactly once, matching how tokens already accumulate. So mixing catalog-priced and provider-priced responses in one session stays consistent.

One non-blocking suggestion: the streaming-merge fix is the subtle part, and right now it's only covered by the Usage::add unit test (test_cost_merge_takes_latest_non_none_not_sum) — which exercises the operator, not the actual EVENT_MESSAGE_DELTA code path that rebuilds usage via Usage::new(...).with_cost(...). A regression here (someone dropping .with_cost again) would slip past the existing test. A small SSE-level test would lock it in. Here's one that passes on 3b942bb and fails if the .with_cost(...) is removed — happy for you to drop it into mod tests in crates/goose/src/providers/formats/anthropic.rs (it reuses the same response_to_streaming_message entrypoint the other streaming tests use):

#[tokio::test]
async fn test_streaming_preserves_provider_cost_from_delta() {
    use futures::StreamExt;
    // message_start has input tokens but no cost; the final message_delta
    // carries the output tokens AND the provider-reported total cost.
    let events = concat!(
        r#"data: {"type":"message_start","message":{"id":"m1","role":"assistant","content":[],"model":"glm-4.7","usage":{"input_tokens":100,"output_tokens":0}}}"#,
        "\n",
        r#"data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}"#,
        "\n",
        r#"data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"hi"}}"#,
        "\n",
        r#"data: {"type":"content_block_stop","index":0}"#,
        "\n",
        r#"data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":50,"cost":0.0123}}"#,
        "\n",
        r#"data: {"type":"message_stop"}"#,
    );
    let lines: Vec<Result<String, anyhow::Error>> =
        events.lines().map(|l| Ok(l.to_string())).collect();
    let stream = Box::pin(futures::stream::iter(lines));
    let mut msg_stream = std::pin::pin!(response_to_streaming_message(stream));
    let mut final_usage = None;
    while let Some(Ok((_msg, usage))) = msg_stream.next().await {
        if usage.is_some() {
            final_usage = usage;
        }
    }
    let usage = final_usage.expect("stream should yield final usage");
    // The provider cost from the delta survives the message_start merge,
    // and is the single total — not doubled, not dropped.
    assert_eq!(usage.usage.cost, Some(0.0123));
    assert_eq!(usage.usage.input_tokens, Some(100));
    assert_eq!(usage.usage.output_tokens, Some(50));
}

Everything else — #[serde(skip_serializing_if)] keeping cost out of JSON when absent, the CostTracker.tsx change preferring the authoritative provider cost over "Pricing data unavailable" while suppressing the misleading $0.000000 per-token breakdown when the catalog has no prices — reads clean and well-scoped. I'd love to see it land.

Reviewed with the help of an AI agent (Claude Code); I ran the branch and verified the outputs above myself.

[DOsinga]

Thanks for this, and thanks @kimnamu for verifying the branch end-to-end.

I dug into the design and I think the current shape is narrower than where I would like this to land, so I want to flag the direction before we go further.

Right now Usage.cost means provider-reported cost only — it is None for any chunk the provider does not price, and the catalog estimate is computed on the fly inside Agent::accumulate_cost and added straight into the session-level accumulated_cost total. So the catalog cost never actually lives on the chunk, and the total is a separately-stored counter accumulated in parallel with the tokens.

The design I would like instead:

• Every chunk should carry its best cost estimate. Usage.cost should be populated for every chunk: provider-reported when the provider gives it, catalog-derived otherwise. It should be filled in once, where the Usage is constructed/finalized — not special-cased in the accumulator. The field then means "best estimate of the cost of this chunk", which is a much cleaner contract.
• The session total should be derived from the per-chunk costs, not independently accumulated. If we store it at all, it should be recomputed from the chunk costs so it cannot drift from them. Right now the total is its own stored column accumulated separately, which is exactly the kind of thing that drifts.

This also resolves the model-switching case cleanly: if a session uses an OpenRouter model that reports cost and later a direct model that only has catalog pricing, each chunk just records its own best estimate from whichever source applies, and the total falls out as the sum.

So the catalog computation wants to move out of accumulate_cost and into wherever the Usage is finalized, and cost should stop being a passthrough-only field. (Minor: the doc comment on cost is also longer than our style wants — but that becomes moot once the field meaning changes.)

This is a bigger change than the current PR, so I would like to align on it before you invest more. Happy to discuss the approach here. Snoozing for a few days to give you a chance to weigh in.

[DOsinga]

Hi @henrydeclety, and thank you for this — provider-reported cost is genuinely useful, and I appreciate you jumping on the codex feedback so quickly.

As I mentioned in my earlier comment, I'd like this to go in as a slightly broader change rather than special-casing the cost in the accumulator: every chunk should carry its best cost estimate (provider-reported when available, catalog-derived otherwise), filled in where Usage is finalized, and the session total should be derived from those per-chunk costs rather than accumulated independently. That's a bigger refactor and I wanted to align on it before you invested more.

Since we haven't heard back on that direction, I'm going to close this for now to keep the queue tidy. This isn't a "no" to the feature at all — if you'd like to take on the redesign (or want to discuss it further), please reopen this or open a new PR and we'll pick it right back up. Thanks again!