feat: Add comprehensive cost tracking display for LLM usage #2992

jackjackbits · 2025-06-18T19:28:27Z

Cost Tracking Display Feature

Overview

This PR implements a comprehensive pricing and cost tracking system for Goose, allowing users to see real-time costs for their LLM usage across different providers and models.

Key Features

1. Real-time Cost Display

Shows cost in the bottom bar with 6 decimal places for precision
Updates dynamically as tokens are consumed
Supports both local token estimation and backend token counts

2. Multi-Model Session Tracking

Tracks costs across entire session even when switching models
Accumulates costs per model with detailed breakdown in tooltip
Preserves cost history when changing providers/models

3. Pricing Data Management

Integrates with OpenRouter API for up-to-date pricing
Caches pricing data locally with 7-day TTL
Pre-loads pricing on server startup to avoid UI delays
Memory-optimized to store only recently used models (last 20)

4. Settings UI

New "Cost Tracking" section in settings
Toggle to enable/disable cost display
Shows OpenRouter connection status
Manual refresh button for pricing data

5. Backend Integration

New /config/pricing endpoint serves cached pricing data
Pricing data persisted to disk for reliability
Support for 300+ models from OpenRouter

Technical Implementation

Backend (Rust)

pricing.rs module handles OpenRouter API integration
Disk caching in .goose/pricing_cache.json
Pre-loads all pricing data on server startup
Serves cached data to frontend for fast response

Frontend (TypeScript)

CostTracker component displays real-time costs
costDatabase.ts manages pricing data with localStorage caching
Memory optimization: only stores recently used models
Session-wide cost accumulation logic

Performance Optimizations

Recently used models tracking (last 20 models)
On-demand fetching for models not in cache
Smart refresh only updates active models
Cache cleanup removes unused data

User Experience

Accurate cost tracking across model switches
Detailed cost breakdown per model in tooltip
Clean monospace display prevents number jumping
Respects user preference to hide/show costs

Testing

All TypeScript/ESLint checks pass
Rust compilation successful
Tested with multiple providers and model switching
Verified cost accumulation across sessions

Files Changed

Backend: crates/goose-server/src/routes/reply.rs, config.rs, pricing.rs
Frontend: ChatView.tsx, CostTracker.tsx, costDatabase.ts, SettingsPage.tsx
Types: Updated generated types for pricing data
Settings: New Cost Tracking settings component

Screenshots

Future Enhancements

Cost limits/budgets
Historical cost analytics
Export cost reports
Per-project cost tracking

- Created CostTracker component that displays cost based on input/output tokens - Added local cost database with pricing for major providers (OpenAI, Anthropic, Google, Groq, Deepseek) - Integrated cost tracker into BottomMenu, displayed to the left of model selector - Updated ChatView to fetch and pass accumulated input/output tokens from session metadata - Extended SessionMetadata interface to include accumulated token fields - Added Coins icon for the cost display - Cost display shows currency symbol and formats cost appropriately (more decimals for small amounts)

- Removed debug console.log statements - Cost tracker now properly hides when no cost data is available - Separator only shows when we have token data - Cost updates depend on backend providing accumulated_input_tokens and accumulated_output_tokens in session metadata

- Added local token estimation (roughly 4 chars per token) when backend tokens aren't available - Cost tracker now updates in real-time as messages are sent/received - Uses locally calculated tokens as fallback when session metadata doesn't have accumulated tokens - Added tooltip showing input/output token counts on hover - Cost display is now functional even without backend token tracking

- Modified useMessageStream hook to fetch session metadata after messages complete - Added sessionMetadata state and return value to useMessageStream - Updated ChatView to use sessionMetadata from message stream for real-time token updates - Fixed linting errors in useMessageStream (case declarations and any type) - Cost tracker now shows accumulated tokens from backend instead of local estimates

- Added debug logging to trace token data flow from sessionMetadata to CostTracker - Fixed CostTracker positioning by adding separator and padding - Made CostTracker always render an element (even when no cost data) for debugging - Added px-2 padding to CostTracker for proper spacing - Debug logs will show in console to help diagnose why cost isn't displaying

- Always show cost tracker with coin icon when model/provider is set - Show loading state (...) while fetching cost data - Show bash.00 when cost data is not available for a model - Added more model name variations for better matching (e.g., claude-3.5-sonnet) - Added detailed logging for cost data lookup to help debug issues - Cost tracker now shows between token count and model selector as intended

- Moved separator to be between cost tracker and model selector - Improved tooltip to show token breakdown and individual costs when available - Added support for local/ollama models (shows bash.00 without 'not available' message) - Better handling of free/local providers - Enhanced tooltip shows model name when cost data is unavailable

- Fixed condition checking to properly handle models with 0 cost for input or output - Added detailed logging for cost calculation debugging - Fixed costInfo check to use undefined instead of falsy values - This should now properly show costs for Claude 3 Opus and other models

- Added claude-opus-4-20250514 and claude-opus-4 to the cost database - Added special handling for opus models in partial matching - Cost tracker will now properly show pricing for Claude Opus 4 models

- Created costDatabase.ts utility for centralized cost management - Supports loading costs from localStorage for custom models - Automatically updates costs for all configured models on app launch - Removed leftmost separator in bottom menu for cleaner look - CostTracker now uses centralized database instead of hardcoded costs - System can now handle any model the user configures, not just hardcoded ones

- Changed cost formatting to always show 6 decimal places (e.g., bash.003915) - Prevents cost display from jumping around as token counts change - Updated all fallback displays to also show bash.000000 for consistency

- Added font-mono class to all cost display text - Fixed-width font ensures numbers don't shift horizontally as they change - Provides a more stable, professional appearance for the cost tracker

- Set fixed height (h-6) for bottom menu to ensure consistent alignment - Wrapped each component in flex containers with h-full for proper vertical centering - Reduced separator margins from mx-2 to mx-1.5 for tighter spacing - Removed extra padding from CostTracker component (px-2 removed) - Added ml-1 spacing between alert indicator and cost tracker - All elements now properly vertically aligned in the status bar

- Removed Coins icon component and its import - Cost now displays as plain monospace text (e.g., bash.003915) - Removed gap-1 class since icon is gone - Cleaner, more minimal appearance in the status bar

…racking - Add backend pricing cache that fetches from OpenRouter and stores to disk - Initialize pricing cache on server startup for all models - Update frontend to pre-load all pricing data on app launch - Implement localStorage caching to prevent red flash on startup - Add session-wide cost tracking that accumulates across model switches - Show total session cost with detailed breakdown by model in tooltip - Support lead/worker model cost tracking - Ensure pricing data is available immediately on app launch

- Only store recently used models (last 20) in memory instead of all 300+ - Pre-fetch only common models on startup - Implement on-demand fetching for models not in cache - Add cache cleanup to remove unused models - Fix TypeScript null vs undefined type issues

joanna2028 · 2025-06-18T22:41:31Z

directly speaking truth to power kills rabbit code

wendytang

in testing, noticing that openrouter pricing fetches fail for
anthropic claude-3-5-sonnet-20241022 but not for openai gpt-4o -- seems like the model name sent to openrouter may not be entirely accurate?

wendytang

I restarted a session and noticed that the cost begins at $0 again. Should it be changed so that cost tracking continues where it left off?

abatkin · 2025-06-24T19:42:30Z

Is this exposed when using the Goose CLI?

jackjackbits · 2025-06-25T12:05:42Z

Is this exposed when using the Goose CLI?

no but would be easy to add.

jackjackbits · 2025-06-25T12:07:56Z

I restarted a session and noticed that the cost begins at $0 again. Should it be changed so that cost tracking continues where it left off?

eventually you could add comprehensive all time tracking across models. given people mainly use one at the moment, the provider's billing dashboard is probably better for this.

but eventually this is a good foundation for autopilot to help manage cost budgets.

spencrmartin · 2025-06-25T14:40:26Z

Thoughts around just placing these additional details within the current LLM context window hover. Would love to eventually leverage cost alerts like AWS (but better)

chaitanyarahalkar · 2025-06-25T17:21:17Z

the CLI PR can be found here #2941. I didn't use the Open Router API costing but had the pricings hardcoded but using their pricing API makes obvious sense. I can build off the costing part coded here.

baxen · 2025-06-26T08:10:10Z

Thoughts around just placing these additional details within the current LLM context window hover. Would love to eventually leverage cost alerts like AWS (but better)

Looks great @spencrmartin! To get this unblocked i suggest we do that as a followup PR

baxen · 2025-06-26T08:11:03Z

the CLI PR can be found here #2941. I didn't use the Open Router API costing but had the pricings hardcoded but using their pricing API makes obvious sense. I can build off the costing part coded here.

Yeah this sounds like a good idea, once this is merged we can reuse that and have it update more dynamically - will review

- fixed a conflict - removed an unused file - fixed a small bug with the tooltip

baxen

LGTM! I updated from main and added a small fix - left a few comments but I think they're better as followup PRs

baxen · 2025-06-26T08:25:16Z

crates/goose/src/providers/pricing.rs

+
+/// Convert OpenRouter model ID to provider/model format
+/// e.g., "anthropic/claude-3.5-sonnet" -> ("anthropic", "claude-3.5-sonnet")
+pub fn parse_model_id(model_id: &str) -> Option<(String, String)> {


for a future version, might be better if we update our naming to follow this external standard

baxen · 2025-06-26T08:28:59Z

ui/desktop/src/components/ChatView.tsx

  }, [filteredMessages]);

+  // Simple token estimation function (roughly 4 characters per token)
+  const estimateTokens = (text: string): number => {


this should average out pretty well as an estimate but we can do an upgrade to propagate the output through from the backend which knows the exact number of tokens. however it'll take adding another notification type to the reply stream so i think that can be a separate issue

* main: Add flag for showing cost tracking (#3090) Improve config file editing and recovery fallback mechanisms (#3082) Cleanup Temporal debug files (#3089) feat: Add comprehensive cost tracking display for LLM usage (#2992)

patrickReiis · 2025-07-04T20:24:12Z

Is this closed then?

#2933

Co-authored-by: jack <jack@deck.local> Co-authored-by: Bradley Axen <baxen@squareup.com> Signed-off-by: Soroosh <soroosh.sarabadani@gmail.com>

Co-authored-by: jack <jack@deck.local> Co-authored-by: Bradley Axen <baxen@squareup.com>

jack added 18 commits June 18, 2025 21:08

fix: Add support for claude-opus-4-20250514 model

dc642b3

- Added claude-opus-4-20250514 and claude-opus-4 to the cost database - Added special handling for opus models in partial matching - Cost tracker will now properly show pricing for Claude Opus 4 models

fix: Use consistent 6 decimal places for cost display

03648e5

- Changed cost formatting to always show 6 decimal places (e.g., bash.003915) - Prevents cost display from jumping around as token counts change - Updated all fallback displays to also show bash.000000 for consistency

style: Use monospace font for cost display to prevent jumping

84f9190

- Added font-mono class to all cost display text - Fixed-width font ensures numbers don't shift horizontally as they change - Provides a more stable, professional appearance for the cost tracker

style: Remove coin icon from cost tracker for cleaner display

6ee03b2

- Removed Coins icon component and its import - Cost now displays as plain monospace text (e.g., bash.003915) - Removed gap-1 class since icon is gone - Cleaner, more minimal appearance in the status bar

feat: add pricing data to provider models and update generated types

7abaa53

feat: add pricing settings UI with status and refresh functionality

acb355e

sambradbury mentioned this pull request Jun 19, 2025

feat(cli): add token cost estimation per provider and make configurable #2941

Closed

wendytang reviewed Jun 23, 2025

View reviewed changes

lily-de mentioned this pull request Jun 25, 2025

Extensions menu #3050

Closed

Merge branch 'main' into feature/cost-tracking-display

8a455b9

vioricavg approved these changes Jun 26, 2025

View reviewed changes

cleanup from merge from main

df3581e

- fixed a conflict - removed an unused file - fixed a small bug with the tooltip

baxen approved these changes Jun 26, 2025

View reviewed changes

baxen merged commit d2ff4f3 into block:main Jun 26, 2025
6 checks passed

vioricavg approved these changes Jun 26, 2025

View reviewed changes

jamadeo mentioned this pull request Jul 3, 2025

fix: Add back lazy_static #3243

Merged

chaitanyarahalkar mentioned this pull request Jul 5, 2025

Macos-only Sandboxing chaitanyarahalkar/goose#9

Open

zanesq mentioned this pull request Jul 8, 2025

Add back session cost tracker #3305

Merged

This was referenced Jul 9, 2025

(draft)feat: add cost estimation per provider for Goose CLI #3312

Closed

feat(cli): add cost estimation per provider for Goose CLI #3330

Merged

cbruyndoncx pushed a commit to cbruyndoncx/goose that referenced this pull request Jul 20, 2025

feat: Add comprehensive cost tracking display for LLM usage (block#2992)

b9eab9b

Co-authored-by: jack <jack@deck.local> Co-authored-by: Bradley Axen <baxen@squareup.com>

feat: Add comprehensive cost tracking display for LLM usage #2992

feat: Add comprehensive cost tracking display for LLM usage #2992

Uh oh!

Conversation

jackjackbits commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Cost Tracking Display Feature

Overview

Key Features

1. Real-time Cost Display

2. Multi-Model Session Tracking

3. Pricing Data Management

4. Settings UI

5. Backend Integration

Technical Implementation

Backend (Rust)

Frontend (TypeScript)

Performance Optimizations

User Experience

Testing

Files Changed

Screenshots

Future Enhancements

Uh oh!

joanna2028 commented Jun 18, 2025

Uh oh!

wendytang left a comment

Choose a reason for hiding this comment

Uh oh!

wendytang left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abatkin commented Jun 24, 2025

Uh oh!

jackjackbits commented Jun 25, 2025

Uh oh!

jackjackbits commented Jun 25, 2025

Uh oh!

spencrmartin commented Jun 25, 2025

Uh oh!

chaitanyarahalkar commented Jun 25, 2025

Uh oh!

baxen commented Jun 26, 2025

Uh oh!

baxen commented Jun 26, 2025

Uh oh!

baxen left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

baxen Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

baxen Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

patrickReiis commented Jul 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

jackjackbits commented Jun 18, 2025 •

edited

Loading

wendytang left a comment •

edited

Loading

baxen left a comment •

edited

Loading

baxen Jun 26, 2025 •

edited

Loading