feat(lens): estimate OTEL costs with shared Rust pricing

[tin-berri]

TLDR

• Direct calls show tokens but leave Lens cost empty
• Estimate supported costs using shared Rust pricing
• Keep incomplete search, tool and routing charges unknown

Intentional product change: eligible direct calls display labeled estimates; unsupported charges remain unknown

User Flow

Before: a direct cached call has no visible cost

1. Send a recorded Moyai trace to POST /v1/traces
2. Open /lens and select the model call
3. See 3,010 input tokens and an empty cost

After: the same trace has a labeled estimate

1. Send the recorded Moyai trace to POST /v1/traces
2. Open /lens and select the model call
3. See Estimated $0.0013 and its pricing source

Relevant issues

• Estimate direct-call costs from model, token and cache usage
• Share the Rust cost crate and LiteLLM pricing catalog
• Builds on gateway spend reconciliation from merged fix(lens): refresh runs until gateway costs are complete #45105

Changes

• Select rates for recorded cache durations, reasoning, service tiers and call times
• Preserve GenAI/OpenInference evidence across list, detail and batch queries
• Set catalog supports_token_only_pricing for hosted search and routing entries
• Resolve deployment paths and aliases to routing requirements independently of served token rates
• Preserve Python gateway/SDK billing and recorded spend precedence, including zero

Screenshots / Proof of Fix

Local gateway, rebuilt standalone Rust Lens and real ClickHouse; labeled synthetic controls and previously captured provider usage

Local demo and actual HTTP recording · Screenshot · Receipt

Before (d3e867a)

1. Export azure_ai/model_router/my-deployment, served gpt-4o, with 100 input and 10 output tokens
2. Read GET http://127.0.0.1:4691/v1/traces/000000000000000000000000000e7201
3. The incomplete token-only total is $0.00035, omitting the routing fee

After (8dbee5f)

1. Read the same persisted spans through http://127.0.0.1:4691/v1/traces
2. Unsupported search, tool and routing charges remain unknown in list and detail
3. Ordinary text, optional search, zero usage and client tools retain estimates
4. Cached direct usage remains $0.00125758; recorded gateway spend remains $0.00002455 and $0

Validation

• 1,370 focused Rust tests and 98 Python catalog/schema tests pass
• 20 routed-identity regressions fail before the fix; all pass afterward
• 26 live controls agree between list and detail
• Three cache-price comparisons agree with existing Python billing
• Rust Clippy, formatting, make check and the unchanged base catalog validator pass
• Greptile 5/5 on 8dbee5ff; Bugbot found no issues; Veria passed

CI status

• Catalog guard, Python schema/metadata tests, code quality and Rust lint pass
• Rust beta-registry failure also exists in the exact base sources
• Push wheel retention assertion is already corrected on the current base
• Vertex memory benchmark measured 0.62 against a 0.60 threshold; unchanged sources passed the previous run
• Other CI jobs remain in progress; this PR is not all green

Pre-Submission checklist

• Added meaningful tests for estimation and unknown costs
• Focused tests and live verification pass
• Current-head CI completed
• Scope is Lens cost estimation
• Current-head Greptile confidence is at least 4/5

Caveats

Medium

• Estimates use bundled catalog prices, excluding private billing arrangements
• Missing evidence or unsupported charges keep cost unknown

Low

• Demo and recording are local artifacts
• Other CI jobs are still running

[tin-berri]

@greptileai Please review this direct OTEL pricing change, including gateway precedence, cache token accounting, duplicate estimates, and list/detail integration.

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lens_otel_cost (8dbee5f) with main (406514f)}

[greptile-apps]

Confidence Score: 5/5

[High impact] The PR appears safe to merge, with both numbered findings addressed and no new blocking issue established

Summary

Lens adds labeled cost estimates for eligible direct calls using the shared Rust calculator and bundled model prices

• Requested routing requirements remain attached when the served model differs
• Unsupported charges and incomplete pricing evidence keep costs unknown
• Recorded gateway spend, including zero, takes precedence
• Named regression cases address both numbered findings

This review inspected code and tests; it did not run tests or live UI checks

_{Reviews (17) · Last reviewed commit: "feat(lens): estimate OTEL costs with sha..." · Reviewed by Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[tin-berri]

@greptileai Please review a8ca357, which addresses all previous findings and adds offline pricing, consistent call grouping, and batch regressions

[tin-berri]

bugbot run

Please review the current tip a8ca357 after the cost identity, batching, and offline pricing fixes

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai Please review this final tip for tier-only pricing, strict selected-rate validation, and used cache-duration handling; all focused tests pass

[tin-berri]

bugbot run

Please review the final tip with tier-only pricing, strict selected-rate validation, and cache-duration regressions passing locally

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai Please review 52ca0e2 for GenAI compatibility, captured call time, and reasoning pricing. All three findings have regression coverage

[tin-berri]

bugbot run

Please review 52ca0e2 for GenAI alias compatibility, stored timestamps, and reasoning pricing after the latest fixes and regression tests

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai Please review 71d7e0a against gateway base e3d70ec, focusing on native pricing ownership, catalog parity, and late gateway reconciliation.

[tin-berri]

bugbot run

Please review 71d7e0a against gateway base e3d70ec, including standalone native estimation, bundled catalog assets, and authoritative gateway amounts

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai Please review the missing search-charge fix, including canonical model admission, reported usage, ordinary text controls, and all four storage projections

[tin-berri]

bugbot run

Please recheck search cost admission and verify normal token estimates and recorded gateway spend precedence remain intact

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai please review b64de19: catalog-owned pricing requirements replace model-name heuristics, preserving request context and declining incomplete tool charges

[tin-berri]

bugbot run

Please review b64de19 for catalog pricing requirements, requested-model context, and unsupported tool usage; gateway billing execution remains unchanged

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai please review d3e867a: pricing eligibility uses catalog supports_token_only_pricing metadata, compatible with base validation; request context and tool-usage regressions pass

[tin-berri]

bugbot run

Please review d3e867a, including catalog schema compatibility, requested-model pricing requirements, and tool usage; Python gateway billing remains unchanged

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai Please review 8dbee5f, including catalog routing contexts and named regression cases; gateway billing remains unchanged and live controls pass

[tin-berri]

bugbot run Please review 8dbee5f for routing context preservation, catalog admission, and regressions; gateway and SDK billing remain on Python

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8dbee5f. Configure here.}