feat: adds provider cache and semantic cache attributes in metrics export

[roroghost17]

Summary

Adds provider-side prompt cache token metrics (cache_read, cache_write, cache_write_5m, cache_write_1h) to both the Prometheus and OTEL telemetry plugins. These are distinct from the existing cache_hits_total counter, which tracks Bifrost's own semantic-cache hits. Also fixes the OTEL plugin's PostLLMHook (previously a no-op) to correctly emit cache-hit metrics for semantic-cache responses that short-circuit before any provider span is created.

Changes

• Prometheus plugin: Added four new CounterVec metrics — bifrost_cache_read_input_tokens_total, bifrost_cache_write_input_tokens_total, bifrost_cache_write_input_tokens_5m_total, and bifrost_cache_write_input_tokens_1h_total. A new extractProviderCacheTokens helper reads these values from PromptTokensDetails (Chat/TextCompletion) or InputTokensDetails (Responses API) and emits them in PostLLMHook.
• OTEL plugin: Implemented the same four provider cache token counters and their Record* methods on MetricsExporter. The previously no-op PostLLMHook now emits the semantic cache-hit metric using a new buildContextAttrs helper that sources dimensions from request context rather than a span (since cache hits have no provider span). Provider cache token counts are recorded in recordMetricsFromTrace from span attributes, covering both the unified read/write keys and the Anthropic-specific 5m/1h TTL breakdown keys.
• Schema mux: Propagated CachedWriteTokenDetails through both ToResponsesResponseUsage and ToBifrostLLMUsage conversion functions so the TTL breakdown is preserved across API format conversions.
• The 5m/1h counters are explicitly documented as subsets of the write total and should not be summed with it.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

go test ./...

• Configure a provider that supports prompt caching (e.g. Anthropic, OpenAI).
• Send requests that exercise prompt cache reads and writes.
• Verify the following metrics are emitted with non-zero values:
  • bifrost_cache_read_input_tokens_total
  • bifrost_cache_write_input_tokens_total
  • bifrost_cache_write_input_tokens_5m_total (Anthropic only)
  • bifrost_cache_write_input_tokens_1h_total (Anthropic only)
• Trigger a semantic cache hit and verify bifrost_cache_hits_total increments exactly once (not double-counted).

Breaking changes

• Yes
• No

Related issues

Security considerations

None.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[CLAassistant]

All committers have signed the CLA.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d1e11026-cf45-4a6a-8896-557983578fb1

📥 Commits

Reviewing files that changed from the base of the PR and between 4f6a8b2 and ba11b9a.

⛔ Files ignored due to path filters (1)

• flake.lock is excluded by !**/*.lock

📒 Files selected for processing (4)

• core/schemas/mux.go
• plugins/otel/main.go
• plugins/otel/metrics.go
• plugins/telemetry/main.go

🚧 Files skipped from review as they are similar to previous changes (4)

• plugins/otel/metrics.go
• core/schemas/mux.go
• plugins/otel/main.go
• plugins/telemetry/main.go

---

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Added telemetry metrics for provider-side prompt-cache token operations: cache reads, cache writes, and TTL-specific (5m/1h) write breakdowns
  • Emit cache-hit events to observability systems for semantic-cache monitoring

• Improvements

  • Propagated richer token-usage details through usage conversions to improve cache analytics and reporting

Walkthrough

Adds provider-side prompt-cache token metrics and cache-hit recording: schema conversions propagate CachedWriteTokenDetails; OpenTelemetry exporter gains cache read/write counters and hook recording; Prometheus plugin adds matching CounterVec metrics and records provider cache token counts.

Changes

Provider-side prompt cache token metrics tracking

Layer / File(s)	Summary
Cached write token detail propagation core/schemas/mux.go	BifrostLLMUsage.ToResponsesResponseUsage() and ResponsesResponseUsage.ToBifrostLLMUsage() now map CachedWriteTokenDetails in both directions.
OpenTelemetry metrics infrastructure plugins/otel/metrics.go	MetricsExporter adds four new counters for provider-side cache read/write input tokens and 5m/1h write-token subsets; initMetrics() registers them and new methods record counts.
OpenTelemetry hook and trace metrics plugins/otel/main.go	Adds bifrost import; PostLLMHook() records cache-hit metrics when present using buildContextAttrs(); recordMetricsFromTrace() records provider cache token metrics from final attempt spans.
Prometheus metrics infrastructure plugins/telemetry/main.go	PrometheusPlugin adds four CounterVec fields for cache read/write input tokens and Anthropic 5m/1h write breakdowns; Init() registers them and wires them into the plugin.
Prometheus hook integration plugins/telemetry/main.go	Adds extractProviderCacheTokens() to pull cached-read/cached-write token counts (and optional TTL breakdowns) from multiple response shapes; PostLLMHook() increments the new counters when values are present.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

• akshaydeo
• danpiths

Poem

🐰 Cache tokens dance in metrics light,
Write and read, both tracked just right,
OTel and Prometheus unite,
Five-minute TTLs burning bright! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: adding provider cache and semantic cache attributes to metrics export, which aligns with the core objectives of the PR.
Description check	✅ Passed	The PR description is comprehensive and well-structured, covering all major sections of the template including Summary, Changes, Type of change, Affected areas, How to test, Breaking changes, and Security considerations.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 05-27-feat_adds_provider_cache_and_semantic_cache_attributes_in_metrics_export

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[roroghost17]

• feat: adds provider cache and semantic cache attributes in metrics export #3816 👈 (View in Graphite)
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[greptile-apps]

Confidence Score: 5/5

This PR is safe to merge — the changes are additive metric instrumentation with no modifications to request routing, auth, or data persistence paths.

All four new counters are guarded before recording, the OTEL PostLLMHook and recordMetricsFromTrace touch strictly disjoint code paths, and the schema propagation in mux.go mirrors the existing field-mapping pattern exactly.

No files require special attention.

Important Files Changed

Filename	Overview
core/schemas/mux.go	Propagates CachedWriteTokenDetails through both ToResponsesResponseUsage and ToBifrostLLMUsage; straightforward field additions mirroring the existing mapping pattern with no issues.
plugins/otel/metrics.go	Adds four new syncInt64Counter fields and Record* methods for provider-side prompt cache tokens; follows existing initialization and method patterns consistently.
plugins/otel/main.go	Implements PostLLMHook for semantic-cache hit metrics and adds provider cache token recording in recordMetricsFromTrace; no double-counting concern.
plugins/telemetry/main.go	Adds four new CounterVec metrics and extractProviderCacheTokens helper; all counters are guarded with > 0 checks before recording.
flake.lock	Routine nixpkgs pin bump; no code changes.

_{Reviews (2): Last reviewed commit: "feat: adds provider cache and semantic c..." | Re-trigger Greptile}

[akshaydeo]

Merge activity

• May 27, 7:16 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• May 27, 7:16 PM UTC: @akshaydeo merged this pull request with Graphite.