smenatic router: budgeting+telemetry management

[kohlivrinda]

Summary

Adds observability and cost attribution for the embedding calls made by the semantic complexity router. Previously, routing classification embeddings were invisible to telemetry and never counted toward budgets. This PR stamps a RoutingDebug field onto responses when a routing embed ran, wires warmup embed usage through a new observer callback, and exposes two new Prometheus counters for routing overhead.

Changes

• BifrostRoutingDebug schema: New struct added to BifrostResponseExtraFields carrying the embedding provider, model, input token count, and a count_toward_budgets flag for the routing classification embed that ran before provider selection.
• Cost calculation: CalculateCost now composes routing embedding cost on top of the base/cache cost when RoutingDebug.CountTowardBudgets is set. A new exported RoutingEmbeddingCost method on the datasheet Store (and CalculateRoutingEmbeddingCost on ModelCatalog) prices routing overhead unconditionally so telemetry can report it regardless of the budget flag.
• Governance plugin — request path: embedComplexityText records embed usage on the triggering request's BifrostContext via a context key. stampRoutingDebug (called from PostLLMHook before cost calculation) reads that usage and writes RoutingDebug onto the response. For streams, only the final chunk is stamped, matching the semantic cache pattern.
• Governance plugin — warmup path: embedComplexityTexts and the single-input warmup fallback in embedComplexityText route usage to a new settleWarmupEmbedUsage method instead of recording on a request context. settleWarmupEmbedUsage fires a WarmupEmbedUsageObserver callback (for telemetry) and, when count_toward_budgets is on, attributes cost to the admin-owned provider/model-level budgets via UpdateProviderAndModelBudgetUsageInMemory.
• WarmupEmbedUsageObserver / WarmupEmbedUsageObserverSetter: New interface and callback type, wired by the HTTP server at bootstrap and plugin reload, forwarding warmup usage to the telemetry plugin's ObserveWarmupRoutingEmbedding method.
• Telemetry plugin: Two new Prometheus counters — bifrost_routing_embedding_requests_total and bifrost_routing_embedding_cost_total — labeled by provider, model, and phase (request for per-request classification embeds, warmup for boot/config-change exemplar embeds). Request-phase metrics are recorded in PostLLMHook off the RoutingDebug stamp; warmup-phase metrics are recorded via ObserveWarmupRoutingEmbedding.
• HTTP server: Bootstrap and ReloadPlugin wire the WarmupEmbedUsageObserver to observeWarmupRoutingEmbedding, which resolves the telemetry plugin per call so a reloaded telemetry instance is picked up without re-wiring governance.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

go test ./core/schemas/... ./framework/modelcatalog/... ./plugins/governance/... ./plugins/telemetry/... ./transports/bifrost-http/...

Key scenarios covered by the new tests:

• TestCalculateCost_RoutingDebugFlagOff / TestCalculateCost_RoutingDebugFlagOn: verifies routing embed cost is added only when count_toward_budgets is set.
• TestCalculateCost_RoutingDebugComposesWithCacheDebug: verifies both debug structs can coexist on one response and their costs compose correctly.
• TestRoutingEmbeddingCost_Standalone / TestRoutingEmbeddingCost_MissingFields: standalone pricing helper, including nil/missing-field guards.
• TestEmbedComplexityTextRecordsRoutingUsage: classification embed on a request context records usage for RoutingDebug stamping.
• TestEmbedComplexityTextWarmupPathObservesInsteadOfRecording: warmup path fires the observer, never the request context key.
• TestStampRoutingDebug: stamping logic, stream final-chunk gating, and nil-result safety.
• TestSettleWarmupEmbedUsageAttributesProviderBudget / TestSettleWarmupEmbedUsageFlagOffLeavesBudgetsUntouched: warmup budget attribution gated on the flag.
• TestRoutingEmbeddingCounters / TestObserveWarmupRoutingEmbedding / TestRoutingEmbeddingCountersAbsentWithoutStamp: Prometheus counter correctness for both phases and the no-stamp case.

Breaking changes

• Yes
• No

Related issues

Security considerations

No new auth surfaces. The RoutingDebug field is included in API responses under extra_fields (already an internal/debug envelope); no PII is introduced — only provider name, model name, and token counts are recorded.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a2d12392-0d43-4731-97c8-a26bc1771de3

📥 Commits

Reviewing files that changed from the base of the PR and between 15d7581 and 7fb8358.

📒 Files selected for processing (10)

• core/schemas/bifrost.go
• framework/modelcatalog/datasheet/cost.go
• framework/modelcatalog/datasheet/cost_test.go
• framework/modelcatalog/pricing.go
• plugins/governance/embedding.go
• plugins/governance/embedding_test.go
• plugins/governance/main.go
• plugins/telemetry/main.go
• plugins/telemetry/main_test.go
• transports/bifrost-http/server/server.go

🚧 Files skipped from review as they are similar to previous changes (5)

• plugins/governance/main.go
• core/schemas/bifrost.go
• framework/modelcatalog/datasheet/cost.go
• framework/modelcatalog/datasheet/cost_test.go
• plugins/governance/embedding_test.go

---

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Added optional routing details, including embedding provider, model, token usage, and budget attribution.
  • Routing-embedding costs can be calculated and included in budget totals when enabled.
  • Added monitoring for routing-embedding requests, costs, and warmup activity.

• Bug Fixes

  • Improved combined cost tracking across requests and semantic-cache usage.
  • Improved warmup timeouts, token validation, and budget attribution accuracy.
  • Routing details are recorded only on final streaming responses and avoid duplicate classification charges.

Walkthrough

The change records routing-classification embedding usage, attaches it to responses, calculates routing embedding costs, settles eligible warmup attribution, and exposes request and warmup Prometheus metrics.

Changes

Routing embedding attribution

Layer / File(s)	Summary
Routing debug contract and cost calculation core/schemas/bifrost.go, framework/modelcatalog/datasheet/cost.go, framework/modelcatalog/datasheet/cost_test.go, framework/modelcatalog/pricing.go	Adds RoutingDebug response metadata and routing embedding cost calculation. Tests cover cost composition, missing data, and negative-token handling.
Governance embedding usage recording and response stamping plugins/governance/embedding.go, plugins/governance/embedding_test.go, plugins/governance/main.go	Records request and warmup usage, applies phase-specific timeouts, stamps final routing metadata, and settles eligible warmup usage.
Routing embedding telemetry plugins/telemetry/main.go, plugins/telemetry/main_test.go	Adds request and warmup Prometheus counters for routing embedding usage and cost. Tests cover stamped, unstamped, and warmup observations.
Warmup observer server wiring transports/bifrost-http/server/server.go	Bootstrap and reload paths connect governance warmup usage to the active telemetry plugin.

Estimated code review effort: 4 (Complex) | ~45 minutes

Mergeability Score: 🔵 Low · up to 7fb83

Routing embedding telemetry and budget attribution can be understated when fallback attempts or validation-error paths consume additional tokens, and non-fatal attribution failures should use warning-level logging. The PR is mergeable with explicit owner follow-up on these bounded accounting and logging issues.

Sequence Diagram(s)

sequenceDiagram
  participant Request
  participant GovernancePlugin
  participant ModelCatalog
  participant PrometheusPlugin
  Request->>GovernancePlugin: classify with routing embedding
  GovernancePlugin->>Request: stamp RoutingDebug
  Request->>ModelCatalog: calculate response cost
  ModelCatalog-->>Request: return composed cost
  GovernancePlugin->>PrometheusPlugin: record request or warmup metrics

Loading

Possibly related PRs

• maximhq/bifrost#5655: Adds the semantic routing configuration used by this change.
• maximhq/bifrost#5656: Introduces embedding execution paths extended by this change.
• maximhq/bifrost#5791: Overlaps with warmup routing-embedding observer wiring.

Suggested reviewers: akshaydeo, pratham-mishra04, danpiths

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 38.46% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title identifies semantic router budgeting and telemetry changes, but it contains a spelling error in “semantic.”
Description check	✅ Passed	The description covers the purpose, changes, affected areas, testing, breaking changes, security, and checklist requirements.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 07-30-smenatic_router_budgeting_telemetry_management

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[kohlivrinda]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• semantic router: incremental embedding + embed usage accounting #5896 : 2 dependent PRs (#5901 , #5924 )
• semantic router ui revamp #5865
• move complexity exemplars to configstore with backfill migration #5842
• feat(complexity): harness-aware complexity input extraction for cc and codex #5810
• server: export ObserveWarmupRoutingEmbedding for enterprise #5791
• weaviate + redis adapter fixes #5790
• semantic routing ui #5743
• semantic router: dimension probe + message history window #5748
• semantic cache: min similarity + fallback logging #5742
• smenatic router: budgeting+telemetry management #5686 👈 (View in Graphite)
• semantic routing #5: added semantic classifier for complexity router + handlers changes #5680
• semantic router #3: embedding executor #5656
• semantic router #2: semantic routing config #5655
• semantic router #1: add chromem backend #5628
• complexity : reducing keyword lists from 4 -> 3 lists #5602
• helm and docs changes #5530
• update complexity boundaries #5529
• expose tier/mechanism on every surface #5499
• structured logging #5497
• complexity tier merge #5494
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@framework/modelcatalog/datasheet/cost.go`:
- Around line 52-74: Update RoutingEmbeddingCost to use an embedding-specific
copy of lookupScopes whose Provider is always routingDebug.ProviderUsed before
calling resolvePricing, while preserving any other scope fields supplied by
callers. Ensure provider-scoped pricing overrides for the routing/cache
embedding provider and model are matched even when request-scoped scopes are
passed.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 4d0a7c23-0c0e-420c-99fc-4eb115bafac0

📥 Commits

Reviewing files that changed from the base of the PR and between fad421f and 5b1211b.

📒 Files selected for processing (10)

• core/schemas/bifrost.go
• framework/modelcatalog/datasheet/cost.go
• framework/modelcatalog/datasheet/cost_test.go
• framework/modelcatalog/pricing.go
• plugins/governance/embedding.go
• plugins/governance/embedding_test.go
• plugins/governance/main.go
• plugins/telemetry/main.go
• plugins/telemetry/main_test.go
• transports/bifrost-http/server/server.go

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[coderabbitai]

🧹 Nitpick comments (1)

plugins/telemetry/main.go (1)

726-730: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Use bifrost.Ptr for unmodified values.

Replace address-of expressions with bifrost.Ptr(...) for consistency with repository pointer construction.

• plugins/telemetry/main.go#L726-L730: use bifrost.Ptr(provider), bifrost.Ptr(model), and bifrost.Ptr(inputTokens).
• plugins/telemetry/main_test.go#L218-L221: use bifrost.Ptr(provider), bifrost.Ptr(model), and bifrost.Ptr(inputTokens).

Based on learnings, “prefer using bifrost.Ptr() to create pointers instead of the address operator (&) even when & would be valid syntactically.”

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/telemetry/main.go` around lines 726 - 730, Replace direct address-of
pointer construction with bifrost.Ptr for provider, model, and inputTokens in
plugins/telemetry/main.go lines 726-730 and plugins/telemetry/main_test.go lines
218-221, preserving the existing values and behavior.

Source: Learnings

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@plugins/telemetry/main.go`:
- Around line 726-730: Replace direct address-of pointer construction with
bifrost.Ptr for provider, model, and inputTokens in plugins/telemetry/main.go
lines 726-730 and plugins/telemetry/main_test.go lines 218-221, preserving the
existing values and behavior.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0b13e297-3579-4808-86c8-121dc8cfc97b

📥 Commits

Reviewing files that changed from the base of the PR and between ee0a136 and 3b2841d.

📒 Files selected for processing (10)

• core/schemas/bifrost.go
• framework/modelcatalog/datasheet/cost.go
• framework/modelcatalog/datasheet/cost_test.go
• framework/modelcatalog/pricing.go
• plugins/governance/embedding.go
• plugins/governance/embedding_test.go
• plugins/governance/main.go
• plugins/telemetry/main.go
• plugins/telemetry/main_test.go
• transports/bifrost-http/server/server.go

🚧 Files skipped from review as they are similar to previous changes (8)

• plugins/governance/main.go
• framework/modelcatalog/pricing.go
• framework/modelcatalog/datasheet/cost.go
• framework/modelcatalog/datasheet/cost_test.go
• core/schemas/bifrost.go
• plugins/governance/embedding.go
• transports/bifrost-http/server/server.go
• plugins/governance/embedding_test.go

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.