feat(dogfood): promote cost_usd + token counts for the FinOps loop - #646
Conversation
|
Warning Review limit reached
Next review available in: 52 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. 📝 WalkthroughWalkthroughUpdates ChangesDogfood telemetry configuration
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR updates the local dogfood server configuration to promote additional Claude Code telemetry attributes so they can be queried via Ourios’ promoted-column path (RFC 0022) and used in scalar aggregates (RFC0002.17), specifically to support spend/usage rollups for the FinOps loop.
Changes:
- Adds
cost_usd,input_tokens, andoutput_tokenstostorage.promoted_attributes.log. - Extends the header examples to include
sum(attr.cost_usd) by attr.model. - Documents intent/assumptions for FinOps-related aggregations in the config comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…nt) (#647) * docs(rfc): draft RFC 0042 — typed numeric promotion (RFC 0022 amendment) Enacts RFC 0022 §7.1, whose deferral clause ("deferred until a consumer demands it") is now met: the FinOps loop's headline query is sum(attr.cost_usd) by attr.model, and cost_usd arrives as a double AnyValue, which §3.1 projects as NULL. Found via review on #646 — the config-only promotion would have made an always-NULL column and a silently empty sum. Two classes (i64, f64) as per-key type declarations; f64 widens ints, nothing parses strings; the cross-file re-typing conflict resolves by the same rule §3.3 gives pre-amendment files (mismatched column reads as absent, via the schema-adapter seam, no coercion); float equality is typed-arm-only because JSON has no canonical float text; aggregates read the typed column cast-free; an all-NULL group sums to NULL, never a fabricated zero cost. Nine §5 criteria incl. projection totality by proptest over AnyValue variants x classes, the three-file absent/mismatched scan, compaction re-projection across a re-typing, and the dogfood consumer query as the corpus gate. Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org> * docs(rfc): disambiguate RFC 0022 cross-references + align the SUMMARY title Seven bare §-references read as self-references where the referent is RFC 0022 (or, in one case, genuinely this RFC); each now names its document. SUMMARY link text matches the RFC title per the convention of dropping only the amendment parenthetical. Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org> --------- Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org>
…lasses The FinOps loop's headline query is sum(attr.cost_usd) by attr.model, but #608's promotion set carries no cost or usage key, so it errors with the promotion hint. This PR's first draft promoted the keys bare — and review caught that RFC 0022 §3.1's string-only projection would have made always-NULL columns and a silently EMPTY sum (worse than the loud error). That finding became RFC 0042; with its typed classes now green in-repo (#649-#653), the promotion lands the way it must: classes matching the AnyValue variants Claude Code actually emits, verified against captured api_request events — - { key: cost_usd, type: f64 } # doubleValue - { key: input_tokens, type: i64 } # intValue - { key: output_tokens, type: i64 } # intValue cost_usd_micros is skipped as redundant under f64 aggregation; cache token counts are additive later if a query wants them. Cost stays the source's responsibility (Claude Code computes and stamps it); Ourios stores, attributes, and aggregates. Pricing tables and FOCUS-shaped output belong to consumers of the query surface. Verified: the server boots the config through the RFC0042.6 validation path (fails only on the port the running dogfood server holds). Promotion is a write-side projection — RFC0042.9's live sum lands with the first capture after the dogfood server restarts on this config. Signed-off-by: Jens Holdgaard Pedersen <jens@holdgaard.org>
69670b9 to
b8e5ddc
Compare
…n empty result The comment claimed sum over an always-NULL column silently returns empty; the RFC0002.18 semantics is a NULL aggregate (every input excluded, value null). Same silence, precise shape. Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org>
…657) The dogfood corpus gate answered for the first time: with the #646 typed promotion set active, a fresh Claude Code capture (10 api_request events) returned sum(attr.cost_usd) by attr.model over the MCP surface from the typed Float64 column — claude-fable-5: 35.27878 USD across 10 requests, one row group scanned — and sum(attr.output_tokens) exercised the Int64 class alongside (2,881 tokens). All nine §5 criteria now hold; accepted stays a maintainer flip. Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org>
Why (take two — the RFC 0042 arc closed the gap)
The FinOps loop's headline query is
sum(attr.cost_usd) by attr.model; #608's promotion set carries no cost or usage key, so it errors with the promotion hint.This PR's first draft promoted the keys bare — and review caught that it would have made things worse: RFC 0022 §3.1 projects only string
AnyValues, so bare promotion ofcost_usd(adoubleValue) meant always-NULL columns and a silently empty sum. That finding became RFC 0042 (typed numeric promotion, §7.1's reserved extension), now green in-repo across #649–#653: typed columns, config classes, the no-coercion scan, typed predicates, cast-free aggregation, and compaction re-typing.What
The promotion, the way it must land — classes matching the
AnyValuevariants Claude Code actually emits (verified against capturedapi_requestevents):cost_usd_microsskipped (redundant underf64aggregation); cache token counts additive later.Scope boundary (unchanged)
Cost is the source's to compute — Claude Code stamps it. Ourios stores, attributes, and aggregates. Pricing tables and FOCUS-shaped output belong to consumers of the query surface; this PR is deliberately the whole of Ourios's share.
Verification
sumreturning real spend) lands with the first capture after the dogfood server restarts on this config — promotion is a write-side projection. That query is also what flips RFC 0042 togreen.Invariants
Dev-recipe config only; rides RFC 0042's shipped, tested seams. No product default, schema, or hot path touched.
Summary by CodeRabbit