chore(dev): promote GenAI attributes in dogfood for native count-by aggregation - #608
Conversation
Adds dogfood-config.yaml (RFC 0020 file front-end) and points
dogfood-server at it via --config, replacing the OURIOS_* env vars.
--config is the sole config source and storage.promoted_attributes
(RFC 0022) has no env-var form, so the switch is required to promote.
Promotes the agent GenAI log attributes model, tool_name, decision to
columns, so a source can aggregate its own telemetry natively through
the loop — e.g. 'count by attr.model' (calls per model), 'count by
attr.tool_name' (tool-use distribution) — instead of paging raw rows.
Verified end-to-end on a throwaway instance: synthetic GenAI logs ->
flush -> 'count by attr.model' returns {opus: 2, haiku: 1} and 'count by
attr.tool_name' returns {Bash: 2, Read: 1}, no unpromoted-attribute
error; the committed file's env-subst + schema also validated.
Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org>
|
Warning Review limit reached
Next review available in: 9 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Promotes selected GenAI-related log attributes in the local dogfood workflow so a telemetry source can run native count by aggregations (via storage.promoted_attributes) instead of paging raw rows.
Changes:
- Switches
just dogfood-serverfromOURIOS_*env-var configuration to--config dogfood-config.yaml. - Adds
dogfood-config.yamldefining local storage/WAL paths, loopback receiver/querier binds, MCP enablement, and promoted log attributes.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
justfile |
Updates the dogfood-server recipe to export OURIOS_DOGFOOD_ROOT and run ourios-server with --config. |
dogfood-config.yaml |
New RFC0020 config file enabling promoted attributes and local loopback-only dogfood settings. |
Comments suppressed due to low confidence (1)
dogfood-config.yaml:25
storage.promoted_attributes.logcurrently promotesmodel/tool_name, but the GenAI semconv keys used by RFC 0037 and the querier test suite are dotted (e.g.gen_ai.request.model). With the current config,count by attr.gen_ai.request.modelwill still be rejected as “not promoted”. Consider promoting the canonicalgen_ai.*keys (optionally alongside the source-specific ones) so standard GenAI queries work out of the box.
promoted_attributes:
log:
- model
- tool_name
- decision
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…t semconv Note in dogfood-config.yaml that model/tool_name/decision are Claude Code's actual (flat) attribute keys, verified against its OTLP output — not the OTel GenAI semconv dotted form (gen_ai.request.model, …), which it does not emit and which would promote always-null columns. Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org>
…lasses The FinOps loop's headline query is sum(attr.cost_usd) by attr.model, but #608's promotion set carries no cost or usage key, so it errors with the promotion hint. This PR's first draft promoted the keys bare — and review caught that RFC 0022 §3.1's string-only projection would have made always-NULL columns and a silently EMPTY sum (worse than the loud error). That finding became RFC 0042; with its typed classes now green in-repo (#649-#653), the promotion lands the way it must: classes matching the AnyValue variants Claude Code actually emits, verified against captured api_request events — - { key: cost_usd, type: f64 } # doubleValue - { key: input_tokens, type: i64 } # intValue - { key: output_tokens, type: i64 } # intValue cost_usd_micros is skipped as redundant under f64 aggregation; cache token counts are additive later if a query wants them. Cost stays the source's responsibility (Claude Code computes and stamps it); Ourios stores, attributes, and aggregates. Pricing tables and FOCUS-shaped output belong to consumers of the query surface. Verified: the server boots the config through the RFC0042.6 validation path (fails only on the port the running dogfood server holds). Promotion is a write-side projection — RFC0042.9's live sum lands with the first capture after the dogfood server restarts on this config. Signed-off-by: Jens Holdgaard Pedersen <jens@holdgaard.org>
) * feat(dogfood): promote cost_usd + token counts under RFC 0042 typed classes The FinOps loop's headline query is sum(attr.cost_usd) by attr.model, but #608's promotion set carries no cost or usage key, so it errors with the promotion hint. This PR's first draft promoted the keys bare — and review caught that RFC 0022 §3.1's string-only projection would have made always-NULL columns and a silently EMPTY sum (worse than the loud error). That finding became RFC 0042; with its typed classes now green in-repo (#649-#653), the promotion lands the way it must: classes matching the AnyValue variants Claude Code actually emits, verified against captured api_request events — - { key: cost_usd, type: f64 } # doubleValue - { key: input_tokens, type: i64 } # intValue - { key: output_tokens, type: i64 } # intValue cost_usd_micros is skipped as redundant under f64 aggregation; cache token counts are additive later if a query wants them. Cost stays the source's responsibility (Claude Code computes and stamps it); Ourios stores, attributes, and aggregates. Pricing tables and FOCUS-shaped output belong to consumers of the query surface. Verified: the server boots the config through the RFC0042.6 validation path (fails only on the port the running dogfood server holds). Promotion is a write-side projection — RFC0042.9's live sum lands with the first capture after the dogfood server restarts on this config. Signed-off-by: Jens Holdgaard Pedersen <jens@holdgaard.org> * docs(dogfood): a wrong-class promotion yields a NULL aggregate, not an empty result The comment claimed sum over an always-NULL column silently returns empty; the RFC0002.18 semantics is a NULL aggregate (every input excluded, value null). Same silence, precise shape. Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org> --------- Signed-off-by: Jens Holdgaard Pedersen <jens@holdgaard.org> Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org>
Promote the GenAI attributes in dogfood → native
count byFollow-up to #605/#606. Those wired the dogfood loop; this makes the aggregation half first-class. A source can now group its own telemetry natively through the loop instead of paging raw rows:
Why a config file (not more
OURIOS_*env vars)storage.promoted_attributes(RFC 0022) has no env-var form, and--configis the sole config source (mutually exclusive withOURIOS_*). So promoting anything requires the RFC 0020 file front-end.dogfood-config.yamlcarries everything the env vars did (loopback binds, open mode, local store/WAL, MCP on) plus the promoted set; paths resolve from${env:OURIOS_DOGFOOD_ROOT}, exported by the recipe.Scope / safety
dogfood-config.yaml+ thedogfood-serverrecipe switches to--config.authsection → RFC 0026 open mode).Verified end-to-end (throwaway instance, alt ports)
Synthetic GenAI logs → graceful-stop flush → restart → query:
count by attr.model→{claude-opus-4-8: 2, claude-haiku-4-5: 1}— exact input distribution, no "add tostorage.promoted_attributes" error.count by attr.tool_name→{Bash: 2, Read: 1}(rows_excluded: 3= the api_request rows with notool_name).${env:OURIOS_DOGFOOD_ROOT}substitution +promoted_attributesschema also validated (server boots, resolves the paths).Note:
sum/avgover these columns (total cost, total tokens) is still an RFC 0002 deferred obligation — a follow-up. This PR lands the grouped-count half.🤖 Generated with Claude Code