docs(rfc): specify RFC 0037 — GenAI / structured-event log handling

[jensholdgaard]

RFC 0037 — GenAI / structured-event log handling (specified)

OpenTelemetry models LLM/agent activity as log events (event_name + a structured AnyValue body such as gen_ai.input.messages). Ourios already handles the base case correctly — structured bodies stored whole as canonical JSON (body_kind = Structured), event_name a first-class column, RFC 0025 covering the event-shaped absent-body record. This RFC scopes three deltas on top:

1. §3.1 — fold event_name into the structured-template key (severity, scope, event_name) so distinct event types get distinct template_ids.
2. §3.2 — close a hazard-docs: apply RFC maturity-model amendments #2 gap (structured bodies bypass the param_byte_limit guard) without violating §3.3 fidelity. Direction resolved (maintainer): Option A — never truncate; guard by a structured_body_bytes metric + per-service alert, not a cap. (Former Option B rejected, §4.)
3. §3.3 — extend count … by to group on a promoted attribute column, so count by attr.gen_ai.request.model is expressible.

Plus §3.5: why a true GenAI vertical slice is rejected (promotion already yields the typed columns), and the SIG-grounded recommended promotion set. §3.4 corpus: a synthetic fixture now, Claude Code's own OTLP export normalized to gen_ai.* via a Collector+OTTL step (the architecturally-correct placement) as a real available-now source, and otel-demo v9 for native gen_ai.* validation.

Status

specified — the §3.2 fork is decided, §5 acceptance criteria are frozen (canonical scenario format), and docs/hazards.md H2 is amended. This PR is the frozen contract; implementation follows in three verified slices (§3.1 → §3.2 → §3.3), each red→green, reaching validated on the v9 corpus.

Invariants/hazards touched (CLAUDE.md §4)

Hazard #1 (miner — template population, re-bases RFC 0024 calibration), #2 (structured-body guard), #6 (DSL group-by surface). Invariant §3.3 (fidelity) preserved unconditionally; §3.5 (schema) additive only.

🤖 Generated with Claude Code

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds RFC 0037 for structured GenAI log events, documenting event-aware templates, uncapped structured bodies, promoted-attribute grouping, calibration, acceptance criteria, and related hazard guidance.

Changes

GenAI structured log events

Layer / File(s)	Summary
RFC model and scope docs/rfcs/0037-genai-structured-log-events.md	Introduces the structured GenAI event model, invariants, and RFC scope.
Design and calibration rules docs/rfcs/0037-genai-structured-log-events.md	Defines event-specific template keys, structured-body observability without truncation, promoted-attribute grouping, and v9 corpus calibration.
Validation and documentation wiring docs/rfcs/0037-genai-structured-log-events.md, docs/hazards.md, docs/SUMMARY.md	Documents rejected alternatives, acceptance and testing criteria, resolved decisions, references, hazard guidance, and the RFC index entry.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

• jensholdgaard/ourios#166: Defines the deterministic canonical encoding invariant referenced by RFC 0037.
• jensholdgaard/ourios#264: Covers template registry and structured-body reconstruction behavior referenced by RFC 0037.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is substantive but does not follow the required template sections for Summary, Related, and Checklist.	Add the missing ## Summary, ## Related, and ## Checklist sections, including the required checklist items and an RFC link if applicable.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly matches the RFC 0037 structured-event log handling change and is concise.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch rfc0037-genai-structured-events

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

Pull request overview

Adds RFC 0037 to specify how Ourios should handle OpenTelemetry GenAI-style structured log events (event_name + structured AnyValue bodies), and updates the hazards doc to explicitly cover structured bodies under the parameter-cardinality hazard.

Changes:

• Add RFC 0037 defining three deltas: event_name-keyed structured templates, structured-body hazard mitigation via observability, and count … by support for promoted attribute columns.
• Link RFC 0037 into the mdBook navigation.
• Amend docs/hazards.md H2 to cover structured bodies and their (proposed) mitigation approach.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
docs/SUMMARY.md	Adds RFC 0037 to the RFC list for mdBook navigation.
docs/rfcs/0037-genai-structured-log-events.md	New RFC specifying intended GenAI/structured-event log handling behavior and acceptance criteria.
docs/hazards.md	Extends hazard H2 to explicitly address structured (non-string) bodies and the intended mitigation strategy.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

Actionable comments posted: 3

🧹 Nitpick comments (1)

docs/rfcs/0037-genai-structured-log-events.md (1)

273-298: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use the repository’s acceptance-scenario blockquote format.

Based on learnings, each §5 scenario should be a separate > **Scenario ...** blockquote with blank lines between scenarios rather than a bullet list.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/rfcs/0037-genai-structured-log-events.md` around lines 273 - 298,
Rewrite the acceptance criteria in §5 from bullet items into separate
blockquotes using the repository’s `> **Scenario ...**` format, with a blank
line between each scenario. Preserve the existing RFC0037.1–RFC0037.5
identifiers and all scenario details unchanged.

Source: Learnings

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/rfcs/0037-genai-structured-log-events.md`:
- Around line 14-19: Correct the status sentence in the RFC so its
acceptance-criteria date is not later than the document’s July 22, 2026 date, or
explicitly state that the document is intentionally postdated; preserve the
existing status and section references.
- Around line 156-167: The promoted-attribute check in field_group_expr must
also require an explicit groupability policy, rather than treating every
promoted column as groupable. Add groupability metadata for promoted fields,
mark gen_ai.conversation.id and gen_ai.response.id as filtering-only, reject
them with the promotion/grouping guidance error, and extend acceptance coverage
for these grouping attempts while preserving grouping for promoted groupable
attributes.
- Around line 124-147: Define an explicit admission/backpressure procedure for
oversized structured bodies in the structured-body policy at
docs/rfcs/0037-genai-structured-log-events.md:124-147, including the
availability behavior when the soft threshold is exceeded; do not rely on
structured_body_bytes alerting or RFC 0036 storage sizing as the resource bound.
Update the corresponding hazard entry at docs/hazards.md:101-114 to describe the
same admission/backpressure control and avoid presenting observation alone as
sufficient protection.

---

Nitpick comments:
In `@docs/rfcs/0037-genai-structured-log-events.md`:
- Around line 273-298: Rewrite the acceptance criteria in §5 from bullet items
into separate blockquotes using the repository’s `> **Scenario ...**` format,
with a blank line between each scenario. Preserve the existing
RFC0037.1–RFC0037.5 identifiers and all scenario details unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 65e13240-d710-4452-ba00-f4e9ff03bef0

📥 Commits

Reviewing files that changed from the base of the PR and between 028d369 and 17aa0bd.

📒 Files selected for processing (3)

• docs/SUMMARY.md
• docs/hazards.md
• docs/rfcs/0037-genai-structured-log-events.md

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

docs/rfcs/0037-genai-structured-log-events.md:135

• This contains an unresolved wiki-style reference [[telemetry-otel-meters-not-prometheus-client]] that doesn’t appear elsewhere in the docs, so it’s likely to render as literal text and not as a link. Suggest replacing it with an explicit reference to the existing “OTel meters, not the Prometheus client” convention (e.g. RFC 0016 §3.6).

- **`structured_body_bytes` metric.** A histogram (OTel meter, per
  [[telemetry-otel-meters-not-prometheus-client]]) observing the
  canonical-JSON byte length of every structured body, dimensioned by
  service so an operator can see which service emits large bodies.

docs/hazards.md:109

• This paragraph explains the structured-body path as avoiding Parquet dictionary encoding because bodies are “unique per record”. In the current implementation, the stronger/clearer reason is that the body column is explicitly configured with dictionary encoding disabled (e.g. crates/ourios-parquet/src/writer.rs:993-1005). It would be more robust to document the actual encoding policy rather than relying on uniqueness assumptions.

column and is **not** capped: truncating it would violate the
bit-identical-reconstruction invariant (`CLAUDE.md` §3.3), and unlike
a runaway `params` slot it never populated dictionary encoding to
begin with (structured bodies are unique per record). The blowup risk
here is raw storage size, not dictionary collapse, so it is guarded by

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.