feat(miner): observe structured-body size via a histogram (RFC 0037 §3.2)

[jensholdgaard]

RFC 0037 slice B — structured_body_bytes observability (§3.2)

Second of three implementation slices (after slice A #599). Implements the hazard-#2 guard for structured bodies.

Design (Option A, maintainer-decided in the spec). A structured (non-string) log body — a GenAI/agent event body — is retained whole and never capped. Truncating would violate §3.3 (bit-identical reconstruction), and the body column has dictionary encoding disabled by design (writer.rs §3.6), so hazard #2's dictionary-collapse failure mode is structurally absent for it. The guard against oversized payloads is therefore observation, not truncation: a new ourios.miner.structured_body.size histogram (unit By, dimensioned by ourios.service) records the canonical-JSON byte length of every structured body, so an operator can spot a service emitting large payloads and fix it at the emitter.

Consolidated, not copy-pasted. The instrument is one Histogram<u64> field added to the existing MinerMetrics, built on the ourios.miner meter, and recorded via a record_structured_body_bytes method that reuses the shared service_attrs helper and mirrors record_overflow. The record call sits in ingest_structured, capturing bytes.len() before the canonical-JSON bytes are moved into the record.

Weaver-authored name. The metric name is defined in semconv/registry/metrics.yaml (copying the ourios.miner.confidence group shape — instrument: histogram, unit: By, ourios.tenant required + ourios.service recommended) and regenerated into crates/ourios-semconv/src/lib.rs via weaver registry generate. Not hand-written, so the CI no-diff and live-check gates stay green. No new attribute needed — ourios.tenant/ourios.service already exist.

Invariants (CLAUDE.md §4)

• Hazard docs: apply RFC maturity-model amendments #2 (parameter cardinality / body size). This slice closes the documented gap (the structured-body amendment to docs/hazards.md H2 that landed with the spec): structured bodies were previously an unguarded ingress; they now have the metric + per-service-alert guard, with fidelity (§3.3) unconditionally preserved.
• §6.3 observability. OTel meter, not a Prometheus client.

Verification

• RFC0037.3 — rfc0037_structured_body.rs asserts both unbounded fidelity (a large structured body — ~200-element array, >8 KB — is retained byte-for-byte with lossy_flag = false) and the metric emission (histogram sum equals the canonical-JSON byte length, data point carries ourios.service). ✅
• fmt + clippy clean; weaver registry check passes; semconv in sync (no-diff).

Refs #546. Slice C (§3.3 group-by-promoted-attribute) follows; RFC flips to green once all §5 pass (RFC0037.2 / .5 verified against existing RFC 0005 / 0025 coverage at that point).

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added monitoring for the canonical JSON size of structured log bodies.
  • Metrics now include tenant and service context for easier analysis.
  • Added semantic convention and registry definitions for the new histogram metric.

• Tests

  • Added coverage confirming structured bodies are preserved accurately and reported with the correct size.

[coderabbitai]

Warning

Review limit reached

@jensholdgaard, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 40 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ce9d7cef-70f1-4b08-abdb-4255d633ba4e

📥 Commits

Reviewing files that changed from the base of the PR and between cae9536 and 1c0d228.

📒 Files selected for processing (2)

• crates/ourios-miner/src/cluster.rs
• crates/ourios-miner/tests/rfc0037_structured_body.rs

📝 Walkthrough

Walkthrough

Structured-body ingestion now records canonical JSON payload sizes in a new per-tenant/service histogram. Semantic-convention metadata, metric registration, ingestion wiring, and an RFC0037 integration test were added.

Changes

Structured body metrics

Layer / File(s)	Summary
Metric contract and registration semconv/registry/metrics.yaml, crates/ourios-semconv/src/lib.rs, crates/ourios-miner/src/metrics.rs	Defines ourios.miner.structured_body.size as a By histogram, registers its semantic-convention constant, and adds the miner metric handle and recording method.
Ingest recording and validation crates/ourios-miner/src/cluster.rs, crates/ourios-miner/tests/rfc0037_structured_body.rs	Records canonical JSON byte length after structured-body encoding and validates record fidelity, lossiness, service attributes, and histogram sum.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the structured-body size histogram change and RFC slice.
Description check	✅ Passed	The description covers the summary, RFC linkage, and verification details, and includes the key checklist evidence even if not in the template’s exact format.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch rfc0037-slice-b-structured-body-metric

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

Pull request overview

Adds observability for structured (non-string) OTLP log bodies in the miner by recording their canonical-JSON size as a per-tenant / per-service histogram, aligning with RFC 0037 §3.2’s “observe, don’t truncate” approach for structured bodies.

Changes:

• Define a new semantic-convention metric ourios.miner.structured_body.size (histogram, unit By) and regenerate the ourios-semconv constant.
• Extend MinerMetrics with a Histogram<u64> and a record_structured_body_bytes helper that reuses the shared (tenant, service) attribute builder.
• Record structured-body byte length during structured ingest, and add an acceptance test asserting both byte-for-byte retention and correct histogram emission (including ourios.service).

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
semconv/registry/metrics.yaml	Adds the metric definition for ourios.miner.structured_body.size with required/recommended attributes.
crates/ourios-semconv/src/lib.rs	Regenerates semconv constants to include OURIOS_MINER_STRUCTURED_BODY_SIZE.
crates/ourios-miner/src/metrics.rs	Adds the histogram instrument and a record_structured_body_bytes method using existing attribute helpers.
crates/ourios-miner/src/cluster.rs	Records the structured-body canonical-JSON size at ingest time before moving bytes into the emitted record.
crates/ourios-miner/tests/rfc0037_structured_body.rs	New RFC0037.3 acceptance test covering unbounded structured-body fidelity and metric emission.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/ourios-miner/tests/rfc0037_structured_body.rs`:
- Around line 59-62: Update the fidelity assertions in the structured-body test
around expected_bytes to retain and compare the full canonical byte sequence,
not just its length. Reuse the output of canonical::encode_any_value as the
expected bytes, and assert byte-for-byte equality against the retained body;
keep the existing length metric assertion as needed.
- Around line 45-122: The existing RFC0037 integration test needs complementary
colocated unit coverage for structured-body ingestion. Add a focused test module
alongside the implementation in cluster.rs or metrics.rs, exercising the
non-trivial structured-body behavior—whole-body retention without lossy marking
and the corresponding size metric—while keeping the existing end-to-end test
unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a6aa3a1b-bb21-4c9b-b32c-0dea74eb0527

📥 Commits

Reviewing files that changed from the base of the PR and between 109933a and cae9536.

📒 Files selected for processing (5)

• crates/ourios-miner/src/cluster.rs
• crates/ourios-miner/src/metrics.rs
• crates/ourios-miner/tests/rfc0037_structured_body.rs
• crates/ourios-semconv/src/lib.rs
• semconv/registry/metrics.yaml

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.