Skip to content

docs(rfc-0001): realign §6.8 telemetry export to OTel SDK + OTLP - #103

Merged
jensholdgaard merged 7 commits into
mainfrom
docs/rfc-0001-otel-telemetry-amendment
Jun 3, 2026
Merged

docs(rfc-0001): realign §6.8 telemetry export to OTel SDK + OTLP#103
jensholdgaard merged 7 commits into
mainfrom
docs/rfc-0001-otel-telemetry-amendment

Conversation

@jensholdgaard

@jensholdgaard jensholdgaard commented Jun 3, 2026

Copy link
Copy Markdown
Owner

What

Amends RFC 0001 §6.8 (Telemetry) to bless the OpenTelemetry SDK + OTLP push exporter as the metrics-export architecture, superseding the Prometheus client/scrape model (maintainer direction 2026-06-03, already pinned for compaction in RFC 0009 §3.6). This is the prerequisite slice the maintainer chose before standing up the MeterProvider bootstrap and instrumenting compaction (epic #94 audit+telemetry).

Decisions blessed here

  • OTel API/SDK dependency split (idiomatic OTel layering):
    • Library crates (ourios-miner, -parquet, -ingester, -querier) depend only on the lightweight opentelemetry API and resolve instruments via global::meter("ourios.<subsystem>") — no SDK / OTLP / transport in any library crate.
    • A new ourios-telemetry crate owns opentelemetry_sdk + opentelemetry-otlp + transport; init() builds the OTLP push MeterProvider (periodic reader, configurable interval), installs it as the process-global provider, returns a shutdown/flush guard. ourios-server calls it at start-up; benches/tests substitute an in-memory reader.
    • The new crate extends the CLAUDE.md §7 target layout; the new-crate commitment is blessed in-RFC per §7's rule.
  • Dimensions are OTel attributes, not Prometheus labels; the service dimension is the service.name resource attribute.

Terminology fixes (maintainer-sanctioned, check preserved)

Registry → meter provider, scrape → OTLP push, labels → attributes — across §§6.8–6.9, the §3.1.2 + RFC0001.8 scenarios, and the §8 metrics-registry test. The §3.1.2 invariant (full mandatory set is exposed) is unchanged — only the mechanism term changes. Roadmap §5 note and RFC 0008's two citations updated to the OTLP-push framing.

Deliberately deferred (NOT in this PR)

Per CLAUDE.md §6.2 I did not silently rewrite the normative mandatory-set contract. The dotted-semconv name redesign (template_countourios.miner.*, joining semconv/registry/ alongside the compaction set), the instrument-type changes, and the p50/p01 quantile question (RFC0001.8 — in-process gauges vs OTLP-native backend-derived quantiles) are genuine contract changes to the §3.1.2 set and are deferred to their own reviewed follow-up. RFC0001.8's assertion is preserved as-is, explicitly flagged for that redesign.

Invariants touched

§3.1 (mandatory metric set): exposure invariant preserved verbatim; only export mechanism + terminology reframed. No metric removed or renamed.

Verification

  • mdbook build
  • Docs-only PR (no code); the ourios-telemetry crate + bootstrap land in the follow-up code slice once these names/architecture are blessed.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Adopt OpenTelemetry SDK with OTLP metric export (push) as the normative telemetry model; Prometheus scraping moved to downstream collectors.
    • Replace “labels” with OTel “attributes”; clarify resource vs data-point attributes and telemetry lifecycle with init-seeded zero-valued measurements to expose full metric set (including replay).
    • Specify confidence_p50/p01 as in-process gauge views derived from histogram quantiles; note planned redesign follow-up.
  • Tests
    • Update tests to use an in-memory SDK collection asserting mandated metric names and matching quantiles.

@jensholdgaard
jensholdgaard requested a review from Copilot June 3, 2026 12:55
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: aeab7c94-f24b-46c6-a5cd-46c8fbe9d619

📥 Commits

Reviewing files that changed from the base of the PR and between 4514966 and b40b448.

📒 Files selected for processing (1)
  • docs/rfcs/0001-template-miner.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • docs/rfcs/0001-template-miner.md

📝 Walkthrough

Walkthrough

The PR updates RFCs and the roadmap to replace Prometheus scrape/registry telemetry language with an OpenTelemetry SDK + OTLP push model, introduces an ourios-telemetry init flow that installs a MeterProvider, converts labels to attributes, updates validation/testing to use an in-memory OTel reader, and clarifies WAL replay metric visibility.

Changes

OpenTelemetry Telemetry Architecture Redesign

Layer / File(s) Summary
OpenTelemetry export architecture and references
docs/rfcs/0001-template-miner.md, docs/roadmap.md, docs/rfcs/0008-wal.md
The §6.8 telemetry section is redesigned to describe an OTel SDK MeterProvider with OTLP push exporter and an ourios-telemetry init/shutdown guard; Prometheus scrape/registry terminology replaced and roadmap/RFC references updated.
Metric terminology and validation contracts
docs/rfcs/0001-template-miner.md
Metric definitions and acceptance criteria now use OpenTelemetry terms (meters, instruments, attributes); overflow metric dimensions reworded to attributes; confidence_p50/confidence_p01 defined as in-process gauge views derived from the confidence histogram quantiles; testing updated to use an SDK in-memory reader.
Replay visibility and export cycles
docs/rfcs/0001-template-miner.md
WAL replay semantics clarified: init-seeded zero-valued data points keep the full §6.8 metric set visible during export cycles while the metric update path is suppressed; wording changed from "scrape" to "metric export cycle".

🎯 3 (Moderate) | ⏱️ ~20 minutes


🐰 From scrape-era labels to attributes bright,

ourios-telemetry wakes at init() light,
Gauges hum p50 and p01 on a tick,
Instruments stand through replay, steady and quick,
Metrics pushed OTLP — a rabbit’s telemetry delight.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely summarizes the main change: realigning RFC 0001's §6.8 telemetry export architecture from Prometheus to OTel SDK + OTLP push model.
Description check ✅ Passed The PR description is comprehensive and well-structured, covering the summary, decisions blessed, terminology fixes, deliberately deferred work, and verification status, but lacks explicit checklist items.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/rfc-0001-otel-telemetry-amendment

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/rfcs/0001-template-miner.md`:
- Around line 1528-1530: The doc incorrectly lists `service` as a per-metric
attribute; update the RFC text and the metric-attribute table so `service` is
clearly a resource attribute only (use the standard OTel `service.name` resource
attribute set on the provider) and not repeated in per-instrument/metric
attributes; either move `service` into a “Resource attributes” column, remove it
from the “Instrument attributes” column, or add an explicit table note that
`service` == `service.name` is resource-scoped and must not be attached
per-metric to avoid double-attaching; ensure the same clarification is applied
to the repeated section around lines referencing 1538–1547.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f6fbb343-bc8a-4dfd-92d2-605a0c3f2a7f

📥 Commits

Reviewing files that changed from the base of the PR and between ce38e55 and 05e69f6.

📒 Files selected for processing (3)
  • docs/rfcs/0001-template-miner.md
  • docs/rfcs/0008-wal.md
  • docs/roadmap.md

Comment thread docs/rfcs/0001-template-miner.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the documentation/specs to realign RFC 0001 §6.8 Telemetry from a Prometheus client + /metrics scrape model to an OpenTelemetry SDK + OTLP push exporter architecture, and propagates the associated terminology updates across referenced docs.

Changes:

  • Update docs/roadmap.md to reflect OTLP push export (and explicitly de-scope scrape endpoints / direct Prometheus client usage).
  • Update docs/rfcs/0008-wal.md to cite the revised telemetry direction and RFC 0001 §6.8 as the normative architecture reference.
  • Amend docs/rfcs/0001-template-miner.md §6.8 telemetry and related scenarios/tests to use OTel terminology (meter provider, attributes, OTLP push) and describe the intended dependency split.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
docs/roadmap.md Updates the post-MVP telemetry roadmap note to OTLP push export framing.
docs/rfcs/0008-wal.md Updates WAL RFC references/wording to match the new OTel + OTLP export direction.
docs/rfcs/0001-template-miner.md Adds the 2026-06-03 telemetry architecture amendment and updates scenarios/terminology accordingly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
jensholdgaard and others added 2 commits June 3, 2026 16:09
Amend RFC 0001 §6.8 to bless the OpenTelemetry SDK + OTLP push
exporter as the metrics-export architecture, superseding the
Prometheus client/scrape model. Specifies the OTel API/SDK
dependency split (library crates depend only on the `opentelemetry`
API via `global::meter`; a new `ourios-telemetry` crate owns the
SDK/OTLP/transport deps and the MeterProvider lifecycle) and blesses
the new crate per CLAUDE.md §7.

Fixes the Prometheus-era terminology (registry -> meter provider,
scrape -> OTLP push, labels -> attributes) across §§6.8-6.9, the
§3.1.2 / RFC0001.8 scenarios, and the §8 metrics test, and updates
the roadmap §5 note and RFC 0008's citations to match.

Deliberately does NOT rename the metrics: the dotted-semconv
redesign (joining semconv/registry/ alongside the compaction set,
RFC 0009 §3.6) and the instrument-type / quantile questions are
genuine contract changes to the §3.1.2 mandatory set and are
deferred to their own reviewed follow-up. RFC0001.8's p50/p01
assertion is preserved, flagged for that redesign.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jensholdgaard
jensholdgaard force-pushed the docs/rfc-0001-otel-telemetry-amendment branch from 05e69f6 to e0c153f Compare June 3, 2026 14:13
@jensholdgaard
jensholdgaard requested a review from Copilot June 3, 2026 14:16
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
@jensholdgaard
jensholdgaard requested a review from Copilot June 3, 2026 14:28
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

Comment thread docs/roadmap.md Outdated
Comment thread docs/rfcs/0008-wal.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
@jensholdgaard
jensholdgaard requested a review from Copilot June 3, 2026 16:48
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
@jensholdgaard
jensholdgaard requested a review from Copilot June 3, 2026 16:55
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
@jensholdgaard
jensholdgaard requested a review from Copilot June 3, 2026 17:02
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
Comment thread docs/rfcs/0001-template-miner.md Outdated
@jensholdgaard
jensholdgaard requested a review from Copilot June 3, 2026 17:11
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@jensholdgaard
jensholdgaard merged commit ce28ee5 into main Jun 3, 2026
12 checks passed
jensholdgaard added a commit that referenced this pull request Jun 3, 2026
…bootstrap) (#104)

* feat(telemetry): scaffold ourios-telemetry crate (OTLP MeterProvider bootstrap)

First slice of the telemetry code work unblocked by RFC 0001 §6.8
(the OTel SDK + OTLP-push amendment, #103) and RFC 0009 §3.6 (#102).

Implements the §6.8 "Export architecture" dependency split: this is
the single crate that carries the heavy OTel SDK + OTLP exporter +
transport, so instrumented library crates can depend only on the
lightweight `opentelemetry` API (`global::meter(...)`).

- `init(&TelemetryConfig)` builds the OTLP push MeterProvider
  (periodic reader, configurable interval), sets `service.name =
  ourios-<role>` as a Resource attribute (not a data-point attribute,
  per §6.8), installs it as the process-global provider, and returns a
  `TelemetryGuard` whose `shutdown()` flushes on exit (Drop flushes
  best-effort).
- `init_in_memory` (behind the `testing` feature) wires an
  InMemoryMetricExporter for other crates' tests (e.g. the miner's §8
  metric-collection test) to collect the exported stream without an
  OTLP endpoint.

No instrumentation yet — that lands in the follow-on slices (weaver
name-constant codegen, then compaction instrumentation). Error type is
hand-rolled to match the workspace convention (no thiserror dep).

Verification: cargo fmt --check, clippy --all-targets --all-features
-D warnings, and cargo test --workspace --all-features all green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fixup! feat(telemetry): scaffold ourios-telemetry crate (OTLP MeterProvider bootstrap)

* fixup! fixup! feat(telemetry): scaffold ourios-telemetry crate (OTLP MeterProvider bootstrap)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants