docs(rfc-0001): pin the structured-body canonical encoding as an ourios-local rule by jensholdgaard · Pull Request #166 · jensholdgaard/ourios

jensholdgaard · 2026-06-09T19:43:54Z

What

Doc-only RFC amendment correcting an overclaim about the structured-body
encoding. RFC 0001 §6.1 / §6.2 step-0 and RFC 0005 §3.3 called it "the
OTLP-canonical JSON encoding per the OTLP specification's HTTP/JSON
binding," implying a spec-defined canonical form.

Why

Per the OTLP spec, the OTel common docs, and a maintainer answer (Josh
Suereth, 2026-06-09): there is no canonical OTLP JSON. OTLP/JSON is
the proto3 JSON mapping plus a short closed list of deviations
(hex trace_id/span_id, integer enums, ignore-unknown-fields,
lowerCamelCase) — with no normative rules on whitespace,
key/field ordering, or number canonicalisation. And OTLP does not
require lossless translation between formats ("we tie to the
JSON/JavaScript spec"). So any faithfulness/round-trip guarantee Ourios
makes for structured bodies is a local Ourios guarantee, not an OTLP
conformance claim.

What changed

Both RFCs are reframed to state the rule precisely. The encoding is
renamed (greppable, consistent across both RFCs) to "the Ourios
canonical body encoding" — a proto3-JSON form, explicitly an
Ourios-local deterministic convention, not an OTLP-mandated canonical
form.

The concrete rule (single-sourced in RFC 0001 §6.1; RFC 0005 §3.3 defers
to it):

proto3 JSON mapping as emitted by opentelemetry-proto's with-serde
via serde_json;
lowerCamelCase fields;
int64/uint64 as decimal strings (decoders accept number or
string);
bytes as base64;
KvlistValue/ArrayValue element order preserved — not sorted
(explicitly not RFC 8785 / JCS);
"deterministic" = byte-identical re-encode of the same in-memory
AnyValue (resolves a prior struct-vs-byte ambiguity);
the faithfulness guarantee (stored_bytes decode back to the original
AnyValue) and lossy_flag = false rest on this Ourios encoder,
not on an OTLP lossless promise;
duplicate keys are non-conforming OTLP (maps require unique keys;
semantics undefined) — Ourios preserves them verbatim (no silent
dedup/reorder) and flags them non-conforming.

Each section carries a dated Amendment 2026-06-09 note citing the
OTel-spec answer. No code touched (the ourios-core otlp::canonical
encoder already implements exactly this — preserve-order serde over the
proto types, int64-as-string, base64; a follow-up PR aligns its doc
comments). No RFC status change.

This unblocks the RFC0001.9 (body_kind = Structured) implementation by
giving it a precise, non-overclaimed encoding contract to build against.

Invariants touched

[§3.3] bit-identical body reconstruction — the reframe sharpens
this: the structured-branch round-trip is now stated as a byte-level
Ourios guarantee, not a vague OTLP promise. No behaviour change, no
schema-byte change.

Verify

mdbook build clean (only the benign mdbook-mermaid version warning).
Doc-only — no cargo.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

Documentation
- Updated RFC specifications for structured logging and Parquet storage to clarify canonical body encoding approaches
- Refined normative descriptions of how attributes and body columns are represented in structured logs
- Enhanced specification clarity for consistent deterministic encoding across systems

…os-local rule RFC 0001 §6.1 / §6.2 step-0 and RFC 0005 §3.3 called the structured-body encoding "OTLP-canonical JSON per the OTLP HTTP/JSON binding," implying a spec-defined canonical form. An OTel-spec answer (no canonical OTLP JSON; OTLP requires no lossless translation) shows there is none. Reframe both RFCs to state the rule is the Ourios canonical body encoding: an Ourios-local deterministic proto3-JSON form (lowerCamelCase fields, int64/uint64 as decimal strings, bytes as base64, kvlist/array order preserved — not sorted / not RFC 8785), byte-deterministic re-encode of the same in-memory AnyValue. The faithfulness / lossy_flag = false guarantee is an Ourios guarantee via the encoder, not an OTLP conformance point. Duplicate keys are non-conforming OTLP and preserved verbatim. RFC 0001 §6.1 stays the source of truth; RFC 0005 §3.3 defers to it. Doc-only; no status change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

jensholdgaard · 2026-06-09T19:44:04Z

@coderabbitai review

coderabbitai · 2026-06-09T19:44:09Z

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai · 2026-06-09T19:45:21Z

Warning

Review limit reached

@jensholdgaard, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 35 minutes and 28 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b1186f6a-cb53-4851-b498-b62abe60b3ab

📥 Commits

Reviewing files that changed from the base of the PR and between 401d838 and c4beb79.

📒 Files selected for processing (1)

docs/rfcs/0001-template-miner.md

📝 Walkthrough

Walkthrough

Two RFC specification documents are updated to redefine the structured body encoding standard from OTLP-canonical JSON to an Ourios-local deterministic proto3-JSON convention. The redefinition, dated 2026-06-09, is propagated through RFC0001 template mining and RFC0005 Parquet storage contracts.

Changes

Structured Body Encoding Standard Redefinition

Layer / File(s)	Summary
Canonical body encoding definition and scenario update `docs/rfcs/0001-template-miner.md`	Scenario RFC0001.9 is updated to state the structured `body` carries Ourios canonical encoding. A substantial amendment section (lines 875–961) explicitly defines the Ourios canonical body encoding standard as a deterministic proto3-JSON serialization convention with specific field naming (`lowerCamelCase`), numeric/string handling, base64 bytes, and order preservation for `kvlist`/`array` elements, plus explicit handling of non-conforming duplicate kvlist keys.
RFC0001 structured body contract propagation `docs/rfcs/0001-template-miner.md`	Record column documentation (§6.1), body representation narrative (§6.1), reconstruction well-definedness rationale, algorithm short-circuit (§6.2 from `canonicalise_to_otlp_json` to `encode_canonical_body`), and reader render contract (§6.6) are all updated to reference Ourios canonical body encoding. Rationale is rewritten to tie byte-determinism to the new Ourios-local encoding definition.
RFC0005 Parquet storage aligned to canonical encoding standard `docs/rfcs/0005-parquet-storage.md`	Parquet `BYTE_ARRAY` storage rule is updated to describe the payload as Ourios canonical body encoding rather than OTLP-canonical JSON. Acceptance criteria for RFC0005.1 are updated to reference the new encoding standard. RFC0001 and OTLP spec references are rewritten to emphasize that OTLP defines no canonical/byte-deterministic JSON and that the Ourios-local encoding ensures byte stability.

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

jensholdgaard/ourios#162: Both PRs update docs/rfcs/0001-template-miner.md's §6.6 reader-render/reconstruction contract around the non-reconstructable body_kind = Structured path (and associated mapping/short-circuit behavior), with the main PR changing it to use the Ourios canonical body encoding framing.
jensholdgaard/ourios#62: The main PR's RFC/spec changes reframe structured/parquet "canonical body encoding" to match the deterministic ourios-core canonical encoder, which is exactly what the retrieved PR implements and wires through ourios-core::otlp::canonical into structured mining and Parquet reader/writer for canonical JSON bytes.
jensholdgaard/ourios#61: The retrieved PR adjusts the bench harness/C1 logic to specially handle BodyKind::Structured records (skipping template-snapshot reconstruction and treating them as storage-layer round-trips), which directly aligns with the main PR's RFC updates that redefine the structured body representation/reconstruction contract around the Ourios canonical body encoding (replacing the earlier OTLP-canonical JSON framing).

Poem

🐰 With whiskers twitched and RFC refined,
The rabbit hops through specs redesigned,
"Ourios canonical," the new chant rings,
No OTLP JSON—deterministic things!
Byte-stable encoding, ordered and true,
Two documents aligned, our vision renewed! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: reframing the structured-body encoding as an Ourios-local rule rather than OTLP-canonical.
Description check	✅ Passed	The description comprehensively explains the RFC amendment, including the what, why, what changed, and verification steps, covering all essential context for understanding the change.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch docs/rfc-canonical-body-encoding

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Copilot

Pull request overview

This PR amends RFC 0001 and RFC 0005 to correct prior language that implied OTLP defines a canonical JSON encoding, and instead pins the structured-body encoding as an Ourios-local deterministic proto3-JSON convention (“Ourios canonical body encoding”), single-sourced in RFC 0001 §6.1.

Changes:

Reframe the structured-body encoding as an Ourios-local, byte-deterministic proto3-JSON convention (not an OTLP conformance/canonicalization claim), with dated amendment notes.
Update RFC 0005 to defer to RFC 0001 §6.1 as the single source of truth for the encoding rule and adjust “faithfulness” rationale accordingly.
Update acceptance-criteria and reference sections to use the new naming and clarify that OTLP defines no canonical/byte-deterministic JSON.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
docs/rfcs/0005-parquet-storage.md	Rewords §3.3 and related references to defer to RFC 0001 §6.1 and replace “OTLP-canonical JSON” with the Ourios-local canonical body encoding framing.
docs/rfcs/0001-template-miner.md	Renames and precisely specifies the Ourios canonical body encoding for `body_kind = Structured`, including amendment rationale and updated algorithm text.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ate keys MinedRecord has only lossy_flag + no non-conformance audit payload, so spec only the behavior that exists: duplicate keys are preserved verbatim with no map-semantic guarantee, not flagged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

…ementation The miner encodes structured bodies at ingest (ingest_structured), not at Parquet-write time — reword RFC0001.9 to be location-agnostic (the emitted record body carries the canonical encoding) rather than contradicting the code. And the §6.1 body type is Option<String> (UTF-8, record.rs), not Option<Bytes>; the BYTE_ARRAY is the RFC 0005 column. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

…1.9) (#167) Implements RFC 0001 scenario RFC0001.9 (structured-body short-circuit) + the §6.6 structured render, against the merged "Ourios canonical body encoding" pin (#166). - Flips RFC0001.9: a Body::Structured(AnyValue) ingests to body_kind=Structured with a structured-template id keyed on (severity_number, scope_name, BodyKind::Structured) (reused across same tuples, distinct across different), the canonical body round-trips via canonical::decode_any_value, lossy_flag=false, params/separators empty, confidence=1.0. - Byte-shape verification test locking the encoder output (int64 decimal-string, base64 bytes, lowerCamelCase, kvlist/array order preserved — not sorted). - §6.6 structured render: BodyKind::Structured with a body → (canonical bytes, Reconstruction::Faithful); missing body → (empty, RetainedVerbatim); Absent → empty + RetainedVerbatim (no stray-byte leak). render no longer panics on non-String. Invariant §3.3: structured reconstruction faithfulness is the Ourios round-trip via the canonical encoding. The RFC 0003 §6.4 ingest-vs-write-time location is a separate maintainer item (not touched). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…174) Reconciles RFC 0003 §6.4 with the merged implementation: structured-body canonicalisation to the Ourios canonical body encoding (RFC 0001 §6.1, #166) happens once at INGEST — ingest_structured encodes the AnyValue the receiver delivered, and the record carries the bytes through WAL→Parquet. The prior write-time text predated the implementation. The original optionality rationale is preserved, not lost: the receiver still hands the miner the decoded AnyValue verbatim, and ingest_structured is exactly where a future inner-field-mining mode would hook — only the STORED form is bytes. Residual write-time claims swept in §1, §2.2, the RFC0003.6 parenthetical, §7.5 and §9; RFC 0001/0005 confirmed already clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ror text The non-finite test now pins both halves of the gap (null shape emitted AND decode rejects). Error Display + doc mentions say Ourios-canonical, matching the post-#166 naming. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ld placeholder The Debug-rendering interim-placeholder claim and the deferred-to- storage-layer claim were both stale (false post-#163/#166/#174); reviewer-flagged, so fixed here rather than the planned follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…anonical (#333) * docs(querier): finish #177's doc sweep — the codec is Ourios-canonical, not OTLP-canonical The last two stale mentions from #177's ride-along list: compile.rs still called the attributes-column encoding "OTLP-canonical-JSON". Post-#166 the codebase convention is "Ourios-canonical" — deliberately so, since the codec deviates from opentelemetry-proto's with-serde exactly where with-serde deviates from the proto3 JSON mapping (non-finite doubles as "NaN"/"Infinity"/"-Infinity" strings, RFC 0018 §3.4 — the #177 core bug, fixed in #271). Doc-comment-only change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: complete the repo-wide Ourios-canonical sweep (miner + parquet) Copilot caught that my "last remaining mentions" claim was wrong — I had grepped for cluster.rs / record_batch.rs under ourios-querier, where those files don't exist (they live in ourios-miner and ourios-parquet), and misread no-matches as already-clean. Repo-wide sweep now: - cluster.rs: rename the §3.3 encoding mention, and rewrite the genuinely stale test note that still described `body` as "the AnyValue's Debug form — an interim placeholder" awaiting a future canonicalisation PR (that PR landed long ago; the body is the Ourios-canonical JSON from `ingest_structured` → `canonical::encode_any_value`). - record_batch.rs: rename both mentions. The remaining "OTLP-canonical" hits in docs/ are historical amendment notes that *describe* the old name (e.g. RFC 0001/0005 "previously called the encoding…") and are correct as written. Comment-only; `cargo grep` for OTLP-canonical in crates/ is now empty. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

jensholdgaard requested a review from Copilot June 9, 2026 19:43

Copilot started reviewing on behalf of jensholdgaard June 9, 2026 19:44 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread docs/rfcs/0001-template-miner.md Outdated

jensholdgaard requested a review from Copilot June 9, 2026 19:52

Copilot started reviewing on behalf of jensholdgaard June 9, 2026 19:52 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread docs/rfcs/0001-template-miner.md Outdated

Comment thread docs/rfcs/0001-template-miner.md Outdated

jensholdgaard requested a review from Copilot June 9, 2026 20:08

Copilot started reviewing on behalf of jensholdgaard June 9, 2026 20:08 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

jensholdgaard merged commit a6e8561 into main Jun 9, 2026
11 checks passed

jensholdgaard mentioned this pull request Jun 9, 2026

feat(miner): structured-body short-circuit + canonical render (RFC0001.9) #167

Merged

jensholdgaard mentioned this pull request Jun 10, 2026

docs(rfc-0003): reconcile §6.4 — canonicalisation happens at ingest, not Parquet-write time #174

Merged

coderabbitai Bot mentioned this pull request Jun 11, 2026

fix(core): make the canonical-encoding f64 round-trip bit-exact (local half of #130) #176

Merged

jensholdgaard mentioned this pull request Jun 11, 2026

non-finite doubles (NaN/±Inf) break the canonical round-trip — null emitted, decode rejects #177

Closed

coderabbitai Bot mentioned this pull request Jun 20, 2026

docs(rfc-0018): OTLP log-spec compliance amendments #265

Merged

coderabbitai Bot mentioned this pull request Jul 3, 2026

docs(rfc): rfc 0022 — queryable attribute columns (rfc 0005 amendment) #343

Merged

coderabbitai Bot mentioned this pull request Jul 22, 2026

docs(rfc): specify RFC 0037 — GenAI / structured-event log handling #598

Merged

Navigation Menu

docs(rfc-0001): pin the structured-body canonical encoding as an ourios-local rule - #166

docs(rfc-0001): pin the structured-body canonical encoding as an ourios-local rule#166
jensholdgaard merged 3 commits into
mainfrom
docs/rfc-canonical-body-encoding

jensholdgaard commented Jun 9, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

jensholdgaard commented Jun 9, 2026

Uh oh!

coderabbitai Bot commented Jun 9, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jun 9, 2026 •

edited

Loading

Review limit reached

Walkthrough

Changes

Possibly related PRs

Poem

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jensholdgaard commented Jun 9, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

What changed

Invariants touched

Verify

Summary by CodeRabbit

Release Notes

Uh oh!

jensholdgaard commented Jun 9, 2026

Uh oh!

coderabbitai Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Walkthrough

Changes

Possibly related PRs

Poem

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jensholdgaard commented Jun 9, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 9, 2026 •

edited

Loading

coderabbitai Bot commented Jun 9, 2026 •

edited

Loading