Skip to content

docs(rfc-0003): reconcile §6.4 — canonicalisation happens at ingest, not Parquet-write time - #174

Merged
jensholdgaard merged 1 commit into
mainfrom
docs/rfc-0003-canonicalisation-at-ingest
Jun 10, 2026
Merged

docs(rfc-0003): reconcile §6.4 — canonicalisation happens at ingest, not Parquet-write time#174
jensholdgaard merged 1 commit into
mainfrom
docs/rfc-0003-canonicalisation-at-ingest

Conversation

@jensholdgaard

Copy link
Copy Markdown
Owner

What

Doc-only RFC amendment reconciling RFC 0003 §6.4 with the merged implementation, per the inconsistency flagged on #166/#167 and the maintainer decision of 2026-06-10. No code, no status change.

§6.4 claimed structured-body canonicalisation "happens once, at Parquet-write time" / "is deferred to the storage layer." The merged code disagrees: ourios-miner's MinerCluster::ingest_structured encodes the AnyValue the receiver delivered via ourios_core::otlp::canonical::encode_any_value at ingest and stores the bytes in MinedRecord.body; the Parquet writer persists them verbatim (RFC 0005 §3.3). RFC 0001 §6.1/§6.2 and RFC 0005 §3.3 were already aligned in #166; §6.4 was the last spot still claiming write-time.

Changes (all in docs/rfcs/0003-otlp-receiver.md)

  • §6.4 — retitled to "AnyValue canonicalisation happens once, at ingest", with a dated > Amendment 2026-06-10 blockquote (mirroring the docs(rfc-0001): pin the structured-body canonical encoding as an ourios-local rule #166 amendment style) noting the prior text predated the implementation. The historical first-amendment blockquote is preserved. The three-point rationale is rewritten; what was already true stays: the receiver never canonicalises, and the miner's step-0 short-circuit dispatches on the discriminator alone before any encode.
  • §1 / §2.2 / RFC0003.6 — residual "deferred to the storage layer" / "Parquet writer's contract" phrasings updated to point at the ingest-time contract. RFC0003.6's actual assertion (AnyValue-tree equality across transports, no byte-level canonicalisation at the receiver layer) is unchanged — only the parenthetical cross-reference moved.
  • §7.5 — dated amendment note: the "canonicalise at the storage layer" conclusion is superseded; variant (a)'s rejection grounds dissolved post-docs(rfc-0001): pin the structured-body canonical encoding as an ourios-local rule #166 (one transport-agnostic encoder over the decoded AnyValue; no transport knowledge needed in the miner). Original evaluation text preserved as the record, with superseded-markers.
  • §9 — the "[x] Where does canonicalisation cost land?" resolution updated to the ingest-time answer.

Untouched because still true: §6.1 step 3, §6.2 ("no separate canonicalisation pass" at the HTTP decode), §6.5 step 2, §6.6 amendment note, RFC0003.7, and the Body::String pass-through paragraph. RFC 0001 / RFC 0005 grepped for residual write-time claims: none found (the "OTLP-canonical severity+scope key" line in RFC 0001 is unrelated and untouched).

The preserved-optionality argument

The original write-time rationale was "optionality for a future mine-inner-fields mode." That optionality is not lost by ingest-time encoding: the receiver still hands Body::Structured(AnyValue) through verbatim, and ingest_structured receives the decoded tree — exactly where an inner-field-mining mode would hook in. Only the stored form is the canonical bytes; the encode point sits after any future inner-field hook, so no option is foreclosed. Meanwhile "single canonicalisation pass" and "miner hot path unchanged" hold either way (one encode per structured record regardless of which layer pays; step-0 dispatch reads only the discriminator), and moving the code to write-time would rework MinedRecord, the §6.9 snapshot serialisation, the §6.6 render, and their tests — churn purchased for a mode with no RFC and no named consumer.

Invariants / hazards

Touches the spec text around [§3.3] (body reconstruction) and hazard H7 only by describing the implemented behaviour more accurately — the stored_bytes ↔ AnyValue round-trip guarantee and the lossy_flag = false promise are restated, not altered. No behaviour, schema, or test changes.

Verification

  • mdbook build clean (only the known benign mdbook-mermaid version warning).
  • git status shows only docs/rfcs/0003-otlp-receiver.md; doc-only, no cargo checks applicable.
  • Re-grepped docs/rfcs/ for write[- ]time / storage layer / canonicalis: every remaining hit in RFC 0003 is inside a historical amendment blockquote or an explicitly superseded-record passage.

🤖 Generated with Claude Code

…not Parquet-write time

Maintainer-decided 2026-06-10 reconciliation of RFC 0003 to the merged
implementation: the miner's ingest_structured encodes the AnyValue the
receiver delivered (Ourios canonical body encoding, RFC 0001 §6.1) at
ingest; the Parquet writer persists the bytes verbatim (RFC 0005 §3.3).
Amends §6.4 with a dated note, rewrites the three-point rationale
(inner-field optionality is preserved because the miner receives the
decoded tree), and updates the residual write-time claims in §1, §2.2,
RFC0003.6, §7.5, and the §9 resolved entry. Doc-only; status unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@jensholdgaard, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 17 minutes and 52 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e45cb15a-2d4e-4b9f-b171-65e329c50675

📥 Commits

Reviewing files that changed from the base of the PR and between 31a5181 and d451f9b.

📒 Files selected for processing (1)
  • docs/rfcs/0003-otlp-receiver.md
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/rfc-0003-canonicalisation-at-ingest

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@jensholdgaard
jensholdgaard requested a review from Copilot June 10, 2026 21:42
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates RFC 0003 to accurately describe the implemented location of AnyValue canonicalisation (performed once in ourios-miner at ingest) and reconciles remaining references that previously claimed canonicalisation was deferred to Parquet-write time.

Changes:

  • Amend §6.4 to state canonicalisation happens at ingest in the miner, preserving historical amendment notes while adding a new dated amendment record.
  • Update related references in §§1, 2.2, RFC0003.6, §7.5, and §9 so they no longer imply write-time/storage-layer canonicalisation.
  • Preserve historical evaluation text while marking previously reached conclusions as superseded by the merged implementation.
Comments suppressed due to low confidence (1)

docs/rfcs/0003-otlp-receiver.md:972

  • Section 7.5(a) currently reads as both “Rejected” and “implemented design,” which is internally contradictory and can confuse readers skimming the alternatives. Consider rephrasing (a) as a historical evaluation that is now implemented, and drop the separate “Superseded … implemented design” line to avoid the contradiction.
**(a) Canonicalise in the miner.** **Rejected** because the
miner's hot path benefits from a constant-time write in the

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jensholdgaard
jensholdgaard merged commit dc9d809 into main Jun 10, 2026
11 checks passed
jensholdgaard added a commit that referenced this pull request Jun 11, 2026
…ld placeholder

The Debug-rendering interim-placeholder claim and the deferred-to-
storage-layer claim were both stale (false post-#163/#166/#174);
reviewer-flagged, so fixed here rather than the planned follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants