Skip to content

docs(rfc): RFC 0033 §3.2 amendment — compressed artifact encoding (run #20 answer) - #521

Merged
jensholdgaard merged 3 commits into
mainfrom
rfc0033-amend-compressed-artifact
Jul 13, 2026
Merged

jensholdgaard merged 3 commits into
mainfrom
rfc0033-amend-compressed-artifact

Conversation

@jensholdgaard

Copy link
Copy Markdown
Owner

What

The RFC 0033 §3.2 amendment answering run #20 (§9.14): zstd-compressed artifact at a new key template_map.v2.json.zst, format_version: 2. Docs only — dated amendment blocks in the RFC 0031 §3.6 style; the v1 design stays readable underneath.

Key decisions, with the rejected fork recorded:

  • Version-in-key over magic-sniffing. Same-key zstd sniffing is correct but lies twice: old readers would classify a healthy v2 artifact Torn (§3.7 pins torn as the corruption page-signal — a mixed fleet would page on healthy state), and §3.3's UnknownVersion disposition becomes unreachable. A new key makes the bump literal absence for old readers — §3.3's unknown-version-is-absent rule realized in its cleanest form. Both audit walks ignore non-*.parquet keys by construction (verified at audit_scan.rs local + S3 arms), so the second key never enters LIST freshness.
  • Publish hygiene: best-effort delete of the stale v1 key after a successful v2 publish; crash between the two is harmless (§3.4 amendment; §7 gains the hygiene open question).
  • Abstention unchanged in spirit: publish iff the compressed size beats the folded audit bytes. §3.7's artifact-size histogram records compressed (published-object) bytes — the GET cost it always meant.
  • §5/§6 scenario text unchanged — RFC0033.6's "artifact object's byte size" is what a warm GET pays, encoding-agnostic. §6 amendment pins validation as comparative run docs(rfc-0001): amend §6.1 to align record schema with OTLP LogRecord #21 from the implementation branch before merge, with the harness printing each pair's publish outcome (published/abstained + would-be size vs fold/lost_race/error) so run docs(architecture): OTLP log-format gap analysis (Half 1 investigation) #20's abstention-vs-failure ambiguity cannot recur.
  • Zero new dependencies: zstd 0.13 is already in every querier build (parquet's zstd feature; ourios-bench binds it directly as the A1 reference codec) and already on the deny.toml allow-list.

Checks run

mdbook build clean. Docs only.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

…#20 answer)

Run #20 (§9.14) showed the uncompressed-JSON artifact never beating
the 513,862 B zstd-Parquet audit fold on otel-demo-v8, so §3.2's size
abstention correctly refused every publish and the RFC0033.6 corpus
arm went unmeasured. This amendment (dated blocks, RFC 0031 §3.6
precedent — v1 design stays readable) pins the answer:

- zstd-compressed JSON body at a NEW key, template_map.v2.json.zst
  (version-in-key: old readers see literal absence — no false 'torn'
  corruption telemetry in mixed-version fleets; the same-key
  magic-sniff alternative is recorded as rejected).
- format_version bumps to 2 in the decompressed body — the version
  names the whole artifact contract including transport encoding;
  the in-body probe stays as defense-in-depth.
- Abstention unchanged in spirit: publish iff the COMPRESSED size
  beats the folded audit bytes; §3.7 artifact-size histogram records
  compressed (published-object) bytes, the GET cost it always meant.
- §3.4 publish gains a best-effort delete of the stale v1 key.
- §5/§6 scenario text unchanged (warm == artifact size still holds:
  the GET is the compressed object); validation = comparative run
  #21 from the implementation branch before merge, harness printing
  each pair's publish outcome (published/abstained/lost_race/error)
  so run #20's abstention-vs-failure ambiguity cannot recur.
- Zero new deps: zstd 0.13 already in every querier build (parquet
  58 zstd feature; ourios-bench binds it as the A1 codec).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
@jensholdgaard
jensholdgaard requested a review from Copilot July 13, 2026 16:06
@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@jensholdgaard, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 14 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3f810f7d-7099-4c28-803e-3f481514edc9

📥 Commits

Reviewing files that changed from the base of the PR and between 1fdbd27 and c4f2c10.

📒 Files selected for processing (1)
  • docs/rfcs/0033-cached-template-map.md
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch rfc0033-amend-compressed-artifact

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates RFC 0033’s cached template-map artifact specification to introduce a zstd-compressed v2 artifact at a new, versioned key, clarifying mixed-version behavior and publish hygiene based on benchmark run #20 findings.

Changes:

  • Amend §3.2 to specify zstd-compressed artifact encoding with format_version: 2 and a new key template_map.v2.json.zst.
  • Add corresponding amendment notes in §3.3 and §3.4 (read dispositions, publish behavior, best-effort v1 key deletion).
  • Add cross-section amendment pointers/notes tying the change to acceptance/validation guidance (incl. comparative run #21 expectations).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/rfcs/0033-cached-template-map.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment thread docs/rfcs/0033-cached-template-map.md Outdated
Comment thread docs/rfcs/0033-cached-template-map.md

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@jensholdgaard
jensholdgaard merged commit ae5cebd into main Jul 13, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants