test(parquet): rfc0005.6 row-group sizing test, flip rfc 0005 to green

[jensholdgaard]

What

Closes the last gap for RFC 0005 (Parquet storage) → green: the one acceptance scenario without a test, RFC0005.6 (row-group size lands inside the H4 target), plus the ladder flip drafted → green.

13 of the 14 §5 scenarios were already implemented and tested — the drafted label was stale (the storage layer landed early under PR #41 + the PR-D..G series and the status was never advanced). This PR adds the missing test and records reality.

RFC0005.6 test (crates/ourios-parquet/tests/sizing.rs)

Per the RFC §6 testing strategy: an #[ignore]d integration test that pushes >256 MiB of mined records through the production writer (Writer::append_records/close, not the corpus-mode single-file path), parses the emitted file's Parquet footer, and asserts every non-final row group's uncompressed total_byte_size ∈ [128 MiB, 1 GiB] (the final group carries the remainder). It also asserts ≥2 row groups, so the 128 MiB flush actually split.

Verified locally (CI does not run #[ignore] tests — §7's open "scheduled cadence" question):

• cargo test -p ourios-parquet --ignored → ok, ~6.8 s dev / ~0.9 s release.

A fixture note worth recording: the body content must be high-entropy. My first cut used repetitive bodies, which compress away in the writer's in-progress buffer so the 128 MiB flush never trips → one row group. High-entropy bodies (same generator shape as no_body_dict.rs) make the buffered uncompressed size track real bytes. So the test also implicitly guards that the sizing path keys off real volume.

Ladder flip

• status: drafted → green + a status note mapping all 14 scenarios to their tests and flagging the §7 non-gating open items (codec tuning, bloom FPR, audit retention, CI cadence).
• docs/roadmap.md §3 status cell updated to match (the authoritative current-state table; the §§4+ narrative is disclaimer-flagged as predating).

Direct drafted → green flip with a documented note (precedent: RFC 0009 #210, RFC 0008's green→accepted) — everything past specified/red was already satisfied.

Invariants

Touches §3.5 (row-group sizing / hazard H4) — this PR adds the acceptance test for that invariant; no behavior change to the writer. No schema change.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Tests

  • Added integration test validating Parquet row-group sizing behavior

• Documentation

  • Updated RFC 0005 (Parquet storage) status to "green"
  • Confirmed all acceptance scenarios have passing tests
  • Updated roadmap to reflect RFC progress

[coderabbitai]

Warning

Review limit reached

@jensholdgaard, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 48 minutes and 2 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 47236dbc-810c-4bdb-aaa4-f9359f7af934

📥 Commits

Reviewing files that changed from the base of the PR and between 24ff575 and 460b80b.

📒 Files selected for processing (3)

• crates/ourios-parquet/tests/sizing.rs
• docs/rfcs/0005-parquet-storage.md
• docs/roadmap.md

📝 Walkthrough

Walkthrough

Adds a new #[ignore]d integration test (rfc0005_6_row_group_size_lands_inside_h4_target) in crates/ourios-parquet/tests/sizing.rs that writes high-entropy synthetic MinedRecord batches and asserts Parquet row-group total_byte_size bounds. Promotes RFC 0005 status from drafted to green in the RFC document and roadmap.

Changes

RFC 0005 Parquet sizing test and green promotion

Layer / File(s)	Summary
Row-group sizing integration test crates/ourios-parquet/tests/sizing.rs	Defines sizing constants (BODY_BYTES, batch/record counts), a high_entropy_body(seed) helper producing deterministic printable-ASCII strings, a sized_record(seed) factory building MinedRecord instances, and an #[ignore]d test that writes records in batches via ourios_parquet::Writer, reads back Parquet row-group metadata, and asserts ≥2 row groups, all groups ≤ 1 GiB, and all non-final groups ≥ 128 MiB.
RFC 0005 status promotion to green docs/rfcs/0005-parquet-storage.md, docs/roadmap.md	Changes RFC 0005 front-matter status from drafted to green, adds a status note section enumerating all §5 scenario-to-test mappings, documents the manually-run #[ignore] sizing test, lists non-gating §7 follow-ups, and updates the roadmap RFC ladder entry to reflect green with the sizing test caveat.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• jensholdgaard/ourios#44: Introduced the ourios_parquet::Writer row-group flushing and sizing logic that the new tests/sizing.rs integration test directly exercises.
• jensholdgaard/ourios#47: Previously updated docs/roadmap.md with RFC 0005.6 row-group sizing deferral narrative, matching the documentation changes in this PR.

Poem

🐇 A row group too big? No, I've tested it right,
With entropy bodies and seeds shining bright.
Two hundred and fifty-six MiB I exceed,
Then check every group for the size that we need.
RFC 0005 now glows a fine green —
The tidiest Parquet the warren has seen! 🌿

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main changes: adding an RFC0005.6 row-group sizing test and flipping RFC 0005 status to green.
Description check	✅ Passed	The PR description covers the main objective (RFC0005.6 test and ladder flip), technical details about the test implementation, and verification steps. However, it does not explicitly address the checklist items from the repository template.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch rfc0005-6-sizing

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

This PR closes the remaining acceptance-test gap for RFC 0005 (Parquet storage) by adding the missing RFC0005.6 row-group sizing integration test and updating documentation to reflect RFC 0005’s current maturity as green.

Changes:

• Add an #[ignore]d heavyweight integration test that writes >256 MiB via the production Parquet Writer and asserts non-final row groups land within the RFC 0005 §3.5 / hazards H4 sizing band.
• Flip RFC 0005 frontmatter status: drafted → status: green and add a status note mapping all §5 scenarios to concrete tests.
• Update docs/roadmap.md to reflect RFC 0005 as green and document how the heavyweight sizing test is run.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
docs/roadmap.md	Updates the RFC ladder table entry for RFC 0005 to green and references the new sizing test.
docs/rfcs/0005-parquet-storage.md	Flips RFC status to green and adds a scenario→test status note including the new RFC0005.6 test.
crates/ourios-parquet/tests/sizing.rs	Introduces the RFC0005.6 heavyweight integration test validating row-group sizing bounds from emitted Parquet metadata.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/roadmap.md`:
- Line 104: The roadmap document has been edited on line 104 to update the
Parquet storage status, but the "Last updated" timestamp on line 5 has not been
bumped to reflect this change. Update the "Last updated" date value to match the
current date to ensure the document metadata accurately reflects when the
content was last modified.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 374fa1e7-c9a9-41ba-9c64-de7aae23a48e

📥 Commits

Reviewing files that changed from the base of the PR and between 2165f6c and 24ff575.

📒 Files selected for processing (3)

• crates/ourios-parquet/tests/sizing.rs
• docs/rfcs/0005-parquet-storage.md
• docs/roadmap.md

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.