feat(bench): RFC 0031 — registry-bearing comparative store (RFC0031.1)

[jensholdgaard]

What

Third increment of the RFC 0031 equivalence-harness slice — closing the gap the Ourios-side extraction (#468) surfaced through local iteration.

The gap: cleanly-mined string bodies reconstruct from the audit-derived template registry (RFC 0017), but build_query_store doesn't persist the audit stream (B1/B2 only ever read row counts), so those bodies rendered empty. RFC0031.1's equivalence check compares rendered bodies, so this had to be fixed.

The fix:

• build_comparative_store — mines the corpus into a queryable store and persists the miner's audit stream into the audit/... partition series (via AuditWriter, mirroring the A1 gate), so the querier's derive_template_registry reconstructs bodies on the read path.
• build_store gains a capture_audit flag and returns the drained audit events; build_query_store / build_b1_store pass false (behaviour unchanged), refactored through a shared inner to avoid duplicating the busiest-template logic.
• The comparative test now builds the registry-bearing store and asserts bit-identical bodies — the Ourios side of RFC0031.1 proven end to end locally: corpus → store → in-process query → byte-exact LineKeys.

Verification

• cargo test -p ourios-bench --lib → 85 passed (incl. the bit-identical-body test).
• store:: tests → 11 passed (build_query_store / build_b1_store behaviour unchanged).
• Workspace clippy --all-targets --all-features -D warnings clean.

Next

The Loki testcontainers side (Dex-style) drives the same OTLP corpus into Loki, queries the equivalent LogQL, and feeds compare_lines → flips the RFC0031.1 stub green. With this increment, the Ourios half is fully local-verifiable, so that's the only remaining piece.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Improved comparative extraction validation to confirm that extracted query content matches the original corpus exactly, including full text bodies.
  • Added support for preserving audit information during comparative store creation, improving the reliability of audit-based analysis.

• Tests

  • Expanded RFC0031.1 coverage to validate timestamps and bit-identical corpus content.
  • Updated comparative test setup for more representative results.

[coderabbitai]

Warning

Review limit reached

@jensholdgaard, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 34 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9047faf8-fd91-492c-a532-a36d49ab187e

📥 Commits

Reviewing files that changed from the base of the PR and between d58d32b and c380990.

📒 Files selected for processing (3)

• crates/ourios-bench/src/a1.rs
• crates/ourios-bench/src/comparative.rs
• crates/ourios-bench/src/store.rs

📝 Walkthrough

Walkthrough

The benchmark store pipeline gains optional audit capture and a comparative builder that persists audit partitions. The RFC0031.1 extraction test now uses this builder and verifies extracted bodies match the original corpus exactly.

Changes

Comparative store construction

Layer / File(s)	Summary
Audit-aware streaming pipeline crates/ourios-bench/src/store.rs	The shared store pipeline accepts capture_audit, forwards it to streaming execution, stores returned audit events, and keeps B1 construction disabled for audit capture.
Comparative audit persistence crates/ourios-bench/src/store.rs	build_comparative_store enables audit capture and writes captured events into canonical audit partitions through AuditWriter.
Public API and extraction validation crates/ourios-bench/src/lib.rs, crates/ourios-bench/src/comparative.rs	The comparative builder is re-exported, and extraction tests compare sorted extracted bodies with the original corpus lines.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ComparativeTest
  participant build_comparative_store
  participant harness_run_streaming
  participant AuditWriter
  ComparativeTest->>build_comparative_store: build comparative store
  build_comparative_store->>harness_run_streaming: enable audit capture
  harness_run_streaming-->>build_comparative_store: return store and audit events
  build_comparative_store->>AuditWriter: write audit partitions
  build_comparative_store-->>ComparativeTest: return built store
  ComparativeTest->>ComparativeTest: compare extracted bodies with corpus lines

Loading

Possibly related PRs

• jensholdgaard/ourios#46: Introduces the audit-stream writer and reader used by the new persistence flow.
• jensholdgaard/ourios#52: Covers related audit-event capture and audit-Parquet writing in the benchmark pipeline.
• jensholdgaard/ourios#274: Reads the persisted audit stream to derive template token versions.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is detailed, but it does not follow the template: it omits the required Summary/Related/Checklist sections and RFC link items.	Add the template sections for Summary, Related, and Checklist, including the RFC link and status for fmt, clippy, tests, and docs.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately describes the main change: adding an RFC0031.1 comparative store for the bench crate.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch rfc0031-green-comparative-store

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

Pull request overview

This PR extends the ourios-bench RFC 0031 equivalence harness to build a registry-bearing comparative store by persisting the miner audit stream into the audit/... partition series, enabling the querier’s read-time template registry derivation (RFC 0017) so rendered string bodies reconstruct correctly for RFC0031.1 comparisons.

Changes:

• Add build_comparative_store, which captures miner audit events and writes them via AuditWriter into audit/... partitions.
• Refactor build_query_store through a shared inner path that can optionally capture audit events, keeping B1/B2 behavior unchanged.
• Update the comparative test to build the comparative store and assert bit-identical extracted bodies.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
crates/ourios-bench/src/store.rs	Adds audit capture plumbing and writes captured audit events into audit/... partitions for registry-bearing comparative stores.
crates/ourios-bench/src/lib.rs	Exports build_comparative_store as part of the public bench API surface.
crates/ourios-bench/src/comparative.rs	Updates the Ourios-side extraction test to validate bit-identical reconstructed bodies using the registry-bearing store.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.