Skip to content

docs(benchmarks): §9.8 — baseline ingest/recovery + real-corpus A1/C1/C2 + B1/B2 - #250

Merged
jensholdgaard merged 5 commits into
mainfrom
docs/benchmarks-baseline-9-8
Jun 18, 2026
Merged

docs(benchmarks): §9.8 — baseline ingest/recovery + real-corpus A1/C1/C2 + B1/B2#250
jensholdgaard merged 5 commits into
mainfrom
docs/benchmarks-baseline-9-8

Conversation

@jensholdgaard

@jensholdgaard jensholdgaard commented Jun 18, 2026

Copy link
Copy Markdown
Owner

What

Records the 2026-06-18 authoritative baseline-8vcpu-32gib run as a new docs/benchmarks.md §9.8 (curated narrative, outside the managed BENCH-RESULTS region). Two dedicated-vCPU cloud hosts (8 vCPU EPYC-Milan / 32 GiB), git d3f2cae, torn down immediately after.

(a) Ingest write-path + recovery — supportive criterion wall-clock (the metrics-fill + bench arc, #247/#248):

  • wal_append 372 µs · sink_write/10000 12.24 ms (817 K rec/s) · recovery/{1,4,16} 169 µs→507 µs→1.87 ms (O(N)).

(b) Thesis gates A1/C1/C2 on real corporaLogHub HDFS_v1 (11.18 M real Hadoop production lines / 1.58 GiB, above §8's 1 GiB minimum; Zenodo md5-pinned, never redistributed) + frozen OTel-Demo v1:

  • C1 = 1.000000 (11,175,578 / 11,175,578 rows reconstructed bit-for-bit) — PASS
  • C2 = 0.825, 40 templates — PASS (OTel-Demo abstains, < 1 M lines)
  • A1 6.21× vs zstd 16.0× → FAIL, the recorded diagnostic (RFC 0011), not a gate.

(c) B1/B2 query gates on real corpora — headline: a time-windowed B2 query on HDFS prunes 13/14 row groups (~5×) via Parquet min/max stats — the predicate-pushdown thesis (pillar #1) on real production data.

Notes

  • Per the bench convention, this is the hand-written curated entry; the --update-benchmarks-md managed region is untouched.
  • b1/b2 criterion timings use the reduced --warm-up-time 1 --measurement-time 3 (matching query-bench.yml); structural pruning/template numbers are exact and criterion-setting-independent.
  • mdbook build clean.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Added a new authoritative benchmark section covering ingest write-path and recovery performance on baseline hardware.
    • Included synthetic throughput/latency metrics (e.g., WAL append and sink writes) and WAL replay timing.
    • Extended real-corpus results for LogHub HDFS_v1 and OTel-Demo v1, including updated thesis gate outcomes.
    • Added query benchmark measurements, highlighting windowed query pruning behavior and related gate updates.

…rpus A1/C1/C2 + B1/B2

Records the 2026-06-18 authoritative `baseline-8vcpu-32gib` (Hetzner ccx33)
run: (a) the self-contained `ingest_write_path` (RFC 0014) + `recovery`
(RFC0008.3) criterion benches as supportive wall-clock, and (b) the
`ourios-bench` binary's A1/C1/C2 on two real corpora (LogHub HDFS_v1 — 11.18M
real production lines / 1.58 GiB, above §8's 1 GiB minimum — and frozen
OTel-Demo v1) plus the B1/B2 criterion benches over them.

Headlines: C1 reconstructs all 11,175,578 non-lossy HDFS rows bit-for-bit;
C2 converges (40 templates, ratio 0.825); a time-windowed B2 query on HDFS
prunes 13/14 row groups (~5×) — predicate pushdown on real data; A1 fails as
the recorded diagnostic (RFC 0011), not a gate. Curated narrative outside the
managed BENCH-RESULTS region; VM torn down after the run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@jensholdgaard, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 49 minutes and 22 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f28e1644-9a12-46c4-912a-bd021b9a5d11

📥 Commits

Reviewing files that changed from the base of the PR and between b4f4e64 and 9e953f8.

📒 Files selected for processing (1)
  • docs/benchmarks.md
📝 Walkthrough

Walkthrough

Appends a new §9.8 section (2026-06-18, baseline-8vcpu-32gib) to docs/benchmarks.md. The section records criterion bench timings for the ingest write-path and recovery on synthetic data, and extends A1/C1/C2 and B1/B2 thesis gate tables with results for HDFS_v1 and OTel-Demo v1 real datasets.

Changes

§9.8 benchmark results

Layer / File(s) Summary
§9.8 ingest, recovery, and query gate results
docs/benchmarks.md
Adds 71 lines covering criterion timings for wal_append/batch, sink_write/*, and recovery/* on synthetic data, plus A1/C1/C2 and B1/B2 tables for HDFS_v1 and OTel-Demo v1; A1 is recorded as a diagnostic failure while C1, C2, B1, and B2 pass. Notes the managed BENCH-RESULTS region is updated via ourios-bench --update-benchmarks-md.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

  • jensholdgaard/ourios#248: Adds the ingest_write_path Criterion suite to ourios-bench that directly produces the wal_append/batch and sink_write/* timings recorded in this PR's §9.8 section.
  • jensholdgaard/ourios#55: Implements update_status_section / ourios-bench --update-benchmarks-md, the tooling that writes the managed BENCH-RESULTS region now populated by §9.8.
  • jensholdgaard/ourios#115: Adds the B1 predicate-pushdown benchmark whose reference implementation generates the B1/B2 query-gate numbers reported in this PR's §9.8 tables.

Poem

🐇 Hoppity-hop through the benchmark trail,
WAL appends recorded, no detail too small,
HDFS gates checked, the C1s all pass,
B1 windowed pruning — fast, not last!
A1 fails by design, a diagnostic friend,
This rabbit stamps the §9.8 blend. ✅

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately captures the main addition: documenting §9.8 baseline benchmark results for ingest/recovery and thesis gates (A1/C1/C2/B1/B2).
Description check ✅ Passed The description provides comprehensive detail across all required template sections (Summary, Related, and implicit Checklist items), documenting the benchmark run context and results.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/benchmarks-baseline-9-8

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new curated benchmark entry documenting the 2026-06-18 authoritative baseline-8vcpu-32gib run in docs/benchmarks.md §9.8, capturing ingest write-path + recovery criterion results and real-corpus A1/C1/C2 + B1/B2 thesis-gate outcomes.

Changes:

  • Adds §9.8 narrative documenting baseline hardware, run parameters, and measured ingest/recovery wall-clock figures.
  • Records real-corpus gate results for LogHub HDFS_v1 and OTel-Demo v1 (A1/C1/C2) plus B1/B2 timings and pruning outcomes.
  • Notes bench-tooling boundary (curated narrative outside the managed BENCH-RESULTS region) and criterion setting differences for b1/b2.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/benchmarks.md`:
- Around line 865-866: The HDFS_v1 corpus size representation contains a factual
inconsistency where lines 865-866 and line 889 state the size as 1.58 GiB.
However, the documented byte count at line 749 (1,577,982,906 raw bytes)
calculates to 1.4688 GiB, which should be rounded to 1.47 GiB. Replace all
instances of "1.58 GiB" with "1.47 GiB" at both locations (lines 865-866 and
889) to align with the correct calculation and maintain consistency with the
size representation used throughout sections §1, §9.4, and §9.6 of the document.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9bdc1da4-1d2d-4eb0-afce-f6b3c12acf9e

📥 Commits

Reviewing files that changed from the base of the PR and between d3f2cae and 111f7ba.

📒 Files selected for processing (1)
  • docs/benchmarks.md

Comment thread docs/benchmarks.md Outdated
…s §1/§9.4/§9.6)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment thread docs/benchmarks.md
Comment thread docs/benchmarks.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comment thread docs/benchmarks.md Outdated
…-row note, attached %

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comment thread docs/benchmarks.md Outdated
…odel, match §9.7)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@jensholdgaard
jensholdgaard merged commit f369fb9 into main Jun 18, 2026
14 checks passed
@jensholdgaard
jensholdgaard deleted the docs/benchmarks-baseline-9-8 branch June 18, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants