Skip to content

fix(bench): query the corpus tenant in b2/otel-demo (was always empty) - #116

Merged
jensholdgaard merged 1 commit into
mainfrom
fix/b2-otel-demo-tenant
Jun 5, 2026
Merged

fix(bench): query the corpus tenant in b2/otel-demo (was always empty)#116
jensholdgaard merged 1 commit into
mainfrom
fix/b2-otel-demo-tenant

Conversation

@jensholdgaard

Copy link
Copy Markdown
Owner

What

Fixes a latent bug in the b2/otel-demo bench arm that made it silently measure an empty query, and surfaced it by running the bench against the real frozen otel-demo corpus (the corpus/otel-demo-v* release assets — fetched with gh release download, no container runtime needed).

The bug

b2/otel-demo hardcoded tenant: "a", but the corpus loader writes every record under bench-tenant (corpus.rs BENCH_TENANT). So the query hit the RFC0007.5 empty-result early return and "benchmarked" a 0-row query at ~1.1 µs — a meaningless number the arm had always produced (it's env-gated, so it had never been validated against a real corpus). The b2/synthetic arm was unaffected (it writes its own records under "a").

Fix

  • BuiltStore now carries the tenant it wrote under (&'static str; the loader is single-tenant).
  • template_exact takes a tenant; b2/otel-demo uses built.tenant and probes the query up front, asserting it returns busiest_template_rows — so a future tenant/template mismatch fails loudly instead of silently timing an empty query. The probe also logs scanned/pruned row groups + bytes.

Validated against the real corpus (indicative, laptop — NOT §9)

corpus mined rows files busiest template → result row groups scanned latency
v1 38,782 1 t29 → 4,682 1/1 783 µs
v4 735,377 5 t45 → 89,382 5/5 2.9 ms

Headline finding (the synthetic bench hid it): on real data a template-exact query prunes nothing (v4 scanned 5/5 row groups) — a template recurs in every time partition, so every row group's template_id min/max matches. b2/synthetic isolated each template to its own file, which made pruning look perfect; real logs don't. The inverted-index-collapse locality needs the time-range predicate + partition pruning (the querier's explicitly-deferred refinement). Captured in scratch/b2-otel-demo-findings.md; this shapes the real B2 measurement (a time-windowed query) and is a strong argument for prioritizing partition-level time pruning.

Tests

  • store.rs rejects_a_non_empty_bucket still green; BuiltStore.tenant populated from the loader constant.
  • The otel-demo probe assertion is the regression guard (empty-query result now fails the bench).
  • Local: cargo fmt --all --check, cargo clippy --workspace --all-targets --all-features -D warnings, cargo test --workspace all green.

Part of epic #81. Follow-up: a CI workflow that fetches the frozen corpus and runs b1/b2 (the b1/otel-demo arm wiring + partition-time-pruning are separate).

🤖 Generated with Claude Code

The b2/otel-demo arm hardcoded `tenant: "a"`, but the corpus loader writes
every record under `bench-tenant` (corpus.rs BENCH_TENANT). So the query hit
the RFC0007.5 empty-result early return and "benchmarked" a 0-row query
(~1.1 µs) instead of scanning the corpus — a meaningless number the arm had
always produced (env-gated, never validated against a real corpus).

- `BuiltStore` now carries the `tenant` it wrote under (the loader is
  single-tenant), so the bench queries the right one.
- b2's `template_exact` takes a tenant; the otel-demo arm uses `built.tenant`
  and probes the query up front, asserting it returns `busiest_template_rows`
  so any future tenant/template mismatch fails loudly rather than silently
  timing an empty query. The probe line also logs the scanned/pruned row
  groups + bytes for the run.

Validated on the frozen otel-demo corpora (fetched via `gh release download`):
v1 → 783 µs scanning 1/1 row groups, v4 → 2.9 ms scanning 5/5. (5/5 is itself a
finding — a template recurs across all partitions, so template_id alone prunes
nothing on real data; logged in scratch.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jensholdgaard
jensholdgaard requested a review from Copilot June 5, 2026 06:06
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@jensholdgaard, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 13 minutes and 22 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 10b13115-a917-4c77-b712-8fd0fb3d8539

📥 Commits

Reviewing files that changed from the base of the PR and between 166662b and 3957022.

📒 Files selected for processing (2)
  • crates/ourios-bench/benches/b2.rs
  • crates/ourios-bench/src/store.rs
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/b2-otel-demo-tenant

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the b2/otel-demo Criterion bench arm so it queries the same tenant that the corpus loader writes under, preventing silent “0-row” measurements and adding an up-front probe that fails loudly if the bench is miswired.

Changes:

  • Extend BuiltStore to carry the tenant used during corpus load/write, and populate it from corpus::BENCH_TENANT.
  • Update the b2 bench helper to build QueryRequests with an explicit tenant, and wire b2/otel-demo to use built.tenant.
  • Add an otel-demo probe assertion plus additional logging of query scan/prune stats to surface empty/incorrect queries immediately.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
crates/ourios-bench/src/store.rs Adds BuiltStore.tenant and returns the loader’s BENCH_TENANT so downstream benches can query the correct tenant.
crates/ourios-bench/benches/b2.rs Threads tenant into template_exact, uses built.tenant for otel-demo, and probes/logs query results and scan stats before benchmarking.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jensholdgaard
jensholdgaard merged commit 6a70866 into main Jun 5, 2026
11 checks passed
@jensholdgaard
jensholdgaard deleted the fix/b2-otel-demo-tenant branch June 5, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants