Skip to content

test(bench): RFC 0031 — backdated wide-range Loki interop arm - #541

Merged
jensholdgaard merged 1 commit into
mainfrom
bench-backdated-loki-interop
Jul 18, 2026
Merged

test(bench): RFC 0031 — backdated wide-range Loki interop arm#541
jensholdgaard merged 1 commit into
mainfrom
bench-backdated-loki-interop

Conversation

@jensholdgaard

Copy link
Copy Markdown
Owner

What

#538 item 2: a backdated wide-time-range arm for the per-PR loki interop job.

The existing RFC0031.1 interop test stamps its fixture ~30 s ago — so it never exercises the query-routing regime the real dispatch runs in: a frozen corpus whose entire time range is days old, where Loki's ingester-vs-store routing (query_ingesters_within) decides whether unflushed rows are visible at all. Both characterized dispatch failure modes live in that regime: the L3 trace pair's 0-of-N flicker (runs #20/#22, each costing a 2 h re-dispatch) and the L4 wide-range completeness loss (upstream grafana/loki#10658).

rfc0031_backdated_wide_range_interop pins it per-PR in ~1 minute: nine records spanning ~4 days (12 h apart, one shared trace, two peer values of one template — one fixture serving both an L3 shape and an L4 shape), pushed to a real Loki running the dispatch's exact flags, then:

  • the L3-shaped trace query polled to all 9 rows — a plateau below 9 is the routing flake reproduced at fixture scale (or a routing-config regression), named as such in the assertion message;
  • the L4-shaped count_over_time matrix polled to completeness, then held exactly equal to Ourios's grouped counts via compare_aggregations — no completeness margin at fixture scale, deliberately: fixture-scale loss has never been observed, so exact is the honest assertion, and if the corpus-scale loss ever reproduces down here, the failure is the discovery.

Config-drift pinning

The dispatch's Loki flag list (every documented deviation from stock, each a run-history lesson) is extracted into one LOKI_DISPATCH_FLAGS constant shared by rfc0031_indicative_comparative_run and this test — the config the cheap test validates is by construction the config the expensive run uses. The dispatch call site is byte-for-byte the same flags, now by reference.

Hazards / invariants

Test-only + one workflow line; no production code, no measurement semantics. Touches the RFC 0031 harness's Loki config only by sharing it (no flag values change). This PR's own required loki interop check is the new test's first real execution — a green run here is the validation.

Checks run

cargo fmt --all --check; workspace cargo clippy --all-targets --all-features -- -D warnings; ourios-bench lib (134) + local rfc0031_comparative integration tests (42 passed, 7 ignored — the two Docker tests run in the loki-interop job). Picker-floor math for the fixture verified against pick_bucket_width (96 h span → 1d buckets, cardinality 2, 9 rows, ~48,000 s average interval — every floor clears).

🤖 Generated with Claude Code

https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

Issue #538 item 2: the per-PR loki-interop job's fixture was stamped
~30s ago, so it never exercised the query-routing regime the real
dispatch runs in — a frozen corpus whose entire time range is days old,
where Loki's ingester-vs-store routing decides whether unflushed rows
are visible. Both characterized dispatch failure modes live there: the
L3 trace pair's 0-of-N flicker (runs #20/#22) and the L4 wide-range
completeness loss.

The new rfc0031_backdated_wide_range_interop test pins that regime
per-PR in ~1 minute: nine records spanning ~4 days (12h apart, one
trace, two peer-param values of one template), pushed to a Loki running
the dispatch's exact flags, then (a) the L3-shaped trace query polled
to all 9 rows — a plateau is the routing flake at fixture scale — and
(b) the L4-shaped count_over_time matrix polled to completeness and
held EXACTLY equal to Ourios's grouped counts (no margin at fixture
scale; if the corpus-scale loss ever reproduces down here, the failure
is the discovery).

The dispatch's Loki flag list is extracted into LOKI_DISPATCH_FLAGS,
shared by the dispatch run and this test — the config the cheap test
pins is by construction the config the expensive run uses, so drift
between them is unrepresentable.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
@jensholdgaard
jensholdgaard requested a review from Copilot July 18, 2026 10:55
@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@jensholdgaard, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 19 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: afc29d6d-fb03-4c2d-b5c8-0565f30fc788

📥 Commits

Reviewing files that changed from the base of the PR and between fe82f42 and 7208581.

📒 Files selected for processing (2)
  • .github/workflows/ci.yml
  • crates/ourios-bench/tests/rfc0031_comparative.rs
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch bench-backdated-loki-interop

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a fast, per-PR Loki interop arm that exercises the “days-old / wide time range” query-routing regime (ingester-vs-store) that the long RFC 0031 dispatch run uses, while also pinning the dispatch Loki flag set against drift by sharing a single flags constant.

Changes:

  • Factor the dispatch Loki CLI flags into a shared LOKI_DISPATCH_FLAGS constant and reuse it in the dispatch-path container startup.
  • Add rfc0031_backdated_wide_range_interop: a Docker-only (ignored) test that pushes a ~4-day backdated fixture to Loki and polls both an L3-shaped trace query and an L4-shaped matrix aggregation to completeness, then asserts exact equivalence with Ourios.
  • Extend the loki-interop CI job to run the new ignored test (in addition to the existing RFC0031.1 interop test).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
crates/ourios-bench/tests/rfc0031_comparative.rs Introduces shared Loki dispatch flags and a new backdated wide-range Loki interop test to catch routing/completeness regressions in fast CI.
.github/workflows/ci.yml Updates the loki-interop job to run the new ignored backdated interop test.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/ci.yml
@jensholdgaard
jensholdgaard merged commit f9b6afe into main Jul 18, 2026
26 checks passed
@jensholdgaard
jensholdgaard deleted the bench-backdated-loki-interop branch July 18, 2026 11:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants