test(bench): RFC 0031 — backdated wide-range Loki interop arm - #541
Conversation
Issue #538 item 2: the per-PR loki-interop job's fixture was stamped ~30s ago, so it never exercised the query-routing regime the real dispatch runs in — a frozen corpus whose entire time range is days old, where Loki's ingester-vs-store routing decides whether unflushed rows are visible. Both characterized dispatch failure modes live there: the L3 trace pair's 0-of-N flicker (runs #20/#22) and the L4 wide-range completeness loss. The new rfc0031_backdated_wide_range_interop test pins that regime per-PR in ~1 minute: nine records spanning ~4 days (12h apart, one trace, two peer-param values of one template), pushed to a Loki running the dispatch's exact flags, then (a) the L3-shaped trace query polled to all 9 rows — a plateau is the routing flake at fixture scale — and (b) the L4-shaped count_over_time matrix polled to completeness and held EXACTLY equal to Ourios's grouped counts (no margin at fixture scale; if the corpus-scale loss ever reproduces down here, the failure is the discovery). The dispatch's Loki flag list is extracted into LOKI_DISPATCH_FLAGS, shared by the dispatch run and this test — the config the cheap test pins is by construction the config the expensive run uses, so drift between them is unrepresentable. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
|
Warning Review limit reached
Next review available in: 19 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Adds a fast, per-PR Loki interop arm that exercises the “days-old / wide time range” query-routing regime (ingester-vs-store) that the long RFC 0031 dispatch run uses, while also pinning the dispatch Loki flag set against drift by sharing a single flags constant.
Changes:
- Factor the dispatch Loki CLI flags into a shared
LOKI_DISPATCH_FLAGSconstant and reuse it in the dispatch-path container startup. - Add
rfc0031_backdated_wide_range_interop: a Docker-only (ignored) test that pushes a ~4-day backdated fixture to Loki and polls both an L3-shaped trace query and an L4-shaped matrix aggregation to completeness, then asserts exact equivalence with Ourios. - Extend the
loki-interopCI job to run the new ignored test (in addition to the existing RFC0031.1 interop test).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| crates/ourios-bench/tests/rfc0031_comparative.rs | Introduces shared Loki dispatch flags and a new backdated wide-range Loki interop test to catch routing/completeness regressions in fast CI. |
| .github/workflows/ci.yml | Updates the loki-interop job to run the new ignored backdated interop test. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
What
#538 item 2: a backdated wide-time-range arm for the per-PR
loki interopjob.The existing RFC0031.1 interop test stamps its fixture ~30 s ago — so it never exercises the query-routing regime the real dispatch runs in: a frozen corpus whose entire time range is days old, where Loki's ingester-vs-store routing (
query_ingesters_within) decides whether unflushed rows are visible at all. Both characterized dispatch failure modes live in that regime: the L3 trace pair's 0-of-N flicker (runs #20/#22, each costing a 2 h re-dispatch) and the L4 wide-range completeness loss (upstream grafana/loki#10658).rfc0031_backdated_wide_range_interoppins it per-PR in ~1 minute: nine records spanning ~4 days (12 h apart, one shared trace, twopeervalues of one template — one fixture serving both an L3 shape and an L4 shape), pushed to a real Loki running the dispatch's exact flags, then:count_over_timematrix polled to completeness, then held exactly equal to Ourios's grouped counts viacompare_aggregations— no completeness margin at fixture scale, deliberately: fixture-scale loss has never been observed, so exact is the honest assertion, and if the corpus-scale loss ever reproduces down here, the failure is the discovery.Config-drift pinning
The dispatch's Loki flag list (every documented deviation from stock, each a run-history lesson) is extracted into one
LOKI_DISPATCH_FLAGSconstant shared byrfc0031_indicative_comparative_runand this test — the config the cheap test validates is by construction the config the expensive run uses. The dispatch call site is byte-for-byte the same flags, now by reference.Hazards / invariants
Test-only + one workflow line; no production code, no measurement semantics. Touches the RFC 0031 harness's Loki config only by sharing it (no flag values change). This PR's own required
loki interopcheck is the new test's first real execution — a green run here is the validation.Checks run
cargo fmt --all --check; workspacecargo clippy --all-targets --all-features -- -D warnings;ourios-benchlib (134) + localrfc0031_comparativeintegration tests (42 passed, 7 ignored — the two Docker tests run in the loki-interop job). Picker-floor math for the fixture verified againstpick_bucket_width(96 h span → 1d buckets, cardinality 2, 9 rows, ~48,000 s average interval — every floor clears).🤖 Generated with Claude Code
https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y