feat(bench): RFC 0031 — Loki query-response parser (RFC0031.1) - #470
Conversation
Fourth increment of the equivalence-harness slice: the Loki half of the extraction. `parse_loki_streams` lowers a Loki `query_range` streams response (`data.result[].values[]` = `["<ns-ts>", "<line>"]` pairs) into `LineKey`s keyed identically to the Ourios side, so the two feed `compare_lines`. Locally verifiable, no container: - A synthetic Loki response parses to LineKeys that compare **Equal** to the hand-built Ourios-side equivalent — proving both halves are keyed compatibly (the crux of the whole harness). - Malformed responses (missing `data.result`, non-pair `values`, a non-numeric timestamp) error rather than silently dropping rows. Cross-system data-alignment decision (resolved): the comparative corpus is **OTLP-JSONL** (explicit time_unix_nano + body per record), the single source of truth fed to Ourios (build_comparative_store) and pushed to Loki as the same OTLP records — so LineKeys align by construction, matching RFC 0031's OTLP-native framing. Remaining for the RFC0031.1 green flip: the CI-gated container wiring — spin Loki (testcontainers), push the OTLP-JSONL corpus, run the LogQL query, feed both sides to compare_lines. Every piece around it is now local. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 10 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR advances RFC 0031’s equivalence-harness by adding the Loki-side extraction needed to compare Loki query_range streams results against Ourios query results using the shared (timestamp_unix_nanos, body_bytes) LineKey identity.
Changes:
- Added
parse_loki_streamsto parse Lokiquery_rangestreams JSON intoLineKeys. - Added unit tests covering successful parsing and malformed-response erroring.
- Re-exported the new parser from
ourios-bench’s public API.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| crates/ourios-bench/src/lib.rs | Re-exports parse_loki_streams to expose the Loki parser as part of the bench crate’s comparative API. |
| crates/ourios-bench/src/comparative.rs | Implements Loki streams response parsing into LineKeys and adds focused unit tests for compatibility and error cases. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Copilot review: - Detect a Loki error response (`status == "error"`) and surface Loki's `errorType`/`error` instead of the misleading "missing data.result" — so a rejected LogQL query in the container test shows Loki's actual complaint. Tested. - Add stream + value indices to malformed-entry error messages, for debugging real Loki responses. - Complete the `# Errors` doc to list every failure case (missing `values`, non-string timestamp/line, the error-response path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What
Fourth increment of the RFC 0031 equivalence-harness slice — the Loki half of the extraction, landing as another locally-verifiable piece.
parse_loki_streamslowers a Lokiquery_rangestreams response (data.result[].values[]=["<ns-timestamp>", "<log line>"]pairs) intoLineKeys keyed identically to the Ourios side ((timestamp, body)), so the two feedcompare_lines.Tests (no container)
LineKeys that compareEqualto the hand-built Ourios-side equivalent for the same lines — proving both halves are keyed compatibly, which is the crux of the whole harness.data.result, a non-pairvaluesentry, a non-numeric timestamp) error rather than silently dropping rows.Cross-system alignment (resolved)
For
LineKeyequality to be meaningful, both systems must see byte-identical(timestamp, body). Decision: the comparative corpus is OTLP-JSONL (explicittime_unix_nano+ body per record) — the single source of truth, fed to Ourios viabuild_comparative_storeand pushed to Loki as the same OTLP records.LineKeys then align by construction, and it fits RFC 0031's OTLP-native framing (text corpora would require reverse-engineering the loader's timestamp assignment).Remaining for the RFC0031.1 green flip
Only the CI-gated container wiring: spin Loki (testcontainers, Dex-style), push the OTLP-JSONL corpus, run the LogQL query, feed both sides to
compare_lines. Everything around it — comparator, Ourios extraction, registry-bearing store, and now the Loki response parser — is proven locally.cargo test -p ourios-bench --lib comparative→ all green; fmt + clippy clean.🤖 Generated with Claude Code