Skip to content

feat(bench): RFC 0031 — Loki query-response parser (RFC0031.1) - #470

Merged
jensholdgaard merged 2 commits into
mainfrom
rfc0031-green-loki-parser
Jul 11, 2026
Merged

feat(bench): RFC 0031 — Loki query-response parser (RFC0031.1)#470
jensholdgaard merged 2 commits into
mainfrom
rfc0031-green-loki-parser

Conversation

@jensholdgaard

Copy link
Copy Markdown
Owner

What

Fourth increment of the RFC 0031 equivalence-harness slice — the Loki half of the extraction, landing as another locally-verifiable piece.

parse_loki_streams lowers a Loki query_range streams response (data.result[].values[] = ["<ns-timestamp>", "<log line>"] pairs) into LineKeys keyed identically to the Ourios side ((timestamp, body)), so the two feed compare_lines.

Tests (no container)

  • A synthetic Loki response parses to LineKeys that compare Equal to the hand-built Ourios-side equivalent for the same lines — proving both halves are keyed compatibly, which is the crux of the whole harness.
  • Malformed responses (missing data.result, a non-pair values entry, a non-numeric timestamp) error rather than silently dropping rows.

Cross-system alignment (resolved)

For LineKey equality to be meaningful, both systems must see byte-identical (timestamp, body). Decision: the comparative corpus is OTLP-JSONL (explicit time_unix_nano + body per record) — the single source of truth, fed to Ourios via build_comparative_store and pushed to Loki as the same OTLP records. LineKeys then align by construction, and it fits RFC 0031's OTLP-native framing (text corpora would require reverse-engineering the loader's timestamp assignment).

Remaining for the RFC0031.1 green flip

Only the CI-gated container wiring: spin Loki (testcontainers, Dex-style), push the OTLP-JSONL corpus, run the LogQL query, feed both sides to compare_lines. Everything around it — comparator, Ourios extraction, registry-bearing store, and now the Loki response parser — is proven locally.

cargo test -p ourios-bench --lib comparative → all green; fmt + clippy clean.

🤖 Generated with Claude Code

Fourth increment of the equivalence-harness slice: the Loki half of the
extraction. `parse_loki_streams` lowers a Loki `query_range` streams
response (`data.result[].values[]` = `["<ns-ts>", "<line>"]` pairs) into
`LineKey`s keyed identically to the Ourios side, so the two feed
`compare_lines`.

Locally verifiable, no container:
- A synthetic Loki response parses to LineKeys that compare **Equal** to
  the hand-built Ourios-side equivalent — proving both halves are keyed
  compatibly (the crux of the whole harness).
- Malformed responses (missing `data.result`, non-pair `values`, a
  non-numeric timestamp) error rather than silently dropping rows.

Cross-system data-alignment decision (resolved): the comparative corpus is
**OTLP-JSONL** (explicit time_unix_nano + body per record), the single
source of truth fed to Ourios (build_comparative_store) and pushed to Loki
as the same OTLP records — so LineKeys align by construction, matching
RFC 0031's OTLP-native framing.

Remaining for the RFC0031.1 green flip: the CI-gated container wiring — spin
Loki (testcontainers), push the OTLP-JSONL corpus, run the LogQL query,
feed both sides to compare_lines. Every piece around it is now local.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jensholdgaard
jensholdgaard requested a review from Copilot July 11, 2026 12:30
@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@jensholdgaard, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 10 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 89131d6d-a11c-4f0b-9849-df68c1de8168

📥 Commits

Reviewing files that changed from the base of the PR and between 05ea638 and 6d87dba.

📒 Files selected for processing (2)
  • crates/ourios-bench/src/comparative.rs
  • crates/ourios-bench/src/lib.rs
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch rfc0031-green-loki-parser

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR advances RFC 0031’s equivalence-harness by adding the Loki-side extraction needed to compare Loki query_range streams results against Ourios query results using the shared (timestamp_unix_nanos, body_bytes) LineKey identity.

Changes:

  • Added parse_loki_streams to parse Loki query_range streams JSON into LineKeys.
  • Added unit tests covering successful parsing and malformed-response erroring.
  • Re-exported the new parser from ourios-bench’s public API.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
crates/ourios-bench/src/lib.rs Re-exports parse_loki_streams to expose the Loki parser as part of the bench crate’s comparative API.
crates/ourios-bench/src/comparative.rs Implements Loki streams response parsing into LineKeys and adds focused unit tests for compatibility and error cases.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/ourios-bench/src/comparative.rs Outdated
Comment thread crates/ourios-bench/src/comparative.rs
Comment thread crates/ourios-bench/src/comparative.rs
Copilot review:
- Detect a Loki error response (`status == "error"`) and surface Loki's
  `errorType`/`error` instead of the misleading "missing data.result" — so
  a rejected LogQL query in the container test shows Loki's actual
  complaint. Tested.
- Add stream + value indices to malformed-entry error messages, for
  debugging real Loki responses.
- Complete the `# Errors` doc to list every failure case (missing
  `values`, non-string timestamp/line, the error-response path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@jensholdgaard
jensholdgaard merged commit 2e0c918 into main Jul 11, 2026
25 checks passed
@jensholdgaard
jensholdgaard deleted the rfc0031-green-loki-parser branch July 11, 2026 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants