docs(rfc-0007): draft querier — DataFusion execution frontend for the logs DSL

[jensholdgaard]

What

First draft of RFC 0007 — Querier (status: drafted): the ourios-querier crate (pillar #3, DataFusion). Closes the design gap that left B1/B2 (the query-performance thesis gates) unmeasured.

Tracks #82.

Scope decision

RFC 0002 (logs DSL) is still undecided between its Branch A/B syntax. But both branches compile to the same DataFusion LogicalPlan target (RFC 0002 §5.5), so this RFC pins the branch-independent execution layer — lowering, predicate pushdown, the no-leakage boundary, and the B1/B2 acceptance criteria — without re-deciding or blocking on the DSL surface. It can't advance past specified until RFC 0002 §3 lands, which the Open Questions record.

Highlights

• Read path only — depends on the RFC 0005 Parquet reader contract (shipped), not on the WAL (RFC 0008) or receiver (RFC 0003). Can be built/benchmarked in parallel with the ingest path.
• The thesis mechanism, made testable: partition pruning (tenant/time) → row-group skipping (min/max stats on template_id, time, severity) → bloom filters. QueryResult surfaces row_groups_pruned / bytes_read so B1 can assert pruning happened.
• Acceptance criteria realise B1/B2 as greppable scenarios: RFC0007.1 (pushdown prunes ≥ floor), RFC0007.2 (template-exact latency scales with result size not corpus size — measured by criterion across the corpus/otel-demo-v* series), plus no-DataFusion-leakage (§4.6), forward-compat reads (§3.5), tenant isolation (§3.7).
• B1/B2 wire back into ourios-bench, closing the RFC 0006 §1 deferral.

Process

Per §9, authored by the maintainer with Claude as drafting assistance. This is a draft for review — not implementation. mdBook builds; added to SUMMARY.md.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Documentation
  • Added an RFC for the new "Querier" execution frontend for the logs DSL: describes how DSL maps to query plans, aggressive predicate pushdown (partition/row-group pruning and index use), a strict boundary preventing internal types from leaking to the public API, measurable acceptance criteria and performance gates (B1/B2), testing/benchmark plans, and open implementation questions.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b765e62b-c952-432b-84f5-38c6e7e30b56

📥 Commits

Reviewing files that changed from the base of the PR and between 1831898 and 4514d60.

📒 Files selected for processing (1)

• docs/rfcs/0007-querier.md

✅ Files skipped from review due to trivial changes (1)

• docs/rfcs/0007-querier.md

---

📝 Walkthrough

Walkthrough

Adds RFC 0007 describing the ourios-querier crate: lowering RFC 0002 logs-DSL to DataFusion LogicalPlans for RFC 0005 Parquet with predicate pushdown, measurable acceptance criteria for pruning/latency (B1/B2), a testing plan, open design questions, and a docs/SUMMARY.md TOC entry.

Changes

RFC 0007 Querier Specification

Layer / File(s)	Summary
RFC metadata and scope docs/rfcs/0007-querier.md	Adds RFC header metadata and clarifies scope: querier is the execution-layer frontend; excludes DSL parsing, SQL endpoints, and storage-format decisions.
Crate summary and motivation docs/rfcs/0007-querier.md	Introduces the ourios-querier purpose and why it should own measurable acceptance contracts B1/B2.
Querier role, design, and lowering docs/rfcs/0007-querier.md	Specifies inputs/outputs, DSL→DataFusion LogicalPlan lowering responsibilities, and scoped predicate-pushdown strategies (partition pruning, row-group skipping via stats, bloom/page indexes for template_id, params non-prunable).
Acceptance criteria and testing strategy docs/rfcs/0007-querier.md	Documents B1/B2 acceptance criteria (pruning/read thresholds, latency scaling, forward-compatible reads, tenant isolation) and testing plan: lowering unit tests, boundary no-leakage checks, Parquet integration tests, criterion benchmarks, and proptest properties.
Alternatives, open questions, references, and TOC registration docs/rfcs/0007-querier.md, docs/SUMMARY.md	Lists rejected alternatives, enumerates open questions (table provider choice, params pushdown scope, streaming vs materialized results, caching/footer impact, async model), adds references, and registers RFC 0007 in the docs TOC.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Possibly related issues

• Epic: Querier — DataFusion frontend + logs DSL (RFC 0007) #82: Directly corresponds to adding RFC 0007 and tracking the ourios-querier specification work.

Possibly related PRs

• jensholdgaard/ourios#64: Overlapping docs TOC edits (RFC entries) — both modify docs/SUMMARY.md.

Poem

🐰 I hopped through plans of DataFusion light,
Lowering DSL dreams into rows so bright.
No leaks in my pockets, the API's tight,
B1 and B2 hummed through the benchmark night,
A small RFC leap — soft, clever, and white.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding RFC 0007 for the Querier DataFusion execution frontend, which matches the changeset's core addition of the RFC document and SUMMARY.md entry.
Description check	✅ Passed	The description is comprehensive and addresses all required template sections: Summary (What section explains the RFC and its purpose), Related (tracks #82 and links RFC 0002/0005), and contextual checklist items (mdBook builds, added to SUMMARY.md).
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch docs/rfc-0007-querier

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

This PR drafts RFC 0007 for the ourios-querier design, defining the DataFusion-backed execution frontend for the logs DSL and tying the B1/B2 query-performance thesis gates to concrete acceptance criteria.

Changes:

• Adds RFC 0007 covering querier scope, lowering, pushdown, no-leakage boundary, acceptance criteria, and testing strategy.
• Adds the new RFC to the mdBook summary.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
docs/SUMMARY.md	Adds RFC 0007 to the RFC navigation list.
docs/rfcs/0007-querier.md	Introduces the draft querier RFC and its proposed design/test contracts.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

🧹 Nitpick comments (1)

docs/rfcs/0007-querier.md (1)

132-174: ⚡ Quick win

Align §5 scenario formatting with the RFC convention.

Section 5 currently uses bullet lists for scenarios; in this repo’s RFC convention, each acceptance scenario should be a blockquote scenario (> **Scenario ...**) with blank lines between scenarios. Please reformat RFC0007.1–RFC0007.5 to match that pattern for consistency and easier review/grep parity across RFCs.

Based on learnings: “In this repo’s RFCs under docs/rfcs/, the §5 ‘acceptance-criteria’ section should use the documented blockquote convention: each scenario should appear as a > **Scenario ...** blockquote, with blank lines separating scenario blocks.”

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/rfcs/0007-querier.md` around lines 132 - 174, Reformat the Acceptance
criteria in §5 by replacing the current bulleted RFC0007.1–RFC0007.5 entries
with the repo’s RFC blockquote scenario pattern: convert each item (RFC0007.1,
RFC0007.2, RFC0007.3, RFC0007.4, RFC0007.5) into a separate blockquote beginning
with > **Scenario —** (or > **Scenario RFC0007.x —**), keep the Given/When/Then
lines inside that blockquote, and ensure there is a blank line between each
scenario block so the section matches the other RFCs’ formatting and is
greppable.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@docs/rfcs/0007-querier.md`:
- Around line 132-174: Reformat the Acceptance criteria in §5 by replacing the
current bulleted RFC0007.1–RFC0007.5 entries with the repo’s RFC blockquote
scenario pattern: convert each item (RFC0007.1, RFC0007.2, RFC0007.3, RFC0007.4,
RFC0007.5) into a separate blockquote beginning with > **Scenario —** (or >
**Scenario RFC0007.x —**), keep the Given/When/Then lines inside that
blockquote, and ensure there is a blank line between each scenario block so the
section matches the other RFCs’ formatting and is greppable.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b0b99fd4-33c2-4e6e-bd0b-7531caf9dc95

📥 Commits

Reviewing files that changed from the base of the PR and between 85f311a and 81f2ca9.

📒 Files selected for processing (2)

• docs/SUMMARY.md
• docs/rfcs/0007-querier.md

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.