feat(querier): add severity_text predicate for the B1 level filter by jensholdgaard · Pull Request #114 · jensholdgaard/ourios

jensholdgaard · 2026-06-04T22:25:35Z

What

First step of the B1/B2 measurement plan: give the throwaway query surface the predicate B1 needs.

B1's thesis-gate query shape (docs/benchmarks.md) is:

count events WHERE tenant=X AND ts BETWEEN t1 AND t2 AND level='ERROR'

…but QueryRequest only carried tenant + time_range + template_id. The level='ERROR' arm couldn't be expressed, which blocks any B1 run. This adds it.

How

QueryRequest gains an optional severity_text: Option<String> equality filter → col(severity_text) = lit(X).
It's the structured counterpart to the B1 reference's grep ERROR: Ourios filters the RFC 0005 severity_text column, and that column's Parquet min/max statistics let DataFusion prune row groups that can't match (an INFO-only file is skipped, not scanned) — which is exactly the pillar-docs: add verification process spec #1 "skip via footer statistics" win B1 measures.
The surface is still the explicitly-mutable, pre-DSL throwaway surface (RFC 0002 deferred until B1/B2 prove out), so this is a free addition.

Tests

New rfc0007_1_severity_filter_counts_and_prunes: a mixed ERROR/INFO file plus an INFO-only file in another hour; a severity_text='ERROR' query counts exactly the ERROR rows and prunes the INFO-only row group (row_groups_pruned >= 1). Confirms Utf8 min/max stats pruning fires for the equality predicate.
Remaining QueryRequest construction sites gain severity_text: None.

Local: cargo fmt --all --check, cargo clippy --workspace --all-targets --all-features -D warnings, cargo test --workspace all green.

Part of epic #81 (thesis-gate benchmark validation). Follow-up PR adds the b1.rs bench + the zstdcat|grep reference comparator.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

New Features
- Severity text filtering now available for queries, enabling users to refine results by specific severity levels.
- Enhanced backwards compatibility: queries filtered by severity text on older datasets lacking this field gracefully return empty results without errors.

coderabbitai · 2026-06-04T22:25:41Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6b20dd46-7a77-4923-b71b-bf41228c61cf

📥 Commits

Reviewing files that changed from the base of the PR and between f6e6ace and a308c30.

📒 Files selected for processing (6)

crates/ourios-bench/benches/b2.rs
crates/ourios-querier/src/lib.rs
crates/ourios-querier/tests/boundary.rs
crates/ourios-querier/tests/execution.rs
crates/ourios-querier/tests/forward_compat.rs
crates/ourios-querier/tests/manifest.rs

📝 Walkthrough

Walkthrough

This PR adds optional severity-level filtering to the querier by introducing a severity_text field to QueryRequest. The implementation includes schema-aware filtering logic that gracefully handles forward compatibility when the column is absent, integration tests validating both filtering correctness and row-group pruning, and updates to all existing test helpers and benchmarks.

Changes

Severity Text Filter Addition

Layer / File(s)	Summary
QueryRequest contract and documentation `crates/ourios-querier/src/lib.rs`	New optional `severity_text: Option<String>` field added to `QueryRequest` with predicate docs, and crate-level execution docs updated to reflect severity equality filtering and pruning stats coverage.
Querier filtering implementation `crates/ourios-querier/src/lib.rs`	`Querier::run` checks inferred schema for `severity_text` column presence; returns empty `QueryResult` if absent (forward compatibility), otherwise applies equality filter before aggregate count and scan statistics.
Severity filter integration test and fixture helpers `crates/ourios-querier/tests/execution.rs`	New `rfc0007_1_severity_filter_counts_and_prunes` test writes mixed-severity and INFO-only partitions, filters to `severity_text='ERROR'`, verifies row count matches ERROR records only, and confirms Parquet row-group pruning. Added `rec_sev()` fixture helper to generate `MinedRecord` instances with coherent OTLP severity values.
Forward compatibility: absent severity column `crates/ourios-querier/tests/forward_compat.rs`	New `rfc0007_4_severity_filter_on_column_absent_everywhere_is_empty` test queries with `severity_text='ERROR'` against old-schema tenant lacking the column, asserts empty result without error.
Test helper updates `crates/ourios-querier/tests/boundary.rs`, `crates/ourios-querier/tests/manifest.rs`	All `req()` helpers across test modules updated to explicitly set `severity_text: None` in `QueryRequest` struct construction.
Benchmark update `crates/ourios-bench/benches/b2.rs`	`template_exact` benchmark query construction updated to explicitly set `severity_text: None`.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

jensholdgaard/ourios#86: Introduces initial QueryRequest scaffold that this PR extends with the optional severity_text field and filtering logic.
jensholdgaard/ourios#90: Modifies the same rfc0007_3_real_engine_error_does_not_leak boundary test and aligns QueryRequest field usage.
jensholdgaard/ourios#91: Adds forward-compatible heterogeneous-schema reads in the same forward_compat.rs test area, providing context for the severity_text-absent handling in this PR.

Poem

🐰 A field hops softly into the query night,
severity_text glows with filtering light,
Old partitions bow—no error, just grace—
Empty results hold forward-compat's place,
Pruning whispers through row groups' embrace.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically summarizes the main change: adding a severity_text predicate to enable B1 level filtering in queries.
Description check	✅ Passed	The description is comprehensive and follows the template well, covering the what/why, how the feature works, test coverage, and local verification steps.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/b1-severity-and-bench

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

jensholdgaard · 2026-06-04T22:25:58Z

@coderabbitai review

coderabbitai · 2026-06-04T22:26:04Z

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot

Pull request overview

Adds an optional severity_text equality predicate to the RFC 0007 “throwaway” query surface so B1’s ... AND level='ERROR' shape can be expressed and measured (including Parquet-statistics pruning).

Changes:

Extend QueryRequest with severity_text: Option<String> and push it down to DataFusion as col(severity_text) = lit(X).
Add an execution test asserting severity_text='ERROR' both counts correctly and triggers row-group pruning via Parquet min/max stats.
Update all existing QueryRequest construction sites (tests + b2 bench) to set severity_text: None.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
crates/ourios-querier/src/lib.rs	Adds `severity_text` to `QueryRequest` and applies it as a DataFusion filter; updates crate docs.
crates/ourios-querier/tests/execution.rs	Adds `rec_sev` helper and new B1 test covering severity filtering + pruning.
crates/ourios-querier/tests/manifest.rs	Updates request helper to include `severity_text: None`.
crates/ourios-querier/tests/forward_compat.rs	Updates request helper to include `severity_text: None`.
crates/ourios-querier/tests/boundary.rs	Updates request construction to include `severity_text: None`.
crates/ourios-bench/benches/b2.rs	Updates benchmark request construction to include `severity_text: None`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jensholdgaard · 2026-06-04T22:39:38Z

@coderabbitai review

coderabbitai · 2026-06-04T22:39:43Z

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

The B1 thesis gate's query shape is `count WHERE tenant AND ts BETWEEN … AND level='ERROR'` (docs/benchmarks.md), but the throwaway QueryRequest only carried tenant + time + template_id, so the `level='ERROR'` arm couldn't be expressed — blocking any B1 measurement. Add an optional `severity_text` equality filter to the (explicitly mutable, pre-DSL) query surface. It's the structured counterpart to the B1 reference's `grep ERROR`: Ourios filters the RFC 0005 `severity_text` column, and the column's Parquet min/max statistics prune row groups that can't match — so an INFO-only file is skipped rather than scanned. The new `rfc0007_1_severity_filter_counts_and_prunes` test confirms both the count and that pruning fires (Utf8 stats pruning works for the equality predicate). The remaining construction sites (the other querier tests + the b2 bench) gain `severity_text: None`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

jensholdgaard · 2026-06-04T22:46:47Z

@coderabbitai review

coderabbitai · 2026-06-04T22:46:52Z

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

jensholdgaard requested a review from Copilot June 4, 2026 22:25

Copilot started reviewing on behalf of jensholdgaard June 4, 2026 22:26 View session

Copilot AI reviewed Jun 4, 2026

View reviewed changes

Comment thread crates/ourios-querier/src/lib.rs

Comment thread crates/ourios-querier/src/lib.rs

jensholdgaard force-pushed the feat/b1-severity-and-bench branch from 0f696dd to 5c105e8 Compare June 4, 2026 22:38

jensholdgaard requested a review from Copilot June 4, 2026 22:39

Copilot started reviewing on behalf of jensholdgaard June 4, 2026 22:39 View session

Copilot AI reviewed Jun 4, 2026

View reviewed changes

Comment thread crates/ourios-querier/tests/execution.rs

jensholdgaard force-pushed the feat/b1-severity-and-bench branch from 5c105e8 to a308c30 Compare June 4, 2026 22:46

jensholdgaard requested a review from Copilot June 4, 2026 22:46

Copilot started reviewing on behalf of jensholdgaard June 4, 2026 22:46 View session

Copilot AI reviewed Jun 4, 2026

View reviewed changes

jensholdgaard merged commit 69befe2 into main Jun 4, 2026
12 checks passed

jensholdgaard deleted the feat/b1-severity-and-bench branch June 4, 2026 22:52

jensholdgaard mentioned this pull request Jun 4, 2026

feat(bench): add the B1 predicate-pushdown bench + zstd|grep reference #115

Merged

Conversation

jensholdgaard commented Jun 4, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Tests

Summary by CodeRabbit

Release Notes

Uh oh!

coderabbitai Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Poem

❌ Failed checks (1 warning)

Uh oh!

jensholdgaard commented Jun 4, 2026

Uh oh!

coderabbitai Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

jensholdgaard commented Jun 4, 2026

Uh oh!

coderabbitai Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

jensholdgaard commented Jun 4, 2026

Uh oh!

coderabbitai Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jensholdgaard commented Jun 4, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 4, 2026 •

edited

Loading

coderabbitai Bot commented Jun 4, 2026 •

edited

Loading

coderabbitai Bot commented Jun 4, 2026 •

edited

Loading

coderabbitai Bot commented Jun 4, 2026 •

edited

Loading