feat(querier): numeric-class predicates + cast-free aggregation (RFC 0042 slice 3b) - #652
Conversation
…0042 slice 3b) Predicate compilation (RFC 0042 §3.4): the union schema's promoted column type — the declared class's type since slice 3a — routes attr/resource comparisons. Numeric classes require numeric literals (the grammar already parses attr.x == 500; compile now types it): - ordering compiles typed-arm-only, prunable via numeric min/max stats - ==/!= on i64 carry the JSON fallback arm; canonical integer formatting is unique (the stored form is "intValue":"<decimal>", proto3-JSON string-encoded i64), so the fragment is exact, with the presence guard mirroring the string two-arm form - ==/!= on f64 are typed-arm-only: JSON text has no canonical float form, so a fallback arm would be wrong both directions - a string literal, a float against i64, or a regex on any numeric class is a compile error naming the declared class (RFC0042.7) - int literals widen into f64, matching the write-side projection Aggregation (RFC0042.3 / §3.5): Float64 columns aggregate bare; Int64 takes a plain numeric cast to the Float64 output (exact widening); Utf8 keeps the RFC0002.17 try_cast. No parse-shaped cast touches a numeric-class column. Tests extend the mixed-schema fixture: ordering covers the declared- class file only; == answers a pre-declaration file via the JSON arm and the typed file via the typed arm; != spans both arms with absent keys excluded; the f64 half (typed-arm-only equality, int widening); and the three compile errors. Completes RFC0042.3/.4/.7 and .5. Verified: cargo fmt --check, clippy clean, nextest 1277/1277, cargo doc -D warnings clean. Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org>
|
Warning Review limit reached
Next review available in: 14 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Implements RFC 0042 slice 3b in the querier by using the post-union schema’s declared promoted-column types to (1) compile numeric-class predicates with correct typed/JSON-arm behavior and (2) avoid parse-shaped casts for numeric aggregation over promoted columns.
Changes:
- Add schema-driven numeric predicate compilation for
i64/f64promoted keys (typed-only ordering;i64==/!=includes JSON fallback;f64equality is typed-only; class-naming compile errors). - Make scalar aggregation cast-free for numeric-class promoted columns (
Float64direct;Int64numeric cast;Utf8keepstry_cast). - Extend mixed-schema integration tests to cover numeric predicate matrix and
f64behaviors.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| crates/ourios-querier/tests/it/rfc0042_mixed_schema.rs | Adds mixed-schema integration tests for numeric predicate typing and f64 semantics. |
| crates/ourios-querier/src/lib.rs | Adds column_type() helper so the compiler can read promoted column classes from the union schema. |
| crates/ourios-querier/src/compile.rs | Routes attribute predicate compilation via numeric-class paths and updates scalar aggregation casting rules. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The comment claimed the Int64 -> Float64 cast is exact widening; it is exact only for |v| <= 2^53 (RFC 0042 §3.1's own stated bound). Name the bound and the consequence instead of overclaiming. Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org>
…ice 4) (#653) RFC0042.8: two input files — the key promoted under the string class in one epoch, unpromoted in the other — consolidate under the current i64 declaration into an Int64 column projected from JSON truth (both epochs' values present), and the rewrite is deterministic: byte-identical across two identical runs, the RFC0036.4 property under a fixed config. The RFC banner records where the ladder stands: .1-.8 green in-repo across #649-#652 + this test; RFC0042.9 (the dogfood corpus gate) outstanding until the first agent capture under the typed set, so the status stays specified with the implementation ahead of the flip. Verified: cargo fmt --check, clippy clean, nextest 165/165 (parquet), mdbook build. Signed-off-by: Jens Holdgaard Pedersen <Jens@holdgaard.org>
What (RFC 0042 slice 3b — completing the querier)
Predicate typing and cast-free aggregation over the typed columns slice 3a's scan makes available. The grammar already parsed
attr.x == 500(Value::Intin the IR, with an existing parser test) — this slice makes the compiler type it against the declared class, read off the union schema.Predicates (RFC 0042 §3.4 / RFC0042.4/.7 + the
==half of .5)i64/f64==/!=oni64"intValue":"<decimal>"(proto3-JSON string-encoded i64 — verified against the canonical encoder's own tests), presence-guarded like the string form==/!=onf64f64i64/ regexAggregation (RFC0042.3 / §3.5)
Float64aggregates bare;Int64takes a plain numericcastto the Float64 output (exact widening — same rule as the writer);Utf8keeps the RFC0002.17try_cast. No parse-shaped cast ever touches a numeric-class column.What made this slice small
Two prior decisions paid off: slice 3a's rule that the union schema carries the declared type means the compiler learns a key's class from the schema it already consults (no new plumbing), and numeric group-keys already rendered (
group_key_stringcovers Int64/Float64), soby attr.input_tokensneeded zero work.Tests
The mixed-schema fixture now drives the full predicate matrix: ordering covers the declared-class file only;
== 7answers a pre-declaration file via the JSON arm while== 40answers the typed file via the typed arm;!=spans both arms with absent keys excluded; the f64 half (typed-arm-only equality — the pre-amendment 0.5 deliberately NOT matched, the documented §3.4 consequence — plus int-widening ordering); and all three compile errors asserting the class is named. RFC0042.3, .4, .5, .7 complete.Remaining (slice 4)
Compaction re-projection across a re-typing (RFC0042.8), status flips, #646 in typed syntax, and the dogfood corpus gate (RFC0042.9).
Verification
cargo fmt --check· clippy clean · nextest 1277/1277 ·cargo doc -D warningsclean.