docs(rfc): rfc 0022 — queryable attribute columns (rfc 0005 amendment) - #343
Conversation
Discharges the typed-attribute reservation in RFC 0005 §3.3 (#147): the RFC 0002 DSL's service / resource.<key> / attr.<key> predicates are the concrete consumer, currently served by an unpruned LIKE over the canonical-JSON columns (#146). Design: per-key promoted OPTIONAL Utf8 columns named literally after the DSL path (service.name always promoted; the rest via an RFC 0020 storage.promoted_attributes knob), dict + page-index + bloom encoded. Predicates on promoted keys compile to a two-arm expression — typed column (full cmp_op set, prunable) OR (IS NULL AND the existing JSON LIKE arm) — which keeps pre-amendment files and non-string values answering identically to today with no history rewrite (§3.5 migration plan = §3.9 additive OPTIONAL + the fallback arm). JSON columns stay the source of truth; the RFC 0017 read path never consumes promoted cells. MAP<STRING,STRING> rejected (no key-scoped stats/bloom → no pruning). Status `specified` (full §5, seven scenarios; RFC0022.5 uses the RFC 0016 scanned/pruned counters as its pruning oracle). Includes the RFC 0005 §3.3 pointer amendment and the SUMMARY.md entry; no schema bytes change until the red/green slices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 52 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds RFC 0022 for queryable promoted attribute columns, updates RFC 0005 to reflect the new promoted-column approach, and links the RFC from the docs summary index. ChangesRFC 0022 documentation
Estimated code review effort: 2 (Simple) | ~15 minutes Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR adds a new “specified” RFC (RFC 0022) that amends the RFC 0005 Parquet storage design to support queryable/pruneable per-attribute columns, aligning storage with the RFC 0002 DSL’s service / resource.<key> / attr.<key> predicates (tracked by #147). It’s a docs-only change intended to be reviewed/signed off before implementation.
Changes:
- Added RFC 0022 describing promoted per-key OPTIONAL Utf8 columns (plus compile-time fallback semantics) to enable pruning and a fuller operator set for promoted keys.
- Amended RFC 0005 §3.3 to explicitly point at RFC 0022 as the discharge of the previously reserved typed-attribute amendment.
- Registered RFC 0022 in the mdBook table of contents.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| docs/SUMMARY.md | Adds RFC 0022 to the RFC index so it appears in the rendered book navigation. |
| docs/rfcs/0022-queryable-attribute-columns.md | Introduces the RFC 0022 specification (schema projection rules, predicate compilation strategy, hazards/telemetry, acceptance criteria). |
| docs/rfcs/0005-parquet-storage.md | Adds an in-place amendment note referencing RFC 0022 as the fulfillment of the reserved §3.3 future change. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/rfcs/0022-queryable-attribute-columns.md`:
- Around line 132-137: The fenced pseudo-code block in the RFC is missing a
language label, which triggers markdownlint MD040. Update the fenced block
around match_expr(op, v) to use a text-style language tag so the documentation
remains lint-clean, and keep the content and surrounding explanation unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 9351a1fc-a249-4143-a4c9-926f1039f25f
📒 Files selected for processing (3)
docs/SUMMARY.mddocs/rfcs/0005-parquet-storage.mddocs/rfcs/0022-queryable-attribute-columns.md
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The promoted-bytes counter follows the existing ourios.storage.parquet.* semconv namespace, and the RFC 0005 pointer note now says schema bytes change at green (red is failing stubs only). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t ordering storage.promoted_attributes doesn't exist in RFC 0020's strict schema today; name it as an extension and call out upgrade-before-config-change (unknown keys are startup errors). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ordering/regex compile to the typed arm only (the JSON arm can't express them); pre-amendment files silently non-match those operators — stated explicitly, and RFC0022.4 now asserts it across an old+new-file scan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ic name, prose - name the exact DataFusion construction for dotted column names (Expr::Column(Column::new_unqualified(...)); col() parses qualified) - rename the telemetry instrument to ourios.storage.parquet.promoted.size per OTel semconv (units in metadata, not names; mirrors file.size) — verified via the OTel MCP - state service.name precisely (Required/Stable identity attribute of the service resource entity), and fix the §3.3 sentence fragment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
§3.4 and RFC0022.7 now say *configured* promoted set explicitly; the implicit service.name promotion cannot be disabled. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…y id Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…not bare equality Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ical type Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… only Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… spelled out http.status_code is commonly an int AnyValue, which projects NULL — swapped examples to http.request.method / http.route and rewrote §7's numeric-promotion question to distinguish string-encoded numbers from true numeric AnyValues (which need typed promotion to participate). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
What
RFC 0022 (
specified) — queryable attribute columns, the RFC 0005 amendment reserved in §3.3 and tracked by #147. Docs-only: the RFC, a pointer amendment in RFC 0005 §3.3, and the SUMMARY.md entry. No code — implementation holds for maintainer sign-off per §5.3 (same protocol as RFC 0019 / #287).The design in five lines
OPTIONALUtf8 columns, named literally after the DSL path (resource.service.name,attr.http.method) — dict + page-index + bloom.service.namealways promoted; the rest opt-in viastorage.promoted_attributes(RFC 0020).(P op v) OR (P IS NULL AND json_like_arm). Steady-state files prune via bloom/stats (null-count 0 kills the fallback arm); pre-amendment files answer identically to today's LIKE stopgap via §3.9's missing-column carve-out — the fallback arm is the §3.5 migration plan, no history rewrite.< <= > >=,=~,!~) on promoted keys only; non-promoted keys keep feat(querier): compile the DSL IR to the execution layer (RFC0002.1/.3/.4/.5/.6) #146's==/!=LIKE behaviour bit-for-bit.Rejected alternatives (§4)
MAP<STRING,STRING>(the §3.3 sketch — map stats/bloom aren't key-scoped, prunes nothing), full flattening (schema explosion), name mangling (collisions + leaked names; literal dots only require unqualified-column construction), query-time JSON UDFs (no pruning), and a cutover backfill (fallback arm makes it unnecessary; compaction converges history for free).Acceptance (§5, seven scenarios)
Projection semantics (.1/.2), old-file result parity (.3), operator gating (.4), pruning via the RFC 0016 scanned/pruned counters (.5, deterministic in the
rfc0007_1_*shape + indicative bench), read-path blindness (.6), and promoted-set drift across deploys (.7).Closes nothing yet — #147 closes when the ladder completes.
🤖 Generated with Claude Code
Summary by CodeRabbit