Skip to content

docs(rfc-0010): audit-stream queries + template drift surface - #164

Merged
jensholdgaard merged 10 commits into
mainfrom
docs/rfc-0010-audit-stream-queries
Jun 9, 2026
Merged

docs(rfc-0010): audit-stream queries + template drift surface#164
jensholdgaard merged 10 commits into
mainfrom
docs/rfc-0010-audit-stream-queries

Conversation

@jensholdgaard

@jensholdgaard jensholdgaard commented Jun 9, 2026

Copy link
Copy Markdown
Owner

What

RFC 0010 — Audit-stream queries & template drift (doc-only,
status: specified; dedicated drift surface maintainer-confirmed
2026-06-09). Adds docs/rfcs/0010-audit-stream-queries.md + the
docs/SUMMARY.md nav entry. No code.

The gap

Three RFCs leave one hole between them:

  • RFC 0001 §6.7 specifies the operator drift query ("templates that
    gained a version in [t1, t2]") as SQL "for spec clarity" and says the
    user-visible form is the RFC 0002 DSL — which was never specified.
  • RFC 0002 §6.3 explicitly defers the audit-stream query surface
    to a future RFC.
  • RFC 0005 already persists audit events to a queryable audit/
    Parquet series (ParquetAuditSink / AuditReader / audit_schema()),
    but nothing turns them into an operator query.

Design

A dedicated, contained drift from <t1> to <t2> verb head over the
per-tenant audit/ stream — NOT RFC 0002's deferred general aggregation
pipeline. Half-open [from, to) window defined locally (RFC 0002 does not
pin range boundaries). §5 carries RFC0010.1–.8 (RFC0010.1 discharges
RFC 0001 H5.3); implementation follows once specified.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Added RFC 0010 detailing a new audit-stream query for detecting template drift. Covers query surface, semantics, tenant isolation, deterministic ordering, acceptance criteria, testing strategy, and handling of edge cases (windows, empty results, tie-breaking). Includes rationale, rejected alternatives, and outstanding questions for future refinement.

Fills the audit-stream query gap RFC 0002 §6.3 deferred: a first-class,
contained `drift` query head over the per-tenant RFC 0005 audit/ Parquet
stream that encapsulates the fixed RFC 0001 §6.7 drift aggregation
(filter widening/type-expansion events in [t1, t2), group by template_id,
project widening_count + version/time bounds, order by count desc).

Deliberately does not require RFC 0002's deferred general count/agg
pipeline; the §5 criteria (RFC0010.1-.8) discharge RFC 0001 scenario
H5.3. Spec-only (status drafted); extends RFC 0002 without reopening it
and reads — does not redefine — the RFC 0005 audit schema. Adds the nav
entry in docs/SUMMARY.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jensholdgaard
jensholdgaard requested a review from Copilot June 9, 2026 10:21
@jensholdgaard

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Drafts RFC 0010 to specify a first-class drift query head over the per-tenant RFC 0005 audit/ Parquet stream, filling the previously-deferred audit-stream query surface (RFC 0002 §6.3) and defining acceptance criteria intended to discharge RFC 0001 hazard scenario H5.3.

Changes:

  • Add new RFC document: RFC 0010 — Audit-stream queries & template drift (status: drafted), including proposed DSL surface, semantics, and acceptance criteria.
  • Add the RFC 0010 entry to the mdBook navigation (docs/SUMMARY.md).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
docs/SUMMARY.md Adds RFC 0010 to the RFCs table of contents.
docs/rfcs/0010-audit-stream-queries.md Introduces the RFC 0010 draft specifying the drift query surface and related criteria.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 17a0872c-3c76-4f69-9876-3c182f261500

📥 Commits

Reviewing files that changed from the base of the PR and between 30af8a4 and f08e37f.

📒 Files selected for processing (1)
  • docs/rfcs/0010-audit-stream-queries.md
✅ Files skipped from review due to trivial changes (1)
  • docs/rfcs/0010-audit-stream-queries.md

📝 Walkthrough

Walkthrough

This PR adds RFC 0010: a specification for a dedicated drift DSL query performing half-open windowed aggregation over tenant-scoped audit Parquet streams to detect templates gaining new versions, with precise semantics, acceptance criteria, DataFusion-level compilation, testing strategy, alternatives, and open questions.

Changes

RFC 0010: Audit-stream drift queries specification

Layer / File(s) Summary
RFC metadata, discovery, and scope
docs/SUMMARY.md, docs/rfcs/0010-audit-stream-queries.md
RFC 0010 header and metadata added; SUMMARY.md index updated; motivation and explicit in-scope vs out-of-scope boundaries for the drift head documented.
Audit background, acceptance criteria, and DSL semantics
docs/rfcs/0010-audit-stream-queries.md
Audit partition layout and column set described; acceptance criteria RFC0010.1–RFC0010.8 enumerate windowing [from,to), widening-only scoping, tenant isolation, deterministic ordering, aggregation bounds, and no-DataFusion/SQL leakage; full drift grammar/JSON surface, DriftRow shape, partition pruning, and Filter→Aggregate→Sort compilation via RFC 0007 specified.
Testing strategy, alternatives, open questions, and references
docs/rfcs/0010-audit-stream-queries.md
Integration test plan replacing H5.3 miner stub; parser/unit/boundary/golden/no-leakage tests; rejected alternatives listed; open questions enumerated; references to prior RFCs and audit interfaces provided.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • jensholdgaard/ourios#83: RFC 0010’s audit-stream drift query RFC explicitly builds on the DataFusion “Querier” execution/no-leakage model defined in RFC 0007, so the two documentation changes are directly connected.

Poem

A rabbit reads the audit stream,
Counts widening versions in a dream,
Windows half-open, tidy and neat,
Tenant paths pruned at their feet,
DSL whispers—no SQL leak, sweet. 🐇

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is comprehensive and covers the PR objectives, context, and design decisions. However, it is missing the standard checklist items from the template (cargo fmt, clippy, tests, docs/CHANGELOG, RFC link). Add the checklist section with the required items checked or marked as N/A for documentation-only changes, and explicitly link the RFC file.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding RFC 0010 documentation for audit-stream queries and template drift surface.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/rfc-0010-audit-stream-queries

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

…fix dep statuses

Maintainer confirmed the dedicated drift query head (not the general
aggregation pipeline) on 2026-06-09 — bump drafted→specified. Fix the
§2.1 dependency-status claim: RFC 0001 is specified + RFC 0005 drafted,
not all accepted/green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
docs/rfcs/0010-audit-stream-queries.md (1)

261-271: ⚡ Quick win

Clarify how drift_query integrates into the top-level grammar.

The EBNF production defines drift_query but does not show how it integrates with the existing RFC 0002 grammar. Line 269 states it's "a top-level alternative to RFC 0002's predicate { | stage } query," suggesting the overall grammar now has multiple top-level alternatives. Consider adding a brief note showing the updated top-level production, e.g.:

query = log_query | drift_query ;
log_query = predicate , { "|" , stage } ;
drift_query = "drift" , "from" , time , "to" , time ;

This would make the grammar extension explicit and unambiguous for implementers.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/rfcs/0010-audit-stream-queries.md` around lines 261 - 271, Add an
explicit top-level production to show how drift_query is integrated: state that
the top-level nonterminal query now alternates between the existing log_query
(the RFC0002 predicate with optional "|" stage(s)) and the new drift_query (the
"drift" from time to time form), and clarify that drift_query is a standalone
top-level alternative that does not accept "|" stages; update the grammar
paragraph around drift_query and the examples to reflect this change and
reference the existing predicate and stage nonterminals.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/rfcs/0010-audit-stream-queries.md`:
- Around line 387-393: Update the documentation cross-reference and/or RFC
wording to explicitly state the half-open boundary semantics used by the
querier: mention that Stage::Range(from, to) in
crates/ourios-querier/src/compile.rs compiles to the predicate `TIME_UNIX_NANO
>= start AND TIME_UNIX_NANO < end` (i.e., [from, to) lower-inclusive,
upper-exclusive), and add that exact wording to RFC 0002 §6.1 (or change RFC
0010’s reference to point at a new § documenting the boundary contract) so the
claim that RFC0010.2 and RFC0002 agree is backed by explicit text; reference the
symbols Stage::Range, TIME_UNIX_NANO, RFC 0002 §6.1, and RFC 0010.2 when making
the edits.

---

Nitpick comments:
In `@docs/rfcs/0010-audit-stream-queries.md`:
- Around line 261-271: Add an explicit top-level production to show how
drift_query is integrated: state that the top-level nonterminal query now
alternates between the existing log_query (the RFC0002 predicate with optional
"|" stage(s)) and the new drift_query (the "drift" from time to time form), and
clarify that drift_query is a standalone top-level alternative that does not
accept "|" stages; update the grammar paragraph around drift_query and the
examples to reflect this change and reference the existing predicate and stage
nonterminals.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1b127819-f538-4d55-9a2a-69d7137f1256

📥 Commits

Reviewing files that changed from the base of the PR and between 523adaf and d6773e5.

📒 Files selected for processing (2)
  • docs/SUMMARY.md
  • docs/rfcs/0010-audit-stream-queries.md

Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

Comment thread docs/rfcs/0010-audit-stream-queries.md
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
…laim

Hazards are numbered items under CLAUDE.md §4 / docs/hazards.md, not §4.5
/§4.6 subsections — cite H5/H6. README has no §5 — cite its
Required-sections acceptance-criteria format. And RFC 0002 does not pin
range boundary semantics, so RFC 0010 defines the drift window as
half-open [from,to) on its own (recommending range converge) rather than
claiming RFC 0002 already does.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jensholdgaard jensholdgaard changed the title docs(rfc-0010): draft audit-stream queries + template drift surface docs(rfc-0010): audit-stream queries + template drift surface Jun 9, 2026
@jensholdgaard
jensholdgaard requested a review from Copilot June 9, 2026 11:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
…02 range

Remove the remaining attributions that RFC 0002 §6.1 range specifies
half-open [from,to) (it does not): the RFC0010.2 §5 criterion, the §6.5
parenthetical, and the boundary-test golden reference now state the
half-open semantics this RFC defines.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread docs/rfcs/0010-audit-stream-queries.md
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
…tter

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

Comment thread docs/rfcs/0010-audit-stream-queries.md
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
… throughout

Bare [§6.7] read as an internal ref (this RFC has no §6.7) — now
[RFC 0001 §6.7]. Closed [t1,t2] in the summary + §5 criteria conflicted
with the half-open [from,to) window — now [t1,t2) everywhere for
consistency with §6.5.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
RFC 0002 §7 timestamps are bare lexical tokens, not quoted strings —
unquote the absolute-window example so it parses. And [§3.7]/(§3.7) are
CLAUDE.md (multi-tenancy), not in-document sections — cite explicitly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
…ner-confirmed)

The status note says the verb-head surface was confirmed 2026-06-09, so
the §9 open question for it is now checked [x]; the remaining forks
(window, tie-break, version aggregation) stay open.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
…ing)

RFC 0005 §3.7 is the audit-event file schema; §6 is its testing
strategy. Fix the columns / event-kind-mapping / predicate-pushdown /
references citations to §3.7 (keeping §3.4 for the partition layout).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread docs/rfcs/0010-audit-stream-queries.md Outdated
event_kind is Arrow UInt8 / INTEGER(8, unsigned) (not INT32);
template_id is UInt64; old_version/new_version are UInt32 and OPTIONAL
(relaxed for the compaction kind, required-by-convention for template
kinds) — matching ourios-parquet audit_schema().

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@jensholdgaard
jensholdgaard merged commit d733287 into main Jun 9, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants