Skip to content

chore(miner): add ourios-miner crate with §5 acceptance stubs (specified → red) - #7

Merged
jensholdgaard merged 3 commits into
mainfrom
chore/rfc-0001-red-gate
May 2, 2026
Merged

chore(miner): add ourios-miner crate with §5 acceptance stubs (specified → red)#7
jensholdgaard merged 3 commits into
mainfrom
chore/rfc-0001-red-gate

Conversation

@jensholdgaard

@jensholdgaard jensholdgaard commented May 2, 2026

Copy link
Copy Markdown
Owner

Summary

Move RFC 0001 from specified to red per docs/verification.md §3 by landing the ourios-miner crate skeleton plus 29 todo!() stubs — one per §5 scenario.

The Red gate is mechanical: every §5 scenario has at least one stub with a matching id, the crate compiles cleanly, the stub is tagged #[ignore], and cargo test -- --include-ignored exits non-zero on each.

Two-loop Red signal (the load-bearing decision in this PR)

The original draft of this PR would have failed CI's cargo test job because the stubs panic via todo!(). After the team's "we need granularity for the Red gate" feedback, the second commit (5d74cbf) splits the Red signal into two loops, formalised in docs/verification.md §3:

  • Outer loop (CI / branch protection). Every stub carries #[ignore = "RFC 0001 Red gate — implementation pending"]. Default cargo test skips them → CI green → PR merges through branch protection cleanly.
  • Inner loop (local dev cycle). cargo test -- --ignored (or cargo test <name> -- --ignored) runs the stubs and watches the todo!()s panic. Each panic is one TODO item for the implementor; as a stub gets filled in, its #[ignore] comes off and the test joins the default run.

This split is what lets a Red-stage main be a healthy, mergeable state rather than a half-broken branch.

Changes

File Change
Cargo.toml Add crates/ourios-miner to workspace members; remove its line from the comment block
crates/ourios-miner/Cargo.toml New — mirrors ourios-core's pattern (workspace inheritance, lints, no deps)
crates/ourios-miner/src/lib.rs New — one doc-comment line (matches ourios-core/src/lib.rs)
crates/ourios-miner/tests/hazards.rs New — 12 #[ignore]'d stubs (H1.1–H1.3, H2.1–H2.2, H5.1–H5.3, H7.1–H7.4)
crates/ourios-miner/tests/invariants.rs New — 9 #[ignore]'d stubs (§3.1.x, §3.2.x, §3.3.1, §3.5.x, §3.7.x)
crates/ourios-miner/tests/rfc_internal.rs New — 8 #[ignore]'d stubs (RFC0001.1–RFC0001.8)
docs/rfcs/0001-template-miner.md Frontmatter status: specifiedstatus: red
docs/verification.md §3 maturity-table row + prose updated to formalise the two-loop Red split; §6.3 worked example updated to show the #[ignore] pattern; H1.2 / H1.3 worked-example titles and stub function names synced with RFC 0001 §5 wording (drift from earlier doc revs); both cargo test --include-ignored invocations fixed to cargo test -- --include-ignored (Cargo rejects the missing-separator form)

Stub format

Each stub follows the (now updated) docs/verification.md §6.3 worked example:

/// Scenario H1.1 — Semantically distinct templates do not silently merge.
/// See `docs/rfcs/0001-template-miner.md` §5.
#[test]
#[ignore = "RFC 0001 Red gate — implementation pending"]
fn h1_1_login_and_logout_remain_distinct_at_default_threshold() {
    todo!("RFC 0001 §6.4");
}

The todo!("RFC 0001 §6.x") body points at the spec subsection that defines the contract — gives the future implementor a hyperlink-style hop. The id in the doc comment + function name makes grep -R "H1.1" . resolve bidirectionally between RFC §5 and the test stub.

Out of scope

  • Implementation. That's the Green gate.
  • The Cargo.lock cleanup (still untracked locally; separate tiny PR).

Checklist

  • cargo fmt clean
  • cargo clippy clean (no new warnings; pedantic forced backticks on lossy_flag, template_version, template_widened, template_type_expanded, template_ids, confidence_p50, confidence_p01 in doc comments — fixed in the same series)
  • Tests added (29 #[ignore]'d todo!() stubs)
  • Docs / CHANGELOG.md updated (RFC frontmatter; verification.md two-loop spec; git-cliff regenerates CHANGELOG.md at release)
  • RFC linked: this PR is the Red-gate transition for RFC 0001

Verification (CLAUDE.md §6.6, run locally on the latest commit)

  • cargo fmt --all --check
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo test --all-features → 29 ignored, 0 failed (outer loop / CI-equivalent) ✓
  • cargo test --all-features --no-fail-fast -- --ignored → 29 failed (12 + 9 + 8), 0 passed (inner-loop signal) ✓
  • mdbook build
  • Greppability spot-check on H1.1, H7.4, §3.7.2, RFC0001.8 — each resolves to both RFC §5 and the matching test stub

Commit history

  • 23f3a92 — initial Red landing (would have turned CI red)
  • 5d74cbf#[ignore] the stubs + formalise the two-loop Red split in docs/verification.md §3 (the "we need granularity for the Red gate" feedback)
  • effd842 — fix cargo test --include-ignored syntax (missing -- separator) + sync H1.2/H1.3 worked-example titles and stub names with RFC 0001 §5

🤖 Generated with Claude Code

…ied → red)

Move RFC 0001 from specified to red per docs/verification.md §3 by
landing the ourios-miner crate skeleton plus 29 todo!() stubs, one
per §5 scenario. The gate is mechanical: every scenario in §5 has
a stub with a matching id, the crate compiles, and cargo test
exits non-zero on every stub.

Layout follows the existing ourios-core crate: workspace-inherited
edition / rust-version / license / repository, [lints] workspace =
true so the pedantic clippy and unsafe_code = deny rules apply.
No dependencies and no library code yet — Green gate's job.

Tests are split into three files mirroring §5's subsections so
grep-by-id is fast and reviews stay scannable:

  tests/hazards.rs       — 12 stubs (H1.1–H1.3, H2.1–H2.2,
                            H5.1–H5.3, H7.1–H7.4)
  tests/invariants.rs    — 9 stubs (§3.1.x, §3.2.x, §3.3.1,
                            §3.5.x, §3.7.x)
  tests/rfc_internal.rs  — 8 stubs (RFC0001.1–RFC0001.8)

Each stub follows verification.md §6.3's worked example exactly:
doc comment carrying the scenario id and title, See `docs/rfcs/
0001-template-miner.md` §5 link, todo!("RFC 0001 §6.x") body
pointing at the spec subsection that defines the contract. The id
in the doc comment + function name makes grep -R resolve
bidirectionally between RFC and tests as docs/verification.md §2.3
prescribes.

Verification (CLAUDE.md §6.6):
- cargo fmt --all --check ✓
- cargo clippy --all-targets --all-features -- -D warnings ✓
- cargo test --all-features --no-fail-fast → 29 failed (12 + 9 +
  8), 0 passed; exit non-zero satisfies the Red signal
- mdbook build ✓
- Greppability spot-check on H1.1, H7.4, §3.7.2, RFC0001.8 — each
  resolves to both RFC §5 and the matching test stub

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Transitions RFC 0001 to the Red stage by adding the new ourios-miner crate skeleton and introducing failing acceptance-criteria test stubs that map 1:1 to RFC 0001 §5 scenarios, per docs/verification.md.

Changes:

  • Add crates/ourios-miner as a workspace member with a minimal crate skeleton.
  • Add 29 todo!()-based failing test stubs in crates/ourios-miner/tests/ for hazards, invariants, and RFC-internal commitments.
  • Update RFC 0001 frontmatter status: from specifiedred.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Cargo.toml Adds ourios-miner to workspace members (causes workspace-wide cargo test to include miner stubs).
crates/ourios-miner/Cargo.toml New crate manifest matching existing workspace patterns.
crates/ourios-miner/src/lib.rs New minimal library entrypoint doc comment.
crates/ourios-miner/tests/hazards.rs Adds failing test stubs for hazard scenarios H1/H2/H5/H7.
crates/ourios-miner/tests/invariants.rs Adds failing test stubs for invariant scenarios §3.1/§3.2/§3.3/§3.5/§3.7.
crates/ourios-miner/tests/rfc_internal.rs Adds failing test stubs for RFC-internal scenarios RFC0001.1–RFC0001.8.
docs/rfcs/0001-template-miner.md Updates RFC status to red.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Cargo.toml
Comment thread crates/ourios-miner/tests/rfc_internal.rs Outdated
Address PR #7 review:

- Copilot (Cargo.toml:3) flagged that landing the Red-stage stubs
  as plain todo!() turns the cargo test CI job red and breaks
  branch protection. The right fix is structural, not a workflow
  hack: tag every stub `#[ignore]` so default `cargo test` skips
  them (CI green), and document the inner-loop signal —
  `cargo test -- --ignored` — as the implementor's TODO list.
- Copilot (rfc_internal.rs:2) flagged that the file-level rustdoc
  hyphenates "Acceptance-criteria" awkwardly across two lines.
  Drop the hyphen on every test file's intro paragraph.

Resulting two-loop split for the Red stage:

  Outer loop (CI / branch protection):
    - cargo test --all-features  → all stubs reported as ignored,
                                    exit zero, CI green
    - Per-scenario coverage check is structural: for each §5 id,
      grep -R "<id>" .  must resolve to both the RFC line and a
      test stub line (the docs/verification.md §2.3 contract)
  Inner loop (local dev):
    - cargo test -- --ignored  → 29 todo!() panics, exit non-zero;
                                  one panic per scenario the
                                  implementor still owes
    - As each stub is filled in, its #[ignore] comes off and the
      test joins the default run; the inner-loop count drops by
      one and the outer-loop default-run count rises by one. When
      the last #[ignore] comes off the gate has crossed
      Red → Green.

This split is now formalised in docs/verification.md §3 (table
row + prose, including a paragraph titled "The Red signal lives
at two granularities, deliberately"), and the §6.3 worked example
shows the #[ignore] pattern verbatim. The §2 cross-doc paragraph
on "Acceptance criteria → Red tests" picks up a one-sentence
pointer to §3's two-loop spec.

The user's framing — "we are in the outer loop" — is the
load-bearing observation: a Red-stage main is healthy, with the
outer loop green and the inner loop fully populated with the work
that needs doing.

Verification (CLAUDE.md §6.6):
- cargo fmt --all --check ✓
- cargo clippy --all-targets --all-features -- -D warnings ✓
- cargo test --all-features  → 29 ignored, 0 failed (outer loop) ✓
- cargo test --all-features --no-fail-fast -- --ignored
    → 29 failed (12 + 9 + 8), 0 passed (inner loop signal) ✓
- mdbook build ✓

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/verification.md Outdated
Comment thread docs/verification.md Outdated
Comment thread docs/verification.md Outdated
Comment thread docs/verification.md
…ample with RFC 0001 §5

Three more PR #7 review hits, all in docs/verification.md:

- Copilot (line 251): `cargo test --include-ignored` is missing
  the `--` separator, so Cargo rejects it as an unknown Cargo
  flag. Confirmed locally:
    error: unexpected argument '--include-ignored' found
      tip: to pass '--include-ignored' as a value, use
           '-- --include-ignored'
  Fixed both occurrences (lines 239 and 251 pre-edit) to use
  `cargo test -- --include-ignored`.

- Copilot (line 430): the §6.3 worked-example stub for H1.2
  used the title "Lossy match retains body" while RFC 0001 §5
  defines H1.2 as "Lossy-zone match retains body". The stub
  function name (`h1_2_lossy_match_retains_body`) also drifted
  from the actual stub in `crates/ourios-miner/tests/hazards.rs`
  (`h1_2_lossy_zone_match_retains_body`). Sync title and
  function name; do the same for the §6.2 GWT block above.

- Copilot (line 438): the §6.3 H1.3 stub used "Every merge
  emits an audit event" while RFC 0001 §5 says "Every widening".
  The §6.2 GWT title was also stale ("merge"), and the bullet
  body said "the reason" where RFC 0001 §6.4's audit schema now
  uses `event_type`. Sync the worked example to match the RFC
  word-for-word, and fix the post-block prose ("audit every
  merge" → "audit every widening"; "absence of merge events" →
  "absence of widening events").

The greppability contract relies on these names being
consistent across RFC, worked example, and stubs — Copilot's
checks caught what reviewers using `grep -R "H1.3"` would have
hit later.

Verification (CLAUDE.md §6.6): cargo fmt clean, cargo clippy
clean, cargo test passing (29 ignored), mdbook build clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@jensholdgaard
jensholdgaard merged commit 69eed13 into main May 2, 2026
8 checks passed
@jensholdgaard
jensholdgaard deleted the chore/rfc-0001-red-gate branch May 2, 2026 18:54
jensholdgaard added a commit that referenced this pull request Jul 11, 2026
* feat(bench): rfc 0031 — floor-direction gate for l6/l7 reporting

L6 (broad scan) and L7 (ingest) are FLOOR/parity classes in RFC 0031
§2 — Ourios may be worse by at most F_L6/F_L7 — but the indicative
report pushed the time-window pairs through bytes_must_win, the wrong
direction (run #7 printed advantage 0.02 as a must-win fail instead of
answering "is Ourios within 3x of Loki?").

Adds bytes_within_floor(ourios, loki, factor): pass iff
ourios <= factor x loki, mirroring bytes_must_win's honesty guards
(factor==0 and zero measurements are Invalid, never a pass). The
overflow arm differs deliberately: an overflowing factor x loki budget
would be a mathematically true pass on an implausible measurement, so
checked_mul fails it closed as Invalid rather than letting a saturated
budget pass everything. advantage keeps the loki/ourios orientation so
both gates' tables read the same way; only the pass rule differs.

The indicative report gains a GateKind on PairSpec: the severity pair
stays must-win (m_l2), the time-window slices report via the floor
gate (f_l6), and the gate lines name the direction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 — reason-field doc covers all invalid arms, param rename

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
jensholdgaard added a commit that referenced this pull request Jul 11, 2026
…havior (#483)

* test(bench): rfc 0031 — pin promoted service.name parity in the comparative store (rfc 0022)

Grounding for the "promote service.name in the comparative store" slice
found the premise already satisfied: RFC 0022 §3.1 promotes service.name
implicitly and non-removably on every Writer constructor, including the
Writer::open path build_comparative_store uses, and the querier's
two-arm compile is union-schema-gated — so the comparative store's files
already carry a stats-bearing resource.service.name column and the DSL
`service ==` predicate already compiles to the promoted arm, not the
JSON-substring LIKE fallback. No writer-side change is needed (threading
PromotedAttributes through build_comparative_store would be a behavioral
no-op), so this lands the parity pin as tests instead.

Parity rationale (RFC 0031 §2 anti-strawman discipline): Loki's stock
OTLP ingest promotes service.name to the service_name stream label by
default, so service-scoped LogQL queries are label-indexed. The tests
pin that Ourios holds the same ground — if a writer or store-builder
change ever demoted the promoted column, the comparative harness would
silently hold Ourios to a harder version of the service-scoped question
than Loki answers.

Two fixture-based tests (no containers), two-service two-hour corpus:

- comparative_store_promotes_the_service_name_column: every published
  data file's footer declares resource.service.name, field-identical to
  the writer's default promoted schema.
- service_predicate_prunes_on_the_promoted_column: `service == "svc-a"`
  matches exactly the 3 svc-a rows with row_groups_pruned = 1 (svc-b's
  row group skipped via promoted-column statistics) and bytes_read 284
  vs 322 for the full-window scan.

Measured while grounding, for the run #7 "service queries read whole
partitions" observation: with both services interleaved in one hour
partition the same query shows row_groups_pruned = 0 — matching rows
sit in every row group, so its min/max spans both services and nothing
can be skipped. The whole-partition reads are a data-layout property
(no service clustering within row groups), not a predicate-compilation
gap; a fix belongs to write-time layout (sorting/clustering), which is
RFC-level, not a harness knob.

Adds parquet (same workspace major, arrow feature) as a dev-dependency
for the footer-level schema assertion. B1/B2's build_query_store path
and the single-service fixture tests are untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(bench): rfc 0031 — exhaustive service match + enable_all runtime

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
jensholdgaard added a commit that referenced this pull request Jul 15, 2026
…m regressions

PR review findings #5, #6, #7.

#5: the RFC0002.7 round-trip generator's `bare_field()` — shared by
`path_field()`, `group_term()`, and the `project` field list — omitted
`Field::EventName`, so grouped-query round-trips never covered
`count by event_name`. It's a valid bare field everywhere the real
grammar's `bare_field` (parse.rs) allows it, so fixed in place.

#6: `rfc0002_12_count_by_matches_naive_oracle`'s foreign-tenant fixture
is written via `write_all`, which partitions by the record's own
`tenant_id` — so the "b" row lands under tenant "b"'s own directory and
the row-level `tenant_id == tenant` backstop in `execute_aggregate`
(CLAUDE.md §3.7) is never exercised, only directory-level scoping. Adds
`rfc0002_12_aggregation_tenant_backstop_excludes_misplaced_row`, which
plants a tenant "b" row *inside* tenant "a"'s partition directory (the
shape a partitioning bug or on-disk corruption would produce — the
`ourios-parquet` writer's RFC 0005 §3.9 row-vs-path contract refuses a
mismatched tenant_id at write time, so the row is written honestly then
relocated) and asserts the backstop filter, not partitioning, keeps it
out of both the count and the group map. Manually verified this test
fails without the backstop filter, confirming it exercises the guard.

#7: RFC0002.15 covered a `params` list shorter than `n + 1`, but not the
distinct case of a list that HAS slot n whose own `value` decodes as
Parquet-level NULL (the field is nullable — RFC 0005 §3.2 — even though
`Param.value` is a non-`Option` Rust `String`, so only a raw/corrupted
writer can produce it). Adds
`rfc0002_15_present_but_null_param_slot_excluded_and_tallied`, built
with a raw arrow-array batch (mirroring `forward_compat.rs`'s
schema-drift fixtures) so the disposition is proven on the actual
`decode_aggregate` code path rather than assumed from the short-list
case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jensholdgaard added a commit that referenced this pull request Jul 15, 2026
…bucket(w) (#533)

* feat(querier): rfc 0002 green — count-by execution with param(n) and bucket(w)

The aggregation-execution slice of the RFC 0002 amendment 2026-07-15
(RFC 0031 L4): `count [by …]` now executes end-to-end, discharging §5
scenarios RFC0002.12/.13/.15/.16. RFC0002.14 (the grammar/compile error
contracts) stays an ignored red stub for its own slice.

Surface (§7 v1.1 / §6.4 amendment):
- IR: the aggregation `by`-list widens from `Vec<Field>` to
  `Vec<GroupTerm>` (field | `param(n)` | `bucket(duration)`).
- Parser: `group_list`/`group_term` productions, confined to `by`-lists;
  positive + negative parse tests per production.
- Structured surface: `{"param": n}` / `{"bucket": "<duration>"}`
  by-elements (widths validated by the string-DSL lexer, RFC0002.2);
  `structured_query.schema.json` gains the additive `group_term` def
  (snapshot-gated by RFC0002.11, which also gains instances).
- Serializer: group terms round-trip (corpus + proptest generator).

Compile (§6.3/§6.5 amendment):
- `compile::validate` lifts the `count` rejection ONLY — sum/min/max/avg,
  sort, project, render keep the explicit rejection. Enforces the
  single-template pinning rule for `param(n)` (top-conjunctive
  `template_id == N`, all naming one N; `resolves_to` does not pin),
  positive bucket widths, and the duplicate-term rules.
- Group terms lower as expressions inside the existing Aggregate row:
  `param(n)` = `array_element(params, n+1).value` (stored string form,
  no type promotion); `bucket(w)` = floor division of the effective
  timestamp (with the §3.9 `time_unix_nano` fallback) into half-open
  epoch-aligned UTC windows; `service` = the RFC 0022 promoted column.

Execute:
- One grouped-count scan per aggregation query (Filter → Aggregate,
  the drift precedent) with a row-level `tenant_id` guard mirroring
  drift's (§3.7 — group values are row contents). `rows` stays the
  total matching count, derived from the same scan.
- Short/NULL `param(n)` rows are EXCLUDED from every group (no synthetic
  absent key) and tallied on the new `QueryStats.rows_excluded`,
  surfaced on the RFC 0016 stats DTO (RFC0002.15).
- Result carrier: `QueryResult.aggregate: Option<Vec<AggregateGroup>>`
  (`key: Vec<String>` per by-term in query order — bucket keys RFC 3339
  UTC window starts — sorted, engine-free per hazard §4.6); the RFC 0016
  response gains the additive `aggregate` field so the HTTP surface
  cannot silently drop the map.
- RFC0002.16 honest bytes: the total is the group-column scan alone —
  zero row materialization, zero template-map acquisition (the RFC 0033
  acquisition was already lazy; the aggregation path never renders).

Invariants: hazard §4.6 (no DataFusion/arrow/SQL crosses the surface —
plain strings/ints only); §3.7 multi-tenancy (partition scope + the new
row-level tenant filter on the aggregation plan). Contract changes
sanctioned by the maintainer-merged amendment (#531): the `count` case
moves out of rfc0002_6_unsupported_stage_rejected (RFC0002.12 names the
lift), and rfc0005_14's error-precedence probe switches from `count` to
the still-rejected `render`.

Verified: cargo fmt --check; workspace clippy --all-targets
--all-features -D warnings; strict rustdoc (ourios-querier); full
cargo nextest run (1107 passed); .12/.13/.15/.16 force-run green,
.14 still ignored-failing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(querier): structured by-list rejects resource/attr paths + param u32::MAX bound

PR review findings #1 and #3. The structured JSON surface's group_term
accepted {resource|attr} field objects in a by-list, but the string DSL's
group_term = field production (§7 v1.1) is bare-field-only — so the
structured surface could express count/aggregate-by queries the string
grammar cannot, which then failed in planning instead of at validation.

RawGroupTerm::into_ir now rejects a {resource|attr} object with a clean
DslError, and the schema gains a bare_field $defs entry so schema
validation itself rejects the shape instead of only the runtime
converter. The schema's param integer also gets an explicit maximum
(u32::MAX) so an out-of-range param slot fails schema validation
cleanly rather than succeeding the schema and then failing Rust
deserialization.

Adds schema instance-list cases (resource/attr group term, param past
u32::MAX) and a structured.rs unit test for the runtime rejection.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(querier): reject count+limit; property-test the §6.3 planner invariants

PR review findings #2 and #4.

#2: `count [by …] | limit n` silently dropped the `limit` — execution
terminates in `Terminal::Aggregate`, which never consults `plan.limit`
(the aggregation map is the whole result; group-limiting semantics
aren't implemented). `validate()` now rejects the combination with a
clear QueryError::InvalidQuery instead of quietly returning the wrong
thing.

#4: pin detection (top-conjunctive `template_id == N`; `or`/`not`/
`resolves_to` don't pin), param-position duplication (at most one
`param(n)` per distinct n), and bucket constraints (positive width, at
most one `bucket(...)`) were covered only by hand-picked examples.
Adds a proptest generating arbitrary predicates and by-lists, checked
against an independently tracked ground truth (ground truth recorded
alongside generation, not derived from the code under test), covering
both `pinned_template_id` and `validate()`'s accept/reject decision.
The hand-picked examples stay as-is (CLAUDE.md §6.2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(querier): event_name in group-term generator; tenant + NULL-param regressions

PR review findings #5, #6, #7.

#5: the RFC0002.7 round-trip generator's `bare_field()` — shared by
`path_field()`, `group_term()`, and the `project` field list — omitted
`Field::EventName`, so grouped-query round-trips never covered
`count by event_name`. It's a valid bare field everywhere the real
grammar's `bare_field` (parse.rs) allows it, so fixed in place.

#6: `rfc0002_12_count_by_matches_naive_oracle`'s foreign-tenant fixture
is written via `write_all`, which partitions by the record's own
`tenant_id` — so the "b" row lands under tenant "b"'s own directory and
the row-level `tenant_id == tenant` backstop in `execute_aggregate`
(CLAUDE.md §3.7) is never exercised, only directory-level scoping. Adds
`rfc0002_12_aggregation_tenant_backstop_excludes_misplaced_row`, which
plants a tenant "b" row *inside* tenant "a"'s partition directory (the
shape a partitioning bug or on-disk corruption would produce — the
`ourios-parquet` writer's RFC 0005 §3.9 row-vs-path contract refuses a
mismatched tenant_id at write time, so the row is written honestly then
relocated) and asserts the backstop filter, not partitioning, keeps it
out of both the count and the group map. Manually verified this test
fails without the backstop filter, confirming it exercises the guard.

#7: RFC0002.15 covered a `params` list shorter than `n + 1`, but not the
distinct case of a list that HAS slot n whose own `value` decodes as
Parquet-level NULL (the field is nullable — RFC 0005 §3.2 — even though
`Param.value` is a non-`Option` Rust `String`, so only a raw/corrupted
writer can produce it). Adds
`rfc0002_15_present_but_null_param_slot_excluded_and_tallied`, built
with a raw arrow-array batch (mirroring `forward_compat.rs`'s
schema-drift fixtures) so the disposition is proven on the actual
`decode_aggregate` code path rather than assumed from the short-list
case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(querier): rfc 0002 — non_exhaustive QueryStats, typed group-null literal

QueryStats gains #[non_exhaustive] matching QueryResult's convention.
The rows_excluded doc comments now scope to any NULL group key, not
just param(n). The absent-OPTIONAL-column NULL substitute in the
aggregate group-term compiler now carries the field's real Arrow
type (Binary/Timestamp/FixedSizeBinary/Utf8) instead of always Utf8,
so the plan's output schema does not depend on which columns happen
to be present. Regression test covers grouping by an entirely-absent
FixedSizeBinary column (trace_id).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(server): rfc 0016 — skip the §7 default-limit injection for count-by queries

apply_limit ran unconditionally, but compile::validate now rejects
count+limit combined (RFC 0002 amendment 2026-07-15). Every
aggregation query sent to the HTTP endpoint was therefore a clean
400. Skip the injection when a Stage::Count is present. Regression
test confirmed via revert: fails without the fix, passes with it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(querier): rfc 0002 — execute_aggregate doc names the tenant backstop scan input

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(querier): rfc 0002 — checked_add for the excluded-row tally

Matches the existing pattern on rows: an overflow surfaces as an
error rather than silently wrapping in release builds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(querier): rfc 0002 — mark AggregateGroup non_exhaustive

Matches QueryResult/QueryStats' convention for public response types.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(querier): rfc 0002 — reject i64-overflowing bucket widths at validate time

bucket_expr's execution lowering casts the width to i64, but
validate_group_terms only checked positivity — a width between
i64::MAX and u64::MAX ns passed validation and failed later during
planning with a different error path. Moved into validate() for one
compile-time contract. Regression test confirmed via revert.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
jensholdgaard added a commit that referenced this pull request Jul 16, 2026
…line theory

Runs #4/#6/#7 all converged L4 to ~93-96% of expected rows, independent
of poll deadline (300s vs 900s made no measurable difference) — ruling
out both a results-cache echo (already disabled in #fe5915a) and a
"just needs more time" theory (the deadline widening from the prior
commit). A stable, time-independent shortfall points at something being
permanently excluded, not merely delayed.

Pulled the frozen otel-demo-v8 corpus locally and checked every log line
matching the L4 pair's needle ("Wrote producer snapshot at offset")
against its capture regex directly: all 11,525 matches parse cleanly.
The regex/content isn't the problem — some matching lines are never
being scanned at all.

That points at Loki's default -validation.max-entries-limit (5000):
count_over_time with a |regexp stage has to scan every raw kafka log
line in a query-frontend split before the line filter narrows it down,
and kafka's per-split volume exceeds 5000 lines often enough to
silently truncate the scan before every matching line is reached.
Raised the limit well past the corpus's noisiest single template's
volume (~971K rows).

Reverted the 900s deadline back to 300s (matching loki_measure_pair) —
the widened deadline never addressed the actual bottleneck, and keeping
it would misattribute the fix in a way that'd mislead the next reader.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
jensholdgaard added a commit that referenced this pull request Jul 16, 2026
…gone

Run #8 (max-entries-limit raised) moved L4 from a hard ~93% plateau to
97.1% (11192/11523) — real progress, and unlike runs #4/#6/#7 the
remaining gap now plausibly behaves like genuine ingest settle time
rather than a fixed ceiling, since the artificial cap that made the
prior 300s vs 900s test inconclusive is gone. Widened the deadline to
600s to test that directly before assuming a third factor is at play.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
jensholdgaard added a commit that referenced this pull request Jul 17, 2026
* feat(bench): rfc 0031 l4 — wire into the live dispatch loop

Live-wires PairClass::L4 into rfc0031_indicative_comparative_run, the
#[ignore]d container-based dispatch test. The previous slice proved
the L4 machinery (ourios_aggregate_answer, parse_loki_matrix,
pick_frequency_pair, compare_aggregations) only at the fixture level,
against a hand-built Loki matrix response — this slice makes it real
against a running Loki container and the actual corpus.

L4 is picked and measured as its own step, kept OUT of the
`Picks`/`specs: Vec<PairSpec>` pipeline the L1/L2/L3/L6 classes share:
an aggregation's (bucket, group) -> count map is not a LineKey
multiset, and forcing it through OuriosAnswer/compare_lines would
misrepresent the state rather than model it (the same "make invalid
states unrepresentable" reasoning the miner/parquet layers already
follow). Concretely: pick_frequency_pair runs post-store-build like
pick_template_pair; its PairSpec is built with the exact dsl/logql
shape the fixture-level test already pinned; loki_query_matrix issues
a real query_range metric call with `step` pinned to the bucket
width so evaluation instants land on parse_loki_matrix's documented
bucket-alignment convention (t = bucket_start + width);
loki_measure_frequency_pair polls it to completeness the same way
loki_measure_pair does for line-returning pairs. Both share the same
Loki container and corpus replay as the existing pairs.

Equivalence-required-but-bytes-unasserted: RFC0031.1 (result-set
equivalence) is never optional, so run_l4_pair asserts
compare_aggregations(...).is_equal() unconditionally — an L4 mismatch
fails the run exactly like every other class's equivalence check. Only
the bytes RATIO stays unasserted (M_L4 is still §7-DEFERRED, no frozen
margin to gate against yet): print_l4_report reuses
print_pair_bytes_gates, which already prints L4's ratio with no
verdict. L4 is measured, equivalence-checked, and reported LAST — after
the L1-L3/L6 evidence has printed and their frozen gates have already
asserted — so an L4-only failure cannot destroy that evidence (the same
run #11 salvage lesson the rest of the harness follows). A missing
candidate is reported loudly at pick time, never silently skipped.

Purely additive: class_pair_specs, build_pair_specs, frozen_gate_failures,
print_pair_bytes_gates, print_indicative_report, PairSpec, and PairClass
are unchanged — no frozen-gate behavior for L1/L2/L3/L6 is touched.

Verification: cargo fmt --all --check, cargo clippy --all-targets
--all-features -- -D warnings (workspace), cargo nextest run -p
ourios-bench (165 passed, 7 skipped) and cargo test -p ourios-bench
--all-features all green, including the untouched fixture-level
rfc0031_5_l4_frequency_aggregation_bytes. The corpus-scale dispatch
test itself needs Docker + OURIOS_COMPARATIVE_CORPUS, neither available
in this sandbox — its first live proof is the comparative-bench
dispatch workflow, same as every other slice in this harness's history.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — backtick-delimit the loki regexp argument

The dispatch's first-ever run failed: capture_regex's own Go RE2
escapes (\s+, \S+) were embedded inside a double-quoted LogQL string
literal, which tried to interpret those backslashes as its own escape
sequences (\s is not a valid one) and Loki rejected the query with
"invalid char escape" before the pattern reached the regex engine.
Fixed by switching to a backtick-delimited (LogQL/Go raw string)
regexp argument, which passes the pattern through literally.

Extracted the duplicated PairSpec-construction block (present
independently in the fixture test and the live-wiring loop) into one
shared l4_pair_spec helper, closing the drift risk and centralizing
the fix. Added a backtick guard: a capture_regex containing a
backtick (regex_escape does not escape backticks) would prematurely
close the raw string, so the candidate is now rejected loudly instead
of emitting a malformed query.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — measure L4 before, not after, the L1-L6 failure asserts

The second dispatch run failed on the pre-existing, documented L3
Loki-side flake (0 of 9 rows before timeout) — but the run never even
attempted L4: the failures.is_empty() assert for L1-L6's own salvaged
measurement failures sat textually BEFORE the L4 measurement/report
code, so any earlier pair's failure aborted the test before L4 was
ever reached. This inverted the design intent (an L4-only failure
should not destroy L1-L6 evidence, not the other way around).

Moved L4's measurement to run immediately after the report prints,
before the gate/failures assertions. run_l4_pair now pushes a Loki-side
measurement failure (flake) into the same failures vec the other
classes salvage into, instead of panicking immediately — so a flaky
L4 measurement no longer aborts before the L1-L6 evidence is captured,
symmetric with the fix for the reverse direction. A genuine L4
equivalence MISMATCH still hard-panics immediately, unchanged:
RFC0031.1 equivalence is never optional, matching L1-L6's own
compare_lines assertion.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — cap the picker's row-count ceiling at 100K

The third dispatch got past the control-flow fix and genuinely
measured L4 — but the picked candidate (a service's dominant,
near-catch-all template) summed to ~971K matching rows, and Loki
returned only 811,775 of them before the 300s poll deadline (the
same budget every other class's loki_measure_pair uses). L4_MIN_ROWS
was a floor with no ceiling, so the picker had no reason to prefer a
smaller, still-meaningful candidate. Added L4_MAX_ROWS=100_000
(comfortable margin at the observed ~2.7K rows/s Loki throughput) to
frequency_shape_rejection, so the picker moves on to a candidate the
poll can actually finish measuring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — disable loki's query-range results cache

Run #4's L4 pair plateaued at 11,053/11,523 rows across every 10s poll
instead of climbing to completeness. The picker's row ceiling (run #3's
fix) had already ruled out "too large to finish in time" — the count
never moved at all, which points at a cache serving the same stale
answer on every retry rather than a slow ingest.

Loki's bundled local-config.yaml enables the embedded results cache for
query_range's metric/matrix path (L4's loki_query_matrix), keyed by the
query+start+end+step tuple that loki_measure_frequency_pair repolls
unchanged. The first (still-incomplete) response gets cached and echoed
back on every subsequent poll. Plain log queries (loki_query_range, used
by L1-L3/L6) aren't extent-cached the same way, so they self-heal across
polls untouched by this.

-query-range.cache-results=false trades Loki's own query latency for
correctness of the harness's completeness poll — in Loki's favour, same
as the other operator-tuning flags already on this container.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — correct the results-cache disable flag name

Run #5 never got past container startup: `-query-range.cache-results`
doesn't exist ("flag provided but not defined"), so Loki's /ready check
timed out on a container that failed to start at all.

Checked the pinned v3.5.3 source directly instead of guessing again:
queryrangebase.Config.CacheResults is registered under the `querier.`
flag prefix in roundtrip.go, not `query-range.`. Correct flag is
-querier.cache-results=false.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — widen the loki poll deadline to 900s

Run #6 (with the corrected -querier.cache-results=false flag from the
prior commit) proved the results-cache theory wrong: L1-L3/L6 all
measured cleanly, but L4 still plateaued — 10752/11523 rows (93.3%),
even slightly worse than run #4's 95.9% pre-fix, and the shortfall
varies run to run rather than repeating a fixed cached answer.

That points at genuine, variable completion time rather than a bug:
L4's LogQL runs a `| regexp` capture over every candidate line before
grouping and counting, a real per-line cost the other classes' plain
stream/count queries never pay. Widened loki_measure_frequency_pair's
deadline from 300s to 900s — well inside the CI job's unset (360 min
default) timeout given the whole run has taken ~95-100 min so far.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — raise loki's max-entries-limit, revert deadline theory

Runs #4/#6/#7 all converged L4 to ~93-96% of expected rows, independent
of poll deadline (300s vs 900s made no measurable difference) — ruling
out both a results-cache echo (already disabled in #fe5915a) and a
"just needs more time" theory (the deadline widening from the prior
commit). A stable, time-independent shortfall points at something being
permanently excluded, not merely delayed.

Pulled the frozen otel-demo-v8 corpus locally and checked every log line
matching the L4 pair's needle ("Wrote producer snapshot at offset")
against its capture regex directly: all 11,525 matches parse cleanly.
The regex/content isn't the problem — some matching lines are never
being scanned at all.

That points at Loki's default -validation.max-entries-limit (5000):
count_over_time with a |regexp stage has to scan every raw kafka log
line in a query-frontend split before the line filter narrows it down,
and kafka's per-split volume exceeds 5000 lines often enough to
silently truncate the scan before every matching line is reached.
Raised the limit well past the corpus's noisiest single template's
volume (~971K rows).

Reverted the 900s deadline back to 300s (matching loki_measure_pair) —
the widened deadline never addressed the actual bottleneck, and keeping
it would misattribute the fix in a way that'd mislead the next reader.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — widen poll deadline now the entries cap is gone

Run #8 (max-entries-limit raised) moved L4 from a hard ~93% plateau to
97.1% (11192/11523) — real progress, and unlike runs #4/#6/#7 the
remaining gap now plausibly behaves like genuine ingest settle time
rather than a fixed ceiling, since the artificial cap that made the
prior 300s vs 900s test inconclusive is gone. Widened the deadline to
600s to test that directly before assuming a third factor is at play.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — revert unhelpful deadline widening, add diagnostics

Run #9 (600s) measured 96.5% (11123/11523), statistically the same as
run #8's 97.1% at 300s — deadline widening does nothing here, so the
remaining shortfall after the entries-limit fix is a second stable
cap, not settle time. Reverted the deadline back to 300s to match
loki_measure_pair rather than keep an unjustified change.

Wired the existing dump_loki_diagnostics helper (already used by
loki_measure_pair on a deadline miss) into loki_measure_frequency_pair
too — it's built around spec.logql + stats parsing, which is
query-shape-agnostic, so it works unmodified for the matrix path. If
L4 still falls short, the next run's failure carries the raw Loki
stats (chunk-fetch counts, any warnings) instead of another guess.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — epoch-align the loki query window to bucket boundaries

Run #10's diagnostics (wired in the prior commit) confirmed the L4
query itself is well-formed and Loki answers it successfully — no
error, no chunk-fetch shortfall visible in the sampled response. That,
combined with runs #6-#10 all converging to a stable ~93-97% regardless
of poll deadline (300s/600s/900s all statistically indistinguishable),
rules out both a timing race and a malformed query.

The real mismatch: Loki's query_range evaluates a step-grid starting
exactly at `start` (start, start+step, ..., end), but Ourios's own
bucket(width) semantics are epoch-aligned (floor(ts/width)*width) —
`min_effective_time_unix_nano` (the corpus's raw earliest timestamp)
has no reason to already be a bucket-boundary multiple. Unless
(end - start) is an exact multiple of the bucket width, the step-grid
leaves a fractional sliver at the tail of the range with no evaluated
window covering it at all — real, ingested, settled data that's simply
never queried, independent of poll duration. That's exactly the shape
every run has shown.

Snap `start` down and `end` up to the nearest bucket-width boundary in
l4_pair_spec — costs nothing (no data exists outside [min, max] to
inflate the count) and guarantees the step-grid fully covers the range.

Verified: cargo fmt --all --check, workspace cargo clippy -D warnings,
local (non-container) rfc0031_comparative unit tests — 39 passed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* diag(bench): rfc 0031 l4 — add an ingest-vs-query-side split probe

Run #11 (bucket-aligned query window) measured 96.6% (11133/11523) —
narrower than the pre-fix ~93% plateau, but still in the same stable
band as runs #6-#10, all independent of poll deadline, entries-limit,
and now bucket alignment. Six straight dispatches without closing the
gap means continuing to guess at query-side LogQL/config knobs isn't
warranted anymore.

Added a decisive probe: on a deadline miss, loki_measure_frequency_pair
now also runs a PLAIN line-filter count (no count_over_time, no
regexp) for the same needle + window via the new
loki_query_range_uncapped (limit sized to expected_rows, unlike the
shared loki_query_range's fixed 5000 cap — which is below this pair's
11523 expected rows and would itself lie about the count). If that
plain count also falls short by the same margin, the shortfall is
ingest-side (Loki never stored those lines) and no further query
tuning will fix it; if it's ~complete, the loss is specific to the
aggregation path.

Diagnostic-only change — no behavior change to the measurement itself,
just evidence gathering on the existing failure path.

Verified: cargo fmt --all --check, workspace cargo clippy -D warnings,
local (non-container) rfc0031_comparative unit tests — 39 passed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — reject high-frequency candidates prone to loki dedup

Run #12's decisive diagnostic confirmed the L4 shortfall is ingest-side:
a plain unaggregated line-filter count for the same needle+window came
back just as short (11160/11523) as every aggregation-path attempt.
Loki's ingester silently drops a log entry that collides with another
on (timestamp, body) within the same stream — a drop invisible to the
OTLP push response's partial_success (push_otlp already asserts that
field is clean on every push in every run so far). No query-side fix
was ever going to close this gap; the picker was choosing a candidate
Loki structurally can't ingest identically.

kafka's template_id=16 ("Wrote producer snapshot") fires roughly every
15s. A local exploration against the real frozen corpus (offline, no
Loki container — pick_frequency_pair only touches Ourios's own
pipeline) found candidates at much lower frequency clear of the same
floors: template_id=60 ("Periodic task") at ~144s average cadence, ~10x
the failing candidate's margin.

Added L4_MIN_AVG_INTERVAL_SECONDS (100s) to frequency_shape_rejection
as a durable picker rule, not a one-off override — this protects any
future re-run of the picker against landing on another collision-prone
high-frequency template, not just this specific dispatch.

Updated two pre-existing tests (pick_frequency_pair_finds_a_moderate_
cardinality_group, rfc0031_5_l4_frequency_aggregation_bytes) whose
synthetic sub-3s timelines — convenient for test speed, not meant to
model real timing risk — tripped the new floor; scaled their
timestamps 1000x (preserving cardinality/row-count/needle assertions
unchanged) so they represent a realistic, non-collision-prone example.

Verified: cargo fmt --all --check, workspace cargo clippy -D warnings,
local (non-container) rfc0031_comparative unit tests — 40 passed
(39 prior + 1 new: frequency_shape_rejection_enforces_the_average_
interval_floor).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* diag(bench): rfc 0031 l4 — dump loki's own container logs on a deadline miss

Run #13's lower-frequency candidate (template_id=60, ~144s average
cadence) still fell short (1144/1197), and a corpus-side check ruled
out the leading theory entirely: every one of the 1197 matching
records has a UNIQUE timestamp AND a unique body (verified via jq
against the frozen otel-demo-v8 corpus locally) — zero exact
(timestamp, body) collisions possible. Loki's documented dedup rule
cannot be the mechanism here, which means it likely wasn't the full
story for the prior candidate either, even though lowering the
frequency floor did measurably help (17.5% loss -> 4.4% loss).

Also checked push_corpus_to_loki/push_otlp end to end for a harness-
side drop: the batching loop appends every non-empty corpus line's
resource_logs to `pending` before any flush, with a final flush after
the read loop — no line is skippable, and push_otlp's retry resends
the identical Bytes payload, so no bug found there either.

Everything checkable from the client side (query responses, corpus
content, our own push code) is now ruled out or confirmed clean. The
next place to look is Loki itself: on a deadline miss,
loki_measure_frequency_pair now also dumps the Loki container's own
stderr, filtered to warn/error/drop/reject/rate-limit/stream-limit
lines — the ingester logs these for exactly the mechanisms still on
the table (rate limiting, out-of-order rejection, stream-limit drops),
none of which are visible in a query response or push_otlp's already-
clean partial_success check.

Diagnostic-only — no change to measurement behavior.

Verified: cargo fmt --all --check, workspace cargo clippy -D warnings,
local (non-container) rfc0031_comparative unit tests — 40 passed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* diag(bench): rfc 0031 l4 — scrape loki's discarded-samples metrics

Run #14's level=warn/level=error stderr scan came back with zero
matches in 6618 total lines — whatever is causing the L4 shortfall
(still 1147/1197 with the lower-frequency candidate), Loki doesn't
consider it log-worthy. That rules out rate limiting, out-of-order
rejection, and stream-limit drops as commonly logged at WARN.

(The first attempt at the stderr filter was a naive "contains warn"
substring match, which drowned in false positives from query text
like `severity_text="WARN"` appearing inside level=info lines — fixed
to match on the `level=` field precisely.)

Loki's distributor increments loki_discarded_samples_total/
loki_discarded_bytes_total (labeled by reason) even for discards that
don't warrant a log line — its own dedicated counter for exactly this
question. Added dump_loki_discard_metrics, scraping /metrics on a
deadline miss (extracted as its own function, alongside
dump_loki_diagnostics, to keep loki_measure_frequency_pair under
clippy's line-count lint).

Diagnostic-only — no change to measurement behavior.

Verified: cargo fmt --all --check, workspace cargo clippy -D warnings,
local (non-container) rfc0031_comparative unit tests — 40 passed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* feat(bench): rfc 0031 l4 — documented completeness margin (§7, 2026-07-17)

Sixteen dispatches (runs #1-#16) exhausted every mechanism checkable
from the harness's side without ever reaching exact L4 completeness.
Runs #13-#16, specifically, ruled out: query-shape artifacts (a plain
line-filter count matched the aggregation-path shortfall exactly),
Loki's documented same-(timestamp,body) dedup (zero exact collisions
found via direct corpus analysis), interleaving between a genuine
mid-corpus kafka restart's two service-instance periods (cleanly
sequential), a harness-side push/batching bug (read end to end, none
found), anything Loki logs at WARN/ERROR (zero matches bar one
harmless startup transient), and Loki's own discarded-samples
Prometheus accounting (zero discards of any kind, any reason).

This matches an open, unresolved upstream Loki issue
(grafana/loki#10658 and related): wide-time-range queries silently
missing a small, consistent percentage of lines, with no error, no
discard signal, and no maintainer-identified root cause. It's a
documented, external, currently-unfixable characteristic of the
comparison partner, not an Ourios or harness defect.

Adds L4_COMPLETENESS_MARGIN = 0.90 (real headroom over the observed
3.9-4.4% loss band) and compare_aggregations_within_margin — narrowly
scoped: it still hard-fails, at any margin, on Loki reporting MORE
than Ourios for any cell or a cell absent from Ourios's own answer,
the two signals that would actually indicate a correctness bug. Only
aggregate under-counting up to the margin is tolerated.
compare_aggregations (exact) is untouched and still gates the
RFC0031.5 fixture-level test's synthetic Loki answer.

Wires the margin into both loki_measure_frequency_pair's poll-complete
threshold (accept short-of-exact within margin instead of always
retrying to a hard timeout) and run_l4_pair's equivalence assertion.

RFC 0031 amended: RFC0031.1's L4 clause now states the margin
explicitly, and §7's L4-query-shape entry (previously open) is closed
with the full evidence trail and the margin decision. M_L4 (bytes-read)
stays deferred — this unblocks a measurement, it doesn't freeze that
margin.

Verified: cargo fmt --all --check, workspace cargo clippy -D warnings,
ourios-bench lib tests (32 passed, comparative module) + local
rfc0031_comparative integration tests (40 passed), mdbook build.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — margin check is total-level, not per-cell

Run #17 validated the completeness-margin design and immediately
refined it: the poll-completion check passed cleanly (1153/1197,
96.3%), but the equivalence check then hard-failed on a single cell
landing 1 row OVER Ourios's count (114 vs 113 for one bucket/value)
while the aggregate total stayed a solid under-count — consistent
with the same step-grid boundary imprecision already characterized
(a record landing in an adjacent bucket), not fabrication.

The original compare_aggregations_within_margin checked "Loki > Ourios"
per cell, which was too strict for that kind of noise. Refined to
check for phantom cells (a (bucket, group_key) Loki reports that
Ourios's own answer doesn't contain at all) and Loki's TOTAL exceeding
Ourios's total instead — this still catches the failure mode that
would actually indicate a bug (wrong regex or wrong bucket math would
produce cells Ourios never produced at all, or push the total over)
while tolerating single-cell boundary noise on keys both systems agree
exist.

Added a test for the exact run #17 shape (a single cell over, total
still under, must pass) alongside the existing phantom-cell and
net-overcount tests (renamed from "overcount" now that the check is
total-level). RFC 0031 §7's L4 entry updated with the refinement and
its rationale.

Verified: cargo fmt --all --check, workspace cargo clippy -D warnings,
ourios-bench lib tests (33 passed, +1 new) + local rfc0031_comparative
integration tests (40 passed), mdbook build.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — per-group_key margin, address PR #536 review

PR #536's code review (14 fresh findings across Copilot + CodeRabbit's
post-run-18 passes) surfaced one substantive correctness gap and
several real documentation/robustness issues in the completeness-
margin work. All verified against current code before fixing.

Substantive fix — cross-key redistribution gap (CodeRabbit, Major):
compare_aggregations_within_margin's grand-total-only check (from the
run #17 fix) let Loki over-count one group_key while under-counting
another by the same amount and still read as 100% complete: Ourios
{A: 100, B: 100} vs Loki {A: 190, B: 10} sums to a "complete" 200/200
while hiding A being fabricated to compensate for B being nearly
lost. Refined to aggregate ourios/loki BY group_key first (summing
each key across every bucket it appears in), then apply the
phantom/overcount/margin checks per-key. This still tolerates run
#17's exact shape (a single bucket's +1 doesn't change a key's own
total across its buckets) while rejecting the redistribution a pure
grand-total check missed. Added regression tests for both shapes.

Also populates real per-key examples in mismatch reports (Copilot: the
old design returned examples: Vec::new() on both mismatch paths,
despite the function accepting examples_cap and RFC0031.1 calling for
example keys on a failed comparison).

Documentation/robustness fixes (all verified against current code,
none required a runtime-behavior change beyond the fix above):
- Two stale comments still asserted Loki's same-(timestamp, body)
  ingester dedup as the shortfall's mechanism, contradicting the
  nearby docs that say this was directly disproven and the true
  mechanism is uncharacterized (Copilot, 6 threads pointing at 2 real
  sites: the frequency_shape_rejection rejection message and one test
  comment — the other 4 threads were already-accurate historical
  narrative, verified and left alone).
- Missing `//` justification comment on one #[allow(cast_precision_loss)]
  (CodeRabbit).
- L4 picker silently continues when no viable candidate exists
  (l4_spec.is_none()) — only an eprintln, no run failure, despite L4
  being a must-win class (CodeRabbit). Now pushes into `failures`.
- PR description inaccurately described L4's equivalence assertion
  (exact compare_aggregations, ordered after the frozen gates) —
  rewritten to match actual behavior (margin-based, before the frozen
  gates, matching the run #11 salvage design already documented inline).
- RFC 0031 §7's L4 entry claimed the picker "prefers the lowest-
  frequency viable candidate" — the actual algorithm is first-fit in
  ascending (template_id, param) order, not an exhaustive ranking
  (CodeRabbit, tagged Heavy Lift). Reworded to describe actual
  behavior and the deliberate scope decision (a real ranking pass
  would cost a query per candidate against a corpus with tens of
  thousands of templates; not built given first-fit has now found a
  validated candidate three real dispatches running).
- Double-backtick delimiters for a LogQL code span containing literal
  backticks (CodeRabbit, MD038).

Verified: cargo fmt --all --check, workspace cargo clippy -D warnings,
ourios-bench lib tests (34 passed, +2 new) + local rfc0031_comparative
integration tests (40 passed), mdbook build.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — absolute row tolerance, not pure percentage

Run #19 (the verification dispatch for the round-1 review fixes) found
a real edge case in the per-group_key percentage margin: a group_key
with exactly 1 total Ourios row, where Loki captured 0 (0%). A pure
ratio has no meaningful middle ground at n=1 — it's binary, 0% or
100% — yet losing one isolated occurrence is fully consistent with
the already-characterized ~4-8% aggregate loss rate this whole margin
exists to tolerate.

Converted the per-key check from a ratio (loki/ourios >= margin) to an
absolute row tolerance floored at 1: ceil(ourios_key_total *
(1 - margin)).max(1). This tolerates a cardinality-1 key losing its
only row while still catching a real shortfall on a large key (100
rows, tolerance 10, losing 20 still rejects) — the phantom-cell and
per-key-overcount hard-fail checks are unaffected. Two new regression
tests cover both ends.

Verified: cargo fmt --all --check, workspace cargo clippy -D warnings,
ourios-bench lib tests (36 passed, +2 new) + local rfc0031_comparative
integration tests (40 passed), mdbook build.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — margin comparator precision, review triage

compare_aggregations_within_margin's tolerance formula (ceil(o*(1-margin))
.max(1)) was itself miscalibrated for small-but-not-1 totals, per two
independent Copilot review threads (o=2 at 90%: tolerance=1 permits 50%
completeness, not 90%). Replace the subtract-then-round row tolerance
with a direct, epsilon-guarded comparison — loki_total >= ourios_total *
margin — which also sidesteps a second bug the naive floor() fix
introduced: 1.0 - 0.9 isn't exactly 0.1 in f64, so floor(40.0 * (1.0 -
0.9)) truncated to 3 instead of 4, tightening the tolerance at exact
90%-boundary cases (caught by the existing
margin_comparison_tolerates_undercount_within_margin test's svcB case).

Extract phantom_cells and aggregate_by_group_key helpers to bring the
function back under clippy's line limit, and add a # Panics section for
the margin-validation assert.

Fix several accumulated PR #536 review findings: run_l4_pair's doc
comment claimed L4 runs after the L1-L3/L6 frozen gates assert (it
actually runs before, printing first); three "ingest-vs-query" overclaims
(the plain line-filter probe still calls query_range, so it can rule out
"specific to the metric-aggregation path" but not prove ingest-side
loss); L4_COMPLETENESS_MARGIN's own doc comment still described the
superseded total-level design. Add a clarifying comment on l4_pair_spec's
step-grid reasoning (the first evaluated instant decodes to an empty
phantom bucket, not a lost real one) and fix the RFC's LogQL code span,
which kept the backslash escaping needed for single backticks even after
switching to a double-backtick delimiter that makes it unnecessary.
Reconcile RFC0031.5's must-win predicate with M_L4 staying deferred —
add a note that the predicate is the target contract, not currently
gated.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — panic-safe diagnostic probe, more review triage

loki_query_range_uncapped used expect()/assert!() internally, but it
runs on the L4 deadline-miss diagnostic path inside the same
runtime.block_on that gathers L1-L3/L6's evidence — a panic there
(a real Loki error response, a malformed body) would unwind the whole
async block and lose all of it, defeating the print-before-assert
salvage design (Copilot). Converted to return Result<u64, String>
instead of panicking, matching the already-panic-free sibling
diagnostics (dump_loki_diagnostics et al.).

Also: fix a test comment that said "one row under the ceiling" for a
fixture that actually lands exactly at the ceiling; fix an unreachable!
message's imprecise invariant claim (the real gating condition is
l4_spec.is_some() implies l4_loki.is_some(), not "iff frequency is
Some"); document loki_query_matrix's whole-second/bucket-alignment
precondition and verify it against l4_pair_spec, its only caller;
reorder loki_measure_frequency_pair's deadline-miss diagnostics to run
only when the completeness margin is actually missed, not on every
deadline-miss regardless of outcome; fix two fixture comments claiming
"~300s average spacing" that don't match their own timestamps (actually
~580s) and a comment attributing the L4 shortfall to ingest-side dedup
after that theory was directly disproven elsewhere in the same file.

Verified the remaining ~40 accumulated review threads (mostly a
recurring "shared 300s deadline" doc/code mismatch and the run_l4_pair
ordering claim, duplicated across many review rounds) against current
code: all already correct, superseded by earlier commits in this
investigation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — round-4 review triage on the margin comparator

Validate margin at function entry rather than after the phantom/overcount
checks, so an invalid margin always panics per the documented contract
instead of potentially returning a data-shaped mismatch first
(CodeRabbit). Fix the doc comment paragraph still describing the
superseded floor-based tolerance (Copilot). Reword the L4-skip
diagnostic and failure message to name both reasons l4_spec can be None
(picker bounds vs a backtick in the capture regex) and to make clear the
skip fails the dispatch rather than reading as benign (Copilot, two
sites).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* fix(bench): rfc 0031 l4 — margin=1.0 strictness + panic-safe matrix poll

Two Copilot findings on the previous commit, both verified genuine:

The cardinality-1 exemption applied at any margin, so a caller passing
margin = 1.0 (exact completeness) would still accept Loki returning 0 of
1 for an n=1 key — the exemption now only applies to a genuinely
fractional margin, with a regression test covering both directions at
1.0. Bit-identical behavior at the harness's 0.90.

loki_query_matrix still used expect/assert internally, so a transient
transport error, 5xx, or torn body during the L4 poll — which runs LAST
in the same async block holding every other pair's already-collected
measurement — would panic and unwind all of it. Converted to
Result<L4Measured, String>; the poll loop now retries an Err until its
deadline exactly like an incomplete answer, then surfaces it as the
pair's failure. Extracted the below-margin shortfall diagnostics into
dump_l4_shortfall_diagnostics to stay under clippy's function-length
limit.

Neither change alters the measured semantics run #23 is currently
confirming (the comparator formula is untouched; at margin 0.90 the
exemption gating is unchanged).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 l4 — document why the phantom check is cell-level

Copilot's latest pass proposed weakening phantom detection from
(bucket, group_key) cells to bare group_keys so a boundary-exact record
shifting into an empty adjacent bucket can't read as phantom. Declined:
a systematic bucket-decode error (every cell shifted one width — the
run #11 bug class) leaves every per-key total intact, so the cell-level
check is the only guard that catches it, while the false positive it
risks requires a nanosecond-exact bucket-boundary timestamp that no
real dispatch has ever produced. Documented the trade-off on
phantom_cells instead of changing behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants