jensholdgaard · jensholdgaard · Jul 22, 2026 · Jul 22, 2026
@@ -2168,24 +2168,28 @@ pruning-granularity-over-bytes trade RFC 0036 §3.3 exists to make, now
 measured on realistic data: it is a good trade, and it gets *better* the
 finer the threshold, bounded only by footer/index overhead.
 
-**Finding 3 — 32 and 64 MiB are indistinguishable because arrow's default
-row-group *row* cap dominates, not the byte threshold.** The compacted
-writer sets no `max_row_group_size`, so parquet-rs's default **1,048,576-row**
-cap applies. On 2,160,000 rows that forces ~3 groups of ~1 M rows each
-*regardless* of a byte threshold ≳ 16 MiB: at 32 and 64 MiB the row cap
-trips before the byte threshold does (byte-identical 3-group files), and
-only 16 MiB (~590 k rows/group) is genuinely byte-capped. So above ~16 MiB
-the compacted **byte** threshold does not actually govern granularity on a
-high-row corpus — the row cap does, silently. This is a real gap versus
-RFC 0036 §3.3's byte-driven-rotation premise (pruning still works — the
-RFC0036.2 gate passes — just at coarser granularity than the byte threshold
-implies). **Actionable follow-up:** to let the byte threshold control
-rotation (and realise the finer pruning at 32 MiB, not only at 16 MiB),
-raise or remove the row cap (`WriterProperties::set_max_row_group_size`) on
-the compacted writer. Separating 32 from 64 MiB by *bytes* alone then needs
-the baseline corpus's volume too. What this run establishes regardless:
-**finer effective groups win the materialization trade at a sub-1% disk
-cost.**
+**Finding 3 — 32 and 64 MiB are near-identical because arrow's default
+row-group *row* cap sets a ~30 MiB granularity floor, finer than either
+byte threshold.** The compacted writer sets no `max_row_group_size`, so
+parquet-rs's default **1,048,576-row** cap applies. At this corpus's ~30
+encoded bytes/row that cap fills a group at **~30 MiB**, which trips
+*before* the 32 MiB byte flush (and well before 64 MiB): so 32 and 64 MiB
+are both **row-capped** at ~3 groups of ~30 MiB (byte-identical files), and
+only 16 MiB (~590 k rows → ~15 MiB, under the cap) is genuinely
+byte-governed → 5 finer groups.
+
+**Correction to an earlier framing.** A prior draft of this finding called
+the row cap a "gap" and suggested raising `max_row_group_size` "so the byte
+threshold bites." That is backwards: raising the cap would let the 32/64 MiB
+byte flush govern and produce *fewer, coarser* groups (~2 of ~32 MiB) — the
+**wrong** direction for pruning. The row cap is a granularity *floor* that
+is currently *helping*. The lever that unambiguously improves window
+pruning is a **smaller byte threshold** (16 MiB halves window
+materialization here), the §7 authoritative-sweep question — not a larger
+row cap. Making group sizing byte-*uniform* (raising the cap) is a separate
+predictability choice, not a pruning win. Net: **finer effective groups win
+the materialization trade at sub-1% disk cost, and the byte-threshold
+*value* — not the row cap — is the lever.**
 
 **Conclusion.** 32 MiB stays a reasonable, defensible shipped default (it
 already delivers the pruning mechanism the RFC0036.2 gate enforces). The

@@ -1,7 +1,7 @@
 ---
 rfc: 0031
 title: Comparative evaluation against Grafana Loki
-status: validated
+status: accepted
 author: Jens Holdgaard Pedersen <jens@holdgaard.org>
 drafting-assistance: Claude
 created: 2026-07-11
@@ -11,7 +11,8 @@ superseded-by: —
 
 # RFC 0031 — Comparative evaluation against Grafana Loki
 
-> **Status note.** **`validated`** (2026-07-21). The §7 gate set is
+> **Status note.** **`accepted`** (2026-07-22, maintainer sign-off — the
+> terminal state). Reached `validated` 2026-07-21: the §7 gate set is
 > frozen and CI-enforcing (`green` via the asserting ci-runner series,
 > §9.13–§9.18), and the first fully authoritative comparative run on
 > `baseline-8vcpu-32gib` (§9.24, PR #583) passed all 11 frozen gate

@@ -1,7 +1,7 @@
 ---
 rfc: 0034
 title: D1 re-scope — ingest throughput is a per-node capacity on baseline hardware, not a per-core rate
-status: specified
+status: accepted
 author: Jens Holdgaard Pedersen <jens@holdgaard.org>
 drafting-assistance: Claude
 created: 2026-07-20
@@ -11,9 +11,10 @@ superseded-by: —
 
 # RFC 0034 — D1 re-scope
 
-> **Status note.** **`specified`**, with all three §5 criteria now in
-> force (2026-07-21); the terminal flip to `accepted` is the
-> maintainer's sign-off, not this note. RFC0034.1 is **enacted** by
+> **Status note.** **`accepted`** (2026-07-22, maintainer sign-off — the
+> terminal state for this re-scope RFC, which has no thesis-gate of its
+> own to `validate`). All three §5 criteria were in force from
+> 2026-07-21. RFC0034.1 is **enacted** by
 > this change: the `docs/benchmarks.md` § D1 block is recast per-node
 > (≥ 100 000 lines/s on `baseline-8vcpu-32gib`, multi-tenant, shared
 > commit stream, p99 ack ≤ 200 ms at the sustained rate) with the old

@@ -1,7 +1,7 @@
 ---
 rfc: 0036
 title: Write-side layout — compacted-partition clustering and row-group sizing
-status: validated
+status: accepted
 author: Jens Holdgaard Pedersen <jens@holdgaard.org>
 drafting-assistance: Claude
 created: 2026-07-21
@@ -11,9 +11,11 @@ superseded-by: —
 
 # RFC 0036 — Write-side layout
 
-> **Status note.** **`validated`** (2026-07-22) — all five §5 scenarios
-> green, plus the comparative evidence below. `accepted` is a maintainer
-> flip. **RFC0036.1** (footer inspection — compacted threshold,
+> **Status note.** **`accepted`** (2026-07-22, maintainer sign-off — the
+> terminal state). Reached `validated` the same day on all five §5
+> scenarios green plus the comparative evidence below; the §7 threshold
+> sweep (§9.28) and its row-cap finding landed with it.
+> **RFC0036.1** (footer inspection — compacted threshold,
 > `sorting_columns`, per-group service min/max — plus the §6 merge
 > property), **RFC0036.3** (D3 file-band + forced-spill memory bound),
 > **RFC0036.4** (shuffled-listing byte-identity rebuild), and
@@ -538,13 +540,14 @@ Mapped to `CLAUDE.md` §6.2; techniques per §5 scenario id:
   one-service window materialises **half** the bytes at 16 MiB (14.66 MiB)
   vs 32/64 MiB (28.94 MiB), same answer, for that +0.80% disk cost — the
   pruning-granularity trade is *good* on compressible data and improves as
-  T shrinks; (3) 32 and 64 MiB are indistinguishable because arrow's default
-  1,048,576-row group cap (the compacted writer sets no
-  `max_row_group_size`) trips before either byte threshold on the 2.16 M-row
-  corpus — so above ~16 MiB the **byte** threshold does not govern
-  granularity, the row cap does (a gap vs §3.3's byte-driven premise;
-  actionable follow-up: `set_max_row_group_size` on the compacted writer so
-  the byte threshold bites). **Disposition:** 32 MiB stands as the
+  T shrinks; (3) 32 and 64 MiB are near-identical because arrow's default
+  1,048,576-row group cap (no `max_row_group_size` is set) fills a group at
+  ~30 MiB on this ~30 B/row corpus and so trips before either byte
+  threshold — a granularity *floor* finer than 32/64 MiB. It is **not** a
+  bug to "fix" by raising the cap: doing so would let 32/64 MiB coarsen (~2
+  groups), the wrong way for pruning. The pruning lever is a *smaller byte
+  threshold* (16 MiB, byte-governed → finer groups), the §7 sweep question —
+  not the row cap. **Disposition:** 32 MiB stands as the
   shipped default (it already delivers the mechanism the RFC0036.2 gate
   enforces); the data leans toward *smaller* thresholds, so **16 MiB is
   flagged as a candidate for the authoritative sweep and a maintainer

@@ -168,12 +168,12 @@ captured by B1/B2 (see `benchmarks.md` §2 / §7).
 | 0028 | Build-feedback program (test-harness + workspace decomposition) | `green` |
 | 0029 | OIDC bearer layer (issuer-agnostic, Dex-validated) | `green` |
 | 0030 | TLS/mTLS on the data-plane listeners | `green` |
-| 0031 | Comparative evaluation against Grafana Loki | **`validated`** — all four must-win classes measured, §7 gates frozen and CI-enforcing; the first fully authoritative run (`benchmarks.md` §9.24, `baseline-8vcpu-32gib`) passed all 11 frozen gate decisions; losses published per §5 (L6 *storage* is a recorded diagnostic, not a win) |
+| 0031 | Comparative evaluation against Grafana Loki | **`accepted`** (2026-07-22) — all four must-win classes measured, §7 gates frozen and CI-enforcing; the first fully authoritative run (`benchmarks.md` §9.24, `baseline-8vcpu-32gib`) passed all 11 frozen gate decisions; losses published per §5 (L6 *storage* is a recorded diagnostic, not a win) |
 | 0032 | Query-schema and cost-model resource for the MCP surface | `green` |
 | 0033 | Cached template-map artifact | `green` |
-| 0034 | D1 re-scope: per-node ingest-throughput bar | `specified` — enacted: RFC0034.1–.3 satisfied by the §9.20–§9.23 measurement series; `accepted` is a maintainer flip |
+| 0034 | D1 re-scope: per-node ingest-throughput bar | **`accepted`** (2026-07-22) — enacted: RFC0034.1–.3 satisfied by the §9.20–§9.23 measurement series (a re-scope RFC with no thesis-gate of its own; `specified`→`accepted`) |
 | 0035 | Ingest concurrency (ordered mining, concurrent encode/publish) | `green` — §9.22 A/B plus the §9.23 asserting soak; the #578 sweep-publish durability window closed alongside |
-| 0036 | Write-side layout (compaction-time service/time sort) | `specified` — awaiting maintainer design review before `red` |
+| 0036 | Write-side layout (compaction-time service/time sort) | **`accepted`** (2026-07-22) — all five §5 green (real compaction); RFC0036.2 scanned-count gate + in-repo before/after (§9.27, 1.43×); baseline no-regression (§9.26) + §7 threshold sweep (§9.28); the comparative harness single-file limit + the row-cap interaction are documented follow-ups |
 
 **Crates — all ten product crates are implemented** (`ourios-core`,
 `-miner`, `-wal`, `-parquet`, `-ingester`, `-querier`, `-server`,