From fdd6999024b500c38072de2d0ea6cdd5c7b115e7 Mon Sep 17 00:00:00 2001 From: Jens Holdgaard Pedersen Date: Fri, 10 Jul 2026 09:40:11 +0200 Subject: [PATCH 1/7] =?UTF-8?q?docs(bench):=20reconcile=20=C2=A79.12=20wit?= =?UTF-8?q?h=20the=20resolved=20#444=20decision?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The C2 gate landed per-service in #451 (option 3), but §9.12 still framed #444 as an open three-way fork ("tokenizer masking vs. length-aware thresholding vs. accept-and-scope — an RFC-level pillar #2 decision") and called it "a decision, not a performance one". Update both spots to the resolution: option 3 shipped (per-service gate, RFC 0006 §3.4.3), masking parked as a future strategic RFC (a Collector transform/redaction processor can polish infra tokens upstream), length-thresholding rejected; #444 accepted the query-capability tradeoff on hostile infra logs. Doc-only. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01X467Druw4cQeEPVp52DTG3 --- docs/benchmarks.md | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 5f686ee6c..eca8a1617 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -1145,10 +1145,14 @@ token is a unique offset-bearing path** 2/3 ≈ 0.67, below the strict 0.7 threshold (§3.1 no-silent-merges), so each line mints a template; the 4-token siblings of the same family (0.75) merge fine. The failure-flag confound turned out to be -a red herring. Tracked as **#444** (tokenizer masking vs. -length-aware thresholding vs. accept-and-scope-C2-per-service — an -RFC-level pillar #2 decision); the safety story held throughout -(bounded memory per RFC 0023, per-service C1 perfect). +a red herring. **#444** resolved it (2026-07-10, maintainer-approved): +of the three options — tokenizer masking, length-aware thresholding, +and accept-and-scope-C2-per-service — **option 3 shipped** (the +per-service gate below, RFC 0006 §3.4.3, PR #451); masking is parked as +a future strategic RFC (no commitment; a Collector `transform`/ +`redaction` processor can polish high-cardinality infra tokens +upstream) and length-thresholding was rejected. The safety story held +throughout (bounded memory per RFC 0023, per-service C1 perfect). The per-service decomposition is now the **first-class bench gate** (`ourios-bench --gates c2` prints it whenever any service bucket exists @@ -1179,6 +1183,10 @@ result on a fresh corpus). Fragmentation does **not** cost query *precision*: the `template_id == 1` probe recovers 1.78 M / 2.76 M rows on cart (one template is most of the corpus) but only 11,523 / 136,790 on kafka, because kafka's dominant event is scattered across -~11,651 ids — no single template query recovers it. So #444 is a -**query-capability / thesis-value** decision, not a performance one; -the pruning path degrades to the first-class-column floor unharmed. +~11,651 ids — no single template query recovers it. So the +fragmentation is a **query-capability / thesis-value** tradeoff, not a +performance one; the pruning path degrades to the first-class-column +floor unharmed. #444 **accepted** that tradeoff on hostile infra logs: +the per-service gate makes C2 acceptance honest without masking, and +any future masking is deferred to an upstream Collector processor or a +dedicated RFC. From 2c9d738bcfd609350f16837805030ea022dc80be Mon Sep 17 00:00:00 2001 From: Jens Holdgaard Pedersen Date: Fri, 10 Jul 2026 10:00:48 +0200 Subject: [PATCH 2/7] =?UTF-8?q?docs(bench):=20clarify=20=C2=A79.12=20re-sc?= =?UTF-8?q?ore=20date=20and=20the=20template-probe=20recovery?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CodeRabbit review on #452: - The §9.12 heading is the run date (2026-07-09); the #444 resolution is 2026-07-10. Flag the C2 re-score date in the heading so the two aren't read as contradictory. - "no single template query recovers it" understated the measurement — a single template_id probe recovers that id's slice (11,523 rows), just not the full dominant event. Reworded. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01X467Druw4cQeEPVp52DTG3 --- docs/benchmarks.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index eca8a1617..740eb6611 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -1093,7 +1093,7 @@ pruning compounds with scale (B1), result-bound latency holds (B2), and the mining-fragmentation failure mode is now bounded, observable, and priced. -### 9.12 Results — 2026-07-09 (indicative, local M-series) — otel-demo v8 capture: C1 / C2 +### 9.12 Results — 2026-07-09 (indicative, local M-series) — otel-demo v8 capture: C1 / C2 (C2 re-scored under the per-service gate on 2026-07-10, #444 / RFC 0006 §3.4.3) **Corpus.** `corpus/otel-demo-v8` (published GitHub release): a **48-hour** OTel-Demo 2.2.0 capture at 150 locust users with the @@ -1183,7 +1183,8 @@ result on a fresh corpus). Fragmentation does **not** cost query *precision*: the `template_id == 1` probe recovers 1.78 M / 2.76 M rows on cart (one template is most of the corpus) but only 11,523 / 136,790 on kafka, because kafka's dominant event is scattered across -~11,651 ids — no single template query recovers it. So the +~11,651 ids — a single `template_id` probe recovers only that one id's +slice (11,523 rows), not the full dominant event. So the fragmentation is a **query-capability / thesis-value** tradeoff, not a performance one; the pruning path degrades to the first-class-column floor unharmed. #444 **accepted** that tradeoff on hostile infra logs: From 193713f0fcff9bf935928e6e4de65087f359229c Mon Sep 17 00:00:00 2001 From: Jens Holdgaard Pedersen Date: Fri, 10 Jul 2026 10:09:29 +0200 Subject: [PATCH 3/7] =?UTF-8?q?docs(bench):=20short=20=C2=A79.12=20heading?= =?UTF-8?q?=20+=20note;=20unslash=20transform/redaction?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot review on #452: - The re-score annotation made the §9.12 heading long/noisy (ToC + anchor slug). Reverted to the short heading and moved the note to an italic sentence right below (CodeRabbit's "explicitly state" option). - "transform`/`redaction` processor" read as one processor and broke oddly across the slash line-break; now "transform or redaction". Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01X467Druw4cQeEPVp52DTG3 --- docs/benchmarks.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 740eb6611..788e3219d 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -1093,7 +1093,11 @@ pruning compounds with scale (B1), result-bound latency holds (B2), and the mining-fragmentation failure mode is now bounded, observable, and priced. -### 9.12 Results — 2026-07-09 (indicative, local M-series) — otel-demo v8 capture: C1 / C2 (C2 re-scored under the per-service gate on 2026-07-10, #444 / RFC 0006 §3.4.3) +### 9.12 Results — 2026-07-09 (indicative, local M-series) — otel-demo v8 capture: C1 / C2 + +_The run is dated 2026-07-09; its C2 verdict was re-scored under the +per-service gate on 2026-07-10 (#444 / RFC 0006 §3.4.3), so the +resolution dates below post-date the heading._ **Corpus.** `corpus/otel-demo-v8` (published GitHub release): a **48-hour** OTel-Demo 2.2.0 capture at 150 locust users with the @@ -1149,7 +1153,7 @@ a red herring. **#444** resolved it (2026-07-10, maintainer-approved): of the three options — tokenizer masking, length-aware thresholding, and accept-and-scope-C2-per-service — **option 3 shipped** (the per-service gate below, RFC 0006 §3.4.3, PR #451); masking is parked as -a future strategic RFC (no commitment; a Collector `transform`/ +a future strategic RFC (no commitment; a Collector `transform` or `redaction` processor can polish high-cardinality infra tokens upstream) and length-thresholding was rejected. The safety story held throughout (bounded memory per RFC 0023, per-service C1 perfect). From 03b9aa1f7d31427c33f564d9608d892b6875fcf8 Mon Sep 17 00:00:00 2001 From: Jens Holdgaard Pedersen Date: Fri, 10 Jul 2026 10:44:28 +0200 Subject: [PATCH 4/7] docs(bench): clarify template_id 1 is a run-specific id, not canonical MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot review on #452: "template_id == 1" read as if 1 were a stable canonical template. Reworded to "cart's dominant template (id 1 in this run — a run-specific identifier, not a canonical one)". Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01X467Druw4cQeEPVp52DTG3 --- docs/benchmarks.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 788e3219d..236aa25f3 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -1184,8 +1184,9 @@ service has 2 templates or 14,608 — a 1 h window prunes 48 of 49 row groups either way (reconfirming the RFC 0023 graceful-degradation result on a fresh corpus). Fragmentation does **not** cost query *latency* or pruning. What it costs is template-exact query -*precision*: the `template_id == 1` probe recovers 1.78 M / 2.76 M -rows on cart (one template is most of the corpus) but only 11,523 / +*precision*: probing cart's dominant template (id 1 in this run — a +run-specific identifier, not a canonical one) recovers 1.78 M / 2.76 M +rows (one template is most of the corpus) but only 11,523 / 136,790 on kafka, because kafka's dominant event is scattered across ~11,651 ids — a single `template_id` probe recovers only that one id's slice (11,523 rows), not the full dominant event. So the From 0bd54824fec29e725db8cf08ab8a62cfb3d7ccd5 Mon Sep 17 00:00:00 2001 From: Jens Holdgaard Pedersen Date: Fri, 10 Jul 2026 13:30:52 +0200 Subject: [PATCH 5/7] =?UTF-8?q?docs(bench):=20drop=20positional=20"below"?= =?UTF-8?q?=20in=20the=20=C2=A79.12=20gate=20reference?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot review on #452: "the per-service gate below" is a positional reference that misleads if the paragraph moves (the gate is introduced earlier in the section). Reference it directly. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01X467Druw4cQeEPVp52DTG3 --- docs/benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 236aa25f3..6e29b7ba6 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -1152,7 +1152,7 @@ family (0.75) merge fine. The failure-flag confound turned out to be a red herring. **#444** resolved it (2026-07-10, maintainer-approved): of the three options — tokenizer masking, length-aware thresholding, and accept-and-scope-C2-per-service — **option 3 shipped** (the -per-service gate below, RFC 0006 §3.4.3, PR #451); masking is parked as +per-service gate, RFC 0006 §3.4.3, PR #451); masking is parked as a future strategic RFC (no commitment; a Collector `transform` or `redaction` processor can polish high-cardinality infra tokens upstream) and length-thresholding was rejected. The safety story held From c1fb28f968d50baafdc9648189b66bc7a2f9f437 Mon Sep 17 00:00:00 2001 From: Jens Holdgaard Pedersen Date: Fri, 10 Jul 2026 13:36:41 +0200 Subject: [PATCH 6/7] =?UTF-8?q?docs(bench):=20make=20the=20#444=20antecede?= =?UTF-8?q?nt=20explicit=20in=20=C2=A79.12?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot review on #452: "#444 resolved it" could read as resolving the red-herring failure-flag confound rather than the fragmentation decision. Reword to "#444 settled how to handle the fragmentation". Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01X467Druw4cQeEPVp52DTG3 --- docs/benchmarks.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 6e29b7ba6..c53f08c80 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -1149,8 +1149,9 @@ token is a unique offset-bearing path** 2/3 ≈ 0.67, below the strict 0.7 threshold (§3.1 no-silent-merges), so each line mints a template; the 4-token siblings of the same family (0.75) merge fine. The failure-flag confound turned out to be -a red herring. **#444** resolved it (2026-07-10, maintainer-approved): -of the three options — tokenizer masking, length-aware thresholding, +a red herring. **#444** settled how to handle the fragmentation +(2026-07-10, maintainer-approved): of the three options — tokenizer +masking, length-aware thresholding, and accept-and-scope-C2-per-service — **option 3 shipped** (the per-service gate, RFC 0006 §3.4.3, PR #451); masking is parked as a future strategic RFC (no commitment; a Collector `transform` or From 54fa5ea496f916f481f9d28669228e9a093637db Mon Sep 17 00:00:00 2001 From: Jens Holdgaard Pedersen Date: Fri, 10 Jul 2026 13:42:18 +0200 Subject: [PATCH 7/7] =?UTF-8?q?docs(bench):=20use=20"length-aware=20thresh?= =?UTF-8?q?olding"=20consistently=20in=20=C2=A79.12?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot review on #452: the same #444 option was named "length-aware thresholding" then "length-thresholding". Use the full term both times. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01X467Druw4cQeEPVp52DTG3 --- docs/benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index c53f08c80..e3ba1c0fb 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -1156,7 +1156,7 @@ and accept-and-scope-C2-per-service — **option 3 shipped** (the per-service gate, RFC 0006 §3.4.3, PR #451); masking is parked as a future strategic RFC (no commitment; a Collector `transform` or `redaction` processor can polish high-cardinality infra tokens -upstream) and length-thresholding was rejected. The safety story held +upstream) and length-aware thresholding was rejected. The safety story held throughout (bounded memory per RFC 0023, per-service C1 perfect). The per-service decomposition is now the **first-class bench gate**