docs: reframe A1 as a diagnostic, not a gating thesis-gate (RFC 0011) - #200
Conversation
RFC 0011 (accepted) demoted A1 from a gating thesis-gate to a recorded diagnostic, but only its §7 gate table and §9 entries were updated — the roadmap §2 MVP-gate framing and benchmarks.md §0 / the A1 section still read as if A1 were one of five must-win gates, contradicting §7. Bring the framing in line with the accepted decision: - roadmap §2: the MVP gate is now FOUR gating [THESIS] goals (B1, B2, C1, C2); A1 is described as a demoted diagnostic that no longer blocks MVP-done or any RFC's validated. - benchmarks.md §0: "five must-win" → four gating + A1 diagnostic. - benchmarks.md A1 section: scope/bar restated as diagnostic/informational with a demotion note; the target/falsifier are retained as the diagnostic's reference line. The [THESIS] tag stays (§7 still lists it among the [THESIS]-tagged goals, marked diagnostic). No new decision — doc cleanup enacting the already-accepted RFC 0011. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 24 minutes and 58 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthrough
ChangesA1 Demotion to Diagnostic
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR updates documentation to align with RFC 0011 by reframing benchmark A1 as a recorded diagnostic rather than an MVP-gating thesis gate, ensuring the roadmap and benchmarks docs no longer imply A1 is “must-win” for MVP-done / validated.
Changes:
- Update
docs/roadmap.mdMVP gate definition from five gating[THESIS]goals to four (B1, B2, C1, C2), with A1 explicitly described as non-gating diagnostic. - Update
docs/benchmarks.mdintro text and A1 section to state A1 is diagnostic/informational rather than gating/must-win, while retaining its reference target/falsifier text.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| docs/roadmap.md | Rewords the MVP gate section to list four gating thesis goals and clarify A1 as a non-blocking diagnostic per RFC 0011. |
| docs/benchmarks.md | Adjusts the “How to read” section and A1’s definition to treat A1 as a diagnostic rather than a gating must-win benchmark. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
docs/benchmarks.md (1)
16-18:⚠️ Potential issue | 🟠 Major | ⚡ Quick winUpdate the status summary to match the A1 demotion.
This still says A1 "fails" and is the open gate, which contradicts RFC 0011 and the updated A1 section below. Rewrite it to say A1 is measured/recorded as a diagnostic and no longer blocks MVP-done or
validated.🛠️ Suggested edit
- As of **2026-06-12** the gates have authoritative numbers on - the §1 hardware baseline (§9.4): **B1, B2, C1, C2 pass; A1 fails** and - is the open gate. + As of **2026-06-12** the gates have authoritative numbers on + the §1 hardware baseline (§9.4): **B1, B2, C1, C2 pass; A1 is + diagnostic** and no longer blocks MVP-done or any RFC's + `validated`.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/benchmarks.md` around lines 16 - 18, The status summary currently states that A1 "fails" and is "the open gate", which contradicts RFC 0011 and the updated A1 section below. Rewrite the status summary to reflect that A1 is now measured and recorded as a diagnostic measurement only and no longer blocks MVP-done or the `validated` status. Ensure the updated text aligns with the A1 demotion described in RFC 0011 and any subsequent A1 details in the document.docs/roadmap.md (1)
164-173:⚠️ Potential issue | 🟠 Major | ⚡ Quick winRemove the leftover A1 blocker language.
The MVP gate table above already demotes A1 to a diagnostic, but these two spots still treat it like a thesis gate: the “remaining blockers” table lists A1 under blockers, and Phase 2 says it “unblocks” A1. That reintroduces the old contract and should be rewritten to say Phase 2 only enables A1 measurement, not MVP gating.
🛠️ Suggested edits
-| **A1** | `ourios-bench` corpus runner driving the miner → Parquet path; A1's compression ratio is measured by the bench. Writer + reader are in place. | +| **A1** | Diagnostic only; measured by `ourios-bench` but not an MVP blocker. Writer + reader are already in place. |-**Unblocks:** thesis gate **A1** (compression ratio). The Parquet -column codec earns its share of the 50–200× headline only once -records actually land on disk in this format. +**Enables measurement of:** A1 (compression ratio) as a +non-blocking diagnostic. The Parquet column codec still matters +for the recorded measurement once records land on disk.Also applies to: 247-249
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/roadmap.md` around lines 164 - 173, The documentation contains inconsistent language about A1's status: the MVP gate table treats A1 as a diagnostic (not a thesis gate), but the blockers table and Phase 2 section still treat it as a thesis gate by listing A1 as a blocker and saying Phase 2 "unblocks" it. Remove A1 from the blockers column in the remaining blockers table, and rewrite any Phase 2 language that says it "unblocks" A1 to instead say it only enables A1 measurement or diagnostic capabilities.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@docs/benchmarks.md`:
- Around line 16-18: The status summary currently states that A1 "fails" and is
"the open gate", which contradicts RFC 0011 and the updated A1 section below.
Rewrite the status summary to reflect that A1 is now measured and recorded as a
diagnostic measurement only and no longer blocks MVP-done or the `validated`
status. Ensure the updated text aligns with the A1 demotion described in RFC
0011 and any subsequent A1 details in the document.
In `@docs/roadmap.md`:
- Around line 164-173: The documentation contains inconsistent language about
A1's status: the MVP gate table treats A1 as a diagnostic (not a thesis gate),
but the blockers table and Phase 2 section still treat it as a thesis gate by
listing A1 as a blocker and saying Phase 2 "unblocks" it. Remove A1 from the
blockers column in the remaining blockers table, and rewrite any Phase 2
language that says it "unblocks" A1 to instead say it only enables A1
measurement or diagnostic capabilities.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 5b2520a3-892b-4e7c-80ec-7ef9de803ae2
📒 Files selected for processing (2)
docs/benchmarks.mddocs/roadmap.md
Copilot caught two stale spots the first pass missed: - the doc intro still read "A1 fails and is the open gate" — updated to "the four gating gates pass; A1 fails but no longer gates (RFC 0011)". - §0's Scope/Bar label vocabulary listed only thesis-gate/tuning-goal and must-win/should-win/stretch, but the A1 section now uses `diagnostic`/`informational`; added both values to the definitions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot's re-review found three more spots the A1 demotion hadn't reached: - benchmarks §0 called `[THESIS]` a "scope" — it's a tag; A1's *scope* is now `diagnostic`. Reworded. - the §9 lead-in summary still said A1 "is the open thesis gate" — fixed, with a forward-pointer noting the A1 readings throughout §9 are diagnostic and the dated entries' "open gate"/"must-win" wording is superseded (the dated historical results themselves are left intact). - roadmap §3's "remaining for the thesis gates" table still listed A1 — removed (A1 is a recorded diagnostic, not a gate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What
Reframes A1 as a recorded diagnostic, not a gating thesis-gate, finishing the doc cleanup that RFC 0011 (
accepted) started.Why
RFC 0011 demoted A1 but only its
benchmarks.md§7 gate table + §9 entries were updated. Two spots still read as if A1 were one of five must-win gates, contradicting §7:roadmap.md§2 — "Five[THESIS]-marked goals … hitting all five" with A1 in the must-win table.benchmarks.md§0 ("five of those … marked[THESIS]") and the A1 section (Scope: thesis-gate/Bar: must-win).(The maintainer flagged roadmap §2; benchmarks.md is the shared root it mirrors, so fixing only roadmap §2 would relocate the contradiction.)
Changes
[THESIS]goals (B1, B2, C1, C2); A1 described as the demoted diagnostic that no longer blocks MVP-done /validated.[THESIS]tag stays (§7 still lists A1 among the[THESIS]-tagged goals, marked diagnostic —The five [THESIS]-tagged goalstable is unchanged and accurate).Invariants
Doc-only; no new decision — enacts the already-
acceptedRFC 0011. No code/schema/hot path.🤖 Generated with Claude Code
Summary by CodeRabbit
Documentation