feat(corpus): scale OKF conformance 20 -> 33 (W44-B6, C08 L73 close-out) - #368
Conversation
- scripts/corpus-generate.py: deterministic Python generator; OKF_VERSION keyed; FIXTURE_SPECS (accepted) and FAILURE_FIXTURES (rejected) decoupled. - 13 new accepted fixtures covering: 5 new sources (aider/opencode/continue/ kiro/factory-droid-2), 2 new languages (sql, yaml), 3 stress shapes (100 entities, 7-deep relation chain, 12-intent rapid-fire stream), 3 edge cases (CJK label, embedded JSON label, multi-modal sha256 hint). - docs/ops/corpus-breadth.md: strategy + generator API + acceptance. - tests/replay_breadth.rs: 5 hermetic tests pinning the W44-B6 deliverable (all 13 fixtures on disk, total >= 33, each well-formed, generator + docs present). - docs/reference/conformance/README.md: section listing generator-produced fixtures. Total corpus: 20 hand-vetted + 13 generated = 33 accepted fixtures. Failure-mode fixtures (E01-E03) live in a separate scratch dir and are expected to be rejected by the harness. Refs: WAVE44_SCOPE.md (rank 6), docs/ops/WAVE44_PERT.md (lane B6), C08 L73.
|
Warning Review limit reached
Next review available in: 58 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (17)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Cherry-pick of 95a1049 to main. Updates WAVE44_SCOPE.md and docs/ops/WAVE44_PERT.md to: - Mark R-5 (C01 L16 Fluent migration) as **withdrawn** — was a factual error (C01 L16 closed in Wave-38 #312) - Update W44-B1 (loom-sse-soak) as SHIPPED via PR #372 — merge commit 774a927 already on main - Update W44-B6 (corpus-breadth) as SHIPPED via PR #368 — pending merge - Update realistic post-W44 to 397-399/402 (was 398-401) - Resolve D-W44-3 (R-5 was a factual error, no machine lane needed) Refs: PR #372 (W44-B1), PR #368 (W44-B6), WAVE44_SCOPE.md, docs/ops/WAVE44_PERT.md
…3c974f (#374) - SCORECARD.md: header refreshed (date=2026-07-24, auditor=Wave-44-D, commit=13c974f7); Wave-44 Delta section added (W44-B1 loom deepened C00 L7; W44-B6 corpus deepened C08 L73 20 -> 33 fixtures; PERT correction withdrawn R-5); Held (no score) line expanded with #368, #372, #373; Remaining unpaid rewritten to honestly name C04 L36 / C08 L76 / C11 L110 as the 3 unpaid pillars (6 raw pts). - TRACEABILITY.json: updated 2026-07-23 -> 2026-07-24; commit 41829e8 -> 13c974f; wave Wave-43 -> Wave-44; delta_vs_w42 -> delta_vs_w43. CRLF preserved. - GAP_QA_MATRIX.md: C00 row mentions #372 loom HTTP SSE soak; C08 row mentions #368 corpus breadth and L76 residual; PLAN-W8-B row updated with Wave-44 closure refs and Wave-45 candidate lanes. - CHANGELOG.md: Unreleased Changed entry for Wave-44-D reaudit. Score 396/402 (98% A) held conservative; 2 of 3 machine lanes shipped 2026-07-24 (W44-B1 #372, W44-B6 #368). PERT correction #373. Remaining 6 raw pts across C04 L36 (HUMAN 2FA), C08 L76 (Agent-Eval Pipeline), C11 L110 (Tray/Menubar Client) -- the 3 unpaid pillars the rubric actually scores below 3. Co-authored-by: KooshaPari <koosha@example.com>
1 participant
Summary
Wave-44 close-out lane B6 — production-scale corpus breadth. Scales the OKF
conformance corpus from 20 hand-vetted fixtures to 33 via a deterministic
generator. Closes C08 L73 production-scale corpus breadth residual from
Wave-43 SCORECARD.
What changed
scripts/corpus-generate.pydocs/reference/conformance/fixtures/*-03[7-9].okf.jsondocs/ops/corpus-breadth.mdtests/replay_breadth.rsdocs/reference/conformance/README.mdTotal: 17 files, +3176/-1.
Coverage gained
Test wiring
tests/replay_breadth.rsadds 5 hermetic tests:w44_b6_all_generated_fixtures_are_on_disk— all 13 named slugs presentw44_b6_corpus_total_meets_or_exceeds_target— >= 33 fixturesw44_b6_each_generated_fixture_is_well_formed_okf_v1— each parses; okf='1.0'; entities/provenance presentw44_b6_generator_script_present_and_importable— generator on disk; defines OKF_VERSION, FIXTURE_SPECS, FAILURE_FIXTURESw44_b6_corpus_breadth_doc_present— docs/ops/corpus-breadth.md references C08 L73 and Wave-44The existing
tests/okf_roundtrip.rs::conformance_corpus_fixtures_validate_via_our_parserautomatically picks up the new 13 fixtures (it iterates
*.okf.jsonunderdocs/reference/conformance/fixtures/).Failure-mode fixtures
E01 (malformed-truncated), E02 (missing-provenance), E03 (duplicate-id) live
in a separate scratch dir. They are NOT in the accepted-corpus run; the
harness is expected to reject them. Wiring the negative test is queued for
W45 if needed.
Score disposition
Pre-W44: 396/402 (98% A)
Target post-W44: 402/402 (100% A+); C08 L73 specifically: 29/30 → 30/30
Realistic post-W44: depends on remaining 5 unpaid residuals; see PR #367.
Predecessors
Refs: WAVE44_SCOPE.md (rank 6), docs/ops/WAVE44_PERT.md (lane B6), C08 L73.