feat(bench): land C2 template-count convergence (PR-J2)

[jensholdgaard]

Summary

The third and final writer-side thesis gate. C2 measures whether the template count plateaus within 2× of its steady-state value by 1 M lines (§3.4.3). All three gates (A1, C1, C2) now run in any combination in a single miner pass.

Key design: the live template count at any point equals the number of distinct non-NO_TEMPLATE template_ids seen so far — templates are monotonic (never unmerged) and every template is stamped onto the records of the lines that use it. So C2 is a pure stream accumulator over the harness callback (like C1), needing no cluster or harness change.

Lands

• src/c2.rs — C2Accumulator: tracks distinct template ids, samples the count at the §3.4.3 cadence (N = max(1, ceil(total_lines / 1024)), curve ≤ 1024 points, final line always sampled), and at finalize computes SS (end count), count-at-1M (sample closest to line 1_000_000, floor tie-break), convergence ratio, and pass (ratio ≥ 0.5 on ≥ 1 M-line corpora; abstain → pass=None below 1 M).
• src/lib.rs — run() drops the C2 NotImplemented guard and wires the C2 accumulator into the shared miner pass. The obsolete c2_still_returns_not_implemented marker test is replaced by no_gates_enabled_is_a_cli_error. Crate / run / BenchError docs updated to PR-J2 status.
• tests/c2.rs — rfc0006_3_c2_abstains_on_short_corpus un-ignored (seed corpus, fast, real miner → abstain). The ≥ 1 M-line end-to-end test gets a real bounded-alphabet generator but stays #[ignore]'d as a heavy on-demand check (>1 M miner ingests; consistent with §3.7 "bench runs on-demand, not per-PR"). Verified passing via --ignored (0.77 s release).

Test coverage

• src/c2.rs colocated unit tests drive the convergence math at ≥ 1 M-line scale fast (synthetic ids, no miner/disk): cadence/curve-length, stable-corpus pass, short-corpus abstain, non-converging (linear growth) fail.
• tests/c2.rs abstention runs the real miner end-to-end.

A default all-gates CLI run now completes on the seed corpus (A1 FAIL on the tiny corpus as expected, C1 PASS, C2 ABSTAIN) instead of erroring on C2.

Test plan

• cargo fmt --all --check — clean.
• cargo clippy --all-targets --all-features -- -D warnings — clean.
• cargo test --all-features — 263 passed / 21 ignored (+4 C2 unit tests, +1 un-ignored abstention, −1 removed marker).
• Heavy ≥1M C2 test passes via cargo test -p ourios-bench -- --ignored.

Maturity stage

With all three writer-side gates green, the only remaining RFC 0006 work is the docs/benchmarks.md §9 markdown appender (--update-benchmarks-md) and the RFC0006.7 reproducibility test. The specified → green flip follows once those land.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • C2 benchmark gate is fully implemented: produces a capped convergence curve with guaranteed final-line sampling, reports per-run metrics including 1M-line-derived ratio and pass/fail when applicable, and is included in run outputs when enabled.

• Tests

  • End-to-end C2 tests added for ≥1M stable corpora and sub‑1M abstention; validate sampling cadence, curve length, and convergence behavior. Previously-ignored small‑corpus test now runs by default.

• Bug Fixes

  • CLI gate validation tightened to treat empty gate selection as an error instead of a special-case.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 47db17eb-321d-460c-b752-c4d6960803cf

📥 Commits

Reviewing files that changed from the base of the PR and between 064376b and b3f9d95.

📒 Files selected for processing (2)

• crates/ourios-bench/src/c2.rs
• crates/ourios-bench/src/lib.rs

---

📝 Walkthrough

Walkthrough

Implements the C2 gate: adds C2Accumulator to sample distinct-template counts during mining, wires accumulator into the harness ingest path, finalizes C2Result after mining, updates gate validation/docs, and adds unit and integration tests exercising stable and short corpora.

Changes

C2 Gate Implementation & Harness Integration

Layer / File(s)	Summary
C2 Accumulator Data Structure & Constants crates/ourios-bench/src/c2.rs	Introduces the C2Accumulator struct with module-level sampling budget/constants (SAMPLE_BUDGET, ONE_MILLION) and fields for total lines, sample cadence, distinct template IDs tracking, convergence curve samples, and processed line count.
C2Accumulator Initialization & Record Accumulation crates/ourios-bench/src/c2.rs	Implements C2Accumulator::new() and record() with core observe logic: update max_template_id/template_count monotonically, increment processed line count, sample at fixed cadence with final-line deduplication, and append curve samples.
C2 Finalization & Convergence Pass Decision crates/ourios-bench/src/c2.rs	Implements finalize() to derive end template count from the last curve sample, select the sample closest to 1,000,000 lines (tie-breaking earlier), compute convergence ratio (count_1m / SS) and set pass for ≥1M corpora; returns bounded curve and cadence metadata for shorter corpora with 1M-derived fields None.
C2Accumulator Unit Tests crates/ourios-bench/src/c2.rs	Adds unit tests validating cadence/curve-length invariants (including final-line sampling), stable-corpus pass behavior on 1M+ lines, abstention on <1M corpora, and a non-converging synthetic scenario with expected ratio bounds.
Harness Module Documentation & C2 Wiring crates/ourios-bench/src/lib.rs	Updates top-level module docs to state A1/C1/C2 are implemented and runnable together, adds c2 module, and updates run() doc comment to require at least one gate enabled and to document CLI error on empty GateSet.
Gate Validation & Error Handling Updates crates/ourios-bench/src/lib.rs	Changes gate-selection validation to reject only empty GateSet (returns BenchError::Cli), removes prior c2-specific BenchError::NotImplemented path, revises NotImplemented docs, and updates unit tests accordingly.
Harness C2 Accumulator Integration & Output crates/ourios-bench/src/lib.rs	Initializes C2Accumulator when C2 enabled, forwards emitted MinedRecords to it during the ingest callback, finalizes c2_result after mining, and includes c2 in ResultsFile.
ResultsFile docs and NotImplemented wording crates/ourios-bench/src/lib.rs	Clarifies ResultsFile.c2 nullability semantics and rewords BenchError::NotImplemented docs to reflect current behavior.
End-to-End C2 Integration Tests crates/ourios-bench/tests/c2.rs	Stable corpus test writes a 1_000_001-line stable.txt and asserts convergence/pass; short-corpus abstention test is enabled by default and validates abstention wiring and related assertions.

Estimated Code Review Effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly Related PRs

• jensholdgaard/ourios#52: Overlaps with harness run() wiring changes that integrate C2 alongside existing A1/C1 flow.
• jensholdgaard/ourios#51: Related accumulator/harness callback patterns for per-record metrics (C1) that this PR parallels for C2.
• jensholdgaard/ourios#50: Earlier C2 test stubs and RFC material expanded here into full implementation and end-to-end tests.

Poem

🐇 I hop through lines, a tiny sleuth,

I tally templates with quiet truth,
Cadence marks where my samples fall,
The final line completes the call,
A million whispers tell the proof.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the main change: landing (implementing) the C2 template-count convergence feature as part of PR-J2.
Description check	✅ Passed	The description comprehensively covers the objective, design rationale, landing files, test coverage, and verification; all checklist items are addressed and marked complete.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/ourios-bench-c2

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Implements the RFC 0006 writer-side C2 gate (template-count convergence) in ourios-bench, wiring it into the existing single-pass harness so A1/C1/C2 can run in any combination from one miner ingest.

Changes:

• Added C2Accumulator stream accumulator and convergence metric computation (sampling cadence, SS, count@1M, ratio, pass/abstain).
• Updated run() to enable C2, removed the prior NotImplemented guard, and adjusted the “no gates enabled” contract/test.
• Updated C2 integration tests: real-miner short-corpus abstention now runs by default; ≥1M stable-corpus E2E remains #[ignore] with an on-demand generator.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
crates/ourios-bench/src/c2.rs	New C2 accumulator implementation + colocated unit tests for cadence/ratio math
crates/ourios-bench/src/lib.rs	Wires C2 into the shared miner pass; updates docs and gate-selection test
crates/ourios-bench/tests/c2.rs	Updates C2 integration tests (heavy stable-corpus generator; un-ignores short-corpus abstention)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

🧹 Nitpick comments (1)

crates/ourios-bench/src/c2.rs (1)

227-249: 💤 Low value

Test name and assertion don't match actual behavior.

With 2M lines and linear growth (one new template per line), the closest sample to 1M is at ~1,000,448 lines (cadence ≈ 1954), giving:

• count_1m ≈ 1,000,448
• template_count_at_end = 2,000,000
• ratio ≈ 0.5002

Since 0.5002 ≥ 0.5, pass == Some(true). The test name says "fails_the_gate" but linear growth lands at the pass boundary. The comment mentions "push it under by minting faster in the back half" but that logic isn't implemented.

Consider either:

1. Rename to linear_growth_at_boundary and add assert_eq!(r.pass, Some(true)); to document the actual boundary behavior, or
2. Mint faster in the second half (e.g., 2 templates/line after 1M) to actually fail the gate and assert pass == Some(false).

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/ourios-bench/src/c2.rs` around lines 227 - 249, The test's
name/assertion doesn't match behavior: linear growth (one new template per line)
with total=2_000_000 actually yields pass == Some(true). Rename the test from
non_converging_curve_fails_the_gate to linear_growth_at_boundary (or similar),
and update the assertions to reflect reality by adding assert_eq!(r.pass,
Some(true)); (keep the existing ratio check). Update the test comment to remove
the incorrect "fails the gate" wording and reference C2Accumulator::new,
C2Accumulator::observe, C2Accumulator::finalize, and r.pass so future readers
understand the observed boundary behavior.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/ourios-bench/src/c2.rs`:
- Around line 227-249: The test's name/assertion doesn't match behavior: linear
growth (one new template per line) with total=2_000_000 actually yields pass ==
Some(true). Rename the test from non_converging_curve_fails_the_gate to
linear_growth_at_boundary (or similar), and update the assertions to reflect
reality by adding assert_eq!(r.pass, Some(true)); (keep the existing ratio
check). Update the test comment to remove the incorrect "fails the gate" wording
and reference C2Accumulator::new, C2Accumulator::observe,
C2Accumulator::finalize, and r.pass so future readers understand the observed
boundary behavior.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6a6e5ac1-5141-4ce8-b17c-199f1134021a

📥 Commits

Reviewing files that changed from the base of the PR and between b75b81c and b55687a.

📒 Files selected for processing (3)

• crates/ourios-bench/src/c2.rs
• crates/ourios-bench/src/lib.rs
• crates/ourios-bench/tests/c2.rs

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.