feat(bench): close RFC 0006 — reproducibility + forged-mismatch, flip to green (PR-J4)

[jensholdgaard]

Summary

Lands the last two acceptance criteria and flips RFC 0006 red → green — every §5 scenario now has a passing test. This closes out the writer-side thesis-gate harness.

Lands

• RFC0006.7 (reproducibility) un-#[ignore]'d and passing. Required pinning the miner's clock: the harness now builds the cluster with TestClock::new(<§3.3 baseline>) so audit-event timestamps are deterministic. The default SystemClock stamps each run's audit events with wall-clock time, which would make A1's audit-stream bytes — and so the compression ratio — differ run-to-run; RFC0006.7 demands bit-identical measurements across reruns. The test runs the bench twice on the seed corpus (all gates) and asserts byte-identical JSON modulo timestamp.

• RFC0006.2 (mismatch sub-criterion) — replaced the #[ignore]'d unimplemented!() end-to-end stub with a colocated unit test (c1::tests::reconstruction_mismatch_is_counted_as_failure). The real miner never produces a non-lossy reconstruction mismatch (the H7.1 property), so the path is only reachable via a hand-forged record — a unit-level fixture, not an integration test that would need a fault-injection hook the harness deliberately lacks. The test forges a [Fixed("alpha")] template + a record reconstructing to "alpha" against an input line of "beta" and asserts pass = false / rate = 0. main.rs maps pass = false to a non-zero exit (§3.4.2).

• docs/rfcs/0006-bench-harness.md frontmatter status: red → green; crate docs + a stale c1::finalize doc (still calling main.rs a red-stage scaffold) updated.

Note on the one remaining #[ignore]

The ≥ 1 M-line C2 end-to-end test stays #[ignore]'d for the per-PR loop (§3.7: the bench runs on-demand, not per-PR). It passes via cargo test -p ourios-bench -- --ignored, and its convergence math is covered by default by the colocated c2 unit tests at ≥1M scale. So the criterion is met; green is honest. Validated (terminal stage) comes when the gates are measured on a real corpus + the §1 hardware baseline.

Test plan

• cargo fmt --all --check — clean.
• cargo clippy --all-targets --all-features -- -D warnings — clean.
• cargo test --all-features — 278 passed / 19 ignored (was 276 / 21).
• mdbook build — clean (frontmatter flip renders).
• Reproducibility test passes (two full A1+C1+C2 runs, bit-identical); forged-mismatch unit test passes.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Capture and report per-row reconstruction mismatch diagnostics with summarized counts and capped display.

• Improvements

  • Deterministic timestamps for audit events to stabilize measurement inputs across runs.

• Tests

  • Activated reproducibility test to run in CI.
  • Added unit test ensuring reconstruction-mismatch cases are counted as failures and yield diagnostic output.

• Documentation

  • RFC status updated to green; test documentation clarified gate coverage.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 78a7859b-f7a2-414c-8729-e6e6771d4650

📥 Commits

Reviewing files that changed from the base of the PR and between e276d4e and a15a8fe.

📒 Files selected for processing (4)

• crates/ourios-bench/src/c1.rs
• crates/ourios-bench/src/lib.rs
• crates/ourios-bench/src/main.rs
• crates/ourios-bench/src/report.rs

✅ Files skipped from review due to trivial changes (1)

• crates/ourios-bench/src/report.rs

---

📝 Walkthrough

Walkthrough

Pins the harness clock for deterministic audit timestamps, records bounded per-row C1 reconstruction mismatch diagnostics, exposes C1Mismatch in results (stderr-only), adds a unit test asserting mismatch is a hard failure, enables reproducibility test, and updates RFC/module status to green.

Changes

RFC 0006 Green Gate

Layer / File(s)	Summary
Deterministic harness clock setup crates/ourios-bench/src/harness.rs	Imports Duration, UNIX_EPOCH, TestClock, and TIME_BASELINE_NS; constructs a fixed benchmark clock from UNIX_EPOCH + TIME_BASELINE_NS and wires it into MinerCluster via .with_clock(...) to stabilize audit-stream timestamps.
C1 accumulator mismatch capture crates/ourios-bench/src/c1.rs	Adds pub(crate) const MISMATCH_SAMPLE_CAP: usize = 16 and mismatches: Vec<C1Mismatch> to C1Accumulator; on non-lossy rows compute actual = reconstruct(...), increment success on match, otherwise record a bounded C1Mismatch; finalize() returns cloned mismatches.
Unit test: reconstruction mismatch failure crates/ourios-bench/src/c1.rs	Adds reconstruction_mismatch_is_counted_as_failure unit test that forges a mismatch (template reconstructs "alpha" vs input "beta"), asserting denominator=1, reconstruct_ok=0, rate=0, pass = false, and one captured mismatch with expected template metadata and expected/actual strings.
Public C1Mismatch type and test fixtures crates/ourios-bench/src/lib.rs, crates/ourios-bench/src/report.rs	Adds C1Result { mismatches: Vec<C1Mismatch> } (annotated #[serde(skip)]) and new public pub struct C1Mismatch { template_id, template_version, expected, actual }. Updates sample_results() to include mismatches: Vec::new() for tests.
Reporting: main prints mismatch diagnostics crates/ourios-bench/src/main.rs	When c1.pass == false, print each c1.mismatches element (template id/version and expected vs actual bytes) and, if more failures exist than shown, print a capped "and N more failing row(s) not shown" message.
Tests and docs activation crates/ourios-bench/tests/c1.rs, crates/ourios-bench/tests/reproducibility.rs, docs/rfcs/0006-bench-harness.md	Updates C1 test documentation to reference the unit-level mismatch test and removes the ignored end-to-end stub; enables rfc0006_7_two_runs_produce_bit_identical_measurements by removing #[ignore]; updates module and RFC front-matter status from PR-J3/red to PR-J4/green.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• jensholdgaard/ourios#51: Related prior work on extending C1 accumulator/finalize recon-rate logic and mismatch handling.
• jensholdgaard/ourios#50: Earlier PR that provided an ignored stub test for the reconstruct-mismatch path; this PR implements the unit-level mismatch test and diagnostics.
• jensholdgaard/ourios#53: Adds JSON results writer and influences main.rs exit/reporting behavior which this PR extends with per-row mismatch diagnostics.

Poem

🐰 I counted bytes beneath the moon,
A clock set sure, no drift at all,
When alpha ≠ beta, I sing a tune —
One mismatch logged, a trumpet call.
Green flag waves; the harness stands tall.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: closing RFC 0006 by implementing reproducibility and mismatch detection, and flipping the status from red to green.
Description check	✅ Passed	The description comprehensively covers all required template sections with clear context, test results, and verification of checklist items.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/ourios-bench-green

Warning

Review ran into problems

🔥 Problems

Stopped waiting for pipeline failures after 30000ms. One of your pipelines takes longer than our 30000ms fetch window to run, so review may not consider pipeline-failure results for inline comments if any failures occurred after the fetch window. Increase the timeout if you want to wait longer or run a @coderabbit review after the pipeline has finished.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

This PR advances RFC 0006 bench harness work by enabling reproducibility coverage, replacing the ignored C1 mismatch stub with a unit-level fixture, and updating RFC/crate status documentation toward green.

Changes:

• Pins bench miner audit timestamps with TestClock to make A1 output deterministic across runs.
• Enables the RFC0006.7 reproducibility integration test.
• Replaces the ignored C1 mismatch integration stub with a colocated accumulator unit test and updates related docs.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
docs/rfcs/0006-bench-harness.md	Flips RFC 0006 status from red to green.
crates/ourios-bench/tests/reproducibility.rs	Enables the reproducibility scenario test.
crates/ourios-bench/tests/c1.rs	Removes the ignored forged-mismatch integration stub and documents the unit-test replacement.
crates/ourios-bench/src/lib.rs	Updates crate-level implementation status documentation.
crates/ourios-bench/src/harness.rs	Pins the miner clock to the corpus baseline timestamp for deterministic audit events.
crates/ourios-bench/src/c1.rs	Updates C1 finalize docs and adds a forged reconstruction-mismatch unit test.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.