feat(release): quantify zero-event benchmark safety bounds

[seonghobae]

Partial implementation of #203. This branch defines a fail-closed release-acceptance boundary for benchmark evidence: zero-event safety claims retain exact trial counts/confidence bounds, benchmark execution failures stay causally classified, and missing/non-authoritative evidence cannot be promoted to release success.

Current production and DDD boundary

The release-acceptance contract lives in the dedicated originweave-release bounded-context crate. originweave-core remains the stable cross-context value-contract kernel and does not depend outward on release-specific policy/evidence types.

• ZeroEventSafetyEvidence requires a nonzero exact trial count and confidence in 1..=9999 basis points.
• upper_event_rate() reports the one-sided Clopper-Pearson zero-event upper bound using the bounded numerical implementation.
• Named ZeroEventSafetyObservation values are deterministic and bounded; duplicate metric identities fail closed.
• BenchmarkFailureClass preserves deterministic/stochastic product failures separately from external site drift, outage, unsupported capability, infrastructure failure, and benchmark defects.
• Non-product execution failures remain Inconclusive, never passing evidence.
• Commercial safety policy must declare all mandatory named metrics; incomplete policy cannot produce accepted commercial evidence.
• The combined report remains evidence only and cannot grant merge, tag, publication, or release authority.

This branch intentionally moves release-acceptance ownership out of originweave-core into originweave-release. ADR 0015 records the deliberate pre-GA migration; no compatibility shim may invert the DDD dependency direction or duplicate release contracts across both contexts.

Primary statistical basis: Clopper, C. J., & Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26(4), 404–413. https://doi.org/10.1093/biomet/26.4.404

Current repair and protected-main lineage

The earlier DDD documentation defects remain repaired: ARCHITECTURE.md identifies originweave-release as the active release-acceptance owner, CHANGELOG.md records the pre-GA move out of core, and the temporary self-modifying workflow remains absent. Its historical control-plane defect stays tracked under #123 and must not be restored.

Protected main is exact 87c4daa1830bac5a5228b6036752ad5633232085 through #286. Current exact head is 24930a3a9ee79c0b712ee3df6589b0592eb6e18f, a normal non-force merge adoption of that protected generation. Fresh compare is 89 ahead / 0 behind, merge base exactly protected main, with 32 effective release/product/docs/test paths and no .github/** delta. GitHub reports the PR open, Ready and mergeable. Workflow/lifecycle changes visible inside the topology merge are inherited protected-main content rather than an effective workflow mutation by this lane.

Exact-current evidence

The previous body was stale at predecessor c7d6c205... / protected main@c789b802.... Exact current 24930a3a9ee79c0b712ee3df6589b0592eb6e18f has terminal returned workflow evidence:

• CI 33925598822: success;
• Manifest V3 Compatibility 33925598787: success;
• Security Scan 33925598876: success;
• SAST Semgrep 33925598799: success; and
• CodeQL PR 33925598844: failure.

The failed CodeQL verdict remains fail-closed and is not replaced by native/MV3/security success. Exact-head GREEN elsewhere does not grant merge, tag, publication or release authority and does not transfer predecessor review evidence.

Remaining #203 scope

This slice still does not execute the complete commercial benchmark portfolio, durably bind authenticated benchmark evidence to every release decision, establish every supported browser/profile claim, or complete integrated commercial release acceptance. Those remain dependency-ordered work under #203.

Protected-main AGENTS.md and live GitHub governance remain authoritative. No self-approval, force-push, destructive rebase, workflow/ruleset/secret mutation, gate weakening, predecessor evidence transfer, tag, release, or publication is authorized by this PR.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

제로 이벤트 안전성 증거와 벤치마크 실패 분류 타입을 추가합니다. 릴리스 결정 진입점은 입력을 검증하고 증거를 canonical 순서로 보고서에 저장합니다. 테스트와 문서는 계산, 분류, 오류 계약 및 중복 처리를 검증합니다.

Changes

릴리스 수락 증거

Layer / File(s)	Summary
벤치마크 실패 분류 계약 crates/originweave-core/src/benchmark_failure.rs, crates/originweave-core/src/root.rs, crates/originweave-core/tests/benchmark_failure_classification.rs	BenchmarkFailureClass가 일곱 실패 유형의 안정적 식별자와 BenchmarkSuiteOutcome 매핑을 제공합니다. 제품 계열 실패는 Failed로 매핑하고, 환경 또는 벤치마크 불확실성은 Inconclusive로 매핑합니다.
제로 이벤트 안전성 계약 crates/originweave-core/src/release_acceptance.rs, crates/originweave-core/tests/release_acceptance_zero_event_bound.rs, crates/originweave-core/tests/release_acceptance_zero_event_threshold.rs, crates/originweave-core/tests/release_acceptance_zero_event_report.rs, docs/doctoring.md	ZeroEventSafetyEvidence가 시행 횟수와 신뢰수준을 검증하고 Clopper–Pearson 단측 상한을 계산합니다. 임계값 평가, 메트릭 식별자 및 오류 계약을 추가합니다. 테스트와 문서가 계산값과 입력 경계를 확인합니다.
릴리스 보고서 통합 crates/originweave-core/src/release_acceptance.rs, crates/originweave-core/tests/release_acceptance_zero_event_report.rs, crates/originweave-core/tests/release_acceptance_classified_failure_report.rs	새 진입점이 안전성 메트릭, 제한사항 및 suite 증거의 중복을 검증합니다. 보고서는 안전성 관찰값과 benchmark failure를 canonical 순서로 저장합니다. 실패 분류에 따라 Rejected 또는 Inconclusive 결과를 보존합니다. 기존 decide_release 진입점은 호환 경로로 유지합니다.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: 🟡 Moderate · up to 7ed3c

The PR adds typed benchmark failures and zero-event safety evidence, but it is not merge-ready because the new threshold test does not compile and release acceptance records safety bounds without enforcing the corresponding quantitative policy. The public API and acceptance behavior need to be aligned before merge.

Sequence Diagram(s)

sequenceDiagram
  participant 호출자
  participant ClassifiedEvidenceEntrypoint
  participant ReleaseDecisionPipeline
  participant ReleaseDecisionReport
  호출자->>ClassifiedEvidenceEntrypoint: benchmark evidence와 안전성 관찰값 전달
  ClassifiedEvidenceEntrypoint->>ReleaseDecisionPipeline: 메타데이터와 중복 증거 검증
  ReleaseDecisionPipeline->>ReleaseDecisionReport: canonical 실패와 관찰값 저장
  ReleaseDecisionReport-->>호출자: 릴리스 결정 보고서 반환

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 85.45% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 55 functions across 8 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	제목은 zero-event benchmark safety bounds를 정량화하는 핵심 변경을 정확하고 간결하게 설명합니다. 릴리스 승인 경계와 실패 분류 전체를 포함하지 않지만, 제목이 모든 변경을 포함할 필요는 없으므로 기준을 충족합니다.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/benchmark-release-decision

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[devin-ai-integration]

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

[devin-ai-integration]

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

[devin-ai-integration]

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

[devin-ai-integration]

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

[devin-ai-integration]

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

[devin-ai-integration]

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

[cwl-noema-review]

Noema LLM review

The PR implements a bounded, deterministic release-acceptance contract for benchmark evidence, including a quantitative zero-event statistical safety gate based on the Clopper-Pearson binomial method. The implementation has been refactored to prevent resource exhaustion (memory/CPU) by bounding input collections and terminating iterators upon detecting duplicates. The logic for zero-event upper bounds is mathematically correct, and the fail-closed behavior is verified by comprehensive tests. Documentation in ARCHITECTURE.md and CHANGELOG.md has been updated to reflect the new bounded context in originweave-release.

Reviewed changed lines

• crates/originweave-release/src/release_acceptance.rs:130 (RIGHT): Correctly implements the one-sided Clopper-Pearson upper bound for zero events using exp_m1 to maintain precision for small values.
• crates/originweave-release/src/release_acceptance.rs:618 (RIGHT): Enforces a fixed maximum cardinality for zero-event observations, preventing unbounded memory allocation during cloning and sorting.
• crates/originweave-release/src/release_acceptance.rs:558 (RIGHT): The loop over evidence terminates immediately upon detecting a duplicate suite, protecting against infinite or oversized iterators.
• crates/originweave-release/src/zero_event_safety_gate.rs:215 (RIGHT): Validates that requirements and observations are within the fixed metric budget before populating BTreeMaps, ensuring resource bounds at the policy boundary.
• ARCHITECTURE.md:81 (RIGHT): Accurately describes the new bounded release-evidence contract and explicitly states that it does not grant release authority.

Adversarial validation

• crates/originweave-release/src/release_acceptance.rs:558 (RIGHT) falsified: An infinite iterator yielding the same suite will cause a hang or OOM. — The code checks if outcomes[index].is_some() { return Err(...) } inside the loop, ensuring termination at the second element.
• crates/originweave-release/src/release_acceptance.rs:618 (RIGHT) falsified: A very large slice of ZeroEventSafetyObservation will cause OOM during to_vec() or sort_by_key(). — The validate_release_metadata function is called first and returns Err(ReleaseDecisionError::TooManyZeroEventSafetyObservations) if length > 5.
• Residual risk: Low. The use of f64 for the final rate comparison is mitigated by exact_fixed_point_boundary_matches for small trial counts where precision is most critical.

Findings

• No blocking findings.

• Result: APPROVE

• Head SHA: 86fe70fa7e25347649a4397361143ff86473261e

• Reviewer credential: noema-review-github-app

• Actor: cwl-noema-review[bot]

[seonghobae]

Current-head verification on c7d6c2053e0fb3f63d936e5aaaca01b2f76ce987: all returned inline threads are resolved, the temporary write-capable/self-modifying workflow is absent, and .github/workflows/ci.yml is exact blob 95c2fa1d7cf190ed2c043c5e05e27dfb606b5118, identical to protected main@c789b802.... Exact CI 33726146724 is terminal GREEN (Rust contracts 100555443903, Production coverage 100555443569), and MV3 33726146609, SAST 33726146692, Security Scan 33726146770, OSV 33726146911, and Scorecard 33726146925 also succeeded. The Noema APPROVED review targets predecessor 86fe70fa..., so it remains historical rather than current-head approval under stale-review semantics. This comment records current evidence only; it is not self-approval or merge/release authorization.

[seonghobae]

Current protected-main adoption: exact contributor head is now 76f30fe867ec77ff604de64497889b1026380082, a two-parent merge of prior product head c7d6c2053e0fb3f63d936e5aaaca01b2f76ce987 and protected main@4ed08bfa7c063fc7f2ef9278ee8d281887b8296b. The semantic delta remains the release bounded-context migration, classified benchmark evidence, zero-event statistics/gate, tests, ADR, doctoring, and product-gap baseline update; current-head CI workflow content is byte-identical to protected main. Local exact-head verification passed 78 release tests, 36 core tests, and 18 DDD/documentation/repository contracts. All predecessor checks and approvals are stale under current rules. No self-approval, merge, tag, publication, or release authority is asserted.

[seonghobae]

Current-head verification repair: protected main is now 4ed08bf and this PR's exact head is 76f30fe. The body still records predecessor c7d6c20... and transfers that predecessor's terminal GREEN; do not use it as current-head evidence. Fresh exact-head runs are CI 33886238370, Manifest V3 33886238427, Security Scan 33886238396, SAST 33886238316, and CodeQL 33886238329; all are currently queued/non-terminal. The earlier Noema approval was already predecessor-only and remains non-transferable. Preserve the release-acceptance bounded-context migration and do not merge, tag, publish, or release until current exact checks and live approvals are terminal.

[seonghobae]

Restacked non-force onto protected main 87c4daa1830bac5a5228b6036752ad5633232085; exact head is now 24930a3a9ee79c0b712ee3df6589b0592eb6e18f. The parent-relative release-acceptance delta remains unchanged.

Exact-head local verification:

• all 158 Python repository contracts: PASS
• cargo fmt, workspace/all-target tests, all-target/all-feature Clippy with warnings denied, and rustdoc with warnings denied: PASS
• fresh-worktree production coverage: functions 100%, lines 100%, regions 100%, branches 100%

Hosted checks and counted approval remain independently required; queued, skipped, or predecessor results are not merge authorization.

[seonghobae]

Fresh exact-head review for 24930a3a9ee79c0b712ee3df6589b0592eb6e18f: the release-context ownership, bounded zero-event evidence, fixed-point threshold policy, exact-boundary handling, mandatory five-metric policy, and fail-closed benchmark causality are coherent on the current tree. Current unresolved/non-outdated review threads: 0. Focused verification passed all originweave-release tests plus the release documentation and DDD dependency-direction contracts. Full local verification passed: 158 Python contract tests with warnings treated as errors; cargo fmt --all --check; all workspace tests; Clippy with -D warnings; rustdoc with -D warnings; and 100% production function, line, region, and branch coverage. The PR body cites predecessor heads and runs, which are not treated as current-head evidence. Hosted exact-head checks and eligible independent approval remain required; no merge or release claim is made.

[seonghobae]

Exact-head focused review for 24930a3 found no new source-backed defect in the zero-event release boundary.

The review traced ZeroEventSafetyEvidence through fixed-point threshold evaluation and the combined commercial decision. Zero trials and invalid confidence fail construction; insufficient confidence, missing metrics, and an upper bound above policy remain Inconclusive; a zero-rate threshold cannot turn finite trials into success; known benchmark failures remain Rejected. Satisfied safety evidence preserves the underlying accepted/accepted-with-limitations decision without granting repository or release authority.

Focused verification passed rustfmt and 19 Rust tests across the zero-event bound, threshold, and commercial gate suites. This is a COMMENTED review, not approval. Current central CodeQL compatibility failures are dispatch placeholders awaiting authenticated shard verdicts; remaining in-progress/queued checks and an eligible exact-head approval still block merge.