feat(model): statistical Pareto K gates refuse LLM numerical authority - #67
Conversation
ADR 0012 requires held-out likelihood/complexity comparison before any blinded LLM review. This crate admits K>=2 statistical candidates, drops dominated alternatives, and recovers known truth K with computed RMSE.
|
Warning Review limit reachedNext included review available in 17 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (15)
📝 WalkthroughWalkthrough새 Changes모델 선택 게이트
Estimated code review effort: 3 (Moderate) | ~25 minutes Merge Risk: 🟡 Moderate · up to The PR adds Pareto-based K admission and selection, but the public API still permits invalid K values below 2, and the claimed recovery behavior is not yet validated with realistic repeated synthetic cases. Documentation also describes planned review behavior and supporting evidence more strongly than the current implementation. Merge should wait for these bounded correctness and evidence issues to be addressed or explicitly accepted. Sequence Diagram(s)sequenceDiagram
participant 호출자
participant select_candidate_k
participant ModelCandidate
participant RMSE 계산기
호출자->>select_candidate_k: 후보 목록 전달
select_candidate_k->>ModelCandidate: 통계 후보와 Pareto 전선 검증
select_candidate_k-->>호출자: 선택된 K 반환
호출자->>RMSE 계산기: 선택된 K와 truth K 전달
RMSE 계산기-->>호출자: RMSE 반환
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
# Conflicts: # CHANGELOG.md # docs/adr/README.md # docs/validation/temporal-event-foundation.md
|
Reviewed and integrated current protected main without force-push. I removed the unreachable Current head: Verification at this exact head:
Please run protected current-head Checks and obtain the required independent approval before normal merge. |
|
PR #67 current HEAD d735177 (exact remote head). Root causes repaired: impossible partial-diagnostic coverage branches were collapsed into the valid constructor invariant; Pareto ordering now uses the actual likelihood/K contract; realistic invalid-diagnostic and RMSE edge contracts were added; Python docstring contract now accounts for 11 Rust crates. Local proof: Rust 1.97.1 focused fmt/test/clippy passed; branch coverage 1008/1008 (100%); line coverage 2951/2951 (100%); Python quality 89 tests and statement/branch coverage 100%; workspace and docstring contracts passed. No independent qualifying approval or merge was performed. |
|
Current-head review refresh for d735177:
|
|
Exact-head review receipt: merge-conflict repair was completed and pushed at |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Current-head review receipt
The current CodeRabbit finding is resolved by this head: The exact-head Devin comments are informational only: complexity is enforced while constructing the Pareto front, the Exact-head verification:
Hosted exact-head required Checks are queued with no completed failure observed. There is no qualifying independent approval yet, so this PR remains |
…-pareto-gates # Conflicts: # ARCHITECTURE.md # CHANGELOG.md # Cargo.toml # README.md # docs/TRACEABILITY.md # docs/validation/temporal-event-foundation.md # scripts/check_workspace_contract.py # tests/quality/test_check_docstrings.py
There was a problem hiding this comment.
|
|
||
| sources = docstrings.rust_sources(REPOSITORY_ROOT) | ||
| crate_roots = sorted(REPOSITORY_ROOT.glob("crates/*/src/lib.rs")) | ||
| self.assertEqual(len(crate_roots), 11) |
There was a problem hiding this comment.
🔴 Docstring test hardcodes wrong crate count
The new assertion self.assertEqual(len(crate_roots), 11) hardcodes 11, but adding model_selection brings the workspace to 12 crates, each with a src/lib.rs. The glob returns 12 paths, so the assertion fails and contradicts the next line comparing against len(EXPECTED_CRATES) (12). The quality test suite breaks.
| self.assertEqual(len(crate_roots), 11) | |
| self.assertEqual(len(crate_roots), 12) |
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Summary
model_selection: candidateKmust be at least 2 with finite held-out log-likelihood and complexity.K.Krecovers known truth with computed RMSE (no hardcoded recovery numbers).Claim boundary
Kfor every corpus.#45still owns0007).topic_measurement,psychometric_core,compute_backend,concept_dictionary).Test plan
cargo test -p model_selection --offline --lib --testscargo clippy -p model_selection --all-targets --offline -- -D warningspython3 scripts/check_docstrings.pypython3 scripts/check_workspace_contract.pypython3 scripts/validate_documentation.pySummary by CodeRabbit