test(essay): add governed facets synthetic recovery evidence

[seonghobae]

Buyer-visible scientific gap

Issue #397 requires essay-scoring calibration evidence that treats human/automated scorers as fallible raters and demonstrates parameter recovery rather than relying on agreement/correlation alone. The paired rating-range diagnostic from #748 is protected-integrated, so this Draft evaluates the existing governed scoring-facets estimator without adding a new likelihood or provider integration.

Governed synthetic recovery slice

The deterministic simulation:

• generates ordinal ratings from the repository's existing MFRM/RSM equation with injected respondent standing, task difficulty, shared thresholds, and two asymmetric rater severities;
• constructs source-text-free governed criterion rating records and assembles them through build_scoring_facets_calibration_bundle;
• fits only through fit_scoring_facets_design, which delegates production estimation to the existing Rust facets kernel;
• aligns estimates to the governed task/rater/respondent axes;
• reports and gates test-only bias, MAE, and RMSE for rater severity, task difficulty, and respondent EAP traits;
• checks recovered rater-severity and task-difficulty ordering; and
• keeps simulation/evaluation arithmetic in the test layer. No production Python likelihood, gradient, Hessian, quadrature, optimization, or uncertainty arithmetic is introduced.

Exact-current scientific evidence

Current exact head: b88eb9ad22f7d0cccb104aa4154e42f0915daed3 over protected main c91ae210bc8e0340a5b0587cf5289b82a2e8ab92.

CI run 31551770403 reached the complete Python 3.14 suite. The new governed facets synthetic-recovery test is GREEN on that exact head. Rust/PyO3, package, fuzz, explicit GPU smoke/parity, Security Scan, and SAST also succeeded. The Python aggregate result is 5 failed, 3459 passed, 2 skipped; the five failures are inherited protected-main contract inconsistencies outside this two-file scientific slice:

1. canonical documentation coverage lacks the current ACTIVE PR vocabulary expected by its contract;
2. canonical documentation coverage lacks the expected IMPLEMENTED / PLANNED extensions wording;
3. stale S-X² test expects an obsolete NumPy fallback despite protected-main Rust fail-closed ownership;
4. the related S-X² spatial/dimension-floor test has the same obsolete fallback expectation; and
5. stale person-fit test expects Python fallback despite protected-main Rust ownership.

This evidence therefore supports the bounded recovery test itself but does not establish repository-wide merge readiness. The scientific thresholds were not loosened to obtain GREEN.

Scientific basis

Uto, M., & Aramaki, K. (2024). Linking essay-writing tests using many-facet models and neural automated essay scoring. Behavior Research Methods, 56, 8450–8479. https://doi.org/10.3758/s13428-024-02485-2

The paper explicitly models task difficulty and rater severity in a many-facet essay-scoring setting. This PR does not claim that the repository's current baseline models rater consistency/range restriction, establishes construct validity, or authorizes high-stakes deployment.

Merge boundary

Keep Draft until the protected-main baseline correction integrates. Regenerate every base-sensitive exact-head gate on the resulting live base, require the recovery evidence to remain GREEN without threshold weakening, and merge only when the unchanged head satisfies all live governance and review requirements. No release bump is included.

Advances #397.

[coderabbitai]

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 27 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0bb6e0da-82c9-4790-8b49-048347322fc4

📥 Commits

Reviewing files that changed from the base of the PR and between c91ae21 and b88eb9a.

📒 Files selected for processing (2)

• docs/doctoring/essay-facets-synthetic-recovery.md
• tests/test_scoring_essay_facets_synthetic_recovery.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[seonghobae]

Superseded by GREEN on current main.