feat(ksirt): kernel-smoothing nonparametric IRT (Ramsay 1991)

[seonghobae]

Summary

Iteration 3 of the autonomous paper-implementation loop (stacked on #219): kernel-smoothing nonparametric IRT (Ramsay, 1991) as mlsirm_core::ksirt + fast_mlsirm.ksirt_analysis.

Estimates option characteristic curves (OCCs) by Nadaraya–Watson kernel regression of option indicators on rank-based ordinal ability estimates, plus expected item score and expected total score curves. For LLM-as-a-Judge item-quality management this reveals non-monotone or poorly discriminating evaluation items without assuming a parametric response model.

Algorithm (all verified against read sources)

Component	Formula	Source
Ability estimates	theta_i = qnorm(rank(total_i)/(n+1)), ties first (stable sort, original order within ties)	ksIRT.R:121,127; JSS §2.1
Evaluation grid	q equally spaced points qnorm(1/(n+1)) … qnorm(n/(n+1))	ksIRT.R:134-141
Default bandwidth	Silverman 1.06 · n^(−1/5) (σ=1 on the normal metric)	ksIRT.R:171-179; JSS Eq. 5
NW weights	K((θ_s−θ̂_i)/h) / Σ_i K(·); gaussian / quadratic / uniform	smoother3.cpp:94-130
Zero denominator	OCC row → 0	smoother3.cpp:140-148
Expected score	e_j(θ_s) = Σ_l x_jl · p̂_jl(θ_s); total = Σ_j e_j	ksIRT.R:241

Source verification: READ = Mazza, Punzo & McGuire (2014), JSS 58(6), doi:10.18637/jss.v058.i06 (Sections 2–2.3) + KernSmoothIRT R/C++ source (ksIRT.R, smoother3.cpp, CrossV.cpp, getweight.R) line-by-line. NOT READ (cited only as-cited-in) = Ramsay (1991), Silverman (1986), Nadaraya (1964), Watson (1964).

Adversarial spec review → REDUCED-SCOPE (applied)

• Standard errors CUT from v1: the R implementation's SE accumulator uses order-dependent partial sums (smoother3.cpp:148-150) and disagrees with JSS Eq. 6; no verifiable closed form from read sources.
• CV bandwidth, missing data, DIF, answer keys: out of scope.
• Options are sorted ascending (deliberate divergence from R's first-seen order; math unaffected).
• Row-sums-to-one claimed only for positive NW denominators.

Test evidence

Every assert reads crate outputs against independently derived constants; each test documents "crate value read" + "mutant killed".

Check	Result
Rust unit tests (10 + 1 #[ignore] MC)	✅ pass; crate suite 447 passed
Hand fixture (n=4): rank→θ qnorm literals, grid endpoints, Silverman constant 1.06·4^(−1/5)	✅ exact to 1e-8
Mutation A: .rev() theta iteration	✅ 4 tests FAIL
Mutation B: rank denominator n+1 → n	✅ 9 tests FAIL
Mutation C: Silverman 1.06 → 1.0	✅ 1 test FAILs
500-rep Monte Carlo (2PL, normal + skewed abilities, rank-invariance composition oracle)	✅ RMSE < 0.06 both
Python tests (fixture exactness, flat-OCC marginal under huge bandwidth, monotone recovery, guards)	✅ 2 passed

Unkillable mutants (documented, not claimed): kernel-argument sign flip (symmetric kernels) and multiplicative kernel constants (cancelled by NW normalization) are identities; the test header names them and the discriminating anchors that would require asymmetric kernels.

Adversarial impl review → 2 confirmed defects, both fixed

1. Unbounded nevalpoints could abort the interpreter via Rust handle_alloc_error → wrapper now caps at 100 000 (ValueError), with a guard test.
2. A test comment's "hand-computed" Silverman digits were wrong at the 8th significant figure (assert was correct) → comment corrected.

Stacking

Base = seonghobae-mokken-scale-analysis (#219). Merge order: #218 → #219 → this.

[Copilot]

Pull request overview

Adds a new nonparametric, kernel-smoothed IRT analysis path (ksIRT) to the Rust core and exposes it through a thin Python wrapper, enabling OCC/expected-score curves derived from rank-based ordinal ability estimates for item-quality diagnostics.

Changes:

• Implement mlsirm_core::ksirt kernel-smoothing OCC estimation (Gaussian/quadratic/uniform) with rank→normal-quantile ability surrogates and Silverman-rule bandwidths.
• Add PyO3 binding (ksirt_occ) and Python API (fast_mlsirm.ksirt_analysis, KsirtResult) with validation and marshaling.
• Add Rust + Python tests that pin fixture exactness and basic recovery/guard behavior; update changelog.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/unit/ksirt_tests.rs	New Rust unit tests with hand fixtures and invariants for ksIRT outputs.
tests/test_paper_features.py	Python-level fixture/recovery tests and wrapper validation guards for ksirt_analysis.
python/fast_mlsirm/ksirt.py	New Python wrapper (ksirt_analysis) + KsirtResult dataclass.
python/fast_mlsirm/init.py	Exposes ksirt_analysis / KsirtResult from the package root.
crates/mlsirm-core/src/mokken.rs	Widens visibility of normal_upper_quantile for reuse by ksIRT.
crates/mlsirm-core/src/lib.rs	Registers the new ksirt Rust module.
crates/mlsirm-core/src/ksirt.rs	New Rust implementation of ksIRT OCC smoothing and expected-score curves.
crates/fast-mlsirm-py/src/lib.rs	Adds PyO3 ksirt_occ binding returning dict outputs to Python.
CHANGELOG.md	Documents the new kernel-smoothing nonparametric IRT feature and scope limits.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[seonghobae]

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply the suggestion exactly.

Do not make changes beyond what is described in the linked review thread.

[Copilot]

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested ch...

Addressed in 1589168.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.