feat: Wald SPRT classification for CAT (single-cut binary, reduced scope) - #256
Merged
Conversation
…ope) Single-cut binary-response sequential probability ratio test for IRT classification testing: point hypotheses at theta_cut -/+ delta, cumulative binary log-likelihood ratio under the D=1 logistic 3PL, inclusive first-crossing decisions against log Wald boundaries A=ln((1-beta)/alpha), B=ln(beta/(1-alpha)) -> above/below/continue with 1-based n_used and a full counterfactual llr_trace diagnostic. Verified against READ sources: R catIrt termSPRT.R/logLik.brm.R/p.brm.R and Thompson (2007, doi:10.7275/fq3r-zz60, p. 7). Reckase (1983) and Eggen (1999) cited as historical origins via Thompson (NOT directly read). Adversarial spec review verdict: REDUCED-SCOPE (no multi-cut, polytomous, or D=1.7 compatibility). Tests: pinned 17-digit interior-crossing oracle (also the final-LLR-only discriminator), below-decision first-crossing invariants on a heterogeneous pool, continue path, full error-path coverage, 500-rep Monte-Carlo structural invariants (#[ignore]). Executed mutation kills: swapped boundaries (2 fail), dropped guessing floor (1), collapsed null hypothesis theta0=theta_cut (2), off-by-one n_used (1); restore verified green. Python wrapper validates before casts (complex laundering, exact 0/1 responses) and returns crate outputs only. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…1947) NOT read Adversarial impl review found that extreme-but-valid parameters (e.g. a = 50, delta = 20 -> z = +/-1000) saturate the 2PL response probability to numerical 0/1, making ln(p1/p0) or ln((1-p1)/(1-p0)) non-finite and returning a spurious Err. Rewrite the increments in stable log space: ln(1-P) = ln(1-c) - softplus(z) (the ln(1-c) terms cancel in the wrong- response increment) and ln(P) = -softplus(-z) when c = 0 (for c > 0 the direct form is bounded below by c). Regression test pins the finite +/-1000 LLRs and the ln-bounded c > 0 case, reading crate outputs. Also mark Wald (1947) as NOT read in the Python docstring, PyO3 docstring, and CHANGELOG (it already was in the core module comment). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements single-cut binary-response Wald SPRT classification for CAT (iteration 25 of the paper-implementation loop). Stacked on #255 (EPV). ## Contract (adversarial spec review: REDUCED-SCOPE) -
sprt_classify(a, b, c, responses, theta_cut, delta, alpha, beta)- D = 1 logistic 3PLP_i = c_i + (1-c_i)/(1+exp(-a_i(theta-b_i))); point hypothesestheta0 = theta_cut - delta,theta1 = theta_cut + delta-LLR_k = sum u_i ln(P_i(t1)/P_i(t0)) + (1-u_i) ln((1-P_i(t1))/(1-P_i(t0)))- Log Wald boundariesA = ln((1-beta)/alpha),B = ln(beta/(1-alpha)); INCLUSIVE first-crossing decisions (catIrt convention):>=A-> "above",<=B-> "below", else "continue";n_used1-based - Fullllr_tracereturned as offline diagnostic; entries pastn_useddocumented as counterfactual replay - NOT in scope (per spec review): multi-cut, polytomous, D=1.7 compatibility layer ## Source status (citation governance) READ: R catIrttermSPRT.R/logLik.brm.R/p.brm.R(github swnydick/catIrt); Thompson (2007), PARE 12(1), doi:10.7275/fq3r-zz60, p. 7. NOT read: Reckase (1983), Eggen (1999), Wald (1947) — cited as historical origins via Thompson, marked in code. ## Evidence - Pinned 17-digit oracle (homogeneous 5-item pool, interior crossing at k=4 with counterfactual 5th entry): decision "above", n_used 4, final LLR 2.1826881566381821 — this single oracle also kills the "decide on final LLR only" mutant (final value is inside the band). - Below-decision first-crossing invariants on a heterogeneous 8-item pool; continue path; full error-path coverage (incl.alpha + beta >= 1). - 500-rep Monte-Carlo structural invariants#[ignore]: trace finiteness/length, no crossing beforen_used, crossing entry beyond the matching boundary, continue iff never leaves the open band. PASSED. - 4 executed mutation kills (temp-file backup/restore, green after restore): M1 swapped boundary decisions (2 tests fail), M2 dropped guessing floor (1), M3 collapsed null hypothesistheta0 = theta_cut(2), M4 off-by-onen_used(1). - Rust: 8 passed (sprt+epv filters); pytest: 9 passed (TestSprtClassify 5 + TestEpvSelect 4). PyO3 binding smoke-verified against the oracle. - Python wrapper validates BEFORE dtype casts: complex-input rejection, exact 0/1 response check (no truncation/wrap laundering), bool accepted. Every test assert reads crate-returned values (decision,n_used,llr_trace,llr).Adversarial impl-review outcome (commit 39ff8dc)
ln(1-P) = ln(1-c) - softplus(z), with theln(1-c)terms cancelling;ln(P) = -softplus(-z)for c=0). Regression test pins the finite +/-1000 LLRs and a c=0.2 saturation case, reading crate outputs.Post-fix verification: rust sprt 5 passed + MC-500 passed; pyd rebuilt; pytest 5 passed. Verdict: CLEAN.