Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,38 @@

### Added

- **Wald SPRT classification for CAT** (`fast_mlsirm.sprt_classify`; in
`mlsirm_core::exposure`). Single-cut binary-response sequential probability
ratio test: point hypotheses at `theta_cut -/+ delta`, cumulative binary
log-likelihood ratio under the D=1 logistic 3PL, and inclusive
first-crossing decisions against the log Wald boundaries
`A = ln((1-beta)/alpha)`, `B = ln(beta/(1-alpha))` -> `"above"`/`"below"`/
`"continue"` with 1-based `n_used`; the full `llr_trace` is returned as an
offline diagnostic (entries past `n_used` are counterfactual replay
Comment thread
seonghobae marked this conversation as resolved.
Resolved
values). Verified against R catIrt `termSPRT.R`/`logLik.brm.R`/`p.brm.R`
and Thompson (2007, doi:10.7275/fq3r-zz60); Reckase (1983), Eggen (1999),
and Wald (1947) are cited as historical origins via Thompson (not directly
read). Log-likelihood ratios are computed in stable log space (softplus /
log-sigmoid), so extreme-but-valid parameters that saturate the response
probability to numerical 0/1 yield finite LLRs instead of errors. Pinned
17-digit interior-crossing oracle, error-path and 500-rep Monte-Carlo
structural-invariant tests; 4 executed mutation kills (swapped boundaries,
dropped guessing floor, collapsed null hypothesis, off-by-one `n_used`).
- **Owen-approximate posterior-predictive EPV item selection**
(`fast_mlsirm.epv_select`; in `mlsirm_core::exposure`). Deliberately
reduced scope of van der Linden's (1998, doi:10.1007/BF02294775) minimum
expected posterior variance (MEPV) criterion: the posterior is Owen's
normal approximation `N(mu, sig2)`, the predictive probability is
Comment thread
seonghobae marked this conversation as resolved.
Resolved
`p*_i = c_i + (1-c_i) Phi((mu-b_i)/sqrt(1/a_i^2 + sig2))`, and the outcome
posterior variances come from `owen_update` rather than exact numerical
posteriors; the unadministered item minimizing
`EPV_i = p*_i sig2_i^+ + (1-p*_i) sig2_i^-` is selected (lowest-index
ties). van der Linden (1998) READ as ERIC ED424235 (Research Report
96-01); the exact-MEPV contract additionally verified against R catR
`EPV.R` and mirtCAT `selection_criteria.R` (both READ); Owen (1975) NOT
read (update formulas follow the crate's `owen_update`). Pinned oracles
and a delegation discriminator (argmin EPV vs. max-info vs. b-matching)
Comment thread
seonghobae marked this conversation as resolved.
Resolved
fixed by the adversarial spec review.
- **Kingsbury-Zara constrained CAT (CCAT) content balancing**
(`fast_mlsirm.ccat_select`; in `mlsirm_core::exposure`). Single-step
content-balanced item selection: eligible groups with zero administered
Expand All @@ -139,7 +171,7 @@
b-matching selection (`argmin |b_i - mu|`, ties to the lowest index) and
posterior-variance stopping rule (plus a `test_length` cap). Owen (1975)
itself NOT read (paywalled); formulas implemented as reproduced by
van der Linden (1998, Research Report 98-01, Appendix A.1-A.6) and
van der Linden (1998, Research Report 96-01, Appendix A.1-A.6) and
cross-checked against the R `irt` package `est_ability_owen.cpp`; pinned
oracles verified against exact-posterior numerical integration (~1e-13)
by the adversarial spec review.
Expand Down
109 changes: 106 additions & 3 deletions crates/fast-mlsirm-py/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,9 @@ use mlsirm_core::dif::{
};
use mlsirm_core::exposure::{
a_stratified as core_a_stratified, ccat_select as core_ccat_select,
kl_information as core_kl_information, kl_select as core_kl_select, owen_cat as core_owen_cat,
owen_update as core_owen_update, sympson_hetter as core_sympson_hetter, AStratifiedConfig,
epv_select as core_epv_select, kl_information as core_kl_information,
kl_select as core_kl_select, owen_cat as core_owen_cat, owen_update as core_owen_update,
sprt_classify as core_sprt_classify, sympson_hetter as core_sympson_hetter, AStratifiedConfig,
SympsonHetterConfig,
};
use mlsirm_core::facets::fit_facets as core_fit_facets;
Expand Down Expand Up @@ -2274,7 +2275,7 @@ fn py_kl_select(
/// Statistical Association, 70(350), 351-356.
/// https://doi.org/10.1080/01621459.1975.10479871
/// van der Linden, W. J. (1998). Bayesian item selection criteria for
/// adaptive testing (Research Report 98-01). University of Twente.
/// adaptive testing (Research Report 96-01). University of Twente.
#[pyfunction]
fn py_owen_update(
a: f64,
Expand Down Expand Up @@ -2381,6 +2382,106 @@ fn py_ccat_select(
Ok(out.into())
}

/// Owen-approximate posterior-predictive expected posterior variance (EPV)
/// item selection (`mlsirm_core::exposure::epv_select`). This is NOT van der
/// Linden's (1998) exact MEPV criterion: the posterior is Owen's (1975) normal
/// approximation N(mu, sig2), the predictive probability is
/// p*_i = c_i + (1 - c_i) * Phi(a_i (mu - b_i) / sqrt(1 + a_i^2 sig2)), and
/// the outcome variances come from `owen_update` rather than exact numerical
/// posteriors. Selects the unadministered item minimizing
/// EPV_i = p*_i sig2_i^+ + (1 - p*_i) sig2_i^-; ties go to the lowest index.
/// Returns {selected, epv, predictive}.
///
/// References:
/// van der Linden, W. J. (1998). Bayesian item selection criteria for
/// adaptive testing. Psychometrika, 63(2), 201-216.
/// https://doi.org/10.1007/BF02294775 (read as ERIC ED424235 research
/// report; the exact-MEPV contract was verified against catR EPV.R and
/// mirtCAT, and this routine deliberately substitutes Owen updates).
/// Owen, R. J. (1975). A Bayesian sequential procedure for quantal response
/// in the context of adaptive mental testing. Journal of the American
/// Statistical Association, 70(350), 351-356. (NOT read; update formulas
/// follow the crate's `owen_update`.)
#[pyfunction]
fn py_epv_select(
py: Python<'_>,
a: PyReadonlyArray1<'_, f64>,
b: PyReadonlyArray1<'_, f64>,
c: PyReadonlyArray1<'_, f64>,
administered: PyReadonlyArray1<'_, bool>,
mu: f64,
sig2: f64,
) -> PyResult<Py<pyo3::types::PyDict>> {
let res = core_epv_select(
a.as_slice()?,
b.as_slice()?,
c.as_slice()?,
administered.as_slice()?,
mu,
sig2,
)
.map_err(PyValueError::new_err)?;
let out = pyo3::types::PyDict::new(py);
out.set_item("selected", res.selected)?;
out.set_item("epv", numpy::PyArray1::from_slice(py, &res.epv))?;
out.set_item(
"predictive",
numpy::PyArray1::from_slice(py, &res.predictive),
)?;
Ok(out.into())
}

/// Single-cut binary-response Wald SPRT classification for CAT
/// (`mlsirm_core::exposure::sprt_classify`). D = 1 logistic 3PL; point
/// hypotheses at `theta_cut -/+ delta`; log Wald boundaries
/// A = ln((1-beta)/alpha), B = ln(beta/(1-alpha)) with inclusive
/// first-crossing decisions ("above"/"below"/"continue"). `llr_trace`
/// entries past `n_used` are offline counterfactual replay values.
///
/// References (APA 7th; see the core module comment for read/not-read
/// source status):
/// Thompson, N. A. (2007). A practitioner's guide for variable-length
/// computerized classification testing. Practical Assessment, Research &
/// Evaluation, 12(1). https://doi.org/10.7275/fq3r-zz60 (READ)
/// Nydick, S. W. (2014). catIrt (R package). (READ: termSPRT.R,
/// logLik.brm.R, p.brm.R)
/// Eggen, T. J. H. M. (1999). Applied Psychological Measurement, 23(3),
/// 249-261. (NOT read; historical citation via Thompson)
/// Reckase, M. D. (1983). A procedure for decision making using tailored
/// testing. (NOT read; historical citation via Thompson)
/// Wald, A. (1947). Sequential analysis. Wiley. (NOT read; boundary forms
/// verified through the READ sources above)
#[pyfunction]
fn py_sprt_classify(
py: Python<'_>,
a: PyReadonlyArray1<'_, f64>,
b: PyReadonlyArray1<'_, f64>,
c: PyReadonlyArray1<'_, f64>,
responses: PyReadonlyArray1<'_, u8>,
theta_cut: f64,
delta: f64,
alpha: f64,
beta: f64,
) -> PyResult<Py<pyo3::types::PyDict>> {
let res = core_sprt_classify(
a.as_slice()?,
b.as_slice()?,
c.as_slice()?,
responses.as_slice()?,
theta_cut,
delta,
alpha,
beta,
)
.map_err(PyValueError::new_err)?;
let out = pyo3::types::PyDict::new(py);
out.set_item("decision", res.decision)?;
out.set_item("n_used", res.n_used)?;
out.set_item("llr", res.llr)?;
out.set_item("llr_trace", numpy::PyArray1::from_slice(py, &res.llr_trace))?;
Ok(out.into())
}

/// Horn's parallel analysis for principal-component retention
/// (`mlsirm_core::parallel`; oracle: CRAN paran 1.5.6, PCA path). `data` is
/// a flattened row-major `n_persons * n_items` matrix; `centile` is 0 for
Expand Down Expand Up @@ -6200,6 +6301,8 @@ fn fast_mlsirm_core(m: &Bound<'_, PyModule>) -> PyResult<()> {
m.add_function(wrap_pyfunction!(py_owen_update, m)?)?;
m.add_function(wrap_pyfunction!(py_owen_cat, m)?)?;
m.add_function(wrap_pyfunction!(py_ccat_select, m)?)?;
m.add_function(wrap_pyfunction!(py_epv_select, m)?)?;
m.add_function(wrap_pyfunction!(py_sprt_classify, m)?)?;
m.add_function(wrap_pyfunction!(guttman_lambdas, m)?)?;
m.add_function(wrap_pyfunction!(tenberge_mu, m)?)?;
m.add_function(wrap_pyfunction!(cronbach_alpha, m)?)?;
Expand Down
Loading
Loading