Skip to content

Add DIMTEST (Stout 1987 / Nandakumar & Stout 1993) essential-unidimensionality test - #258

Merged
seonghobae merged 2 commits into
seonghobae-ci-classifyfrom
seonghobae-dimtest
Jul 26, 2026
Merged

Add DIMTEST (Stout 1987 / Nandakumar & Stout 1993) essential-unidimensionality test#258
seonghobae merged 2 commits into
seonghobae-ci-classifyfrom
seonghobae-dimtest

Conversation

@seonghobae

@seonghobae seonghobae commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Implements DIMTEST (Stout, 1987, as refined by Nandakumar & Stout, 1993) — the confirmatory test of essential unidimensionality — as iteration 27 of the paper-implementation series. Stacked on #257 (ci_classify).

Scope (spec-verify: REDUCED-SCOPE APPROVED)

  • Caller-supplied confirmatory AT1/AT2 partitions (equal length >= 4, disjoint); PT = complement.
  • Group examinees by raw PT score; discard groups with J_k < 20.
  • ML (J_k) variance denominators; refined bias-correction denominator S_k^2 = [(mu4_k - sigma_k^4) + delta4_k + 2*sqrt((mu4_k - sigma_k^4)*delta4_k)]/J_k.
  • T_L = K^{-1/2} * sum_k (sigma_k^2 - sigma_U,k^2)/S_k on AT1; T_B analogously on AT2; T = (T_L - T_B)/sqrt(2); one-sided p = 1 - Phi(T).
  • Explicitly OUT: ATFIND, DIMTEST 2 bootstrap (Froelich & Habing, 2008), polytomous, missing data.

Citation governance

  • READ (formulas transcribed): Nandakumar & Stout 1992 ERIC tech report ED351383 (pre-publication of the 1993 JEBS paper); Kieftenbeld & Nandakumar 2015 (original-vs-bootstrap distinction).
  • NOT READ: Stout (1987) original; Stout et al. (2001); Froelich & Habing (2008). Code cites "Stout (1987) as described by Nandakumar & Stout".
  • No open-source DIMTEST implementation exists to compare against (sirt = DETECT; ltm::unidimTest = modified parallel analysis); oracle is an independent Python implementation.

Evidence

  • Pinned oracle (500x18 2D fixture, rho=0.30, seed 20260726): t_l=8.5848469411043151, t_b=3.6579307315481961, t=3.4838558621150524, p=0.000247122791999742, groups_used=8, n_discarded=17. Rust matches at 1e-12 (p at 5e-7 — crate erfc is the Numerical Recipes approx, |err|<1.2e-7; disclosed in tests).
  • Exact-null bits fixture: AT columns = distinct bits of 0..31 give exactly zero pairwise covariance -> t_l = t_b = t = 0 exactly.
  • 5/5 mutation kills EXECUTED: (M1) sign swap in sigma^2 - sigma_U^2; (M2) J_k-1 denominators; (M3) dropped bias-correction term; (M4) Jmin 20->2; (M5) two-sided p. Each fails the pinned oracle/null tests; suite restored green (12 passed).
  • 500-rep Monte Carlo size/power test (#[ignore]) executed: size < 0.10 at nominal 0.05, power > 0.8 for the 2D alternative.
  • Python: TestDimtest 5 passed; validate-before-cast (complex laundering, exact 0/1, index range/overlap) before any dtype conversion.

Adversarial impl-review outcome

  • Round 1 (FINDINGS, 2 MAJOR): (1) Python wrapper validated after dtype casts — string responses/indices silently accepted; (2) Rust tests omitted the spec-required per-group oracle intermediates.
  • Fixes (93e56e8): dtype-kind checks before any astype coercion (+ string-rejection regression tests); grouping and per-group computation factored into dimtest_pt_groups/dimtest_group_diag, with a new test pinning all 8 groups x 2 subtests x 8 intermediates (J_k, mean, sigma_k^2, sigma_U,k^2, mu4_k, delta4_k, S_k^2, contribution) at 1e-12 against the independent oracle.
  • Round 2: CLEAN. Re-verified: refactor preserves numeric operation order; per-group literals match the spec oracle digit-for-digit; bool/int/float still accepted; 13 Rust + 6 Python tests pass.

seonghobae and others added 2 commits July 26, 2026 01:31
…ensionality test

Implements the original (non-bootstrap) DIMTEST with caller-supplied
confirmatory AT1/AT2 partitions, PT raw-score grouping (J_k >= 20),
ML variance denominators, the Nandakumar & Stout (1992/1993) refined
bias-correction denominator S_k, T_L/T_B statistics, and one-sided
p-value. Formulas transcribed from the inspected ERIC ED351383
technical report (pre-publication of Nandakumar & Stout, 1993, JEBS).

- Rust core in mlsirm-core detect.rs with pinned 17-digit oracle
  (independent Python reference), exact-null bits fixture, error
  paths, and 500-rep Monte Carlo size/power test (#[ignore]).
- 5/5 mutation kills executed (sign swap, J_k-1 denominator, dropped
  bias correction, Jmin relaxation, two-sided p).
- PyO3 binding py_dimtest + thin Python wrapper with validate-before-
  cast input checks; exports and pytest coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…group oracle anchors

- Python wrapper now rejects non-numeric (e.g. string) response and index
  arrays via dtype-kind checks BEFORE any astype coercion, with regression
  tests for string responses and string indices.
- Rust: factor dimtest grouping (dimtest_pt_groups) and per-group
  computation (dimtest_group_diag) out of dimtest_stat, and add a test
  pinning all 8x2 per-group intermediates (J_k, mean, sigma_k^2,
  sigma_U,k^2, mu4_k, delta4_k, S_k^2, contribution) for AT1 and AT2
  against the independent oracle at 1e-12, closing the compensating-error
  audit gap.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@seonghobae
seonghobae merged commit dc6f32e into seonghobae-ci-classify Jul 26, 2026
5 checks passed
@seonghobae
seonghobae deleted the seonghobae-dimtest branch July 26, 2026 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant