feat: Owen-approximate posterior-predictive EPV item selection (van der Linden 1998, reduced scope) - #255
feat: Owen-approximate posterior-predictive EPV item selection (van der Linden 1998, reduced scope)#255seonghobae wants to merge 2 commits into
Conversation
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headb2a711a40544ea21465e028a1e233ed0eaa4713c. -
Head SHA:
b2a711a40544ea21465e028a1e233ed0eaa4713c -
Workflow run: 30159258014
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (5 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (5 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Test (2 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (2 files)"]
R2 --> V2["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage Decision
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (5 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (5 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Test (2 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (2 files)"]
R2 --> V2["targeted test run"]
|
4c81ea9 to
efe1acf
Compare
…er Linden, 1998; reduced scope) Adds epv_select to mlsirm_core::exposure: for each unadministered item, the predictive success probability p* = c + (1-c) Phi((mu-b)/sqrt(1/a^2+sig2)) weights the two Owen-update outcome posterior variances, EPV = p* sig2+ + (1-p*) sig2-, and the argmin (lowest-index ties) is selected. Explicitly NOT vdL-98 exact MEPV (which uses point probabilities and exact numerical posteriors); the reduced scope is documented in code, docstring, and CHANGELOG per the adversarial spec review. PyO3 binding py_epv_select + Python wrapper fast_mlsirm.epv_select (arrays validated before lossy casts). Tests: pinned crate-parity oracles (5-item pool, 17-digit spec-review values, 5e-7 abs), delegation discriminator (argmin EPV=2 vs max-info=4 vs b-matching=1), administered-masking invariance, error paths, and a 500-rep Monte-Carlo invariant suite (#[ignore]). Mutation kills executed: argmin->argmax (3 fail), drop guessing floor (2 fail), swap sig2+/- weights (2 fail), posterior-predictive->point probability (2 fail). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…citation Adversarial impl-review findings: (1) complex inputs were laundered through the float64 cast, silently discarding imaginary parts — now rejected with ValueError before any cast, regression-tested; (2) the ERIC ED424235 research-report citation said 98-01 but the read source is Research Report 96-01. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
b2a711a to
3d14e0e
Compare
1 participant
Iteration 24 of the autonomous paper-implementation loop. Stacked on #254 (seonghobae-ccat). ## What Owen-approximate posterior-predictive EPV item selection —
fast_mlsirm.epv_select, Rust coremlsirm_core::exposure::epv_select(+ PyO3py_epv_select). For each unadministered item under the Owen normal posteriorN(mu, sig2): - predictive success probabilityp*_i = c_i + (1 - c_i) Phi((mu - b_i) / sqrt(1/a_i^2 + sig2))(identicaldtoowen_update) -EPV_i = p*_i sig2_i^+ + (1 - p*_i) sig2_i^-with both outcome variances fromowen_update- select argmin over unadministered items; lowest-index ties;epv/predictivereturned for the whole pool. ## Adversarial spec-verify (BEFORE implementing): REDUCED-SCOPE The reviewer confirmed the posterior-predictive formula is mathematically correct but ruled that van der Linden (1998), catREPV.R, and mirtCAT define MEPV with point probabilitiesP_i(mu)and exact numerical posterior variances, not posterior-predictive weights + Owen updates. Accepted contract: an explicitly labeled "Owen-approximate posterior-predictive EPV" with prescribed doc wording (used verbatim in code, docstring, and CHANGELOG). This is deliberately NOT exact vdL-98 MEPV. Source status: van der Linden (1998) READ as ERIC ED424235 (Research Report 96-01); catREPV.Rand mirtCATselection_criteria.RREAD; Owen (1975) and the Psychometrika body NOT read (documented in code). ## Evidence - Pinned crate-parity oracles (spec-review exact arithmetic, 17 digits, 5e-7 abs tol) for bothpredictiveandepvover a 5-item pool; argmin = item 1. - Delegation discriminator (spec-review pool): argmin EPV = 2 while max-Fisher-info = 4 and Owen b-matching = 1 — a delegating mutant cannot satisfy all three. - Administered masking: scoring covers the whole pool; masking affects selection only (runner-up promotion pinned, vectors unchanged to 1e-15). - Error paths: empty pool, length mismatch, nonpositivea, non-finiteb/c/mu,cout of [0,1),sig2 <= 0, all-administered, degenerate Owen posterior propagation. - MC-500#[ignore]invariant suite: predictive finite in[c_i, 1](closed — erfc saturation), EPV finite > 0, selected always unadministered, ties lowest-index. (StrictEPV < sig2deliberately NOT asserted: single-outcome Owen variance can genuinely increase and roundoff inflation of 8.9e-16 was observed.) - 4 executed mutation kills (temp-file backup/restore, re-run verified green): - M1 argmin -> argmax: 3 tests fail - M2 drop guessing floor (p* = Phi(d)): 2 fail - M3 swapsig2+/sig2-weights: 2 fail - M5 posterior-predictive -> point probabilityP_i(mu): 2 fail - Python wrapper validates arrays BEFORE lossy casts; 4 pytest parity tests green.cargo test -p mlsirm-core --lib -- epv: 4 passed (+1 ignored MC-500 passed with--ignored).pytest -k EpvSelect: 4 passed. ## References - van der Linden, W. J. (1998). Bayesian item selection criteria for adaptive testing. Psychometrika, 63(2), 201-216. https://doi.org/10.1007/BF02294775 - Owen, R. J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. JASA, 70(350), 351-356. - Magis, D., & Raiche, G. (2012). catR. JSS, 48(8). https://doi.org/10.18637/jss.v048.i08Adversarial impl-review (AFTER implementing): CLEAN after fixes (b2a711a)
a/b/cwere laundered through the float64 cast (imaginary parts silently dropped). Fixed:np.iscomplexobjrejection before any cast + 3 regression error-path tests.