Skip to content

Add Thurstone (1927) Case V paired-comparison scaling - #290

Merged
seonghobae merged 2 commits into
mainfrom
seonghobae-thurstone
Jul 31, 2026
Merged

Add Thurstone (1927) Case V paired-comparison scaling#290
seonghobae merged 2 commits into
mainfrom
seonghobae-thurstone

Conversation

@seonghobae

@seonghobae seonghobae commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Thurstone (1927) Case V paired-comparison scaling

Iteration 51 of the autonomous paper-implementation loop.

What

mlsirm_core::scaling::thurstone_case_v (NEW module) + PyO3 binding + fast_mlsirm.thurstone_case_v wrapper returning a ThurstoneResult dataclass (scale, gof, model, residual).

Given an n x n choice-probability matrix (choice[i,j] = P(column j preferred over row i), psych convention):

  • z = qnorm(choice); scale_j = colmean_j(z) - min_k colmean_k(z) (minimum exactly 0)
  • fitted model model_ij = pnorm(scale_j - scale_i); residual = model - choice
  • goodness of fit GF = 1 - sse/ssc over the full model matrix including the diagonal — this pins the psych code behavior; the .Rd prose claiming "lower off diagonal" is stale (lower-only GF on fixture A would be 0.99868280449920059, numerically distinct from the pinned 0.99986967677023893).

Citation governance

  • READ: psych thurstone.R, matrix.addition.R, thurstone.Rd (Revelle). The %+% argument was hand-derived: -scale %+% t(scale) gives S_j - S_i (reviewer independently confirmed).
  • NOT READ (as-cited): Thurstone (1927), Psychological Review 34(4), 273-286 — original unacquirable; cited as the model origin per the psych source.
  • Deliberate divergence: entries must be strictly in (0, 1). psych's direct path admits qnorm(0)/qnorm(1) = +-Inf; we reject boundaries as a safety contract.

Verification

  • Oracle: mpmath 50-digit, EXECUTED. Fixtures: A (3x3 asymmetric, nonzero residuals), B (2x2 exactly Case-V-consistent, GF = 1 round-trip), C (4x4 intransitive, min colmean not in first column).
  • Spec-verify (adversarial, before implementation): APPROVED-WITH-CHANGES; all 4 mandatory changes applied (full-matrix GF documented + pinned; MC scales bounded to [0,2]; strict-(0,1) divergence marked; accuracy claims fixtures-only). Reviewer recomputed all pins at 80 digits; measured Rust-kernel error <= 2.2e-8 on fixtures (pins at 1e-6).
  • Mutation kills (all EXECUTED on clean baseline, all KILLED): MU1 transpose index choice[j*n+i]; MU2 drop min-shift; MU3 sign flip S_i - S_j; MU4 GF denominator ssc without mean-centering (killed by fixture C — fixture A's delta 4.3e-7 is sub-tolerance, documented); MU5 residual sign choice - model.
  • Suites: cargo 734 pass (incl. MC-500 #[ignore] verified separately); pytest 267 pass.

Limitation (stated, not hidden)

Fixture B is zero-residual and thus variance-mutation-blind; variance-sensitive mutants are anchored by fixtures A and C nonzero-residual pins.


Adversarial impl-review outcome (post-merge-gate)

VERDICT: CLEAN (first pass, no fix cycles needed).

Independent reviewer executed: Rust scaling tests (5 pass) + MC-500 #[ignore] (pass); 12 Python-binding probes (near-boundary 1e-300/1-1e-16, 50x50 consistent round-trip max scale err 8.54e-10, Fortran-order, non-contiguous transpose, float32, int-matrix rejection, NaN rejection, asymmetric non-complementary input, 1x1/0x0 rejection, raw-binding length/n-mismatch errors); re-verified all 15 numeric pins against the mpmath 50-digit oracle; confirmed citation-governance header honesty (psych READ / Thurstone 1927 NOT READ / .Rd-vs-code GF conflict / strict-(0,1) divergence); confirmed every assert reads crate/binding outputs; rustfmt --check clean.

New module mlsirm_core::scaling with thurstone_case_v: scale values
colmean(qnorm(choice)) - min, fitted model Phi(S_j - S_i), residuals,
and psych's full-matrix goodness of fit 1 - sse/ssc (pinning the psych
CODE behavior; the .Rd 'lower off diagonal' prose is stale). Algorithm
follows psych's thurstone() (Revelle; source READ); Thurstone (1927)
NOT READ, cited as origin per the psych source. Entries must be
strictly in (0, 1) -- deliberate safety divergence from psych's direct
path which admits infinite quantiles.

Pinned against a 50-digit mpmath oracle on three fixtures (asymmetric
nonzero-residual, exactly-consistent round-trip, intransitive 4x4 with
non-first min column). Five mutation kills (MU1-MU5) executed on a
clean baseline; MC-500 consistent-recovery test under #[ignore].
PyO3 binding thurstone_case_v + Python wrapper fast_mlsirm.scaling
with ThurstoneResult dataclass.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 26, 2026 12:25
@coderabbitai

coderabbitai Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 13 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d71b5d4a-b19a-4a1d-bef3-7a0cbbc0982e

📥 Commits

Reviewing files that changed from the base of the PR and between 3c1390e and 6d1fd5b.

📒 Files selected for processing (8)
  • CHANGELOG.md
  • crates/fast-mlsirm-py/src/lib.rs
  • crates/mlsirm-core/src/lib.rs
  • crates/mlsirm-core/src/scaling.rs
  • python/fast_mlsirm/__init__.py
  • python/fast_mlsirm/scaling.py
  • tests/test_paper_features.py
  • tests/unit/scaling_tests.rs
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch seonghobae-thurstone

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Thurstone (1927) Case V paired-comparison scaling feature implemented in the Rust core, exposed through the PyO3 extension, and surfaced as a Python API returning a structured result. This fits the codebase’s pattern of implementing psych-algorithm-compatible psychometrics routines in Rust with Python wrappers and pinned oracle-based fixtures.

Changes:

  • Introduces mlsirm_core::scaling::thurstone_case_v with strict (0, 1) input validation, returning scale/model/residual and a full-matrix GF.
  • Adds PyO3 binding and a Python wrapper (fast_mlsirm.thurstone_case_v) returning a ThurstoneResult dataclass with NumPy-shaped outputs.
  • Adds Rust + Python fixture pins and error-contract tests, and documents the feature in the changelog.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/unit/scaling_tests.rs New Rust fixture pins and error-contract tests for Thurstone Case V scaling.
tests/test_paper_features.py Adds Python-level fixture pins and error tests for fast_mlsirm.thurstone_case_v.
python/fast_mlsirm/scaling.py New Python wrapper + ThurstoneResult dataclass around the Rust core implementation.
python/fast_mlsirm/init.py Exposes thurstone_case_v and ThurstoneResult at the package top level.
crates/mlsirm-core/src/scaling.rs New Rust implementation of Thurstone Case V scaling and result struct.
crates/mlsirm-core/src/lib.rs Exports the new scaling module.
crates/fast-mlsirm-py/src/lib.rs Adds the PyO3 thurstone_case_v function and registers it in the module.
CHANGELOG.md Documents the new scaling feature and its contract/fixtures.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +63 to +65
//! Revelle, W. (2024). *psych: Procedures for psychological, psychometric,
//! and personality research* (R package). Northwestern University.
//! https://CRAN.R-project.org/package=psych
@seonghobae
seonghobae merged commit 853de10 into main Jul 31, 2026
32 checks passed
@seonghobae
seonghobae deleted the seonghobae-thurstone branch July 31, 2026 12:36
seonghobae added a commit that referenced this pull request Jul 31, 2026
…374)

Integrates the remaining paired-comparison, rating, and inter-rater
agreement features from the seonghobae stack tip (seonghobae-ncohen)
onto current main after #290 squash-landed and intermediate stacked
bases could not cleanly retarget under squash rewrites.

Preserves main GPU/MT NLL path (lib.rs unchanged). Python/Rust parity
bindings and unit/paper tests included.
seonghobae added a commit that referenced this pull request Jul 31, 2026
…adge (#375)

Bump package versions after #374 lands the #290#328 paired-comparison,
rating, and inter-rater APIs on main (with #373 README DeepWiki badge).
Restore the v0.1.1 changelog section that the stack tip tree had overwritten
and document the 0.1.2 feature surface without regressing GPU/MT NLL.
cursor Bot pushed a commit to ContextualWisdomLab/.github that referenced this pull request Aug 16, 2026
The 180s NVIDIA NIM candidate timeout killed reviews in three minutes
(ContextualWisdomLab/fast-mlsirm#290). Raise NIM, cadence, dynamic-cap,
and central-fallback run timeouts to 7200s, keep GPT-5/free-tier short,
and omit gpt-5.6-terra plus github-models/* from the dispatch pool.
Leave PR-number concurrency and cancel-in-progress unchanged so the
dispatch queue cannot multiply unbounded parallel two-hour jobs.

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
cursor Bot pushed a commit to ContextualWisdomLab/.github that referenced this pull request Aug 18, 2026
The 180s NVIDIA NIM candidate timeout killed reviews in three minutes
(ContextualWisdomLab/fast-mlsirm#290). Raise NIM, cadence, dynamic-cap,
and central-fallback run timeouts to 7200s, keep GPT-5/free-tier short,
and omit gpt-5.6-terra plus github-models/* from the dispatch pool.
Leave PR-number concurrency and cancel-in-progress unchanged so the
dispatch queue cannot multiply unbounded parallel two-hour jobs.

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
seonghobae added a commit to ContextualWisdomLab/.github that referenced this pull request Aug 19, 2026
The 180s NVIDIA NIM candidate timeout killed reviews in three minutes
(ContextualWisdomLab/fast-mlsirm#290). Raise NIM, cadence, dynamic-cap,
and central-fallback run timeouts to 7200s, keep GPT-5/free-tier short,
and omit gpt-5.6-terra plus github-models/* from the dispatch pool.
Leave PR-number concurrency and cancel-in-progress unchanged so the
dispatch queue cannot multiply unbounded parallel two-hour jobs.

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
seonghobae added a commit to ContextualWisdomLab/.github that referenced this pull request Aug 20, 2026
The 180s NVIDIA NIM candidate timeout killed reviews in three minutes
(ContextualWisdomLab/fast-mlsirm#290). Raise NIM, cadence, dynamic-cap,
and central-fallback run timeouts to 7200s, keep GPT-5/free-tier short,
and omit gpt-5.6-terra plus github-models/* from the dispatch pool.
Leave PR-number concurrency and cancel-in-progress unchanged so the
dispatch queue cannot multiply unbounded parallel two-hour jobs.

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
seonghobae added a commit to ContextualWisdomLab/.github that referenced this pull request Aug 20, 2026
The 180s NVIDIA NIM candidate timeout killed reviews in three minutes
(ContextualWisdomLab/fast-mlsirm#290). Raise NIM, cadence, dynamic-cap,
and central-fallback run timeouts to 7200s, keep GPT-5/free-tier short,
and omit gpt-5.6-terra plus github-models/* from the dispatch pool.
Leave PR-number concurrency and cancel-in-progress unchanged so the
dispatch queue cannot multiply unbounded parallel two-hour jobs.

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
opencode-agent Bot pushed a commit to ContextualWisdomLab/.github that referenced this pull request Aug 20, 2026
The 180s NVIDIA NIM candidate timeout killed reviews in three minutes
(ContextualWisdomLab/fast-mlsirm#290). Raise NIM, cadence, dynamic-cap,
and central-fallback run timeouts to 7200s, keep GPT-5/free-tier short,
and omit gpt-5.6-terra plus github-models/* from the dispatch pool.
Leave PR-number concurrency and cancel-in-progress unchanged so the
dispatch queue cannot multiply unbounded parallel two-hour jobs.

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants