feat(irt): expose Rust polytomous predictions

[seonghobae]

Closes #1281.
Closes #1291.
Closes #1296.
Closes #1297.
Closes #1307.
Addresses #1280.
Addresses #1292.

Summary

Expose public GRM/GPCM category probabilities and expected-response predictions while keeping probability and expected-score arithmetic in the Rust numerical owner. Python validates, bounds, normalizes, and marshals only.

The prediction path now enforces one bounded scientific contract across the installed public Python boundary, the package-private raw Python helper, and the direct Rust boundary: trusted/lossless/finite evidence, a 20,000,000 probability-cell request ceiling, the fitter-supported 2..=64 category domain, and preservation of package-owned PolytomousFit metadata. GRM fitting also preserves strict threshold ordering for sparse/collapsed observed-category patterns so returned fits remain inside the scoring/prediction parameter domain.

Test-first lineage

Public prediction and allocation envelope

• feature ddf3cd33df8f2606882619c14cc1c5b57b332fdc: Rust-owned GRM/GPCM prediction surface plus public Python exports and ADR-0016;
• public resource RED a6cfbe67be331bcb0d1b3c29e51ba65ce203ad0d and allocation-order RED 10dc8015a20239ea21db4647be05078ea8a00dab;
• native RED 425e310b5e185fabdffd9a743d4016ca579c9876 -> GREEN ae48db32c6b6c2ed00d0f14afd2b1fc4fa805fce: Rust enforces POLY_MAX_PREDICTION_CELLS = 20_000_000 before allocation/parameter-dependent work.

Callback-free/lossless evidence admission — #1281

• RED 8d6b62abb10ea5981561bd4be84caa3077ddb48e -> GREEN 2a7cf5565a8ac3b468a3a8f575599760454b8260: reject hostile array providers, complex/non-real storage, and lossy float64 conversion while preserving trusted NumPy/built-in evidence;
• installer 941843a9db775f5e18030c76d73669ef86686afc keeps public/legacy exports on the same guard;
• finiteness RED a0f57302be0f929cbe2ffbd439fe0c90b721e702 -> GREEN b51a0720c9a6014656d8f6d07001dc05df4b722a: reject trusted NaN/Inf evidence before raw prediction/native work.

Rank/resource ordering — #1291

• RED 4cb3117e4dda30d5eec7de326a5aac2f3c486368 -> GREEN 854c3dfd89ac49b714096e7cf393a108b611450f: reject provably over-rank evidence before NumPy materialization;
• hosted-CI RCA found slope/category over-rank cases could still materialize theta first; df2922df968200e6a78eedda211cfb6f91ed444f preflights theta, slope, and category parameters before any field reaches NumPy;
• c41c9a5de5194ae032b8b4eaf1bdc0cd25a0f173 -> e8d8335536dd66e1f1d4ed1a3bf2f9a94a76efe5: reject lossy integer promotion and oversized joint grids before float64 conversion.

Category-domain parity — #1292 / #1307

• public RED f69b11c0c982e158de0c7fbe0c816e197a828531 -> GREEN 2ee70059f198bc4186f57e90dfc8c5825410ae6e: manually constructed fits implying n_cat > 64 fail from trusted shape metadata before NumPy/core discovery;
• native RED fc198cf9579175c02bee2822a0dbf461c35e67d5 -> GREEN 0f4c78fe46a02b21e143b80cf4dfad858cfe9b7d: direct Rust prediction enforces 2..=64 before allocation/item-parameter validation;
• raw-Python RED fdce0d6fe45a6ff89f5ba6362435df7821d1516f -> GREEN b9ecf432bc257a9d99e2b3e1358331d72b971295: package-private _polytomous_predictions() now replays MAX_POLYTOMOUS_CATEGORIES after shape validation and before resource/core work;
• compatibility 1dc38914439833b180983cc825e5340034c311b3: the raw helper still delegates the valid 64-category boundary with canonical n_cat=64 payload and output shape;
• governed evidence 84ffcb5bf407aaad79a3ebd89aa1100ebf6e7a54 records the raw-helper defense-in-depth contract.

Fit metadata and GRM domain preservation — #1296 / #1297

• f1d417a0bba572da5a6cc3dff082589fea241b44 -> 0664286a425620faba8c551c778127ece2856b81: admission uses dataclasses.replace so normalization preserves convergence, trace, stopping and threshold metadata;
• 602c8dd7750d032214c6f3130941d5c12da13c6d -> 2eeb3d42479779edcb58890b121ce73f4f321982: smoothed GRM initialization preserves strict cumulative-threshold order without changing Samejima probability arithmetic;
• 9bf2a850a167950d96a5db288eb7a3054fbe233c: sparse-category post-M-step proof keeps returned GRM thresholds inside the native prediction domain and yields finite nonnegative normalized probabilities.

Scientific ownership

GRM/GPCM probability kernels, expected-score arithmetic, parameterization, fit likelihood/EM updates, item-parameter validation inside the numerical core, and extreme-theta behavior remain Rust-owned. Python changes are trust/resource/domain validation and marshalling only; no second production numerical engine is introduced.

Verification boundary

Protected base tip at this body refresh: main@9ef9e1a86f18743323f15da64bba7a80aa0698a4.
Exact source/test/evidence head at the immediately preceding lifecycle refetch: 84ffcb5bf407aaad79a3ebd89aa1100ebf6e7a54, open, non-Draft, mergeable. Relative to the current protected tip the branch is 58 commits ahead and 1 commit behind; that moving-main delta is unrelated report work and must be reconciled non-destructively before landing if it remains outstanding.

All predecessor-head checks and reviews are historical. Exact-current-head CI, Security Scan, SAST Semgrep, CodeQL, ClusterFuzzLite, every applicable protected central coverage/review/package/SBOM/provenance/dependency context, and independent approval must be freshly satisfied before merge. No self-approval, gate weakening, force update, or predecessor-head evidence reuse is authorized.

Consumer

LineageWeave can consume polytomous_expected_response(fit, theta) and polytomous_category_probabilities(fit, theta) without importing fast_mlsirm._core or reproducing GRM/GPCM arithmetic.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: aafbb744-3c66-448b-a5c3-1eefdb1f16bf

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The change adds public GRM and GPCM polytomous prediction APIs. Rust computes batched probabilities and expected scores. Python validates and normalizes evidence, enforces resource and rank limits, exposes the functions, and adds regression coverage and documentation.

Changes

Polytomous prediction flow

Layer / File(s)	Summary
Rust prediction kernel crates/mlsirm-core/src/poly.rs, crates/mlsirm-core/tests/polytomous_predictions_resource.rs	The Rust core adds batched GRM/GPCM predictions, flattened probabilities, expected scores, input validation, and a 20,000,000-cell limit.
Python API and binding crates/fast-mlsirm-py/src/lib.rs, python/fast_mlsirm/polytomous.py, python/fast_mlsirm/_legacy_init.py, docs/adr/0016-public-polytomous-prediction-boundary.md	The PyO3 binding and Python wrappers expose category-probability and expected-response functions. The wrappers validate inputs, delegate to Rust, reshape outputs, and export the public functions.
Prediction evidence admission python/fast_mlsirm/_polytomous_prediction_admission.py, python/fast_mlsirm/__init__.py, docs/changelog.d/1281-polytomous-prediction-admission.md, docs/changelog.d/1291-polytomous-prediction-rank.md	The admission wrapper rejects callbacks, cycles, over-rank structures, complex values, and lossy conversions before NumPy or native dispatch. Package initialization installs the wrapper.
Prediction validation and records tests/test_polytomous_predictions.py, docs/changelog.d/1280-polytomous-prediction-resource.md	Tests cover numerical consistency, validation order, trusted inputs, rank limits, callback rejection, complex inputs, and resource limits. The changelog records the resource safeguards.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: 🟡 Moderate · up to df292

The new prediction APIs can currently change large integer inputs during normalization and briefly materialize oversized broadcast grids before rejecting them, risking incorrect results and avoidable memory use. Merge should wait for these admission checks to be corrected.

Sequence Diagram(s)

sequenceDiagram
  participant PythonCaller
  participant PredictionAdmission
  participant PolytomousAPI
  participant PyO3Binding
  participant RustCore

  PythonCaller->>PredictionAdmission: Submit fit and theta evidence
  PredictionAdmission->>PredictionAdmission: Preflight rank, callbacks, cycles, and size
  PredictionAdmission->>PredictionAdmission: Create contiguous float64 arrays
  PredictionAdmission->>PolytomousAPI: Invoke guarded prediction
  PolytomousAPI->>PyO3Binding: Call polytomous_predictions
  PyO3Binding->>RustCore: Compute GRM/GPCM predictions
  RustCore-->>PyO3Binding: Return probabilities and expected scores
  PyO3Binding-->>PolytomousAPI: Return flattened outputs
  PolytomousAPI-->>PythonCaller: Return reshaped arrays

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes satisfy issues #1281 and #1291 by sealing inputs, enforcing rank and resource limits, and preserving valid prediction behavior.
Out of Scope Changes check	✅ Passed	The implementation, tests, ADR, and changelog entries directly support the linked prediction-boundary objectives.
Docstring Coverage	✅ Passed	Docstring check was indeterminate for this PR — some files could not be analyzed in time. Not blocking.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: exposing Rust-backed polytomous prediction APIs.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/public-polytomous-predictions

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[opencode-agent]

Scheduled review-feedback autofix for this PR head.

• Head SHA: 92f0999ac20252387fe8d8f0b0172980a6a8e33c

[seonghobae]

Merge-forward: resolved python/fast_mlsirm/__init__.py package-surface conflict against main@9c12eab1 by combining this branch's polytomous prediction-admission install with main's independently landed CDM response-safety and inference admission-safety installs (both sides kept, installs ordered before legacy export copies). Regenerated CHANGELOG.md from the merged fragment set via scripts/render_changelog_fragments.py --update (--check passes). Merge commit 7ddfa2c1; no statistical behavior or result schema changed.

[seonghobae]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[seonghobae]

Merge-forward 2: resolved the package-surface conflict and changelog against main@f5d61892 (which now carries the merged CAT administration lane #1345 and fitstats S-X² lane #1029). Both sides kept; installs ordered before legacy export copies; CHANGELOG.md regenerated from the merged fragment set (--check passes). Merge commit 70fff368; predecessor-head reviews/checks are historical.

[opencode-agent]

Scheduled review-feedback autofix for this PR head.

• Head SHA: 8f186b7d9eaad5a023d6dce0ded97955a2ecb381

[devin-ai-integration]

Devin Review found 2 new potential issues.

[devin-ai-integration]

🔍 Reconstruction excludes the residual grand mean

The residual is grand-mean-centered (center at interaction_map.rs) before factorization, so reconstruction reconstructs the centered residual while unexplained = raw - fitted uses the raw residual. A full-rank map therefore leaves the entire grand mean inside unexplained. The cross-term identity still holds and ADR-0021 documents centering, but only grand-mean (not row/column) centering is applied, and public tests exercise only zero-mean residuals.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

📝 Info: Python coordinate budget uses full, not complete-case, dimensions

The Python coordinate budget (rows + columns) * axis_count_value uses the full matrix shape, while the Rust validate_factorization_workspace charges only retained complete-case dimensions. The Python guard is strictly more conservative and can reject a request the core would accept. Not a correctness bug, but the two ceilings bound different quantities.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

Devin Review found 1 new potential issue.

[devin-ai-integration]

📝 Info: Interaction-map API missing from all

residual_interaction_map and ResidualInteractionMap are re-exported (init.py) but never added to __all__, unlike the polytomous prediction functions that arrive via _legacy_init.__all__. Direct imports work, but from fast_mlsirm import * omits them. If a fully public surface is intended, add them to __all__ for consistency.

---

Was this helpful? React with 👍 or 👎 to provide feedback.