Skip to content

feat(interaction-map): own residual factorization in Rust - #1387

Merged
seonghobae merged 45 commits into
feat/public-polytomous-predictionsfrom
feat/residual-interaction-map-contract
Aug 25, 2026
Merged

feat(interaction-map): own residual factorization in Rust#1387
seonghobae merged 45 commits into
feat/public-polytomous-predictionsfrom
feat/residual-interaction-map-contract

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Closes #1388.
Closes #1389.
Closes #1390.

Contract

Moves reusable residual interaction-map computation into fast-mlsirm. Rust owns complete-case admission, Gabriel symmetric factorization, coordinates, singular values, axis inertia, reconstruction, distance, unexplained residual, and the algebraic cross term. Consumers retain identifiers, authorization, persistence, selection, and UI.

No weights, thresholds, or model scores are invented. axis_count is consumer-supplied. NaN is reserved for observed-response missingness; observed infinity is invalid and fitted model expectations must be finite evidence.

Test-first hardening lineage

Evidence/resource admission — #1388

  • Python RED 4f32560a1459f75792717f8a89bea66b4ebb4e95 -> GREEN 0080788ecc1f11dbeb53cff98ef977871d90f3e7: callback-free/lossless evidence and control admission plus 20,000,000-cell logical/coordinate envelopes.
  • Native resource RED 405203c472d7c26599783cdb80205d61143af578 -> GREEN ea33483c673bbb5d1d3407af3463f6320cb91b78: same logical/coordinate ceilings in Rust plus 128 MiB peak symmetric-eigendecomposition workspace ceiling.
  • Review remediation 27dd0ae13d6b065dd4dcdb07a6ce795dc2c99c7a + public proof 1f3e816e4b42bc4d4f03089606649f654ad5a93f + governed evidence 12bd0342a2b3525b8497998be24aa06226e90e37.

Shape-consistent empty complete-case result — #1389

  • Public RED 2cf7c3d665b8a86534aab38287e6b6c372b621a4.
  • Native RED ef09fa71cdce4e2ce9b38dd9cb9db23c9368fcdf -> GREEN ea84f6766f0f09a38372f201613ceb1f03006b4e.
  • Governed evidence 1d27c71c23f7b93a9af657fda0ec6172a996fa98.

The GREEN intentionally does not add a maximal-complete-submatrix selector and does not alter non-empty factorization arithmetic.

Missingness / model-evidence parity — #1390

Current-source review found that the same finite-cell predicate was used for observed responses and fitted expectations. That silently reclassified expected=NaN as response missingness and could change the analyzed rectangle rather than report invalid model evidence. Direct Rust also treated observed infinity as missing although the public contract rejects infinity.

  • Public RED df72c9e7fa6416e42a1aebd2cde56d2c82dc5771: expected=NaN fails before compiled-core dispatch while observed NaN remains admissible.
  • Native expected RED 070ce177ca565556cdf79e742cc3cd28e8e75abf -> Python GREEN 94c0243af1e269269d5c4b23982cbb605b4f7185 -> Rust GREEN 7805a79c4cb9182e70fecc61f3fffef920938f4b.
  • Native observed-infinity RED 4cec5b64af444b52e76d1553e72f69eb084d0a52 -> GREEN a317ccfb1957fd96c0e5c54f073d9f0384c46ee6: direct Rust now preserves the same NaN-only missingness semantics instead of turning infinity into a dropped cell.
  • Governed evidence/current source head f9e80c1565a8244ed7543e9706d9628299cc3f23.

Scientific basis

  • Gabriel, K. R. (1971). The biplot graphic display of matrices with application to principal component analysis. Biometrika, 58(3), 453-467. https://doi.org/10.1093/biomet/58.3.453
  • Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping unobserved item-respondent interactions: A latent space item response model with interaction map. Psychometrika, 86(2), 378-403. https://doi.org/10.1007/s11336-021-09762-5

Scientific ownership

No production psychometric arithmetic moved to Python. Gabriel factorization, singular values, symmetric coordinates, reconstruction, distance, unexplained residual and cross-term arithmetic remain Rust-owned; Python changes are validation, lossless marshalling, bounded materialization, result shaping and tests.

Fresh integration boundary

Protected main advanced normally during this invocation to 897e1c7b2e64d74066cdc7cb460f610c625b39d2 via #998. The intervening main delta is logistic-DIF/release work and does not modify interaction-map Rust/Python/tests, though both lineages touch shared CHANGELOG.md / package-export context. Fresh ancestry showed this branch diverged from merge base d77505e9dc8eee5780bf354e13dd4f4bf83df453; GitHub currently reports it mergeable. Main movement is not a writer lease; require fresh merge-result/base-sensitive evidence before landing.

Exact source/test/evidence head immediately before this body update: f9e80c1565a8244ed7543e9706d9628299cc3f23, open and non-Draft. Every predecessor-head workflow/review is historical after these writes. Require fresh exact-current-head repository CI, Security Scan, SAST Semgrep, CodeQL, ClusterFuzzLite, all applicable protected-central coverage/review/package/fuzz/SBOM/provenance/dependency evidence, resolved conversations and independent approvals before merge. No self-approval, review dismissal, gate weakening, force update, destructive rebase or Python numerical substitute is used.


Open in Devin Review

claude and others added 19 commits August 17, 2026 11:41
PRs merged after the v0.8.0 release-cut PR (#966) added new
docs/changelog.d fragments without re-running the renderer, and the
already-released release-0.8.0-cut.md fragment was left in place,
duplicating the 0.8.0 section under Unreleased. Delete the stale
fragment and re-render so render_changelog_fragments.py --check passes
again, unblocking the v0.8.0 release-tag dispatch.
Ten more PRs merged into main after this fix's original commit, each
adding docs/changelog.d fragments without re-running the renderer.
Re-run render_changelog_fragments.py --update so --check passes again
and the v0.8.0 release-tag dispatch is no longer blocked.
* test(dif): expose executable logistic controls

* fix(dif): normalize controls before data and Rust

* fix(dif): install hardened public DIF adapters

* docs(changelog): record logistic DIF control boundary

* docs(doctoring): trace logistic DIF control trust boundary

* fix(changelog): classify logistic DIF control boundary

---------

Co-authored-by: Claude <noreply@anthropic.com>
…rrent-review-971

Resolves the changelog/init-module conflicts introduced by main's
independent progress since this branch diverged:

- CHANGELOG.md: regenerated the authoritative Unreleased block via
  scripts/render_changelog_fragments.py --update so it reflects every
  currently-present docs/changelog.d fragment (including this branch's
  958-logistic-dif-control-boundary.md) without reintroducing the
  already-released, already-removed release-0.8.0-cut.md duplicate.
- python/fast_mlsirm/__init__.py: combined this branch's DIF
  control-safety installation with main's independently added exposure
  array/flexilevel control-safety installation.
…rappers

safe_logistic_dif and safe_logistic_dif_purified normalized max_iter with
minimum=0, admitting zero at the Python boundary even though the native
logistic_sweep (crates/mlsirm-core/src/dif.rs) rejects max_iter == 0 with
"max_iter must be >= 1", and the module's own doctoring already documented
max_iter as a positive usize control. A zero cap therefore cleared control
validation, materialized caller response/group arrays, and dispatched to
the compiled core before failing there instead of being rejected up front
(Devin AI review finding on PR #998).

Raise the minimum to 1 for both wrappers so the rejection happens before
data materialization or core discovery, matching the documented and
native domains. Updated the compatibility regressions accordingly: the
former zero-iteration dispatch proofs now use the true minimum (1), and
new regressions prove max_iter=0 is rejected before core discovery for
both logistic_dif and logistic_dif_purified.
…rrent-review-971

Pulls in #1324 (grm-recovery toolchain pin + metadata scalar test
realignment), now merged to main, resolving the two pre-existing CI
regressions this branch had been carrying independently of its own
diff. Regenerated CHANGELOG.md via
scripts/render_changelog_fragments.py --update to resolve the
conflict from the merged fragment set.
# Conflicts:
#	python/fast_mlsirm/__init__.py
@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The change adds a Rust-backed residual interaction-map implementation with complete-case filtering and Gabriel factorization. It exposes the results through a validated Python API, documents the contract, and adds core and Python tests.

Changes

Residual Interaction-Map Contract

Layer / File(s) Summary
Core factorization and diagnostics
crates/mlsirm-core/src/entrypoint.rs, crates/mlsirm-core/src/factor.rs, crates/mlsirm-core/src/interaction_map.rs, docs/adr/0021-residual-interaction-map-boundary.md
The Rust core adds residual_interaction_map, complete-case filtering, Gabriel coordinates, singular values, axis shares, reconstruction, unexplained residuals, and cross-share values. Core tests cover rank-one reconstruction and incomplete-cell handling.
Python binding and package API
crates/fast-mlsirm-py/src/lib.rs, python/fast_mlsirm/interaction_map.py, python/fast_mlsirm/__init__.py, tests/test_interaction_map.py, CHANGELOG.md
The Python extension and top-level package expose the new API. The wrapper validates arrays and axis_count, reshapes results, maps missing cross-share values to NaN, and tests validation and output contracts.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: 🟠 High · up to 4b8e0

The new interaction-map API accepts unbounded axis_count values, and an extreme input can cause an impossible allocation that aborts the host process; validation is needed before merge. Two RUF002 lint warnings also remain.

Sequence Diagram(s)

sequenceDiagram
  participant PythonCaller
  participant PythonWrapper as fast_mlsirm.interaction_map
  participant RustBinding as fast-mlsirm-py
  participant RustCore as mlsirm_core::interaction_map
  PythonCaller->>PythonWrapper: Call residual_interaction_map
  PythonWrapper->>RustBinding: Pass validated arrays and axis_count
  RustBinding->>RustCore: Compute residual interaction map
  RustCore-->>RustBinding: Return coordinates and diagnostics
  RustBinding-->>PythonWrapper: Return result data
  PythonWrapper-->>PythonCaller: Return ResidualInteractionMap
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 7 files. (2 skipped: 2…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: moving residual interaction-map factorization into the Rust core.
Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 7 files. (2 skipped: 2 unsupported.)

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/residual-interaction-map-contract

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as resolved.

@seonghobae
seonghobae changed the base branch from main to feat/public-polytomous-predictions August 25, 2026 11:20
@seonghobae
seonghobae merged commit cbb0df1 into feat/public-polytomous-predictions Aug 25, 2026
22 of 23 checks passed
@seonghobae
seonghobae deleted the feat/residual-interaction-map-contract branch August 25, 2026 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants