Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog.d/856-crm-control-callback-safety.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Continuous-response-model control callback safety

## Fixed

- Validate CRM quadrature, iteration, and tolerance controls before native-core discovery; accept only exact built-in or supported genuine NumPy scalar identities; preserve the Rust quadrature domain and convergence tolerance contract; and reject hostile scalar subclasses before caller-controlled conversion, comparison, ufunc, or representation callbacks can execute.
40 changes: 40 additions & 0 deletions docs/doctoring/crm_control_callback_safety.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Continuous-response-model control callback safety

## Scope

The public `fit_crm()` API marshals caller controls for a Rust-owned Samejima continuous response model. This boundary change does not alter the logit transform, Gauss-Hermite nodes or weights, observed-data likelihood, EM posterior bookkeeping, weighted-least-squares item M-step, convergence rule, identification/sign convention, derived discrimination/difficulty parameters, EAP scoring, estimands, or interpretation. Production mathematical and psychometric arithmetic remains in `mlsirm-core`.

Protected-main behavior discovered the compiled core before validating scalar controls. `max_iter` was compared directly before trusted identity was established, `tol` was sent through NumPy finiteness and comparison operations, and `q_theta`/`max_iter`/`tol` were normalized with `int(...)` or `float(...)` only at native dispatch. Caller-defined Python or NumPy scalar subclasses can override these conversion/comparison/ufunc hooks, so nominal numeric controls could execute caller code before the PyO3 boundary.

The corrected boundary validates controls before native-core discovery. Exact built-in Python integers and exact supported NumPy integer scalar classes are accepted for integer controls; exact built-in numerics and supported genuine NumPy numeric scalar classes are accepted for the tolerance. Subclasses and booleans are rejected before conversion. `q_theta` is constrained to the exact embedded Rust Gauss-Hermite domain `{7, 11, 15, 21, 31, 41}`, `max_iter` retains the existing repository bound, and `tol` retains the Rust finite strictly-positive contract. Only admitted controls are normalized to built-in values for PyO3 dispatch.

## Security and reliability interpretation

This is specified-input-type validation and trust-boundary hardening, not a new psychometric estimator. Exact runtime type identity prevents caller-controlled subclass hooks from becoming part of control validation. Validation errors do not interpolate rejected objects, so hostile representation callbacks are not required to produce package-owned evidence. Unsupported quadrature orders now fail before compiled-core discovery while preserving the same domain already enforced by Rust.

NIST SP 800-218 recommends secure design and validation of externally supplied inputs before potentially dangerous processing. That guidance is used as secure-development context only; this bounded change does not claim certification or conformance.

## Verification contract

Regression evidence must establish that:

- hostile Python integer, NumPy-integer, and floating subclasses fail before conversion, comparison, ufunc, representation, equality, or hash callbacks execute;
- malformed trusted controls fail before native-core discovery;
- exact Python booleans are not accepted as numeric CRM controls;
- genuine supported NumPy controls are normalized to built-in values before the PyO3 call;
- every Rust-embedded Gauss-Hermite order (`7`, `11`, `15`, `21`, `31`, `41`) remains available and unsupported orders fail closed;
- `max_iter` remains bounded by the existing repository limit;
- zero, negative, non-finite, and non-representable tolerances fail as package-owned validation errors;
- CRM mathematical results remain governed by existing Rust estimator, convergence, recovery, and backend evidence rather than new Python arithmetic.

## References

Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. *Journal of the Royal Statistical Society: Series B (Methodological), 39*(1), 1–22. https://doi.org/10.1111/j.2517-6161.1977.tb01600.x

National Institute of Standards and Technology. (2022). *Secure software development framework (SSDF) version 1.1: Recommendations for mitigating the risk of software vulnerabilities* (NIST Special Publication 800-218). https://doi.org/10.6028/NIST.SP.800-218

Samejima, F. (1973). Homogeneous case of the continuous response model. *Psychometrika, 38*(2), 203–219. https://doi.org/10.1007/BF02291114

Wang, T., & Zeng, L. (1998). Item parameter estimation for a continuous response model using an EM algorithm. *Applied Psychological Measurement, 22*(4), 333–344. https://doi.org/10.1177/014662169802200402

Wu, C. F. J. (1983). On the convergence properties of the EM algorithm. *The Annals of Statistics, 11*(1), 95–103. https://doi.org/10.1214/aos/1176346060
73 changes: 63 additions & 10 deletions python/fast_mlsirm/crm.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,54 @@
from .config import MAX_MAX_ITER


_SUPPORTED_Q_THETA = (7, 11, 15, 21, 31, 41)
_NUMPY_INTEGER_TYPES = tuple(
np.dtype(name).type
for name in ("int8", "int16", "int32", "int64", "uint8", "uint16", "uint32", "uint64")
)
_NUMPY_FLOAT_TYPES = tuple(
np.dtype(name).type for name in ("float16", "float32", "float64", "longdouble")
)


def _is_exact_type(value_type: type, trusted_types: tuple[type, ...]) -> bool:
"""Return whether ``value_type`` is one trusted scalar type without callbacks."""

return any(value_type is trusted_type for trusted_type in trusted_types)


def _trusted_integer(value: object, name: str) -> int:
"""Normalize an exact built-in or genuine NumPy integer scalar."""

value_type = type(value)
if value_type is int:
return value
if _is_exact_type(value_type, _NUMPY_INTEGER_TYPES):
return int(value)
raise ValueError(f"{name} must be an integer")


def _positive_real(value: object, name: str) -> float:
"""Normalize a trusted finite positive numeric scalar without subclass hooks."""

value_type = type(value)
trusted = (
value_type is int
or value_type is float
or _is_exact_type(value_type, _NUMPY_INTEGER_TYPES)
or _is_exact_type(value_type, _NUMPY_FLOAT_TYPES)
)
if not trusted:
raise ValueError(f"{name} must be finite and positive")
try:
result = float(value)
except (OverflowError, ValueError) as exc:
raise ValueError(f"{name} must be finite and positive") from exc
if not np.isfinite(result) or result <= 0.0:
raise ValueError(f"{name} must be finite and positive")
return result


@dataclass
class CrmFit:
"""Fitted continuous response model (Samejima, 1973).
Expand Down Expand Up @@ -79,31 +127,36 @@ def fit_crm(
"""
from .fitstats import _core_module

core = _core_module()
if core is None or not hasattr(core, "fit_crm"):
raise RuntimeError("fit_crm requires the compiled Rust core")
q_theta_value = _trusted_integer(q_theta, "q_theta")
if q_theta_value not in _SUPPORTED_Q_THETA:
raise ValueError("q_theta must be one of 7, 11, 15, 21, 31, or 41")
max_iter_value = _trusted_integer(max_iter, "max_iter")
if not 1 <= max_iter_value <= MAX_MAX_ITER:
raise ValueError(f"max_iter must be in 1..={MAX_MAX_ITER}")
tol_value = _positive_real(tol, "tol")

y = np.asarray(responses, dtype=np.float64)
if y.ndim != 2:
raise ValueError("responses must be a 2-D persons x items array")
n_persons, n_items = y.shape
if n_persons == 0 or n_items == 0:
raise ValueError("responses must contain at least one person and one item")
if not 1 <= max_iter <= MAX_MAX_ITER:
raise ValueError(f"max_iter must be in 1..={MAX_MAX_ITER}")
if not np.isfinite(tol) or tol <= 0.0:
raise ValueError("tol must be finite and positive")

observed = np.isfinite(y)
yy = np.where(observed, y, 0.5).reshape(-1)

core = _core_module()
if core is None or not hasattr(core, "fit_crm"):
raise RuntimeError("fit_crm requires the compiled Rust core")

res = core.fit_crm(
yy,
observed.reshape(-1),
int(n_persons),
int(n_items),
int(q_theta),
int(max_iter),
float(tol),
q_theta_value,
max_iter_value,
tol_value,
)
return CrmFit(
slope=np.asarray(res["slope"], dtype=np.float64),
Expand Down
160 changes: 160 additions & 0 deletions tests/test_crm_control_callbacks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
"""Fail-closed tests for public continuous-response-model control marshalling."""

from __future__ import annotations

import numpy as np
import pytest

import fast_mlsirm.fitstats as fitstats
from fast_mlsirm import crm


_RESPONSES = np.array([[0.25], [0.75]], dtype=np.float64)
_SUPPORTED_Q = (7, 11, 15, 21, 31, 41)


def _unexpected_core() -> object:
"""Fail if native-core discovery happens before CRM control validation."""

raise AssertionError("compiled core discovered before CRM control validation")


def _result(tol: float = 1e-6) -> dict[str, object]:
"""Return the smallest shape-consistent trusted-core result fixture."""

return {
"slope": [1.0],
"intercept": [0.0],
"resid_sd": [1.0],
"discrimination": [1.0],
"difficulty": [0.0],
"theta": [0.0, 0.0],
"loglik_trace": [0.0],
"n_iter": 1,
"converged": True,
"n_parameters": 3,
"termination_reason": "tolerance",
"final_delta": 0.0,
"stopping_tolerance": tol,
}


def test_hostile_scalar_controls_fail_before_callbacks_and_core(monkeypatch):
"""Rejected scalar subclasses execute neither callbacks nor native discovery."""

class HostileMeta(type):
def __eq__(cls, other): # pragma: no cover - must never execute
raise AssertionError("hostile type equality executed")

def __hash__(cls): # pragma: no cover - must never execute
raise AssertionError("hostile type hash executed")

class HostileInt(int, metaclass=HostileMeta):
def __int__(self): # pragma: no cover - must never execute
raise AssertionError("hostile integer conversion executed")

def __le__(self, other): # pragma: no cover - must never execute
raise AssertionError("hostile integer comparison executed")

def __ge__(self, other): # pragma: no cover - must never execute
raise AssertionError("hostile integer comparison executed")

def __repr__(self): # pragma: no cover - must never execute
raise AssertionError("hostile integer repr executed")

class HostileNumpyInt(np.int64):
def __int__(self): # pragma: no cover - must never execute
raise AssertionError("hostile NumPy integer conversion executed")

def __repr__(self): # pragma: no cover - must never execute
raise AssertionError("hostile NumPy integer repr executed")

class HostileFloat(float):
def __float__(self): # pragma: no cover - must never execute
raise AssertionError("hostile float conversion executed")

def __le__(self, other): # pragma: no cover - must never execute
raise AssertionError("hostile float comparison executed")

def __ge__(self, other): # pragma: no cover - must never execute
raise AssertionError("hostile float comparison executed")

def __array_ufunc__(self, *args, **kwargs): # pragma: no cover
raise AssertionError("hostile NumPy ufunc executed")

def __repr__(self): # pragma: no cover - must never execute
raise AssertionError("hostile float repr executed")

monkeypatch.setattr(fitstats, "_core_module", _unexpected_core)
cases = (
({"q_theta": HostileInt(41)}, "q_theta"),
({"q_theta": HostileNumpyInt(41)}, "q_theta"),
({"max_iter": HostileInt(5)}, "max_iter"),
({"tol": HostileFloat(1e-6)}, "tol"),
)
for kwargs, field in cases:
with pytest.raises(ValueError, match=field):
crm.fit_crm(_RESPONSES, **kwargs)


def test_invalid_exact_controls_fail_before_core(monkeypatch):
"""Malformed trusted controls fail at the Python marshalling boundary."""

monkeypatch.setattr(fitstats, "_core_module", _unexpected_core)
cases = (
({"q_theta": 9}, "q_theta"),
({"q_theta": True}, "q_theta"),
({"max_iter": 0}, "max_iter"),
({"max_iter": True}, "max_iter"),
({"tol": 0.0}, "tol"),
({"tol": -1.0}, "tol"),
({"tol": float("nan")}, "tol"),
({"tol": float("inf")}, "tol"),
({"tol": 10**400}, "tol"),
({"tol": True}, "tol"),
)
for kwargs, field in cases:
with pytest.raises(ValueError, match=field):
crm.fit_crm(_RESPONSES, **kwargs)


def test_genuine_numpy_controls_are_normalized_before_native_dispatch(monkeypatch):
"""Supported genuine NumPy scalars remain trusted after exact-type admission."""

captured: dict[str, tuple[object, ...]] = {}

class CapturingCore:
def fit_crm(self, *args):
captured["args"] = args
return _result(float(args[6]))

monkeypatch.setattr(fitstats, "_core_module", lambda: CapturingCore())
fitted = crm.fit_crm(
_RESPONSES,
q_theta=np.uint16(21),
max_iter=np.int32(3),
tol=np.float32(1e-5),
)

args = captured["args"]
assert type(args[4]) is int and args[4] == 21
assert type(args[5]) is int and args[5] == 3
assert type(args[6]) is float and args[6] > 0.0
assert fitted.n_parameters == 3


def test_supported_quadrature_orders_reach_native_boundary(monkeypatch):
"""Python validation preserves every Gauss-Hermite order embedded by Rust."""

captured: list[int] = []

class CapturingCore:
def fit_crm(self, *args):
captured.append(args[4])
return _result()

monkeypatch.setattr(fitstats, "_core_module", lambda: CapturingCore())
for q_theta in _SUPPORTED_Q:
crm.fit_crm(_RESPONSES, q_theta=q_theta, max_iter=1)

assert tuple(captured) == _SUPPORTED_Q
Loading