Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/changelog.d/872-config-integer-callback-safety.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Harden configuration integer trust boundaries

## Fixed

- Reject caller-defined integer subclasses and arbitrary `__index__` providers before public simulation and fit configuration validation can dispatch caller-controlled coercion.
- Preserve exact built-in integers and genuine NumPy integer scalars while validating simulation size, optimizer-work, quadrature, latent-integration, seed, and verbosity controls through built-in integer values.
- Store those trusted integers back on the frozen configs so later size products and `seed + restart` cannot wrap narrow NumPy scalars.
- Normalize `dimensionality_diagnostics` `k_folds`, `seed`, and `latent_dims` to built-in integers before the candidate-by-fold budget product or `seed + fold_idx` can wrap a narrow NumPy scalar.
- Normalize `fit_diagnostics` `parameter_count` and `m2_q_*` to built-in integers before AIC/BIC arithmetic or `int(q_*)` can dispatch caller `__index__` hooks.
- Run the same simulation and fit validators at construction so memory-safety bounds cannot be bypassed by skipping an explicit `validate()` call.
29 changes: 29 additions & 0 deletions docs/doctoring/config_integer_callback_safety.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Configuration integer callback safety

## Problem

Public configuration validation accepted Python's generic integer protocol. Calling `operator.index()` or comparing caller-controlled integer-like objects before trust was established allowed arbitrary `__index__` implementations or integer subclasses to participate in validation. Construction-time validation closed the memory-budget bypass, but `seed` and `verbose` still stored untrusted objects, and admitted narrow NumPy integers were left on the frozen fields. Those stored scalars wrap on ordinary Python/NumPy arithmetic: `np.uint8(16) * np.uint8(16)` becomes `0`, so `n_items` and `simulate()` can disagree with the validated size product, and `config.seed + restart` can wrap a restart index.

## Boundary decision

Configuration validation is marshalling and trust-boundary work, not psychometric arithmetic. The package now accepts only exact built-in `int` values and exact supported NumPy integer scalar types for validated integer controls. Accepted NumPy scalars are converted to built-in integers for bounds and work-budget calculations; booleans, caller-defined `int` subclasses, and arbitrary index providers are rejected without invoking their coercion hooks. After those checks pass, the trusted built-in integers are written back onto the frozen dataclass so later size products, RNG seeding, and save-time `int(seed)` cannot dispatch caller callbacks or wrap.

`MLS2PLMConfig` and `FitConfig` run that same validator from `__post_init__`, so invalid or untrusted integer controls cannot exist as constructed objects. `validate()` remains public and idempotent for callers that already invoke it at simulate/fit entry points.

The hardened surface covers simulation sizes, latent dimension, and simulation `seed`, plus fit latent dimension, optimizer iteration/restart/history controls, quadrature node counts, marginal M-step count, latent-space integration point/seed controls, fit `seed`, and `verbose`. The same trusted-integer marshalling is applied to `dimensionality_diagnostics` `k_folds`, `seed`, and each `latent_dims` candidate before the diagnostic fit-budget product or `seed + fold_idx` can wrap a narrow NumPy scalar, and to `fit_diagnostics` `parameter_count` and `m2_q_*` before AIC/BIC arithmetic or save-time `int(q_*)` can dispatch caller `__index__` hooks. Numerical model ownership and Rust-first computation are unchanged.

## Test evidence

`tests/test_config_integer_callback_safety.py` provides hostile `__index__` regressions, valid-valued caller `int` subclasses, genuine NumPy scalar compatibility, write-back of narrow unsigned sizes, a 2-by-256 simulation shape check that would fail if `n_items` wrapped, dimensionality-diagnostic wrap/`__index__` checks for `k_folds`, fold `seed`, and `latent_dims`, and `fit_diagnostics` `__index__` checks for `parameter_count` and `m2_q_*`. The original RED commit is `4c81e4dc465312d13f044b9b47e14d839af6cc1a`; exact-head hosted CI/security/package/coverage/review evidence remains authoritative as the branch advances.

## References

Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M., Haldane, A., del Río, J. F., Wiebe, M., Peterson, P., ... Oliphant, T. E. (2020). Array programming with NumPy. *Nature, 585*(7825), 357–362. https://doi.org/10.1038/s41586-020-2649-2

MITRE. (n.d.). *CWE-1287: Improper validation of specified type of input*. https://cwe.mitre.org/data/definitions/1287.html

National Institute of Standards and Technology. (2022). *Secure software development framework (SSDF) version 1.1: Recommendations for mitigating the risk of software vulnerabilities* (NIST Special Publication 800-218). https://doi.org/10.6028/NIST.SP.800-218

OWASP Foundation. (2025). *OWASP application security verification standard 5.0.0*. https://owasp.org/www-project-application-security-verification-standard/

Python Software Foundation. (n.d.). Emulating numeric types. In *The Python language reference*. https://docs.python.org/3/reference/datamodel.html#object.__index__
2 changes: 1 addition & 1 deletion fuzz/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ codegraph explore "neg_loglik_and_grad config Params ModelConfig"
| --- | --- | --- |
| `load_factor_csv` | `python/fast_mlsirm/io.py` | Reads an on-disk item→factor CSV via `numpy.loadtxt`; reached from the CLI. |
| `render_diagnostics_report` | `python/fast_mlsirm/report.py` | Parses an **arbitrary JSON** diagnostics file and renders it to HTML. |
| `MLS2PLMConfig` / `FitConfig` `.validate()` | `python/fast_mlsirm/config.py` | Every CLI / API call funnels user numeric parameters through these validators. |
| `MLS2PLMConfig` / `FitConfig` construction and `.validate()` | `python/fast_mlsirm/config.py` | Every CLI / API call funnels user numeric parameters through these validators, including construction-time checks. |
| `neg_loglik_and_grad` | `crates/mlsirm-core/src/lib.rs` | The core numeric kernel — widest Rust blast radius; consumes response data + parameter vectors. |

## Tools & Licenses
Expand Down
57 changes: 29 additions & 28 deletions fuzz/atheris/fuzz_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,12 @@

``MLS2PLMConfig`` and ``FitConfig`` are the request/DTO validators of the
package: every CLI invocation and every public API entry point funnels
user-supplied numeric parameters through ``.validate()``. The contract is that
``validate()`` is a *total* function on arbitrary field values -- it must
either return normally or raise ``ValueError`` with a message. Any other
exception (``TypeError`` from an unexpected type, ``OverflowError``,
``ZeroDivisionError`` in the equicorrelation bound, ...) is a bug.
user-supplied numeric parameters through construction-time ``.validate()``.
The contract is that construction and ``validate()`` are *total* on arbitrary
field values -- they must either return normally or raise ``ValueError`` with a
message. Any other exception (``TypeError`` from an unexpected type,
``OverflowError``, ``ZeroDivisionError`` in the equicorrelation bound, ...) is
a bug.

This harness draws arbitrary field values from the fuzzer, builds both configs,
and asserts the validator only ever rejects with ``ValueError``. When a config
Expand Down Expand Up @@ -44,38 +45,38 @@ def _maybe_nan_float(fdp: "atheris.FuzzedDataProvider") -> float:
def _test_one_input(data: bytes) -> None:
fdp = atheris.FuzzedDataProvider(data)

sim = MLS2PLMConfig(
n_persons=fdp.ConsumeIntInRange(-8, 4096),
n_dims=fdp.ConsumeIntInRange(-8, 512),
items_per_dim=fdp.ConsumeIntInRange(-8, 512),
latent_dim=fdp.ConsumeIntInRange(-8, 512),
phi=_maybe_nan_float(fdp),
gamma=_maybe_nan_float(fdp),
seed=fdp.ConsumeInt(8),
dtype=fdp.PickValueInList(["float64", "float32", "int8", "", "FLOAT64"]),
)
try:
sim = MLS2PLMConfig(
n_persons=fdp.ConsumeIntInRange(-8, 4096),
n_dims=fdp.ConsumeIntInRange(-8, 512),
items_per_dim=fdp.ConsumeIntInRange(-8, 512),
latent_dim=fdp.ConsumeIntInRange(-8, 512),
phi=_maybe_nan_float(fdp),
gamma=_maybe_nan_float(fdp),
seed=fdp.ConsumeInt(8),
dtype=fdp.PickValueInList(["float64", "float32", "int8", "", "FLOAT64"]),
)
sim.validate()
except ValueError:
pass
else:
# n_items is a pure product of two validated positive ints.
assert sim.n_items >= 1, f"validated config produced n_items={sim.n_items}"

fit = FitConfig(
model=fdp.PickValueInList(["MLS2PLM", "mls2plm", "MIRT", "bogus", ""]),
latent_dim=fdp.ConsumeIntInRange(-8, 512),
optimizer=fdp.PickValueInList(["adam", "lbfgs", "adam_lbfgs", "sgd", ""]),
max_iter=fdp.ConsumeIntInRange(-8, 100000),
n_restarts=fdp.ConsumeIntInRange(-8, 512),
learning_rate=_maybe_nan_float(fdp),
seed=fdp.ConsumeInt(8),
eps_distance=_maybe_nan_float(fdp),
init_gamma=_maybe_nan_float(fdp),
backend=fdp.PickValueInList(["numpy", "rust", "auto", "", "NUMPY"]),
penalty=PenaltyConfig(),
)
try:
fit = FitConfig(
model=fdp.PickValueInList(["MLS2PLM", "mls2plm", "MIRT", "bogus", ""]),
latent_dim=fdp.ConsumeIntInRange(-8, 512),
optimizer=fdp.PickValueInList(["adam", "lbfgs", "adam_lbfgs", "sgd", ""]),
max_iter=fdp.ConsumeIntInRange(-8, 100000),
n_restarts=fdp.ConsumeIntInRange(-8, 512),
learning_rate=_maybe_nan_float(fdp),
seed=fdp.ConsumeInt(8),
eps_distance=_maybe_nan_float(fdp),
init_gamma=_maybe_nan_float(fdp),
backend=fdp.PickValueInList(["numpy", "rust", "auto", "", "NUMPY"]),
penalty=PenaltyConfig(),
)
fit.validate()
except ValueError:
pass
Expand Down
Loading
Loading