Add Light's kappa (kappam.light, irr 0.85) with chance-product z test - #313
Closed
seonghobae wants to merge 48 commits into
Closed
Add Light's kappa (kappam.light, irr 0.85) with chance-product z test#313seonghobae wants to merge 48 commits into
seonghobae wants to merge 48 commits into
Conversation
New module mlsirm_core::scaling with thurstone_case_v: scale values colmean(qnorm(choice)) - min, fitted model Phi(S_j - S_i), residuals, and psych's full-matrix goodness of fit 1 - sse/ssc (pinning the psych CODE behavior; the .Rd 'lower off diagonal' prose is stale). Algorithm follows psych's thurstone() (Revelle; source READ); Thurstone (1927) NOT READ, cited as origin per the psych source. Entries must be strictly in (0, 1) -- deliberate safety divergence from psych's direct path which admits infinite quantiles. Pinned against a 50-digit mpmath oracle on three fixtures (asymmetric nonzero-residual, exactly-consistent round-trip, intransitive 4x4 with non-first min column). Five mutation kills (MU1-MU5) executed on a clean baseline; MC-500 consistent-recovery test under #[ignore]. PyO3 binding thurstone_case_v + Python wrapper fast_mlsirm.scaling with ThurstoneResult dataclass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Implements bradley_terry_mm in mlsirm_core::scaling following the choix 0.4.1 opt.mm pairwise algorithm (source READ; Hunter 2004 and Bradley & Terry 1952 NOT READ, cited as described by choix): centered log-worths from an n x n win-count matrix, alpha regularization, exp-scale weights summing to n, L1 tol*n convergence over consecutive updates. Zero-wins items at alpha=0, all-zero matrices (deliberate divergence from choix's uniform fallback), and Ford-condition non-convergence raise errors. Pinned against an EXECUTED 50-digit mpmath oracle cross-checked with choix (max diff <= 1.4e-12): asymmetric 3x3, exact +/-ln(3)/2 closed form, zero-pair 4x4, alpha=0.5 MAP, iterations==18 convergence pin. Five mutation kills EXECUTED (MU1 winner accumulation, MU2 denominator symmetry, MU3 centering, MU4 weight normalization, MU5 tol*n semantics) plus 500-rep Monte-Carlo recovery (#[ignore]). PyO3 binding, Python wrapper with pre-cast validation, exports, pytest class, CHANGELOG. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Impl-review probe follow-up: symmetric finite 1e300 counts are valid input whose MLE is exactly the zero vector -- pinned as accepted with correct params. Row sums overflowing f64 (2e308 -> inf) are pinned to trip the non-finite update guard with an error instead of returning a bogus fit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Implements choix 0.4.1's lsr.py dense pairwise path (source READ;
Maystre & Grossglauser 2015 NOT READ, cited as described by choix):
one-shot spectral estimate and iterative MLE from an n x n win-count
matrix. Rust core (mlsirm_core::scaling::{lsr_pairwise, ilsr_pairwise})
with Gaussian-elimination statdist guarded by positivity, sum, and
residual checks; overflow from huge counts/alpha raises instead of
returning NaN. I-LSR at alpha=0 reproduces the Bradley-Terry MLE
(cross-algorithm anchor vs bradley_terry_mm); alpha>0 regularization
semantics deliberately differ (chain-rate vs Dirichlet-MAP, per source).
Pins from an EXECUTED exact-Fraction/mpmath oracle cross-checked with
pip choix 0.4.1 (<= 2.2e-13). Six mutation kills EXECUTED (chain
transpose, dropped diagonal subtraction, dropped centering,
sum-n normalization via weights pins, denominator collapse via I-LSR
pins with the one-shot-unobservable limitation documented, tol*n vs tol
via a separating iteration-count fixture). MC-500 recovery test
(#[ignore]) passing. cargo 745 pass; pytest 273 pass.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… rejection The relative pivot threshold in the stationary-distribution solve compared an O(1) sum-constraint pivot against an O(max count) global scale, falsely rejecting validly connected win matrices with globally huge counts (impl-review finding). Normalize the generator to unit max magnitude after the overflow guard; the stationary distribution is invariant under global rescaling of transition rates. Regression asserts: base vs base*1e20 params/weights equal to 1e-12; asymmetric 1e150 ILSR finite. The overflow fixture is now a genuinely overflowing n=4 matrix (row sums -> inf); the previous n=3 all-1e308 fixture never overflowed and is correctly accepted post-fix. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…atrix Mirror of the Rust-side fixture change in 343fc2f: the n=3 all-1e308 matrix never overflowed and is correctly accepted after the scale-invariance fix; use n=4 at 1.7e308 (row sums -> inf) and add a Python-side scale-invariance regression assert. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Port choix 0.4.1's rank_centrality (Negahban, Oh, & Shah, 2017, as implemented by choix's continuous-time win-ratio chain; choix source READ, paper's discrete-time walk NOT implemented) to the Rust core: - scaling::rank_centrality builds the Markov chain with regularized win-ratio rates (alpha + w_ij) / (2*alpha + w_ij + w_ji) from a pre-transform counts snapshot, with explicit Err on disconnected graphs at alpha = 0 and on overflowing counts or ratio denominators (choix silently degrades to near-zero ratios there). - Extract the shared stationary-distribution solver statdist_params (normalization, partial-pivot Gaussian elimination, guards, centered log transform) from the LSR pass and reuse it, so LSR/I-LSR and Rank Centrality share one verified solver. - PyO3 binding rank_centrality; Python wrapper returning LsrResult. - Tests pin EXECUTED exact-Fraction oracle anchors (3x3 and 4x4 at alpha 0 and 1/2, one-sided, disconnected-with-alpha), exact scale invariance at alpha = 0 only, error contract, and an ignored 500-rep Monte-Carlo recovery check (measured worst MAE 0.0912, bound 0.15); cross-checked against pip choix 0.4.1 to 2e-16. Six mutation kills executed (transposed ratio, missing ratio transform, half-updated denominator, denominator-c-only, dropped normalization, dropped centering). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…lsr_rankings) Rust cores scaling::lsr_rankings (one-shot) and scaling::ilsr_rankings (iterative MLE) for full/partial rankings in CSR layout, porting choix 0.4.1 exactly (source READ; Maystre & Grossglauser 2015 NOT READ, cited as-cited per choix docstrings). Python wrappers accept lists of rankings, validate before unsigned casts (negatives, non-integers, length<2 rejected), and return LsrResult. Documented divergences from choix: length<2 rankings rejected (choix no-ops), within-ranking duplicates rejected (choix accepts if connected), negative indices rejected (Python would wrap). Tests: exact rational anchors from an executed exact-Fraction/mpmath oracle (full + partial fixtures; the partial fixture is the only one that can see a wrong all-items denominator), bit-exact length-2 equivalence with lsr_pairwise, I-LSR fixed-point pins (atol 1e-7 = oracle-measured margin) + iteration-count pins (8/11 at tol=1e-8) + weights==exp_transform(params) invariant, full error contract incl. disconnected graph and alpha overflow, MC-500 recovery (#[ignore], bound 0.2 vs measured worst 0.1440). Five mutation kills executed (MU1 stale denominator, MU2 transpose, MU3 full-ranking losers, MU4 all-items sum, MU5 uniform I-LSR worths). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Cap n at 10000 in rankings_validate: the dense O(n^2) chain would otherwise attempt terabyte allocations and abort the process on tiny inputs like lsr_rankings([[0,1]], 1_000_000) (finding 1, High). - Reject np.bool_ items alongside Python bool (finding 2, Medium). - Catch OverflowError from int(x) so infinite items raise ValueError per the wrapper contract (finding 3, Low). Regression tests added on both the Rust error contract and the Python validation test. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Rust cores scaling::lsr_top1 / scaling::ilsr_top1 with CSR (winners, losers, starts) layout, PyO3 bindings, and Python wrappers lsr_top1/ilsr_top1 taking (winner, losers) pairs. Pins from an executed exact-Fraction/mpmath oracle cross-checked against pip choix 0.4.1 (<= 1.2e-16). Documented divergences: empty loser sets, winner-in-losers, and duplicate losers are rejected. Single-loser observations bit-match lsr_pairwise. Mutation kills MU1-MU5 all executed; MC-500 recovery bound measured at 0.2580 (pinned 0.3). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Impl-review finding (LOW): n or item indices beyond u64 leaked a raw OverflowError from the uint64 cast instead of ValueError. Mirror the Rust dense-chain 10000-item cap in _rankings_to_csr/_top1_to_csr BEFORE any cast; regression-tested for huge n, huge loser, and huge winner. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Implements the Kendall & Babington Smith (1940) circular-triad consistency test and coefficient of agreement u, as implemented by the eba R package 1.10-0 (circular.R / kendall.u.R, source READ; the 1940 paper and Alway's exact tables NOT READ, cited as origins per eba's manual pages). - Rust core scaling::circular_triads: T = C(n,3) - sum_j C(d_j,2) (integer arithmetic), T_max, T_exp = C(n,3)/4, zeta = 1 - T/T_max; EXACT null p-values for n <= 10 from embedded distributions (dyadic rationals, assert_eq!-pinned), continuity-corrected chi-square for n >= 11 (df = n(n-1)(n-2)/(n-4)^2). Documented divergences from eba: n = 2 and malformed/incomplete tournaments are rejected. - Rust core scaling::kendall_u: Sigma, u = 2*Sigma/(C(m,2)*C(n,2)) - 1, min_u, RAW chi-square (can be negative under continuity correction; only the p-value clamps), df = C(n,2)m(m-1)/(m-2)^2. Stricter than eba: every pair must have the same m >= 3 judges. - PyO3 bindings + Python wrappers circular_triads / kendall_u with CircularTriadsResult / KendallUResult dataclasses; input validation before casts. - 11 Rust tests (exact-Fraction oracle pins: 1940 dog example, n = 12 chi-square path vs scipy, table integrity sum = 2^C(n,2), negative raw chi2, error contracts, MC-500 invariants #[ignore]); 8 Python tests. Five mutants (drop pairing, T_max parity swap, drop opposite-tail, corr sign flip, drop Sigma correction) all EXECUTED and killed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Implements the Elo (1978) rating system as specified by the CRAN PlayerRatings 1.1-0 package's elo() (R/ratings.R + elo_c C kernel, both READ): batch-per-period updates where every expected score within a rating period uses the period-start ratings, per-game white advantage gamma, and PlayerRatings win/draw/loss and lag bookkeeping. Elo's 1978 book was NOT read and is cited as the origin per PlayerRatings. - Rust core mlsirm_core::scaling::elo_rating (EloResult with ratings, games, wins, draws, losses, lag); periods may be unsorted (grouped by ascending label, matching R split() ordering); self-play rejected and scalar K factor only (documented divergences). - PROVED: E_w + E_b = 1 identically for any finite gamma (the exponents are exact negations), so rating sums are conserved at n*init and an E_b = 1 - E_w refactor is a documented unkillable mutant. - Tests anchored to an executed exact-rational oracle: exact-fraction single/two-period fixtures (batch-semantics proof at kfac=400), float regression, closed-form nonzero-gamma pin, kfac=0, unsorted periods, fractional-score bookkeeping, saturation, error contract, and an MC-500 invariant suite (ignored by default). Five mutation kills executed: sequential-update, black-score flip, gamma sign flip, lag reset drop, logistic divisor. - PyO3 binding elo_rating; Python wrapper fast_mlsirm.elo_rating with (g, 4) [period, white, black, score] schedule, scalar gamma broadcast, and PlayerRatings defaults init=2200, kfac=27. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Impl-review finding (High): the wrapper coerced the whole games array to float before the uint64 period cast, so distinct integer period labels above 2**53 silently merged into one rating period (wrong batching, wrong ratings), and out-of-u64 labels were accepted with only a NumPy warning. Fix: take period labels losslessly from integer-dtype input arrays, and reject float-path labels >= 2**53 (float(2**53+1) already rounds to 2**53, so that value is ambiguous). Regression test pins the crate's sequential-update ratings for labels 2**53 / 2**53+1 passed as uint64 and asserts ValueError on the float path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…integer bound Round-2 review finding (High): np.float32 games arrays lose integer fidelity above 2**24 (float16 above 2**11) before the float64 promotion, so distinct period labels could still silently merge under the previous 2**53-only guard. The float-path bound is now derived from the input dtype's mantissa (np.finfo(dtype).nmant). Regression test pins ValueError for float32 labels at 2**24 and exact crate ratings below the bound. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Implements the Glicko rating system as a Rust core (mlsirm_core::scaling::glicko_rating) with a thin PyO3 binding and Python wrapper. Sources READ: Glickman's 'The Glicko system' technical note (worked example reproduced to full float64 precision) and CRAN PlayerRatings 1.1-0 glicko()/glicko_c. Glickman (1999), the derivation paper, was NOT read and is cited as the origin per both READ sources. - Batch-per-period Step 2 updates with opponent-g weighting and the new-variance rating step; participant-only Step 1b inflation RD = min(sqrt(RD^2 + (lag+1) c^2), rdmax). - Per-player init_rating/init_dev arrays (heterogeneous RDs); results cover ALL 0..n players (documented no-status divergence from R). - Documented non-identity: no rating-sum conservation (pinned by test). - Tests anchored to an executed float64 oracle: Glickman worked-example anchor, two-period inflation/lag/idle-player full-vector pins, rdmax clamp, gamma exact pins, unsorted periods, fractional score, error contract, MC-500 (#[ignore]). - Seven executed mutation kills: opponent-g swap, inflation off-by-one, clamp drop, stale-variance update, missing q^2, gamma sign, all-player inflation. - Python wrapper inherits the Elo period-label fidelity contract (integer-dtype lossless u64 path; dtype-derived float bound). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
np.finfo(dtype).nmant excludes the implicit leading bit, so the exact-integer ceiling of a float dtype is 2**(nmant + 1), not 2**nmant. The elo/glicko wrappers were rejecting exactly representable period labels one power of two early (float32 at 2**23, float64 at 2**52). Bound is now 2**(nmant + 1) with the >= comparison kept (2**53 itself is ambiguous because 2**53 + 1 rounds onto it). Boundary tests pin acceptance of 2**24 - 1 (float32) and 2**53 - 1 (float64) via crate game tallies, killing a 2**nmant mutant. Found by adversarial implementation review of PR #298. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…glicko2) Rust core glicko2_rating with batch-per-period updates on the Glicko-2 scale: participant-only pre-period variance inflation phi^2 <- min(phi^2 + lag * sigma^2, (q rdmax)^2) (Glicko-2 lag, not Glicko-1 lag+1; R source comment pinned), per-player volatility via Glickman's Step-5 Illinois iteration (eps 1e-6, endpoint A; DERIVED: f(x) = -1/2 d/dx of PlayerRatings' penalized nllh, so the Illinois root matches R's optimum), tau == 0 volatility freeze, volatility ceiling q * rdmax, per-game white advantage gamma, W/D/L and lag bookkeeping. Documented R-vs-note deviation: idle players get no per-period Step-6 growth; lag * sigma^2 applies at next participation. Anchored to an executed float64 oracle: Glickman worked-example anchor (r'=1464.05, RD'=151.52, sigma'=0.059996) with heterogeneous init, two-period inflation/lag/idle pins, rdmax + volatility-ceiling clamps, gamma, unsorted-period, fractional-score + tau-0, return-after-idle. Nine executed mutation kills (own-g swap, lag off-by-one, variance-clamp drop, volatility-clamp drop, skipped volatility update, stale-sigma inflation, Illinois endpoint swap, gamma sign, rating-before-deviation). PyO3 binding glicko2_rating; Python wrapper fast_mlsirm.glicko2_rating returning Glicko2Result, inheriting the Elo/Glicko period-label fidelity contract. cargo 794 pass; pytest 305 pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Python wrapper: wrap int(n_players), gamma asarray, and float(tau)/ float(rdmax) in narrow try/except -> ValueError so callers see the documented exception type instead of leaked TypeError/OverflowError (red-green mutation kill EXECUTED: reverting the n_players guard fails the error-contract test). - PyO3 binding: usize::try_from for white/black player ids instead of 'as usize' truncation on 32-bit targets; PyValueError on overflow. - Error-contract coverage: Rust g2_error_contract adds non-finite init arrays, non-finite/negative rdmax, negative score, and at/above the ln(10)/400*rdmax volatility ceiling boundary; Python TestGlicko2 adds 18 cases (out-of-range/negative index, score bounds, gamma shape/ non-finite/complex/object, non-finite tau/rdmax, inf/None n_players, None tau/rdmax, init NaN/length mismatch, 10000-player cap). cargo g2_: 11 pass; pytest TestGlicko2: 5 pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Rust core stephenson_rating extending Glicko with per-game neighborhood variance (ngames*hval^2), per-game bonus bval/100 on both sides, participants-only lambda drift toward opponents, and (lag+1)*cval^2 per-period deviation-variance inflation clamped at rdmax^2. Normative source: CRAN PlayerRatings 1.1-0 R driver (ratings.R 591-737) + C kernel (ratings.c stephenson_c 157-202), both READ and line-cited; no journal paper exists (Kaggle-2010 provenance noted as NOT independently verifiable). PyO3 binding + NumPy wrapper with PlayerRatings defaults. Tests anchored to an EXECUTED faithful oracle port (S1 heterogeneous init, S2 two-period draw/lag, S3 full knobs, S4 rdmax clamp + prior-run continuation, S5 bval symmetry, lambda=0 contrast; 1e-12 pins), 500-rep MC invariants (#[ignore]), and five EXECUTED mutation kills: bval drop, lambda sign flip, per-game hval scaling drop, (lag+1)->lag, opponent-g->own-g. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Enforce the 2..=10000 n_players cap in the Python wrapper BEFORE any length-n allocation (High: huge n_players previously attempted the allocation instead of raising ValueError). - Preserve integer fidelity for white/black player-id columns: integer dtypes cast directly to u64; float/object inputs are rejected at or above the dtype's exact-integer bound before the uint64 cast, matching the existing period-label contract (Medium). - Guard u64 counter overflow in the Rust core: init_games/init_lag values that could overflow across the run's increments now return Err instead of panicking (debug) or wrapping (release) (Medium). Regression tests: Rust st_error_contract overflow cases; Python fidelity + pre-allocation cap cases. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Implements elom_rating: nn-seat event ratings with rank base scores, per-period single update K*(actual - expected) with expected summing (r_p - event mean rating)/40, constant-K or kriichi experience-decay K factor, placing mode, and games/places/lag bookkeeping. Normative sources (READ): CRAN PlayerRatings 1.1-0 R/ratings.R lines 739-932 (elom driver), src/ratings.c lines 45-80 (elom_c kernel), R/ratings.R lines 1006-1020 (kriichi). No journal paper exists for this system. Faithfully reproduces the R quirk that partial events shrink the ORIGINAL base exactly once regardless of empty-seat count (sbase <- basev resets inside the shrink loop, R:855-866). REDUCED-SCOPE vs R (documented in the core header): player == -1 iff score is NaN (jointly enforced), sorted periods required, in-event duplicate players rejected, kriichi bounds gv > 0 and 0 < kv <= 1. Tests: exact-value anchors E1-E9 (dyadic rationals, hand-derived oracle executed), error contract, MC-500 permutation invariance (#[ignore]); 6 mutation kills executed (K-scaling, event-mean, cumulative-shrink, kriichi games-timing, tie-rank, per-event-update). PyO3 binding elom_rating + numpy-validating Python wrapper. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
An unsigned id above i64::MAX wrapped to a negative value through the int64 narrowing cast (uint64::MAX -> -1) and was silently treated as the empty-seat sentinel instead of being rejected. Impl-review finding 1 (Medium). Adds a regression test. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Rust core reimplementation of CRAN PlayerRatings 1.1-0 metrics() (R/ratings.R 936-957, READ; no journal paper exists -- CRAN source is the normative reference): per predictor column, binomial deviance on the cap-clamped predictions and RMSE/MAE on the RAW uncapped predictions (the R source quirk at lines 949/951), each times 100 and optionally divided by the 0.5-constant-predictor baseline. Numerator NaN removal is elementwise per pair; scaled baselines use the act-only row set (a different set when the predictor column has NaNs). The bdev baseline is implemented as the exact constant ln 2 (algebraic identity documented in the source, sub-ulp divergence from R's summation for non-0/1 act). Reduced scope (documented): which/sort/digits/drop presentation arguments are not implemented (full unrounded np x 3 matrix returned); na.rm=FALSE not implemented; Inf, empty per-column row sets, out-of- domain caps, and scale=TRUE with an all-0.5 act baseline are rejected where R would recycle or emit NaN/Inf. Tests: six exact-Fraction-oracle anchors (unscaled, cap quirk, scaled, NaN row sets, baseline-row-set killer, two-column stride pins with both columns pinned), error contract, MC-500 #[ignore] (scaled-vs- unscaled column-constant ratios with bdev ratio exactly ln 2; bitwise column-permutation invariance). Five mutation kills EXECUTED: cap-on-mse/mae, bdev-uncapped, stride transpose, missing sqrt, baseline pair-removal. Algebraic rearrangements (sqrt(a)/sqrt(b) vs sqrt(a/b)) are documented as unobservable. PyO3 binding metrics_rating (plain name) + Python wrapper with complex/object-dtype/shape validation before casts; TestMetrics; CHANGELOG. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Impl-review findings: numeric strings (plain, object-dtype, and cap tuples) were silently parsed as floats, bool arrays were accepted as 0/1, and 0-D scalars were promoted to 1-D by ascontiguousarray before the shape check, bypassing the documented contract. Validation now checks dtype kind and ndim before any cast; regression tests added. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round-2 review finding: np.bool_ is not a bool subclass, so object arrays of np.bool_ bypassed the bool rejection and cast to 0/1 floats. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round-3 adversarial review found that object-dtype arrays containing None were silently cast to NaN by astype(float64), bypassing the explicit-missing contract (missing values must be passed as np.nan). None (and any element that is str/bytes/bool/np.bool_) is now rejected with ValueError before the cast, with regression tests for both act and pred paths. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round-4 adversarial review found that np.ma.MaskedArray inputs lost their mask through np.asarray, so masked missing values were silently counted as observed. Masked arrays are now rejected with ValueError before conversion; missing values must be encoded as explicit np.nan. Regression tests cover both act and pred paths. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Rust core fide_rating reimplements CRAN PlayerRatings 1.1-0 fide() (R/ratings.R 125-272) with kfide() (959-972): per-period batch Elo, per-player K from PERIOD-START games/elite state (kv triple, defaults 10/15/30), sticky elite flag from POST-update ratings >= 2400, and a running mean of POST-update opponent ratings. REDUCED-SCOPE: no status/history frames, kfide-only K schedule, self-play rejected, 30/2400 thresholds hard-coded. kv=(k,k,k) reduces bitwise to elo_rating(kfac=k) (fd_mc_500_elo_reduction anchor). Exact-oracle fixtures F1-F5 (F1 Fraction-exact); every assert reads crate outputs; 5 mutants EXECUTED-killed (post-period K, non-sticky elite, pre-update opponent, swapped running-mean weights, kv swap). PyO3 binding, pyd rebuild, hardened Python wrapper (masked-array / object-dtype / complex rejection, u64 period fidelity), TestFide suite. cargo 825 pass; pytest 322 pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Impl-review round-1 findings: (1) nested-list bools, bool arrays, and datetime64 game values were silently coerced into legal-looking rated games — the wrapper now rejects non-fiu ndarray dtypes and scans original list/object elements for bool/datetime before the float cast; (2) complex kv/init leaked TypeError instead of the documented ValueError — realness is now checked before conversion. Decimal and Fraction inputs still coerce as before. Regression tests added. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round-2 adversarial review finding: bool and np.bool_ values in kv tuples/arrays and init were silently coerced to 1.0/0.0. Reject bool dtype arrays, probe object-dtype and tuple elements for bool/complex before float coercion. Decimal/Fraction coercion preserved (regression-verified). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round-3 adversarial review finding: init=np.array(True) and 0-d object-dtype bool arrays coerced to 1.0. Reject bool-kind and 0-d object bool init values. Decimal/Fraction init coercion preserved. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ting) Rust core predict_rating_two/predict_rating_multi in mlsirm-core scaling (Elo logistic, deviation-shrunk Glicko-family with qip3 = 3(ln10/400/pi)^2, EloM rowmean branch with optional min-tie placing), PyO3 bindings, Python wrappers, exact-oracle anchor tests P1-P9, 7-mutant EXECUTED kill map, MC-500 invariants, TestPredict pytest coverage, CHANGELOG entry. Normative source: CRAN PlayerRatings 1.1-0 R/ratings.R lines 1056-1133 (READ). REDUCED-SCOPE: index-based players, per-game/scalar gamma. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Impl-review round-1 findings: (1) games/tng went through a float round-trip in the Python wrappers, losing exact integer counts at or above 2^53 and silently shifting the strict games < tng cutoff — new _predict_games_u64/_predict_tng_u64 keep integer inputs lossless and reject float inputs at/above the source dtype's exact-integer bound; (2) predict_rating_multi computed nr*np unchecked, so a wrapping product could pass the length check and panic on indexing — now checked_mul with a checked error. Regression tests at both the Rust error contract and pytest levels. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Implements the VGAM bratt() family model (VGAM 1.1-14 R/family.categorical.R, READ and normative): P(i beats j) = alpha_i / (alpha_i + alpha_j + alpha0), P(tie) = alpha0 / (...), fitted by a hand-derived supporting-hyperplane MM ascent with a joint likelihood-preserving rescale of alpha AND alpha0. This is the additive-alpha0 ties model, NOT Rao-Kupper or Davidson. - Rust core bratt_mm + BrattResult in mlsirm-core scaling.rs with full error contract (n cap 10000 before O(n^2), symmetric ties, zero-win and tie-free rejection directing to bradley_terry_mm). - bt2_ test block: exact-Fraction oracle anchors B1-B4 (iter-1 pins [1, 27/40, 3/4], alpha0 = 9/14; converged pins at 1e-13 with an independent spec-gradient check), permutation equivariance, rescale anchor, 14-path error contract, and an MC-500 log-likelihood dominance test (#[ignore]). - 5 mutation kills EXECUTED (W-includes-ties, alpha0 denominator double-count, rescale-skips-alpha0, D-missing-alpha0, and an alpha0-blind convergence check killed by a tol-separated convergence anchor pinning iterations == 2). - PyO3 binding bratt_mm + Python wrapper/BrattResult dataclass with hardened input validation; TestBratt (4 tests). cargo test -p mlsirm-core --lib: 840 passed. pytest tests/test_paper_features.py: 333 passed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- MAJOR: finite-but-huge counts could overflow derived aggregates (w_tot, t_tot, pair totals n_ij) to +inf, letting the MM update return exactly-zero parameters and a NaN log-likelihood as Ok. Now every derived aggregate is checked for finiteness up front, updated parameters must be finite AND strictly positive, and a non-finite log-likelihood is rejected. Regression bt2_huge_counts_overflow_rejected covers both the pair-total and row-total overflow paths (asserts read the crate Err). - MINOR: object-dtype arrays of Python bools bypassed the boolean rejection at the Python boundary (cast cleanly to float64). The object path now rejects bool/np.bool_ elements before casting; regression added to TestBratt.test_validation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reimplements CRAN irr 0.85 kappam.fleiss() (R source READ and normative; Fleiss 1971 and Conger 1980 NOT READ, cited as model origins only) in the Rust core (mlsirm_core::agreement::fleiss_kappa) with a thin PyO3 binding and NumPy wrapper: - classification-table agreement, classic (sum p_j^2) and exact (sum p_j^2 - (1/nr) sum s2_j) chance agreement; kappa, Fleiss' z test, and category-wise kappas (classic mode; NaN for empty categories, matching R's 0/0) - listwise row drop for missing ratings (negative code / NaN) - documented API deviations: index codes 0..k-1 with explicit/inferred k, error on degenerate 1 - chanceP = 0 (R returns NaN), size caps Evidence: exact-Fraction oracle anchors FK1-FK5 (classic kappa 139/399, exact 37/102, category kappas [1/21, 31/91, 43/63]); 6 mutants EXECUTED and killed (agreeP centering, row-vs-column chance sums, exact==classic, variance sign, missing-as-category, pjk centering); MC-500 subject/rater permutation-invariance test (#[ignore], executed); cargo 847 pass; TestFleiss pytest pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Impl-review round 1 findings: - MAJOR: uint64 values above i64::MAX wrapped negative via astype(int64) and were silently dropped as missing; now rejected before conversion. - MINOR: explicit k accepted lossy coercions (3.9, '3', bool); now requires a true integer (int or np.integer, bool excluded). Regression tests added to TestFleiss.test_validation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Transcribed from CRAN irr 0.85 R/icc.R (READ, normative source; Shrout & Fleiss 1979, McGraw & Wong 1996, Bartko 1966 NOT READ, cited as origins only). Rust core computes all six variants (oneway/twoway x consistency/agreement x single/average) from one-pass ANOVA mean squares, the F test of H0: icc = r0 (two-way agreement via Satterthwaite df, preserving the R quirk that both units' CI bounds reuse the nr-scaled plug-in form, icc.R lines 139-141), and unclamped confidence bounds. Listwise NaN row drop; Inf rejected; degenerate zero-variance / icc=1 pivots error instead of leaking non-finite output. Evidence: exact-Fraction oracle on Shrout-Fleiss Table 2 (all six coefficients + scipy F CI pins), 6 EXECUTED mutation kills (MSw divisor, quantile df order, agreement denominator, r0-in-F, dimension map, CI plug-in), MC-500 permutation invariance + Spearman-Brown single/average bridge for all three families. cargo 853 pass; pytest paper suite 340 pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Impl-review finding (MINOR): object arrays of Python/numpy bools were silently coerced to 0.0/1.0 before the bool-dtype check, bypassing the boolean rejection contract. Scan object arrays for bool elements before the float64 conversion; regression test added. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round-2 impl-review finding (MINOR): an object array whose element is a 0-D np.ndarray of dtype bool coerced to 1.0 and bypassed the boolean rejection. The object-array scan now also rejects ndarray elements with boolean dtype; regression test added. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round-3 impl-review finding (MINOR): 0-D object-dtype ndarrays wrapping a bool still bypassed the scan. The guard now iteratively unwraps 0-D ndarray elements of any dtype via .item() before the bool check, closing the wrapper-nesting family of bypasses at the root; regression tests for object-wrapped bool and np.bool_ added. 0-D float ndarray elements remain accepted and bitwise-match the plain-float result. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replace the boolean blacklist with a numeric whitelist: after unwrapping 0-D ndarrays, only int/float/np.integer/np.floating scalars are accepted (bool excluded as an int subclass). This closes the np.void structured- scalar bypass and any future exotic-scalar coercion path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Five review rounds produced successive bypasses of per-element vetting (object bools, nested 0-D wrappers, np.void, timedelta64, self- referential 0-D arrays hanging the unwrap loop, __float__-lying int/float subclasses). Numeric data never requires object dtype, so the wrapper now rejects it categorically, eliminating the entire coercion attack class. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Rust reimplementation of CRAN irr 0.85 kripp.alpha() (R/kripp.alpha.R, READ and normative; Krippendorff 1980 NOT READ, cited as method origin only). Coincidence matrix over unordered rater pairs with the irr divisor quirk preserved verbatim (mc = #nonmissing-1 per column only when any value is missing, else 1), all four metrics (nominal, ordinal half-endpoint weights, interval, ratio), alpha = 1 when fewer than two observed levels. Documented deviations: all-missing, infinities, and ratio zero-sum level pairs are explicit errors. Evidence: exact-Fraction oracle anchors K1-K4 (nominal 113/152, ordinal 108577/133160, interval 951/1120, ratio 18222619/22852465, nmv=40; no-NA quirk pin 43/72 vs m-1 mutant's 11/18), 6-mutant EXECUTED kill map, MC-500 permutation invariance. cargo 858 pass, pytest paper suite 344 pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Review finding (MAJOR): int64/uint64 rating labels beyond 2**53 are not exactly representable as float64, so distinct levels silently collapsed during the cast — complete disagreement returned alpha=1 with a single level. The wrapper now rejects any integer array with values outside [-2**53, 2**53]; the boundary itself remains accepted (exactly representable). Regression test executes the reviewer's repro and was red-green verified against the unguarded version. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Rust core light_kappa + LightKappaResult in mlsirm-core::agreement, reusing cohen_kappa per rater pair after full-matrix level compaction (level-set invariance of the unweighted kappa proven in the oracle). Light's z: disrater = m^2 - sum(c1*c2), chanceP = 1 - npairs*prod(dis/m^2) (overflow-safe form of irr's B/ns^(2*npairs)), varkappa, erfc p-value. Deviations: pe==1 pairs and chanceP<=0 (reachable on valid data with disjoint rater level sets) are explicit errors where R emits NaN. Exact-Fraction oracle anchors L1-L5; 5 mutants EXECUTED-killed (mean->sum, disrater diagonal, npairs factor, pair-loop bound, listwise drop); MC-500 rater-permutation invariance. PyO3 binding, NumPy wrapper with strict dtype policy (negative floats rejected; NaN/negative int = missing), TestLight suite. cargo 864 pass; pytest paper suite 349 pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
The epsilon guard |1-chanceP| < 1e-12 rejected valid high-agreement data (e.g. 100k subjects x 3 raters with one dissent: chanceP within 2.4e-14 of 1 yet z finite in R). Only chanceP <= 0 and chanceP >= 1 are degenerate; the trailing is_finite check backstops residual overflow. Regression test red-green verified (old guard -> FAIL; fix -> pass). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
3 tasks
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Light's kappa (
light_kappa)Implements
kappam.light()from CRAN irr 0.85 plus the unweighted branch ofkappa2()(both R sources READ in full; algorithm source of truth). Light, R. J. (1971), Psychological Bulletin, 76(5), 365–377 is cited as method origin only (NOT READ).Stacked on #311 (
seonghobae-kripp).What it computes
C(nr,2)rater pairs after listwise missing-row drop (value, plus the per-pairkappasin(i,j),i<jorder).disraterₚ = m² − Σₐ c1[a]c2[a],chanceP = 1 − npairs·Π(disraterₚ/m²)(overflow-safe, algebraically identical to R's1 − B/ns^(2·npairs)),varκ = chanceP/(m(1−chanceP)),z = value/√varκ,p = erfc(|z|/√2).Fidelity and documented deviations
cohen_kappa. Equivalence PROVEN per pair in the exact-Fraction oracle (unweighted kappa is invariant to unused levels). The corresponding mutant is an unkillable identity — documented honestly in the spec, not hidden.pe == 1,chanceP ≤ 0(reachable on VALID data — e.g. raters using disjoint level sets givechanceP = −2; R silently emits NaN z), all rows dropped, and< 2observed levels are explicit errors.Evidence chain
lk_Rust tests (anchors L1–L5, non-contiguous label regression, error contract incl. the chanceP=−2 valid-data case, MC-500 rater-permutation invariance#[ignore]); PythonTestLight(anchor, NaN vs integer −1 missing equivalence, 10 input rejections incl. 2.5 / −1.0 float / masked / object / complex / bool). Every assert reads crate outputs.-p mlsirm-core --lib864 passed / 109 ignored; pytest paper suite 349 passed.Surface
mlsirm_core::agreement::light_kappa(ratings, ns, nr) -> LightKappaResult_core.light_kappa(dict)fast_mlsirm.light_kappa(ratings) -> LightKappaResult— strict dtype policy (negative floats rejected loudly; NaN or negative int = missing)Adversarial implementation review
(1.0 - chance_p).abs() < 1e-12guard rejected valid high-agreement data (100,000x3 raters with one dissenting row gives chanceP within 2.4e-14 of 1 yet R computes a finite z). Fixed in 124d3f8: guard now errors only on exact degeneracy (chance_p <= 0.0 || chance_p >= 1.0) with anis_finitebackstop; regression testlk_near_unit_chance_p_is_validadded and red-green verified (FAILs under the old guard).Review log: session files/light_impl_review.md.