Add K-index answer-copying statistic (CopyDetect-faithful port) - #260
Merged
Conversation
Implement the K-index of matching incorrect answers exactly as in the CRAN CopyDetect package's internal k() (R/similarity1.r, READ; corroborated by R/similarity2.r): number-incorrect subgroup keyed on the copier's score (including the copier and, when scores match, the source - CopyDetect convention, source exclusion NOT applied), p = mean(emp_agg)/ws, and the binomial upper tail P(Bin(ws, p) >= m) computed with a coefficient-free f64 term recurrence (no factorial overflow, smaller-tail summation, [0,1] clamp). Holland (1996, ETS RR-96-07) and Sotaridona & Meijer (2002, JEM 39(2)) were NOT read; the statistic is cited only as implemented by CopyDetect. Sotaridona & Meijer (2001, RR-01-07, ERIC ED467373) was read for background corroboration of the binomial framework only. The aberrance package was checked and contains no K-index. Rust core mlsirm_core::security::k_index + KIndexResult; PyO3 py_k_index; Python fast_mlsirm.k_index with pre-cast validation (complex/bool/non-binary rejected, ws==0 degenerate error). Tests: pinned oracle (independent exact-binomial Python oracle, all 7 result fields), subgroup-membership invariants (copier always included; source included when its score matches), degenerate branches (m==0 -> K=1, p==1 -> K=1), error paths, and a 500-rep Monte Carlo (#[ignore]) checking null size < 0.10 and power > 0.5 under 90% full-response copying. Three executed mutation kills documented: tail flip, p denominator wc-for-ws, subgroup keyed on ws. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The linear-space term recurrence started from (1-p)^n, which underflows to zero for cases like n=1000, p=0.99 (K wrongly returned 0 instead of 0.58304080330109709). Sum the upper-tail terms directly in log space (log-term recurrence + max-shifted exponential sum): no complement subtraction, no cancellation, no underflow. Regression test pins the extreme-p case against an exact fractions.Fraction reference computed on the exact binary value of 0.99, plus the moderate oracle case. Found by adversarial implementation review. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
seonghobae
added a commit
that referenced
this pull request
Jul 26, 2026
* Add K-index answer-copying statistic (CopyDetect-faithful port) Implement the K-index of matching incorrect answers exactly as in the CRAN CopyDetect package's internal k() (R/similarity1.r, READ; corroborated by R/similarity2.r): number-incorrect subgroup keyed on the copier's score (including the copier and, when scores match, the source - CopyDetect convention, source exclusion NOT applied), p = mean(emp_agg)/ws, and the binomial upper tail P(Bin(ws, p) >= m) computed with a coefficient-free f64 term recurrence (no factorial overflow, smaller-tail summation, [0,1] clamp). Holland (1996, ETS RR-96-07) and Sotaridona & Meijer (2002, JEM 39(2)) were NOT read; the statistic is cited only as implemented by CopyDetect. Sotaridona & Meijer (2001, RR-01-07, ERIC ED467373) was read for background corroboration of the binomial framework only. The aberrance package was checked and contains no K-index. Rust core mlsirm_core::security::k_index + KIndexResult; PyO3 py_k_index; Python fast_mlsirm.k_index with pre-cast validation (complex/bool/non-binary rejected, ws==0 degenerate error). Tests: pinned oracle (independent exact-binomial Python oracle, all 7 result fields), subgroup-membership invariants (copier always included; source included when its score matches), degenerate branches (m==0 -> K=1, p==1 -> K=1), error paths, and a 500-rep Monte Carlo (#[ignore]) checking null size < 0.10 and power > 0.5 under 90% full-response copying. Three executed mutation kills documented: tail flip, p denominator wc-for-ws, subgroup keyed on ws. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Fix binomial tail underflow in k_index at extreme p / large n The linear-space term recurrence started from (1-p)^n, which underflows to zero for cases like n=1000, p=0.99 (K wrongly returned 0 instead of 0.58304080330109709). Sum the upper-tail terms directly in log space (log-term recurrence + max-shifted exponential sum): no complement subtraction, no cancellation, no underflow. Regression test pins the extreme-p case against an exact fractions.Fraction reference computed on the exact binary value of 0.99, plus the moderate oracle case. Found by adversarial implementation review. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
seonghobae
added a commit
that referenced
this pull request
Jul 26, 2026
* Add K-index answer-copying statistic (CopyDetect-faithful port) Implement the K-index of matching incorrect answers exactly as in the CRAN CopyDetect package's internal k() (R/similarity1.r, READ; corroborated by R/similarity2.r): number-incorrect subgroup keyed on the copier's score (including the copier and, when scores match, the source - CopyDetect convention, source exclusion NOT applied), p = mean(emp_agg)/ws, and the binomial upper tail P(Bin(ws, p) >= m) computed with a coefficient-free f64 term recurrence (no factorial overflow, smaller-tail summation, [0,1] clamp). Holland (1996, ETS RR-96-07) and Sotaridona & Meijer (2002, JEM 39(2)) were NOT read; the statistic is cited only as implemented by CopyDetect. Sotaridona & Meijer (2001, RR-01-07, ERIC ED467373) was read for background corroboration of the binomial framework only. The aberrance package was checked and contains no K-index. Rust core mlsirm_core::security::k_index + KIndexResult; PyO3 py_k_index; Python fast_mlsirm.k_index with pre-cast validation (complex/bool/non-binary rejected, ws==0 degenerate error). Tests: pinned oracle (independent exact-binomial Python oracle, all 7 result fields), subgroup-membership invariants (copier always included; source included when its score matches), degenerate branches (m==0 -> K=1, p==1 -> K=1), error paths, and a 500-rep Monte Carlo (#[ignore]) checking null size < 0.10 and power > 0.5 under 90% full-response copying. Three executed mutation kills documented: tail flip, p denominator wc-for-ws, subgroup keyed on ws. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Fix binomial tail underflow in k_index at extreme p / large n The linear-space term recurrence started from (1-p)^n, which underflows to zero for cases like n=1000, p=0.99 (K wrongly returned 0 instead of 0.58304080330109709). Sum the upper-tail terms directly in log space (log-term recurrence + max-shifted exponential sum): no complement subtraction, no cancellation, no underflow. Regression test pins the extreme-p case against an exact fractions.Fraction reference computed on the exact binary value of 0.99, plus the moderate oracle case. Found by adversarial implementation review. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the K-index of matching incorrect answers as a faithful port of the CRAN CopyDetect package's internal
k().Stacked on #259 (Wollack omega;
seonghobae-wollack-omega).Scope (adversarial spec-verify: REDUCED-SCOPE APPROVED)
R/similarity1.r" (READ; corroborated byR/similarity2.r). NOT claimed as a transcription of Holland (1996, ETS RR-96-07) or Sotaridona & Meijer (2002, JEM 39(2)) — neither was read. Sotaridona & Meijer (2001, RR-01-07, ERIC ED467373) READ for background corroboration only.aberrancechecked: no K-index.p = mean(emp_agg)/ws,K = P(Bin(ws, p) >= m)with the upper tail summed directly in log space (log-term recurrence + max-shifted exponential sum; no factorial overflow, no extreme-p underflow, no complement cancellation; [0,1] clamp; explicit p==0/p==1 branches).ws == 0→ error.Surfaces
mlsirm_core::security::k_index+KIndexResultpy_k_indexfast_mlsirm.k_index/KIndexResult(pre-cast validation: complex/bool/non-binary rejected)Evidence
math.combPython oracle): wc=6, ws=5, m=2, subgroup=[1,2,5], emp_agg=[3,2,3], p=0.5333…, K=0.85139489711934158 (1e-12), all 7 fields asserted from crate output.#[ignore]: null size < 0.10 (K discrete/conservative), power > 0.5 under 90% full-response copying. (Copying only incorrect answers inflates wc into a sparse subgroup where self-inclusion makes K conservative — documented in the test.)Adversarial implementation review
(1-p)^n, underflowing at extreme p / large n (n=1000, p=0.99, m=990 returned K=0). Fixed in 721ae19 by log-space upper-tail summation + pinned regression.