Skip to content

fix(rubric): harden testlet pilot control callbacks - #884

Merged
seonghobae merged 6 commits into
mainfrom
fix/rubric-testlet-pilot-control-callbacks-883
Aug 16, 2026
Merged

fix(rubric): harden testlet pilot control callbacks#884
seonghobae merged 6 commits into
mainfrom
fix/rubric-testlet-pilot-control-callbacks-883

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Scope

Fix #883 at the generated-item testlet pilot handoff validation boundary without changing testlet arithmetic or Rust numerical ownership.

Test-first lineage

  • RED 669e944f9779ea61225cb0d8da91f772a6e149bd adds public-handoff regressions for arbitrary __index__ providers and caller-defined Python/NumPy integer, floating, boolean, and string subclasses. Rejected controls must execute zero caller conversion, normalization, hashing, equality, or representation callbacks; genuine NumPy scalar controls remain supported.
  • The implementation line through 6fdced1abf88dcdae78076abac6453b51ad48045 adds exact built-in / supported NumPy scalar admission and identity-only scalar-type checks while preserving the public control domains.
  • Exact-head CI on 6fdced1... ran the complete Python suite and proved the production/control tests green, but failed the repository release contract because the new changelog fragment started at ### Security instead of the required level-one title plus classified section.
  • Exact current head 976c3a90b9f1835dff216794c21ea344428e66b8 repairs only that canonical fragment grammar. The Python failures were both downstream consequences of the same first-line format defect.

Boundaries

  • preserve models rasch / 2pl, MAX_MAX_ITER, and Rust quadrature vocabulary {7, 11, 15, 21, 31, 41};
  • preserve finite non-negative tol / init_sigma2 semantics and strict boolean controls;
  • preserve genuine NumPy integer/floating/boolean scalar compatibility;
  • Python remains validation/marshalling only;
  • no formula, estimator, provider, dependency, database, workflow, release-version, or gate change.

Exact current state

  • Protected base: fb67ced09d8ee00542c05d56374537a9a7239751.
  • Exact source head: 976c3a90b9f1835dff216794c21ea344428e66b8.
  • GitHub reports the PR mergeable and Ready for review.
  • Exact-head CI completed successfully: Python 3.12/3.14, Rust, package, GPU smoke, fuzz, and aggregate Python all passed.
  • Exact-head Security Scan and SAST Semgrep completed successfully; current check-runs contain no failure or in-progress result.
  • No formal submitted review or unresolved inline review thread was present immediately before Ready transition.
  • Integration still requires the protected-main qualifying independent non-author approval and all live required contexts at merge time. No gate is weakened or bypassed.

Fixes #883.

@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 15 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d3ce9502-7d32-47cf-9ce2-a6c562b1f69d

📥 Commits

Reviewing files that changed from the base of the PR and between fb67ced and 976c3a9.

📒 Files selected for processing (4)
  • docs/changelog.d/883-rubric-testlet-pilot-control-callbacks.md
  • docs/doctoring/rubric_testlet_pilot_control_callbacks.md
  • python/fast_mlsirm/rubric/testlet_pilot.py
  • tests/test_rubric_testlet_pilot_control_callbacks.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 15, 2026 07:38
@seonghobae
seonghobae enabled auto-merge (squash) August 15, 2026 07:40
@seonghobae
seonghobae merged commit 98e93dd into main Aug 16, 2026
35 checks passed
@seonghobae
seonghobae deleted the fix/rubric-testlet-pilot-control-callbacks-883 branch August 16, 2026 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Harden rubric testlet pilot controls before caller callbacks

1 participant