Enforce multi-item IRT and improve local judge schema reliability - #816
Conversation
…egression' into codex/fast-judge-accepted-type-regression
…770) * test(jmle): add optimizer-mode recovery evidence * docs(doctoring): define JMLE optimizer recovery evidence * test(jmle): identify recovery scale before error gates * docs(jmle): document affine recovery identification * test(jmle): avoid quasi-separated recovery fixture * test(jmle): eliminate person-separation confound in recovery fixture * fix(jmle): recognize accepted L-BFGS objective convergence
…pted-type-regression-rebased # Conflicts: # docs/documentation_coverage.md # tests/test_architecture_documentation_contract.py # tests/test_cov_c_fitstats.py # tests/test_documentation_coverage_fitness.py
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headd661b97ac99cf784b8d62db65ee1f435da6e0f9d. -
Head SHA:
d661b97ac99cf784b8d62db65ee1f435da6e0f9d -
Workflow run: 31861353078
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (17 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (17 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (7 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (7 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (12 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (12 files)"]
R3 --> V3["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage evidence job did not run or did not publish coverage evidence. Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (17 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (17 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (7 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (7 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (12 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (12 files)"]
R3 --> V3["targeted test run"]
|
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head2ef29e72f6aeede8961cde31001429156f825365. -
Head SHA:
2ef29e72f6aeede8961cde31001429156f825365 -
Workflow run: 31863493124
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (18 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (18 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (7 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (7 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (13 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (13 files)"]
R3 --> V3["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage evidence job did not run or did not publish coverage evidence. Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (18 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (18 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (7 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (7 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (13 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (13 files)"]
R3 --> V3["targeted test run"]
|
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current heade750a44181ebd079d72f90d8f698a0361a2e25e7. -
Head SHA:
e750a44181ebd079d72f90d8f698a0361a2e25e7 -
Workflow run: 31867996825
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (18 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (18 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (7 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (7 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (13 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (13 files)"]
R3 --> V3["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage evidence job did not run or did not publish coverage evidence. Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (18 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (18 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (7 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (7 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (13 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (13 files)"]
R3 --> V3["targeted test run"]
|
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
Admin green-wave: product CI SUCCESS; dismiss bot request.
github-project-automation
Bot
moved this from In Progress
to Done
in naruon Platform Roadmap
1 participant
Scope
This PR hardens fast-mlsirm's provider-neutral LLM-as-a-Judge and public IRT response contract:
Exact current identity
main:fb67ced09d8ee00542c05d56374537a9a7239751.f3249f613ad6a87e261dc2938ee1af2552bb67ca.main.MERGEABLEbutBLOCKED, auto-merge disabled, aggregate reviewREVIEW_REQUIRED.Every review, status, approval, or check anchored to another head is historical only.
Implemented controls
NaN,-1, and masks.docs/papers/and citation-only records where redistribution was not established.Current evidence
ContextualOrchestratorJudge -> _FastMLSIJudgeAdapter -> TaskOrchestrator -> ModelClient -> mlx-lm, with a Gemma 4 e4b worker, strict binary boundaries, two criteria, and an IRT row[2,2]. This is transport/shape evidence only, not semantic quality or unbiasedness evidence.31800704581, dynamic CodeQL31800701211, Strix31800702566, package/Rust/Python/fuzz/GPU jobs, security fan-out31800704465, Semgrep31800704450, and required bootstrap/Noema/queue jobs. Queued or pending is not passing evidence.31797316247/ artifact9219034211ended success but had noevidence-binding.json; nestedrun.jsonhad null repository/head/commit/report metadata. Preserve it as provider/content evidence only, not as a clean security gate..githubPR fix(model-comparison): replay casewise trust boundary on current review workflow #1009 is Ready at2833d8a1c2f2cbb02387a2af752db51298cc64c4, but remainsBLOCKEDwith aggregateREVIEW_REQUIRED, queued hosted checks, and no qualifying independent current-head approval. Its aggregate-review scheduler gate and structured Strix artifact-hold fix still require normal protected integration; structured same-head evidence must be regenerated after protected-main integration.Remaining protected gates
No predecessor, bot-comment, CodeRabbit pass, local-only suite, unbound artifact, status-only result, self-approval, Admin bypass, keyword match, positional repair, or silent drop is Merge authority.
Fresh routed recheck — 2026-08-15
Latest exact-head update — 2026-08-15
f3249f613ad6a87e261dc2938ee1af2552bb67carecords the terminal Strix run31819951865/job94830599923: zero reported vulnerabilities, but artifact9227098928is unbound and its scan workspace reported tracked package files as missing.Terminal exact-head audit — 2026-08-15