Enforce multi-item IRT and improve local judge schema reliability

README.md

@@ -93,14 +93,57 @@ print(fixed_item_calibration.best)
   strict structured parsing. A judge result becomes an IRT row only through
   LLMJudgeResult.to_irt_row() with at least two criteria, followed by
   validate_irt_response_matrix() for a multi-item dichotomous or explicitly
-  categorized polytomous matrix. Equal-width direct K-way projection is
-  experimental; opt-in `category_method="cumulative_threshold"` evaluates each
-  ordered boundary with a strict Boolean vector and derives the category in the
-  adapter. Category-count and prompt-perturbation calibration remain required
-  for both methods. See
+  categorized polytomous matrix. When `category_count` is supplied without an
+  explicit method, the adapter defaults to `category_method="binary_threshold"`:
+  each ordered boundary is a bounded Boolean call, and malformed or
+  non-monotone evidence fails closed. Equal-width direct K-way projection is
+  calibration-only and requires explicit `category_method="direct"`;
+  `category_method="cumulative_threshold"` remains an explicit alternative
+  that asks for one strict Boolean vector. Binary threshold judging has a
+  maximum of 64 calls per result.
+  When the injected contextual-orchestrator exposes its bounded
+  `client.local_concurrency`, those independent boundary calls reuse that limit;
+  generic injected orchestrators remain sequential by default.
+  `ContextualOrchestratorJudge` accepts only an adapter declaring
+  `contextual_orchestrator_contract == "contextual-orchestrator-contract-v1"`;
+  an arbitrary direct provider transport fails at construction. This is an
+  architectural provenance marker, not a security credential; production
+  adapters must still route, trace, and validate through contextual-orchestrator.
+  A failed binary boundary raises `JudgeFormatError` with bounded `.evidence`
+  containing call/parse status, partial trace-step counts, usage, and ordered
+  boundary records; callers must retain that failure in calibration results.
+  For semantically interpretable polytomous calibration, callers may provide a
+  `category_anchors` tuple of exactly K definitions on every `JudgeCriterion`;
+  those definitions are carried as rubric data to each binary boundary. Mixed,
+  incomplete, or mismatched anchor sets are rejected. Omitted anchors remain a
+  backwards-compatible exploratory mode and must not be treated as gold
+  calibration evidence. Each binary prompt asks whether the answer meets at
+  least the requested boundary (not exactly that category), requires
+  criterion/task relevance, and rejects generic intent, unrelated detail,
+  missing-control admissions, or rubric repetition as evidence.
+  Category-count and prompt-perturbation calibration remain required for all
+  methods. See
   [ADR 0005](https://github.com/ContextualWisdomLab/contextual-orchestrator/blob/1b7dbd2a46533f41072def1fb94283147134cab5/docs/planning/adrs/0005-irt-response-matrix-contract.md),
   [ADR 0006](https://github.com/ContextualWisdomLab/contextual-orchestrator/blob/1b7dbd2a46533f41072def1fb94283147134cab5/docs/planning/adrs/0006-polytomous-llm-judge-bias-calibration.md), and
   [ADR 0008](https://github.com/ContextualWisdomLab/contextual-orchestrator/blob/1b7dbd2a46533f41072def1fb94283147134cab5/docs/planning/adrs/0008-fast-judge-review-hardening.md).
+  Production and benchmark IRT readiness additionally requires every declared
+  polytomous category to be observed at least once for every item; low-level
+  diagnostic fitters remain available for partial-occupancy fixtures.
+  `build_multiple_choice_calibration_cases()` and
+  `evaluate_paired_calibration()` provide bounded paired controls for
+  baseline, option-only/no-question, shuffled-option, and
+  distractor-replacement variants. They accept an existing
+  `ContextualOrchestratorJudge`, so every calibration call uses the
+  contextual-orchestrator route; provider, parse, semantic, and IRT failures
+  remain in the denominator, with no retry, repair, keyword matching, or
+  positional category inference. Gold categories and contamination status are
+  caller-supplied, and every successful result must project to multiple
+  criterion columns before it can be used as a polytomous row. Paired score
+  deltas are diagnostic sensitivity evidence, not a causal positive-option-
+  count law or a claim of judge debiasing. When a binary judge raises a
+  `JudgeFormatError`, the report retains its bounded `.evidence` (boundary
+  statuses, parse state, trace counts, and usage) while excluding source text
+  and raw model output.
   Cross-repository exact-head review, structured Strix evidence, and merge
   policy are recorded in [contextual-orchestrator ADR 0004](https://github.com/ContextualWisdomLab/contextual-orchestrator/blob/1b7dbd2a46533f41072def1fb94283147134cab5/docs/planning/adrs/0004-pr-review-merge-loop.md) and
   [ADR 0009 dependency cooldown](https://github.com/ContextualWisdomLab/contextual-orchestrator/blob/1b7dbd2a46533f41072def1fb94283147134cab5/docs/planning/adrs/0009-supply-chain-dependency-cooldown.md).

docs/adr/0010-llm-orchestration-and-credentials.md

@@ -12,10 +12,42 @@ Some `fast-mlsirm` features and validation studies may use LLMs for item generat
 ### Product/model calls
 
 - Prefer provider-neutral interfaces.
-- Use `contextual-orchestrator` when a reusable orchestration boundary is appropriate, but do not create a reverse source dependency or write there while its owner loop controls that repository. Every cross-repository call MUST bind a versioned request/result schema (for example, `contextual-orchestrator-contract-v1`) or an immutable artifact digest, and the compatibility policy for that contract must be recorded with the caller.
+- Every LLM-as-a-Judge call MUST use a contextual-orchestrator-backed adapter. The adapter must declare the versioned `contextual-orchestrator-contract-v1` boundary; `ContextualOrchestratorJudge` rejects an unmarked or direct-provider transport at construction. Do not create a reverse source dependency or write there while its owner loop controls that repository. Every cross-repository call MUST bind the versioned request/result schema or an immutable artifact digest, and the compatibility policy for that contract must be recorded with the caller.
 - Treat all model output as untrusted; schema/provenance/semantic validation remains inside the calling workflow.
 - Deterministic tests/gates that do not require a model call shall remain executable without model credentials.
 
+### Cross-repository judge contract
+
+The versioned boundary is also a public package contract: fast-mlsirm exports
+`CONTEXTUAL_ORCHESTRATOR_CONTRACT_V1` from both `fast_mlsirm.llm_judge` and the
+package root with the value `contextual-orchestrator-contract-v1`. The
+contextual-orchestrator same-interpreter preflight is authoritative for live
+judge readiness; a working injected judge with a missing public contract symbol
+is an integration failure, not an available integration. A regression test must
+cover the package-root export before a live MLX judge result is treated as
+cross-repository evidence.
+
+The two repositories remain separately installable and do not acquire a reverse
+source dependency. That boundary is easy to misconfigure: isolated project
+virtual environments cannot import the other checkout. A live source checkout
+therefore MUST use one interpreter with both packages installed (prefer editable
+installs), or explicitly expose both source roots with `PYTHONPATH`, and MUST
+run `python -m contextual_orchestrator check-fast-mlsirm` with that interpreter
+before judging. A missing cross-import is a fail-closed integration error; it
+must not be hidden by a second interpreter, keyword/positional repair, or direct
+provider fallback.
+
+### Failure-evidence boundary
+
+Failed binary-threshold calls may expose bounded operational evidence, but they
+must not retain raw provider output or exception text: either can contain task,
+answer, reference, rubric, or provider diagnostics. Evidence is therefore an
+allowlisted record of criterion/threshold identity, call/parse status, stable
+failure code, exception type, validated Boolean values, trace count, and usage.
+This redaction boundary applies before callers persist or publish
+`JudgeFormatError.evidence`; it is not a keyword, positional, or silent-repair
+mechanism.
+
 ### Credentials
 
 - Model-backed GitHub tests/agents use the existing GitHub Secret `NVIDIA_NIM_API_KEY` when the model call requires NVIDIA NIM.
@@ -55,6 +87,8 @@ Costs:
 - live-model validation requires explicit external service availability;
 - orchestration evidence can be expensive;
 - some end-to-end tests remain bounded/scheduled rather than always-on PR gates.
+- test doubles must explicitly declare the same contract marker; an unmarked
+  fake is not evidence of a valid production transport path.
 
 ## Alternatives considered
 

docs/adr/0013-continuous-execution-and-documentation-governance.md

@@ -74,6 +74,18 @@ paths. The matrix must state target architecture and protected-main behavior
 separately until ADR-0011 is implemented; the active PR applies that wording
 correction.
 
+The cross-repository Strix security dependency exposed an additional evidence
+boundary: a successful status and a named artifact are not enough to establish
+that the report belongs to the exact workflow run being used by this PR. The
+central workflow must bind the target repository, the exact `strix-reports`
+artifact name, the outer GitHub Actions run ID, the full PR head, the report
+path, and the report digest. Consumers must require exactly one non-expired
+artifact of that name before download and reject provider-internal run IDs,
+missing fields, duplicates, expiry, or mismatches. Until that structured
+binding exists on protected `main` and the current fast-mlsirm head is
+rechecked, Strix output remains provider/content evidence only; it cannot
+authorize Judge, IRT, or Merge acceptance.
+
 ## Consequences
 
 ### Positive