feat(routing): add stateless candidate controls

[seonghobae]

Summary

• add strict request-local routing.candidate_id and routing.exclude_candidate_ids controls for virtual chat and Responses requests
• enforce the controls across ordinary, structured/tool, and streaming execution without persisting routing state or expanding /v1/models
• return requested/excluded/attempted/served candidate evidence only when controls are supplied
• keep active controls in the response-cache key and force them onto synchronous execution

Contract

• candidate_id is an exact private agent ID
• exclude_candidate_ids contains unique non-empty private agent IDs; there is no repository-authored candidate-count cutoff
• unknown, disabled, non-chat, non-ZDR, excluded pins and concrete-model combinations fail with HTTP 400 invalid_routing
• omitted controls preserve the existing request and response shape
• serving identity is reported only from exact answering_step_id or explicit served_agent_id provenance; missing identity fails closed instead of being inferred from output equality or trace order

No-heuristics correction — 2026-09-02

Fresh RCA found two decision-affecting rules that were not established by RouteLLM, FrugalGPT, an API standard, or measured deployment evidence: the fixed 32-ID exclusion ceiling and the historical serving-candidate fallback that guessed identity from matching output text / trace position. Both violate the repository's no-heuristics contract.

RED regression commit ef5ae0143a2b19eb32ad3e826a6c21755ba7451d proves that more than 32 exclusions must remain admissible under the normal authenticated request-size boundary and that a historical trace lacking explicit serving identity must omit served_candidate_id. Exact-guarded one-shot repair machinery was added in 3d045c6ee416821e23c6f27963921e5d591e136d and 6d93c16369b967d559e556f1170a1acd561ce1e5; canonical trigger head is 5166349dea1be54d53edc3e33ce185a90b159163. Source-fix run 33585788325 is queued and is not GREEN evidence.

The repair removes the numeric exclusion limit from both HTTP and Python-API validation; normal authenticated request-body controls remain the resource boundary. It also removes output-equality and first/last-row serving inference. Exact answering_step_id remains authoritative for multi-step workflows, explicit served_agent_id remains authoritative for provider-shaped paths, and records with neither identity remain attempt provenance only. ADR 0032, docs/product-technical-gap-baseline.md, README, CHANGELOG, and executable regressions are updated by the same verified one-shot.

Verification

Previous focused verification remains historical only after the new head. Hosted exact-head checks and the source-fix RED→GREEN run are authoritative. The one-shot removes itself only after the focused candidate-control/API regression suite is GREEN and then publishes by ordinary non-force push; branch movement causes a fail-closed stop rather than an overwrite.

Research / architecture

ADR 0032 retains its RouteLLM/FrugalGPT routing basis. Candidate controls are operational evidence inputs, not a learned-routing claim. The 2026-09-02 amendment makes the previously implicit distinction executable: caller-provided exact membership is accepted without an arbitrary cardinality ranking rule, while serving identity is emitted only when provenance identifies it exactly.

---

Summary by CodeRabbit

• 새로운 기능

  • 신뢰된 호출자는 요청별로 특정 후보를 지정하거나 후보를 제외할 수 있습니다.
  • 후보 라우팅은 일반, 구조화, 스트리밍 Chat 및 Responses 요청에 적용됩니다.
  • 응답의 orchestration.routing에서 요청된 후보, 제외 후보, 시도된 후보와 실제 제공 후보를 확인할 수 있습니다.
  • 라우팅 설정은 검증되며 요청 간에 저장되지 않고 모델 카탈로그에도 표시되지 않습니다.

• 문서

  • README, 변경 로그 및 아키텍처 문서에 후보 라우팅 사용법과 제약 사항을 추가했습니다.

• 테스트

  • 후보 검증, 스트리밍, 캐시, 경쟁 실행 및 라우팅 증거 처리에 대한 검증을 강화했습니다.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

요청별 routing.candidate_id 및 exclude_candidate_ids 제어를 추가했습니다. 채팅과 Responses 경로에서 후보를 검증하고 선택에 적용합니다. 동기·구조화·스트리밍 응답에 orchestration.routing 증거를 제공합니다.

Changes

요청별 후보 라우팅

Layer / File(s)	Summary
라우팅 계약과 입력 검증 contextual_orchestrator/api_contract.py, contextual_orchestrator/server.py, fuzz/targets.py, tests/test_api_contract.py	CandidateRoutingControls와 CandidateRoutingEvidence 스키마를 추가했습니다. 채팅 및 Responses 요청에서 후보 ID, 제외 목록, 엔드포인트, 역할, 태그를 검증합니다.
요청별 후보 선택 정책 contextual_orchestrator/orchestrator.py	ContextVar로 후보 정책과 시도를 추적합니다. 후보 고정과 제외를 후보 선택, 워크플로, 트리아지, 파일 제공자, 재시도, 캐시 키에 적용합니다. 실제 답변 단계와 provider 호출을 기준으로 served_candidate_id와 시도 목록을 계산합니다.
동기·직접·스트리밍 경로 통합 contextual_orchestrator/cost_router.py, contextual_orchestrator/orchestrator.py, contextual_orchestrator/server.py	활성 후보 제어가 있으면 batch 및 지연 라우팅을 동기 실행으로 정규화합니다. 채팅, Responses, 직접 호출, 구조화 호출, 오케스트레이션, 스트리밍 경로에서 후보 정책 컨텍스트와 증거를 재사용합니다.
실행 검증과 무휴리스틱 증거 tests/test_candidate_routing_controls.py, tests/test_endpoint_race.py, tests/test_measured_routing_evidence.py, tests/test_model_judge.py, tests/test_candidate_routing_no_heuristic_limits.py	잘못된 입력, 후보 적격성, vision 조건, 동기 정규화, 스트리밍 triage, 동시 시도, 캐시 적중, 구조화 호출, 명시적 serving identity를 검증합니다.
문서와 수정 자동화 README.md, CHANGELOG.md, docs/planning/adrs/0032-model-group-cost-aware-discovery.md, scripts/source_fix_983_no_heuristic_candidate_controls.py, .github/workflows/source-fix-983-no-heuristic-candidate-controls.yml	요청별 후보 제어 계약을 문서화합니다. 후보 제외 목록의 저장소 작성 제한과 serving identity 휴리스틱을 제거하는 일회성 수정 경로를 추가합니다.

Estimated code review effort: 5 (Critical) | ~120 minutes

Merge Risk: 🟠 High · up to b05bf

The PR adds caller-controlled candidate pinning, exclusions, and routing provenance to public request paths. At the current head, valid requests with more than 32 exclusions can still be rejected, ambiguous execution history can produce incorrect serving identity, and a write-enabled workflow can mutate and push source after review, creating contract failures, misleading routing evidence, and unreviewed behavior changes; the PR is not ready to merge.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Server
  participant TaskOrchestrator
  participant CostRoutingCoordinator
  participant Provider
  Client->>Server: routing controls 포함 요청
  Server->>TaskOrchestrator: 후보 정책 검증
  Server->>CostRoutingCoordinator: 검증된 routing 전달
  CostRoutingCoordinator->>TaskOrchestrator: 후보 정책 적용
  TaskOrchestrator->>Provider: 허용된 후보 호출
  Provider-->>TaskOrchestrator: 응답과 trace 반환
  TaskOrchestrator-->>Server: routing evidence 반환
  Server-->>Client: orchestration.routing 포함 응답

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 44.44% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 99 functions across 11 files. (3 skipped:…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	제목은 요청별 상태 비저장 후보 라우팅 제어 추가라는 PR의 주요 변경 사항을 정확하고 간결하게 설명합니다.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 44.44% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 99 functions across 11 files. (3 skipped: 2 unsupported, 1 too large.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/stateless-candidate-controls

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

Exact-head remediation evidence for a426755ab0122bb9ea0714433b174f34fbf43230:

• Scoped auto-stream triage and every triage fallback to the active pin/exclusion policy.
• Rejected provider-bound file affinity conflicts and filtered final failover candidates.
• Recorded actual request-local provider attempts, including failed advances.
• Rejected candidate controls on legacy Completions and embedding endpoints.
• Kept empty exclusions behavior-identical, hardened conduct fallback, added fuzz coverage, and documented the already-attached RouteLLM/FrugalGPT PDFs.
• Focused contract/regression suite: 94 passed in 128.44s.
• Ruff and git diff --check: passed.

Hosted exact-head gates are pending.

[seonghobae]

Exact environment classification for the two local full-suite import failures at e564871d792adfbc04d8ad7657eb3ef3ca25a5dc (base main@c6c3a0c9d6c2ec8f3509ccdc6db24b1562a32e7b):

• The ambient Python 3.14 environment contains mcp==1.23.3 at ~/.local/lib/python3.14/site-packages/mcp, but that distribution does not export mcp.Client. Because a module named mcp exists, pytest.importorskip("mcp") correctly does not skip; the subsequent patch target fails. This package is absent from pyproject.toml, uv.lock, and requirements.lock by design.
• The same ambient environment reports fast-mlsirm==0.9.1, but resolves a namespace with no module file and no fit_irt_experiment export. This is not the repository-locked artifact.
• In fresh isolated worktrees, uv run --locked --group dev creates the pinned environment. There mcp is absent (the optional test skips), while the locked fast-mlsirm==0.9.1 artifact exports fit_irt_experiment.
• Exact-main locked proof: affected full files plus the two modality regressions — 15 passed, 1 skipped.
• Exact-PR locked proof: same selection — 15 passed, 1 skipped.
• The focused import/vision selection independently produced 3 passed, 1 skipped on both trees.
• Vision regression confirmed green: test_general_free_serving_candidates_excludes_a_free_vision_only_input_model and test_discovery_and_orchestrator_modality_eligibility_cannot_drift both pass on exact main and exact PR head.

Conclusion: this is local ambient dependency drift/namespace contamination, not a PR-scoped or current-main source defect. No dependency declaration, lock, test suppression, or source was changed. Reproduce with the repository contract (uv run --locked --group dev pytest ...), not bare ambient python -m pytest.

[seonghobae]

Cross-PR integration contract: routing identity is provider-neutral model_group only; do not add or preserve a provider-family abstraction. OpenRouter discovery must retain concrete free model IDs, while the aggregate openrouter/free router is not a serving candidate. OpenCode, Noema, and Strix must call contextual-orchestrator. Do not impose fixed wall-clock deadlines on inference, initial ping, readiness/health, provider discovery, or OpenRouter ZDR-list retrieval; use explicit cancellation and evidence-backed transport failure instead. Reconcile this PR with #971 and central .github #1508 before merge.

[seonghobae]

@coderabbitai review

[seonghobae]

@opencode-agent

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[seonghobae]

Round 10: never trust provider-supplied _candidate_routing/orchestration fields

Pushed b05bf900 fixing two fresh Devin bug findings from the review round after round 9 (e8ed2eab) landed.

1. "Provider fields forge routing evidence"

CostRoutingCoordinator.complete()'s provider_request passthrough branch unconditionally republished a _candidate_routing field popped off the raw provider response as gateway-computed orchestration.routing evidence. TaskOrchestrator.proxy_completion() only ever sets that key itself when an active candidate control made _candidate_routing_evidence return non-None — so with no active control, observing that key can only mean it arrived already-present on the provider's own (untrusted) response body. A coincidentally- or adversarially-named provider field could therefore forge fake served_candidate_id/attempted_candidate_ids into an ordinary response, violating the documented "present only when the request supplied routing controls" contract.

2. "Provider metadata crashes tool responses"

server.py's single-agent tool-loop passthrough (the exact block touched in round 7) had the identical trust gap, plus a second bug: result.setdefault("orchestration", {})["routing"] = evidence assumes a pre-existing "orchestration" field (if the provider happened to return one) is a mapping. A provider response with a non-dict "orchestration" field (a string, list, ...) crashed this line with TypeError: 'str' object does not support item assignment after a successful inference call, turning a working response into a 500.

Fix

Both call sites shared the same underlying question — "was a candidate control genuinely active for this request?" — so this adds TaskOrchestrator._has_active_candidate_controls(routing) as the single source of truth (matching candidate_routing_policy's own no-op condition: key presence for candidate_id, presence-and-non-empty for exclude_candidate_ids), used in three places:

• cost_router.py's has_candidate_controls (refactored to call it, replacing the equivalent inline logic round 9 added — same semantics, one implementation)
• cost_router.py's provider_request branch, gating the _candidate_routing republish
• server.py's proxy_tool_request, gating the republish and checking isinstance(orchestration, dict) before merging into it (falling back to a fresh dict — matching the OpenAPI schema's own "orchestration is an object" contract — when the provider's field isn't one)

Two new regression tests (test_coordinator_never_republishes_provider_supplied_candidate_routing_without_active_controls, test_http_tool_loop_never_republishes_provider_supplied_candidate_routing_or_crashes) inject a forged _candidate_routing field and a non-mapping orchestration field via a custom test client and assert: no crash, and the forged evidence never reaches the response. Verified via git stash A/B: both fail against the pre-fix code (the forged evidence leaks through; the HTTP request 500s with the exact TypeError Devin predicted) and pass with the fix.

Concurrent work note

Pulled in four commits from a separate, already in-flight automated repair (source-fix-983-no-heuristic-candidate-controls, addressing distinct Devin findings about a hardcoded 32-ID exclusion cap and served_candidate_id's text-match fallback) via a clean fast-forward merge — entirely disjoint files from this change, no conflicts. That repair's own code-changing commit had not landed yet as of this push, so its regression file (tests/test_candidate_routing_no_heuristic_limits.py) currently has 2 known, pre-existing, not-yet-fixed failures unrelated to this round — not this change's concern to fix, since a separate automated workflow already owns that repair.

Verification

• python -m pytest tests -q: 3349 passed, 5 failed, 2 skipped — the same pre-existing baseline from rounds 8/9, run before the fast-forward merge landed so it does not yet include the 3 no-heuristics regression tests noted above (2 fail there, 1 passes, tracked separately by the other in-flight repair).
• interrogate: 100% docstring coverage repo-wide.
• git diff --check: clean.

---

Generated by Claude Code

---

Generated by Claude Code

[seonghobae]

Completed this PR's stuck source-fix-983-no-heuristic-candidate-controls repair directly — it had been queued 4.5+ hours with zero progress (severe org-wide Actions capacity congestion; only ~3 jobs run concurrently across the repo at any time while hundreds queue). Waiting longer wasn't productive, and per its own design ("branch movement causes a fail-closed stop rather than an overwrite") it was safe to complete without racing it.

Applied scripts/source_fix_983_no_heuristic_candidate_controls.py's documented repair exactly, then merged current main in (bringing the already-landed stale-base test fix from #1005). Two things needed hand-adjustment beyond a blind script run:

1. The script's two orchestrator.py replace_once calls targeting "exclude_candidate_ids must contain at most 32 agent IDs" had gone ambiguous — an earlier round on this branch independently added a second, differently-conditioned raise using identical wording (a pure type-check guard, unrelated to the 32-cap). Applied both by hand, anchored on full surrounding context, and normalized both messages for consistency.
2. Removing the output-equality/trace-position fallback exposed a real regression: route_once() and stream_route()'s own trace-step construction never set served_agent_id (only route_once's cross-endpoint-failover case did), so the stricter evidence resolution silently stopped reporting served_candidate_id for the overwhelmingly common single-candidate route() request — confirmed via tests/test_cache_hit_reports_no_current_candidate_attempt and 4 others going red. Fixed at the source: both paths now record served_agent_id explicitly (equal to agent_id when there was no internal failover) — a real fact being recorded, not an inference, so the evidence function never needs to fall back to bare agent_id/trace position for these paths. This is why test_served_candidate_evidence_fails_closed_without_explicit_identity's hand-constructed single-row-without-explicit-identity case still correctly fails closed: it never goes through route_once/stream_route, so it never gets that explicit field.

Verified: tests/test_candidate_routing_no_heuristic_limits.py + tests/test_candidate_routing_controls.py + tests/test_api_contract.py + tests/test_orchestrated_responses_stream.py — 76/76 passed. python -m compileall clean. git diff --check clean. Cancelled the now-superseded queued source-fix run (id 33586942966) so it stops holding a scarce Actions slot for nothing. Hosted exact-head checks on the new head remain authoritative before merge.

---

Generated by Claude Code

[seonghobae]

Follow-up: fixed a regression introduced by the no-heuristics repair (commit 76527070)

A full local suite run after the earlier no-heuristics candidate-routing-evidence fix surfaced a real regression it introduced: making route_once/stream_route stamp served_agent_id on every trace row unconditionally (so _candidate_routing_evidence has an explicit serving fact even when the serving agent is unchanged) broke a separate, pre-existing regression guard in test_provider_reliability.py and test_tool_execution_fallback.py — the ordinary, no-candidate-policy path must never carry served_agent_id/failover metadata for an unchanged serving agent.

Root-caused by diffing a clean origin/main worktree (passing) against this branch's head (failing) to rule out full-suite test pollution before concluding it was real.

Fix: the unconditional stamp now only fires while request-local candidate-attempt tracking is actually active (inside a candidate_routing_policy scope). The ordinary path's trace-row shape is unchanged; failover still always stamps it regardless of policy state.

Full local suite: 3355 passed, 2 pre-existing sandbox-only failures (unrelated — fast_mlsirm unavailable under this sandbox's proxy policy, and a known local tokenizer-usage-source artifact in test_spend_analytics that passes on real CI).

---

Generated by Claude Code

[seonghobae]

@jules Fresh exact-head verification found a remaining source/contract contradiction on 765270700e681d18afcf0b39c95e9310ea2477da; repair it directly with ordinary commits. Do not reintroduce the removed source-fix-983 workflow/script, force-push, or transfer GREEN evidence from the earlier one-shot.

Current contextual_orchestrator/api_contract.py still publishes CandidateRoutingControls.exclude_candidate_ids.maxItems = 32 at this exact head, while the PR body, CHANGELOG, ADR direction, runtime contract, and Devin resolution at 2026-09-02 09:05 UTC all state that the unsupported 32-ID cutoff has been removed. This means generated/OpenAPI clients still reject requests the runtime intentionally accepts; the current documentation claim is false at source.

RED/GREEN acceptance:

• Add/retain an exact OpenAPI regression that validates an exclusion list longer than 32 (within the normal authenticated request-body limit) and fails on the current schema.
• Remove the schema-only maxItems: 32; do not replace it with another repository-authored candidate-count heuristic. Keep uniqueItems, lexical ID constraints, request-size/resource boundaries, endpoint/candidate role validation, and synchronous-control semantics intact.
• Verify the runtime Python/API validation also has no hidden 32-item cutoff, and that omitted/empty exclusions preserve the old response/cache contract.
• Re-run the focused candidate/API/stream/cache/file-affinity suite and exact-head required Tests/Fuzz/Security/SAST once hosted capacity is available. Leave the two remaining informational race/cache review threads alone unless they become real defects; the outdated source-fix-script thread is obsolete because that machinery is absent from the current diff.
• Preserve fail-closed served identity: only answering_step_id or explicit served_agent_id, never output-equality/trace-order inference.

Do not mark this GREEN until the live exact head itself, not a bot-plan or resolved-review annotation, contains the schema correction and regression.

[seonghobae]

Repaired in 05eb95f2 (not @jules — this session picked it up directly since I had full context on the earlier served_agent_id/no-heuristics work on this PR).

• Removed maxItems: 32 from CandidateRoutingControls.exclude_candidate_ids in contextual_orchestrator/api_contract.py. No replacement cardinality heuristic. uniqueItems, lexical ID constraints, and normal authenticated request-size bounds are unchanged.
• Added a RED-before/GREEN-after regression in tests/test_api_contract.py validating a 64-ID exclusion list against the schema — confirmed it fails against the pre-fix schema and passes against the corrected one.
• Verified server.py's _validate_routing (the runtime validator) has no other hidden count-based cutoff on this field — grepped for stray 32/length checks near exclude_candidate_ids, none found.
• Left the fail-closed served-identity contract (answering_step_id/explicit served_agent_id only, never output-equality/trace-order inference) untouched, per your note.
• source-fix-983-* machinery is confirmed absent from the current diff (already removed earlier this session).

Full relevant suite (test_api_contract.py, test_candidate_routing_controls.py, test_candidate_routing_no_heuristic_limits.py): 54 passed. CHANGELOG.md and docs/product-technical-gap-baseline.md updated with dated entries.

---

Generated by Claude Code

[seonghobae]

Rebased this PR onto current main (it was stuck against a stale base sha 8839081..., mergeable_state: dirty).

CI status before merge: all checks green (CodeRabbit, Devin Review completed).

Merge: git merge origin/main hit one real conflict, resolved as follows:

• fuzz/targets.py: both this PR and main's fix(rater): preserve generic v1 and add criterion-bound wrapper #917 (governed rater observation) independently appended a new 11. entry to the module docstring's numbered list of fuzzed untrusted-input surfaces — this PR's server._validate_routing, main's rater_observation.RaterInvocation.from_mapping. Kept both entries, renumbered sequentially as 11. and 12.. Purely additive documentation text, no logic conflict — the actual exercise_* functions in the file were untouched and merged automatically with no conflict.

All other files (CHANGELOG.md, contextual_orchestrator/__init__.py, etc.) auto-merged cleanly.

Verification (Python 3.12 venv, pip install --require-hashes -r requirements.lock + pip install --no-deps -e .):

• This PR's own test files (test_candidate_routing_controls.py, test_candidate_routing_no_heuristic_limits.py, test_api_contract.py, test_endpoint_race.py, test_measured_routing_evidence.py, test_model_judge.py) — 145 passed
• Broader sweep of every test touching the heavily-modified files (-k "cost_router or routing or orchestrator or server or api_contract or endpoint_race or model_judge", 516 tests) — 515 passed, 1 failed: test_psychometric_routing.py::test_fast_mlsirm_fit_uses_judge_acceptance_item_for_context_score (ModuleNotFoundError: fast_mlsirm — this is the documented pre-existing sandbox-only failure; the private fast-mlsirm GitHub-archive dependency is blocked by this sandbox's egress policy, unrelated to this PR's diff)
• tests/test_planning_adr_identifiers.py — passed (no ADR numbering collision this time)
• interrogate on all four heavily-touched source files (server.py, orchestrator.py, cost_router.py, api_contract.py) — 100% docstring coverage

Pushed directly to feat/stateless-candidate-controls (no force-push).

---

Generated by Claude Code

[devin-ai-integration]

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 2 new potential issues.

[devin-ai-integration]

🟡 Cache hits misreport serving candidate

When auto triage runs before a response-cache hit, _candidate_routing_evidence reuses the cached trace's serving identity. The response falsely attributes cached output to a provider.

Prompt for agents

Prevent candidate routing evidence from deriving served_candidate_id from a cached trace. In contextual_orchestrator/orchestrator.py, _candidate_routing_evidence currently suppresses explicit serving identity only when tracked_attempts equals an empty list. An auto-mode request can make a fresh triage provider call, leaving tracked_attempts nonempty, and then hit a durable or still-live response cache whose trace belongs to the original request. Use the result's cache_status to ensure cache hits never publish served_candidate_id from historical trace rows, while retaining current attempted_candidate_ids such as a real triage call. Add coverage where the response cache is warm but the triage cache is cold or cleared.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[seonghobae]

Fixed one confirmed-stale invariant in the fuzz target; two other open findings verified as already-moot or too delicate to touch blind

Triaged this PR's ~62 review threads. Most substantive findings were already resolved through this session's earlier rounds. Checked the remaining open ones:

Fixed (commit 1889e1ae): Devin flagged that fuzz/targets.py's exercise_request_body still asserted len(excluded) <= 32 on the real server._validate_routing() output, even though the fixed 32-ID exclusion ceiling was removed from both the OpenAPI schema and the runtime validator earlier in this PR. Confirmed real — the fuzz target was asserting an invariant its own driven code no longer enforces, so a legitimately-valid >32-item exclusion list would report as a false-positive fuzzing crash. Removed the stale assertion and updated the module docstring's matching "bounded" claim.

• RED confirmed: reverted the change, reran the new deterministic regression (40 unique IDs) — AssertionError at the removed line, exactly as described.
• GREEN: new regression passes; full tests/fuzz/test_fuzz_properties.py — 20 passed. interrogate on fuzz/targets.py: 100%. git diff --check: clean. Pushed non-force on top of the existing branch (811ff84b → 1889e1ae).

Verified already moot: the scripts/source_fix_983_no_heuristic_candidate_controls.py uniqueness-check finding (CodeRabbit, marked is_outdated) — confirmed that script and its companion workflow no longer exist at the current head; both were temporary repair machinery already removed in an earlier round.

Left open, not attempted: "Cache hits misreport serving candidate" (Devin, current head, orchestrator.py:4231) — traced this through _candidate_routing_evidence/complete()'s cache path and confirmed the described mechanism is real: when a live triage call populates tracked_attempts inside an active candidate-routing scope but the final answer is then served from the response cache (cache_status: "hit", a copy.deepcopy of an unrelated historical record), the fallback branch can still scan that stale cached trace for served_agent_id and misattribute it as the current response's serving candidate. This is genuine, but _candidate_routing_evidence is a serving-identity trust boundary this PR has already revised ~8 times today with very deliberate, narrow scoping by the owner (fail-closed identity, answering_step_id vs served_agent_id precedence, race-attempt ordering, etc.) — I didn't have enough confidence in exactly how far a cache_status-based guard should reach (both identity branches, or only the fallback one) to make a correctly-scoped change blind, so I'm leaving it for the owner rather than risk a subtly wrong fix in this exact function.

---

Generated by Claude Code

[seonghobae]

Merged stale main; root-caused and fixed the CI-only duplicate-call regression (pre-existing bug, not caused by this PR's own commits)

Base merge: git fetch origin main && git merge --no-edit — clean, no conflicts (fuzz/targets.py and tests/fuzz/test_fuzz_properties.py auto-merged with no manual resolution needed this time). Pulled in main's test_admin_contract.py::test_model_group_mutations_refresh_audit_events import json fix (main PR #1035). Confirmed that test passes post-merge.

The duplicate-call regression (test_http_auto_preflight_accepts_worker_only_pin_when_free_model_always_routes and test_coordinator_auto_route_only_pin_succeeds_for_free_model, both new in this PR): neither failed in any earlier local round, in isolation or full-suite, because this sandbox's blocked fast-mlsirm GitHub-archive download (documented since the first 2026-09-02 entry in docs/product-technical-gap-baseline.md) always short-circuits _model_judge_verification to its "fast-mlsirm judge is unavailable" fail-closed return before a judge is ever selected — masking a real bug that only a hosted run with fast-mlsirm actually importable can exercise. Confirmed with Python 3.12 + the exact lock (minus the blocked archive, matching CI's uv run --locked toolchain) that this is genuinely environment-gated, not test-order-dependent: both tests pass every way I ran them locally.

Root cause (pre-existing — present unchanged at merge-base 212ff437, predates this PR entirely): _ranked_agents deliberately still returns role-ineligible members, appended after every eligible one (per its own docstring). Every other caller in orchestrator.py that wants only role-eligible candidates re-applies role not in agent.provider_exclusions itself (_plan_generated, _parse_workflow_plan) — but _model_judge_verification's judge-selection next(...) was missing that filter. With a single-candidate pool excluded from verifier (this PR's own new orchestrator/free worker-only provable-route fixture), it picked that ineligible agent as judge anyway instead of failing closed, producing an extra, unrequested live call once fast-mlsirm is actually importable. _invoke's own failover path already enforces this exact exclusion for a backup judge (test_fast_mlsirm_judge_failover_honors_verifier_exclusions); this closed the identical gap for the primary selection. This is unrelated to, and does not touch, the _candidate_routing_evidence/served-identity trust boundary the prior round deliberately left open — different function, different bug class (a selection-eligibility check, not identity inference).

Fix (contextual_orchestrator/orchestrator.py): added if "verifier" not in agent.provider_exclusions to the judge-selection generator. One line plus a documentation comment explaining the precedent.

Verification:

• RED-before/GREEN-after: new regression test_model_judge_never_selects_a_verifier_excluded_sole_candidate (tests/test_model_judge.py) — stashed the production fix, reran: AssertionError: assert ['worker_only'] == [] (judge constructed with the excluded agent). Restored the fix: passes, judge never constructed.
• test_model_judge.py + test_candidate_routing_controls.py + test_candidate_routing_no_heuristic_limits.py + test_api_contract.py + test_admin_contract.py: 98 passed.
• Full local suite (Python 3.12, matching CI's toolchain): 3441 passed, 2 pre-existing sandbox-only failures unrelated to this change and already documented in docs/product-technical-gap-baseline.md's 2026-09-02 entry (fast_mlsirm unavailable; test_spend_analytics's local-tokenizer artifact — same missing-fast-mlsirm mechanism, confirmed to flip an unrelated conduct-mode trace's usage_source from mixed to tokenizer when the judge step never runs).
• interrogate on orchestrator.py: 100%. git diff --check: clean.

CHANGELOG.md and docs/product-technical-gap-baseline.md updated with a dated entry per this PR's established practice. Pushed non-force: 1889e1ae → 47011b81.

---

🤖 Generated with Claude Code

---

Generated by Claude Code

[seonghobae]

Fixed: "ZDR pins skip workflow triage" — confirmed real, fixed in 53aa9c5c

Confirmed against exact head 47011b81 before touching anything: Devin's claim was correct, and it was a genuine functional gap, not a nitpick.

Root cause. _compute_triage_verdict's free-only ranking (_ranked_agents(text, "worker", free_only=True)) is always empty when zdr_only pins a paid candidate — the pin restricts every candidate list to that one agent (_request_candidate_allowed), and a paid agent never satisfies free_only. The empty-pool fallback that exists precisely to recover an evidence source in that case was gated behind if not candidates and not _REQUEST_ZDR_ONLY.get(), so it was skipped whenever ZDR was active — even though the fallback's own per-agent filter already re-derives ZDR eligibility (_zdr_agent_allowed) and the active pin (_request_candidate_allowed) itself. The gate discarded the one legitimate evidence source (the pinned candidate) instead of protecting against anything real, so the request silently defaulted to route with zero triage call and zero routing evidence, regardless of task complexity. Traced the call chain (_needs_workflow → would_route → _dispatch) to confirm this isn't caught anywhere else — no other layer re-attempts triage once _compute_triage_verdict returns False.

Fix. Removed the not _REQUEST_ZDR_ONLY.get() gate. The fallback build is now unconditional on an empty free-only pool; it narrows itself to ZDR-eligible/pin-matching agents on its own, so this changes nothing when zdr_only is inactive and costs nothing extra when it's active with genuinely no eligible agent (still returns False, same as before).

Regression test (tests/test_candidate_routing_controls.py::test_auto_zdr_only_paid_pin_still_gets_a_live_triage_decision): posts an auto-mode request with zdr_only=true and routing.candidate_id pinning a single paid privacy:zdr candidate; asserts triage genuinely calls the pinned candidate and the conduct verdict is honored (multi-step workflow runs, not the one-call route path). RED-before/GREEN-after verified via git stash A/B on orchestrator.py alone — without the fix, client.calls == ["paid_zdr_agent"] (one call, the direct route — triage silently skipped); with the fix, the triage call happens first and conduct proceeds.

Verification:

• tests/test_candidate_routing_controls.py — 46 passed (1 new).
• tests/test_candidate_routing_no_heuristic_limits.py + tests/test_model_judge.py — 44 passed.
• Broader ZDR/routing sweep (test_api_contract, test_batch_embeddings, test_batch_routing_boundaries, test_cost_review_server, test_cost_router, test_cost_router_boundaries, test_generated_workflow, test_model_discovery, test_model_group, test_orchestrated_responses_stream, test_passthrough_provider_failover, test_privacy_policy_analysis, test_provider_bootstrap, test_provider_catalog_bootstrap, test_provider_catalog_store, test_routing_endpoint_constraint, test_measured_routing_evidence) — 518 passed, 1 skipped.
• Full suite (PYTHONPATH=. python -m pytest tests -q): 3442 passed, 2 failed, 2 skipped in 755.44s — the 2 failures are exactly this session's already-documented, sandbox-only baseline (test_fast_mlsirm_fit_uses_judge_acceptance_item_for_context_score: blocked fast_mlsirm archive download; test_exact_output_without_prompt_usage_is_explicitly_unavailable: same missing-fast-mlsirm mechanism shifting usage_source), unrelated to and unaffected by this change.
• interrogate contextual_orchestrator/orchestrator.py: 100%.
• git diff --check: clean.

CHANGELOG.md updated with a dated entry per this PR's established practice. Freshness re-checked immediately before push (head unchanged at 47011b81); pushed non-force: 47011b81 → 53aa9c5c.

---

🤖 Generated with Claude Code

---

Generated by Claude Code

[seonghobae]

Adjudication evidence (host 1 session, 2026-09-06 KST; full report with commands in #1080). Nothing here closes, flips, or retargets anything — the decision is the opener's.

Semantic duplicate mechanism with #1032. This PR adds _REQUEST_EXCLUDED_CANDIDATE_IDS / _REQUEST_ATTEMPTED_CANDIDATE_IDS ContextVars and _record_candidate_attempt at every call site; #1032 adds _REQUEST_ELIGIBLE_AGENT_IDS + _request_eligibility_scope for the same per-request eligibility purpose. Two scopes for one concept must be reconciled by hand before both land. Also: the CHANGELOG bullet is appended at EOF under ## [0.1.0] (line 1253+) — move it under [Unreleased]/[0.2.0]; the ADR-0032 amendment is spliced mid-references (:153-183) with a second copy at EOF.