feat(discovery): add bounded OpenRouter free canary

[seonghobae]

Summary

• add a dry-run-default OpenRouter free-model canary CLI
• select only a freshly discovered chat row with explicit comparable USD zero prompt and completion prices, without pinning a model id
• require explicit positive request, output-token, timeout, evidence-retention, and output-path bounds before the sole opt-in live request
• write atomic evidence without credentials, prompts, or responses; add no scheduler or default live call

Safety contract

• OPENROUTER_API_KEY is resolved only from the KV registry
• dry run performs current catalog discovery but never constructs completion transport
• live uses max_retries=0 and exactly one fixed request
• missing credential, incomplete/ambiguous pricing, or incomplete live bounds fail before completion transport
• tests use mocks; no provider request or spend occurred during development

Verification

• uv run pytest -q — 2861 passed, 1 skipped
• focused discovery/CLI/canary suite — 124 passed
• Ruff changed Python files — passed
• interrogate new production module — 100%
• git diff --check — passed

Product gap

Closes the unclaimed OpenRouter canary slice recorded in docs/product-technical-gap-baseline.md. Existing routing/discovery research and KV boundaries are reused; no new routing-policy claim is introduced.

---

Summary by CodeRabbit

• 새 기능

  • OpenRouter 무료 모델 canary를 추가했습니다. 기본적으로 드라이런으로 카탈로그를 검사하며, 가격이 0인 채팅 모델만 선택합니다.
  • --live 모드에서 요청 수, 출력 토큰, 타임아웃 및 증거 보존 한도를 설정할 수 있습니다.
  • 실행 증거를 안전한 JSON 형식으로 기록하고 만료된 증거를 정리할 수 있습니다.

• 문서

  • canary CLI 사용법과 OpenRouter 자격 증명 설정 방법을 문서화했습니다.
  • 감사 증거 보호 및 보존 관련 기술 기준을 보완했습니다.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 488fbf95-bf72-4d8f-bca3-a27bf5a867d9

📥 Commits

Reviewing files that changed from the base of the PR and between 899e6f8 and e527eaf.

📒 Files selected for processing (3)

• CHANGELOG.md
• README.md
• docs/product-technical-gap-baseline.md

💤 Files with no reviewable changes (1)

• CHANGELOG.md

🚧 Files skipped from review as they are similar to previous changes (2)

• docs/product-technical-gap-baseline.md
• README.md

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

OpenRouter 무료 모델 canary의 dry-run 기본 동작과 opt-in live 운영 조건을 문서화했습니다. 증거 보존, 만료 정리, OpenRouter 자격 증명 사용 및 NIST 통제 관련 설명도 갱신했습니다.

Changes

OpenRouter 무료 canary

Layer / File(s)	Summary
Canary 운영 계약 문서 CHANGELOG.md, README.md, docs/product-technical-gap-baseline.md	기본 dry-run의 모델 선택 조건과 live 모드의 실행 제한, 증거 파일 처리, 만료 정리 동작을 문서화했습니다. OpenRouter 자격 증명 예시와 NIST SP 800-53 관련 기준도 갱신했습니다.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to e527e

The change documents the bounded, dry-run-default OpenRouter free-model canary contract. No concrete unresolved merge risk is identified.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	제목은 변경의 핵심인 제한된 OpenRouter 무료 모델 canary 추가를 명확하고 간결하게 설명합니다.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/openrouter-free-canary

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[seonghobae]

Merge conflict resolved

Merged current main (4d143601) into this branch. Two real conflicts:

• CHANGELOG.md: additive — kept both this PR's ### Added entry and main's ### Deprecated entry under ## [0.2.0] - Unreleased.
• contextual_orchestrator/__main__.py: main()'s dispatch block had a genuine collision — this branch's own history had already extracted openrouter-free-canary handling (including the later --prune-expired-evidence addition) into the module-level _openrouter_free_canary_command() helper, but a stale duplicate of the old inline dispatch logic (plus a broken zero-arg _check_fast_mlsirm_command() call — the helper requires an argv parameter) was still left in main() from before that extraction. main's side had the correct, already-clean delegated dispatch plus a new nim-benchmark subcommand this branch doesn't have. Resolved by dropping the dead/broken inline block entirely and keeping main's clean dispatch (which now also picks up nim-benchmark), confirming the retained _openrouter_free_canary_command helper already carries this PR's --prune-expired-evidence feature intact.

New head: 899e6f83. Verified: tests/test_openrouter_free_canary.py, tests/test_discover_models_cli.py, tests/test_cli_auth.py, tests/test_nim_benchmark_workflow_contract.py (56 passed), interrogate 100%, py_compile clean.

---

Generated by Claude Code

---

Generated by Claude Code

[seonghobae]

The failing required "Full unit and contract suite" check (KeyError: 'total_tokens' in tests/test_provider_embedding_batch_backend.py::test_unknown_tokenizer_byte_bound_never_becomes_recorded_usage) is not this PR's own defect — this PR's diff is docs-only (CHANGELOG.md, README.md, docs/product-technical-gap-baseline.md), touching no source or test files.

Root-caused it as a genuine, pre-existing CI-only race: two tests in that file call CostRoutingCoordinator.complete_embeddings_batch() against a provider (async, ThreadPoolExecutor-backed) embedding agent without wait_timeout, then assert on total_tokens immediately — unlike sibling tests in the same file that correctly pass wait_timeout=1 for this exact shape. Under CI's heavier concurrent load the background job sometimes hasn't finished by the time the immediate poll runs, so the poll returns a non-terminal document lacking total_tokens. Reproduced deterministically with a temporary artificial delay (RED without the fix, GREEN with it), reverted before committing.

Filed the canonical fix at #1025 and ported the identical two-line change onto this PR's branch (commit 98047635) so this PR isn't left waiting on that one to merge first.

---

Generated by Claude Code

[seonghobae]

Brought this branch current with main (it was stuck behind against the stale base 8839081, abandoned days ago while main advanced to 212ff43+).

• CI checks on the pre-merge head were all queued/skipped/cancelled — a review-infrastructure artifact of the stale/superseded head, not a real code failure.
• git merge origin/main --no-edit: clean auto-merge, no conflicts (CHANGELOG.md merged cleanly).
• Verified locally in a fresh venv (pip install --require-hashes -r requirements.lock && pip install --no-deps -e .):
  • tests/test_openrouter_free_canary.py + tests/test_provider_embedding_batch_backend.py (this PR's own changed test): 35 passed
  • Merge-diff-touched suites (test_rater_observation*.py, test_review_gateway*.py, test_admin_contract.py, test_chat_model_capability_isolation.py): 88 passed, 1 pre-existing failure (test_admin_contract.py::test_model_group_mutations_refresh_audit_events, a NameError: name 'json' is not defined already present on main HEAD, unrelated to this PR, tracked/fixed separately in fix(admin): repair test_model_group_mutations_refresh_audit_events #1029)
• Pushed the merge commit directly to feat/openrouter-free-canary (no force-push).

---

Generated by Claude Code

[seonghobae]

Exact-head follow-up on e527eafb59e56a339972ce9e7e066583692b46a5:

• Merged current protected main (60c562defc81fb1897fa97ebdb5bf8f69eae0c55) without force-push.
• Removed the unrelated embedding-batch race delta; PR fix(tests): wait for provider embedding batch completion before assertions #1044 remains its owner.
• uv run pytest -q tests/test_openrouter_free_canary.py tests/test_model_discovery.py -> 141 passed.
• Live read-only discovery using the configured OpenRouter credential through an ephemeral in-memory KV found 576 models, including 10 serving-eligible free candidates, with 0 discovery errors. No completion or paid request was issued.
• Auto-merge is enabled; exact-head required checks remain authoritative.