MOT-4176 feat(llm-router,providers): provider-side env-var credential fallback

[andersonleal]

Summary

A provider's API key set in the provider's own environment was never loaded: every provider declares credential_env_var (e.g. ZAI_API_KEY) in its ProviderDeclaration, but the only std::env::var read happens inside llm-router's process during router::provider::resolve (registry/resolve.rs). If the router runs under a different supervisor/environment than the provider, the key is present-but-unreachable and the provider fails NotConfigured.

Fix: when the router resolves no credential, the provider now falls back to reading its own declared env var from its own process. Strictly subordinate — any router-resolved credential (config slice or router-process env) always wins, so nothing that works today changes.

Per commit:

• feat(llm-router): pure with_api_key_fallback + env-reading apply_credential_env_fallback in provider_scaffold/router_client.rs. Injects Credential::ApiKey with source=Env, configured=true only when the router returned credential: None and the var is non-empty after trim (whitespace-only keys are rejected — stricter than the router's own is_empty() check).
• refactor(providers): each keyed provider hoists its env-var literal into pub const CREDENTIAL_ENV_VAR, reused by declaration() and the resolve call sites (single source of truth).
• feat(providers): scaffold::resolve + ScaffoldCache::resolve gain a trailing credential_env_var: Option<&str> and apply the fallback before returning/caching. All call sites wired atomically: anthropic/zai/openai/xai/llamacpp pass Some(CREDENTIAL_ENV_VAR) on both hot (stream_fn, embed) and discovery paths (via each crate's router_client wrapper, whose external 2-arg signature is unchanged); openai-codex passes None (OAuth-only); kimi's inlined resolve wraps its response with the shared helper. llamacpp's no-key local flow is untouched (unset var = no-op).

Env vars: ANTHROPIC_API_KEY, ZAI_API_KEY, MOONSHOT_API_KEY, OPENAI_API_KEY, XAI_API_KEY, LLAMACPP_API_KEY.

Out of scope (unchanged): router-side resolve precedence, CredentialSource protocol enum (reuses Env), OAuth handling, version bumps.

Test Coverage

DIFF COVERAGE MAP  (env-var credential fallback)
================================================
with_api_key_fallback (pure — never touches process env)
 +-- router credential present -> unchanged (source kept)   [UNIT] router_credential_wins_over_env
 +-- none + key -> ApiKey, source=Env, configured=true      [UNIT] injects_env_when_router_has_none
 +-- none + no key -> credential stays None                 [UNIT] no_key_leaves_none
 +-- "" / whitespace-only -> no injection                   [UNIT] empty_and_whitespace_are_not_injected
 +-- " sk-abc\n" -> trimmed to "sk-abc"                     [UNIT] injected_key_is_trimmed
CREDENTIAL_ENV_VAR consts (x6)
 +-- const value + declaration() wiring                     [UNIT] declaration_uses_credential_env_var_const

Per-crate (each crate is its own standalone Cargo workspace; built+tested from its own dir):
llm-router 94 lib (+13/+4 int); anthropic 83; zai 62; kimi 61; openai 78; xai 75; llamacpp 63; openai-codex 56 — all green, plus each crate's doc/integration suites.

Known pre-existing failure (not this PR): provider-llamacpp integration provider_registers_with_persisted_token_and_discovers_catalog_without_a_credential (catalog count 0 vs 1) reproduces identically at the pre-branch commit in an isolated worktree; the fallback wiring lands only in the final commit, after the failure already exists.

Pre-Landing Review

Per-task spec+quality reviews (all approved) plus a final whole-branch review: verdict ready to merge, no Critical/Important findings. Deferred minors: (1) the no-credential integration tests are now coupled to ambient env — a CI shell exporting e.g. LLAMACPP_API_KEY could flip them (suggest hermetic remove_var in a follow-up); (2) env-injected keys ride the existing 30s resolve-cache TTL (matches current semantics; informational).

Summary by CodeRabbit

• New Features

  • API credentials can now be supplied through configured environment variables when no credential is returned by the router.
  • Environment-provided credentials are trimmed and clearly marked as the active source.
  • Supported providers now expose standardized credential environment variable settings.

• Bug Fixes

  • Existing router-provided credentials continue to take precedence over environment values.
  • Empty or whitespace-only environment variables are ignored.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
workers	Ready	Preview, Comment	Jul 22, 2026 6:31pm
workers-tech-spec	Ready	Preview, Comment	Jul 22, 2026 6:31pm

[github-actions]

skill-check — worker

0 verified, 47 skipped (no docs/).

Layer	Result
structure	✓
vale	✓
ai	✓
render	✓

Four for four. Nicely done.

[coderabbitai]

📝 Walkthrough

Walkthrough

Provider resolution now accepts an optional credential environment variable. Shared fallback logic injects trimmed API keys when router responses lack credentials, and supported providers pass centralized environment-variable constants through declarations, router clients, and stream or embedding calls.

Changes

Credential environment resolution

Layer / File(s)	Summary
Shared resolver fallback llm-router/src/provider_scaffold/cache.rs, llm-router/src/provider_scaffold/router_client.rs	Resolver and cache signatures accept credential environment variables; missing credentials can be populated from trimmed, non-empty environment values, with precedence and edge cases covered by tests.
Provider credential declarations provider-{anthropic,kimi,llamacpp,openai,xai,zai}/src/register.rs	Providers expose credential environment constants, reference them from declarations, and test declaration consistency.
Provider resolution wiring provider-{anthropic,kimi,llamacpp,openai,xai,zai}/src/{router_client.rs,stream_fn.rs}, provider-{llamacpp,openai}/src/embed.rs, provider-openai-codex/src/{router_client.rs,stream_fn.rs}	Provider resolution and cached calls pass their credential environment constants; OpenAI Codex passes None.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

• iii-hq/workers#443: Introduces the provider-zai worker whose credential resolution is updated here.
• iii-hq/workers#513: Adds the Llama.cpp embedding path wired to the new credential resolution parameter.
• iii-hq/workers#525: Establishes the provider scaffold resolution path extended by this fallback logic.

Suggested labels: no-ticket

Suggested reviewers: ytallo

Poem

I’m a rabbit with a key in my burrow,
Trimmed of spaces, never empty or narrow.
The router speaks first, the env waits in line,
Six providers now share one sign.
Hop, hop—credentials resolve just fine!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 89.29% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: provider-side environment-variable credential fallback across llm-router and providers.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch provider-env-credential-fallback

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}