Repository navigation
feat(registry): migrate cascadia, vllm and llamacpp to registry rows (#1417) - #1421
Conversation
…ozilla-ai#1417) - Register cascadia, llamacpp, and vllm as config-only rows in PROVIDER_REGISTRY with api_key_optional=True - Replace code folders with import shims keeping deep-import paths backward compatible - Honor VLLM_API_KEY from environment in VllmProvider - Adopt LLAMACPP_API_KEY in LlamacppProvider, honoring both explicit keys and env vars - Subclass _OptionalApiKeyProvider in OpenAICompatibleProvider and remove duplicate key-verification method - Add comprehensive test coverage for metadata, key resolution precedence, and capabilities
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (8)
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review. WalkthroughCascadia, llama.cpp, and vLLM now use provider registry rows. Their historical module paths remain available through import shims. OpenAI-compatible key handling uses the registry’s optional-key provider. Tests cover metadata, capabilities, key resolution, and registration. ChangesProvider registry migration
Suggested reviewers: Priority: ➖ Normal Change: Feature · Severity of issue fixed: Medium Merge Risk: ⚪ Minimal · up to The migration centralizes optional API-key handling while preserving provider imports and registration behavior; the supplied tests cover the key paths and the change is mergeable. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests.
... and 30 files with indirect coverage changes 🚀 New features to boost your workflow:
|
njbrake
left a comment
njbrake
left a comment
There was a problem hiding this comment.
Both key-handling defects from #1417 are fixed. Against a llama.cpp server started with --api-key: llamacpp with an explicit key or LLAMACPP_API_KEY, and vllm with VLLM_API_KEY, all returned 401 before this change and complete now. Cascadia is the no-op the issue predicted.
Provider metadata across every provider is unchanged except llamacpp's env_key, "None" to LLAMACPP_API_KEY. Capability flags, tiers and class names hold, so the migration is behavior-preserving outside the two intended fixes.
llamacpp integration: 14 passed, 17 skipped, all skips capability-gated. The vllm and cascadia rows serve completion, streaming and list_models on their row defaults; both sit in CI_EXCLUDED_PROVIDERS, so CI does not cover them.
Non-blocking follow-ups: llamafile still discards caller-supplied keys and publishes None as its key env var; llamacpp's supports_moderation=True is wrong, the endpoint 404s.
Note: this review was drafted by Claude Opus 5 via back-and-forth with @njbrake. The reasoning and decisions are his; the prose is Claude's.
This branch was successfully deployed
2 participants
Description
Migrates three local-runner providers (
cascadia,vllm, andllamacpp) toPROVIDER_REGISTRYrows usingapi_key_optional=Trueadded in #1416, following the plan outlined in #1417.Changes
PROVIDER_REGISTRYrow withapi_key_optional=True. Replacedsrc/any_llm/providers/cascadia/cascadia.pywith an import shim callingget_registry_provider_class("cascadia").PROVIDER_REGISTRYrow withapi_key_optional=True. Fixed defect whereVllmProviderpreviously declaredENV_API_KEY_NAME = "VLLM_API_KEY"but never read it in_verify_and_set_api_key. It now properly honorsVLLM_API_KEYfrom the environment while still defaulting to"no-key-required"when unset. Replacedsrc/any_llm/providers/vllm/vllm.pywith an import shim.PROVIDER_REGISTRYrow withapi_key_optional=True. FixedENV_API_KEY_NAMEwhich was previously the string"None", adopting"LLAMACPP_API_KEY"and honoring both explicit keys and environment variables (useful when llama.cpp server runs with--api-key). Replacedsrc/any_llm/providers/llamacpp/llamacpp.pywith an import shim.src/any_llm/providers/openai/custom.py, subclassed_OptionalApiKeyProviderto drop its duplicate_verify_and_set_api_keyimplementation.test_vllm_provider.pyandtest_llamacpp.pyverifying metadata, capability flags, loader/enum resolution, and key resolution precedence (explicit key > env key >"no-key-required"). Added shim resolution tests forcascadia,llamacpp, andvllmintest_registry.py.PR Type
Relevant issues
Resolves #1417
Checklist
Summary by CodeRabbit
New Features
Bug Fixes