feat: add first-class OpenAI-compatible custom endpoint path

[njbrake]

Description

Adds a first-class, supported way to point any-llm at any OpenAI-compatible endpoint that does not have a dedicated provider, via a new factory:

from any_llm import AnyLLM

llm = AnyLLM.create_openai_compatible(
    name="mygateway",
    api_base="https://mygateway.example/v1",
    api_key="your-key",  # optional for keyless local servers
)
resp = llm.completion(model="some-model", messages=[{"role": "user", "content": "Hello!"}])

Previously the only way to reach an unlisted gateway was AnyLLM.create("openai", api_base=...), which misreports the provider identity as openai, and LLMProvider.from_string rejects any name not in the enum, so users could not represent their gateway under its own name. The returned provider reports the caller's chosen name and is usable exactly like any other provider instance.

Design notes:

• Instance-based, not enum routed. The custom endpoint is represented by a provider instance you hold and call methods on. The provider enum stays the identity registry; named provider:model routing for community gateways is the follow-up registry work in the parent issue.
• Keyless tolerated (mirrors vllm/llamafile): api_key falls back to OPENAI_COMPATIBLE_API_KEY, then to a placeholder, so local servers are never blocked by MissingApiKeyError.
• Capability flags follow the OpenAI-compatible defaults from BaseOpenAIProvider; calling a capability the endpoint does not implement surfaces the endpoint's own error. Explicit per-provider flags are the registry's job in a later PR.

Implementation:

• src/any_llm/providers/openai/custom.py: OpenAICompatibleProvider(BaseOpenAIProvider)
• src/any_llm/any_llm.py: AnyLLM.create_openai_compatible(name, api_base, api_key=None, **kwargs)
• tests/unit/providers/test_openai_compatible_provider.py: 11 tests (identity, base-URL binding, empty-api_base guard, keyless placeholder, explicit/env key precedence, capability defaults, kwargs forwarding, enum exclusion)
• docs/quickstart.md: "Custom OpenAI-compatible Endpoints" section

Testing:

• pytest tests/unit/providers/test_openai_compatible_provider.py: 11 passed
• pre-commit run --files <changed>: ruff, ruff-format, mypy strict, codespell all pass

PR Type

• 🆕 New Feature

Relevant issues

Part of #1197 (first-class OpenAI-compatible custom path). Deliberately scoped to that one step; does not close the issue.

Checklist

• I understand the code I am submitting.
• I have added unit tests that prove my fix/feature works
• I have run this code locally and verified it fixes the issue.
• New and existing tests pass locally
• Documentation was updated where necessary
• I have read and followed the contribution guidelines
• AI Usage:
  • No AI was used.
  • AI was used for drafting/refactoring.
  • This is fully AI-generated.

AI Usage Information

• AI Model used: Claude Opus 4.8

• AI Developer Tool used: Claude Code

• Any other info you'd like to share: Implemented by Claude via back-and-forth with @njbrake, who directed the scope (this is PR 1 of the Provider policy: two tiers, a config registry, and a first-class OpenAI-compatible path #1197 provider-policy rollout) and the design decisions (instance-based custom path, zero-knob registry to follow). Verified locally with unit tests and the pre-commit gate.

• I am an AI Agent filling out this form (check box if true)

Summary by CodeRabbit

• New Features

  • Added support for connecting to arbitrary OpenAI-compatible endpoints via AnyLLM.create_openai_compatible.
  • You can now supply a custom gateway name, base URL, and optional API key.
  • Keyless endpoints are supported using a safe placeholder when no key is provided.
  • Capability flags follow OpenAI-compatible defaults; unsupported calls surface appropriate errors.

• Documentation

  • Updated the quickstart with a new “Custom OpenAI-compatible Endpoints” section and examples.

• Tests

  • Expanded unit, documentation, and integration coverage for custom OpenAI-compatible flows.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

The change adds configurable OpenAI-compatible endpoint support through a factory and provider, including keyless authentication, capability defaults, integration coverage, unit tests, and quickstart documentation.

Changes

OpenAI-compatible endpoint support

Layer / File(s)	Summary
Configurable provider implementation src/any_llm/providers/openai/custom.py	Adds provider identity, API-base configuration, API-key fallbacks, and OpenAI-compatible capability settings.
Factory integration and identity src/any_llm/any_llm.py, tests/unit/providers/test_openai_compatible_provider.py	Adds AnyLLM.create_openai_compatible, validates names, creates per-name provider subclasses, and forwards endpoint configuration.
Validation and documentation tests/unit/providers/test_openai_compatible_provider.py, tests/docs/test_all.py, docs/quickstart.md	Covers configuration, authentication, capabilities, client options, documentation execution, and custom endpoint usage.
OpenAI endpoint integration flow tests/integration/test_openai_compatible.py	Tests non-streaming completion, streaming completion, and model listing against an OpenAI-compatible endpoint.

Suggested labels: 1.22.0

Suggested reviewers: tbille

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and clearly summarises the main change: adding a first-class OpenAI-compatible custom endpoint path.
Description check	✅ Passed	The description follows the template well, covering purpose, PR type, related issue, checklist, tests, docs, and AI usage details.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/openai-compatible-custom-path

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines	Coverage Δ
src/any_llm/any_llm.py	74.25% <100.00%> (-0.86%)	⬇️
src/any_llm/providers/openai/custom.py	100.00% <100.00%> (ø)

... and 35 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/any_llm/any_llm.py`:
- Line 207: The public factory create_openai_compatible in
src/any_llm/any_llm.py:207 and the provider constructor in
src/any_llm/providers/openai/custom.py:39 both trigger ANN401 through open-ended
kwargs annotations. Update both sites to use the repository’s accepted typed
kwargs alias, or add narrowly scoped ANN401 suppressions with rationale that
these client options are intentionally open-ended.

In `@src/any_llm/providers/openai/custom.py`:
- Around line 20-23: Clarify the capability-error wording in
src/any_llm/providers/openai/custom.py lines 20-23: disabled capabilities are
rejected locally by AnyLLM, while enabled OpenAI-compatible defaults are passed
through to the endpoint, where endpoint errors may occur. Apply the same
distinction in docs/quickstart.md line 119; update both sites without changing
the capability flags or provider identity behavior.

In `@tests/unit/providers/test_openai_compatible_provider.py`:
- Around line 66-70: Update test_client_kwargs_are_forwarded to construct the
provider through AnyLLM.create_openai_compatible, passing timeout as a keyword
argument and retaining the timeout assertion. Ensure the test validates the
public factory’s kwargs-forwarding contract rather than directly instantiating
OpenAICompatibleProvider.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: deaa0461-20ab-4900-9e7e-11f1aa09cdc2

📥 Commits

Reviewing files that changed from the base of the PR and between a5a86c0 and bdb9c19.

📒 Files selected for processing (4)

• docs/quickstart.md
• src/any_llm/any_llm.py
• src/any_llm/providers/openai/custom.py
• tests/unit/providers/test_openai_compatible_provider.py

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/any_llm/any_llm.py`:
- Line 207: Update create_openai_compatible to reject name values that are empty
or contain only whitespace before creating the dynamic subclass or assigning
PROVIDER_NAME, raising the established validation error type. Add a focused
error-case test covering blank and whitespace-only names.

In `@src/any_llm/providers/openai/custom.py`:
- Around line 33-42: Update OpenAICompatibleProvider.__init__ and
ENV_API_BASE_NAME consistently: either remove the unused advertised
environment-variable constant, or allow api_base to be optional so the base
resolver can honor OPENAI_COMPATIBLE_API_BASE before enforcing a
missing-endpoint error, and add coverage for the environment-based configuration
path.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c783d476-f5f8-4d84-879d-a64a366487c2

📥 Commits

Reviewing files that changed from the base of the PR and between bdb9c19 and d985692.

📒 Files selected for processing (4)

• docs/quickstart.md
• src/any_llm/any_llm.py
• src/any_llm/providers/openai/custom.py
• tests/unit/providers/test_openai_compatible_provider.py

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Reject empty custom provider names.

An empty or whitespace-only name is accepted and copied to PROVIDER_NAME, producing blank provider metadata and error messages. Validate the identifier before creating the dynamic subclass and add an error-case test.

Proposed validation

     def create_openai_compatible(cls, name: str, api_base: str, api_key: str | None = None, **kwargs: Any) -> AnyLLM:
+        if not name.strip():
+            raise ValueError("name must not be empty")

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	def create_openai_compatible(cls, name: str, api_base: str, api_key: str | None = None, **kwargs: Any) -> AnyLLM:
	def create_openai_compatible(cls, name: str, api_base: str, api_key: str | None = None, **kwargs: Any) -> AnyLLM:
	if not name.strip():
	raise ValueError("name must not be empty")

🧰 Tools

🪛 Ruff (0.15.21)

[warning] 207-207: Dynamically typed expressions (typing.Any) are disallowed in **kwargs

(ANN401)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/any_llm/any_llm.py` at line 207, Update create_openai_compatible to
reject name values that are empty or contain only whitespace before creating the
dynamic subclass or assigning PROVIDER_NAME, raising the established validation
error type. Add a focused error-case test covering blank and whitespace-only
names.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)

src/any_llm/providers/openai/custom.py (2)

37-39: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Reject whitespace-only API bases.

if not api_base accepts values such as " ", deferring an invalid endpoint failure to client initialisation. Use if not api_base.strip() and extend the existing error-case test accordingly.

Suggested validation

-        if not api_base:
+        if not api_base.strip():

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/any_llm/providers/openai/custom.py` around lines 37 - 39, Update the
api_base validation in OpenAICompatibleProvider to reject whitespace-only
strings by checking the stripped value, while preserving the existing
explicit-endpoint error. Extend the existing error-case test to cover a
whitespace-only api_base.

---

36-36: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Add @override to the constructor.

OpenAICompatibleProvider.__init__ overrides AnyLLM.__init__, and overloaded methods in the provider Python code require @override. Add it immediately above the constructor definition.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/any_llm/providers/openai/custom.py` at line 36, Add the required
`@override` decorator immediately above OpenAICompatibleProvider.__init__,
preserving its existing signature and implementation.

Source: Coding guidelines

docs/quickstart.md (1)

101-110: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Document the OPENAI_COMPATIBLE_API_KEY fallback.

When api_key is omitted, the provider reads OPENAI_COMPATIBLE_API_KEY before using the keyless placeholder. The section currently documents only an inline key or keyless mode, so hosted users may miss the supported environment configuration. As per the PR objectives, environment-based key selection is part of the supported contract.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/quickstart.md` around lines 101 - 110, Update the
AnyLLM.create_openai_compatible documentation example or surrounding text to
state that omitted api_key values use OPENAI_COMPATIBLE_API_KEY before falling
back to the keyless placeholder. Preserve the existing inline-key and keyless
usage guidance.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@docs/quickstart.md`:
- Around line 101-110: Update the AnyLLM.create_openai_compatible documentation
example or surrounding text to state that omitted api_key values use
OPENAI_COMPATIBLE_API_KEY before falling back to the keyless placeholder.
Preserve the existing inline-key and keyless usage guidance.

In `@src/any_llm/providers/openai/custom.py`:
- Around line 37-39: Update the api_base validation in OpenAICompatibleProvider
to reject whitespace-only strings by checking the stripped value, while
preserving the existing explicit-endpoint error. Extend the existing error-case
test to cover a whitespace-only api_base.
- Line 36: Add the required `@override` decorator immediately above
OpenAICompatibleProvider.__init__, preserving its existing signature and
implementation.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 57206973-f603-49b5-8d0e-6515feb0c93d

📥 Commits

Reviewing files that changed from the base of the PR and between 7ec2136 and 77fc0bf.

📒 Files selected for processing (4)

• docs/quickstart.md
• src/any_llm/any_llm.py
• src/any_llm/providers/openai/custom.py
• tests/unit/providers/test_openai_compatible_provider.py

[njbrake]

Note: this comment was drafted by Claude via back-and-forth with @njbrake. The reasoning and decisions are his; the prose is Claude's.

@tbille this one is ready for you. Please review and merge whenever you're comfortable with it. Heads up that #1201 (the config registry) is stacked on this branch, so merging this one first keeps that diff clean.