feat(providers): add Inference APIs as OpenAI-compatible provider

[TaylorHawkes]

TLDR

Problem this solves:

• inferenceapis/<model> fails with "LLM Provider NOT provided"
• Users must hand-configure openai/ plus a custom api_base

How it solves it:

• Adds Inference APIs as a JSON-configured OpenAI-compatible provider
• Covers /v1/chat/completions and /v1/responses
• Reads the key from INFERENCEAPIS_API_KEY
• Lists it in Add Model and the provider docs index

User Flow

Before: a developer with an Inference APIs key cannot call it by provider name

1. They set INFERENCEAPIS_API_KEY and call litellm.completion(model="inferenceapis/meta-llama/Llama-3.3-70B-Instruct-Turbo", ...)
2. The call raises BadRequestError: LLM Provider NOT provided
3. The same happens for litellm.responses(...)

After: the same two calls return answers from https://api.inferenceapis.com/v1

1. They set INFERENCEAPIS_API_KEY and call litellm.completion(model="inferenceapis/meta-llama/Llama-3.3-70B-Instruct-Turbo", ...)
2. The call returns a normal chat completion with usage
3. litellm.responses(...) with the same model returns status=completed

Relevant issues

None.

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally: pytest tests/test_litellm/llms/openai_like/test_inferenceapis_provider.py tests/test_litellm/llms/openai_like/test_json_providers.py and tests/test_litellm/proxy/public_endpoints/test_public_endpoints.py pass (57 passed); check_provider_folders_documented.py, ruff check and ruff format --check are clean on the changed files
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• Greptile confidence score of at least 4/5

Screenshots / Proof of Fix

Shared setup: INFERENCEAPIS_API_KEY set to a live key. Script (real calls, no mocks):

import litellm
r = litellm.completion(model="inferenceapis/meta-llama/Llama-3.3-70B-Instruct-Turbo",
                       messages=[{"role": "user", "content": "Reply with the single word: pong"}], max_completion_tokens=5)
print("[chat]", r.choices[0].message.content, r._hidden_params.get("custom_llm_provider"), r.usage.total_tokens)
r = litellm.responses(model="inferenceapis/meta-llama/Llama-3.3-70B-Instruct-Turbo",
                      input="Reply with the single word: pong", max_output_tokens=20)
print("[responses]", r.status, [c.text for o in r.output if o.type == "message" for c in o.content])

Before (1dd4c13)

/v1/chat/completions

1. Run the script's completion call
2. BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=inferenceapis/meta-llama/Llama-3.3-70B-Instruct-Turbo

/v1/responses

1. Run the script's responses call
2. BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=inferenceapis/meta-llama/Llama-3.3-70B-Instruct-Turbo

After (0ae0ceb)

/v1/chat/completions

1. Run the script's completion call
2. [chat] OK content='pong' provider=inferenceapis total_tokens=44

/v1/responses

1. Run the script's responses call
2. [responses] OK status=completed output_text=['pong']

Tests also capture the outgoing HTTP request (URL, Authorization header, JSON body) for both an explicit key and the INFERENCEAPIS_API_KEY / INFERENCEAPIS_API_BASE environment variables.

Type

🆕 New Feature

Caveats (if any)

Low

• osv-scan fails on soupsieve 2.8.4 in uv.lock. Fork PRs may not modify uv.lock, so this PR leaves it alone; main carries the same entry and build(deps): bump soupsieve from 2.8.4 to 2.9.2 #41679 fixes it

• No dashboard logo or enum entry added; the UI falls back to the provider slug

• Model ids contain a slash, e.g. inferenceapis/openai/gpt-oss-120b

• /v1/embeddings and /v1/rerank exist upstream but are not registered here

• No model_prices_and_context_window.json entries yet; follow-up PR

• /v1/messages is not offered by this provider

Disclosure: I run this service.

🤖 Generated with Claude Code

[CLAassistant]

All committers have signed the CLA.

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing TaylorHawkes:feat/add-inferenceapis-provider (9b10720) with main (8fc9c46)}

[greptile-apps]

Confidence Score: 3/5

The PR is not yet safe to merge because the revised request tests can make real provider calls instead of using their MockTransport

Findings

1.  Mock Client Is Ignored ▶
2.  Redundant Provider Comment ▶

Summary

Adds Inference APIs as a JSON-configured OpenAI-compatible provider with chat-completions and Responses API registration, environment-based credentials, provider metadata, and request tests

• Registers provider resolution, endpoint support, and public model-creation fields
• Adds tests for configuration, routing, parameter forwarding, and outgoing chat requests
• The latest test-seam change does not intercept the OpenAI-like request path, leaving the request tests capable of real network access

_{Reviews (3) · Last reviewed commit: "test(providers): inject a recording HTTP..."}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[TaylorHawkes]

Pushed 940fb4d addressing CI and the Greptile review:

• proxy-endpoints (test_every_backend_provider_is_listed_in_add_model_or_frozen_as_unlisted): added an inferenceapis entry to provider_create_fields.json.
• code-quality (provider docs check): added inferenceapis to provider_endpoints_support.json and its backup.
• Greptile P2, tests inspect structure only: added request-level tests that capture the outgoing HTTP request and assert the URL, Authorization header and JSON body, for an explicit key and for the INFERENCEAPIS_API_KEY / INFERENCEAPIS_API_BASE environment variables.
• Greptile P2, redundant comment: removed.
• osv-scan flags soupsieve 2.8.4 in uv.lock. This PR does not touch the lock file and main (1dd4c13) carries the same entry, so I have left it alone.

Live before/after proof in the description was re-run at the new tip.

@greptileai

[TaylorHawkes]

Pushed 2a7c105 for the class-level patching note: the request tests now inject a recording httpx.Client (with httpx.MockTransport) through litellm.client_session, so nothing is monkeypatched on a class. tests/test_litellm/llms/openai_like/ passes locally (202 passed, 5 skipped); ruff check and ruff format --check are clean.

@greptileai

[greptile-apps]

Mock Client Is Ignored

The OpenAI-like handler replaces this client with its own HTTPHandler, so these unit tests can contact the real provider.

Rule Used: What: prevent any tests from being added here that make real network calls - only mock tests can be added to this folder. Exception: tests/e2e folder, this folder can contain network calls. Why: ensure it runs correctly on github ci/cd + for all d... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Redundant Provider Comment

This comment only restates the entry, violating the repository directive limiting source comments. This requirement must be satisfied before merging.

Suggested change

	"inferenceapis", # Inference APIs - JSON-configured provider
	"inferenceapis",

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[TaylorHawkes]

Pushed 0ae0ceb.

• Redundant provider comment in constants.py: removed.

• "Mock client is ignored": the injected client is what serves these requests, and the tests cannot reach the network:

  • each test asserts len(sent) == 1 on the list the MockTransport handler appends to, then asserts that request's URL, Authorization header and JSON body. If the handler were bypassed, sent would be empty and the test would fail;
  • the keys are fakes (sk-test-explicit, sk-from-env), so a real call would come back 401 and the content == "pong" assertion would fail. These tests passed in CI on this PR;
  • I also ran -k TestInferenceAPIsRequests in a process where socket.socket.connect and socket.create_connection raise: 2 passed.

  JSON-configured providers take the OpenAI SDK path, which builds its client on litellm.client_session; that is the seam used here.

@greptileai

[greptile-apps]

You're right. I rechecked the revised test seam and the provider dispatch: JSON providers default to the OpenAIGPTConfig path, which uses litellm.client_session; the MockTransport records the request, each test requires len(sent) == 1, and the socket guard independently confirms there is no network fallback. My earlier claim that this path bypasses the injected client was incorrect. I retract the P1 finding; the revised tests do not make real provider calls.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

[TaylorHawkes]

Pushed b7b6adb, a separate 3-line commit: uv lock --upgrade-package soupsieve (2.8.4 -> 2.9.2), only the soupsieve stanza of uv.lock changes.

osv-scan fails on every PR whose lock still pins soupsieve 2.8.4 (GHSA-gjv8-xp57-g29c, GHSA-j934-xhv5-fg8f, fixed in 2.9.0). Locally, with the workflow's pinned osv-scanner v2.3.8 and the same command: No issues found; uv lock --check passes.

Same intent as dependabot's #41679. If you would rather keep dependency changes out of this PR, say so and I will drop the commit.

[TaylorHawkes]

Dropped b7b6adb (the uv.lock soupsieve bump). I had not seen the "Block fork dependency changes" rule; lockfile changes belong in a branch of this repository, so the PR is back to 0ae0ceb and touches no dependency files.

That leaves osv-scan red for the reason above: main still pins soupsieve 2.8.4, and #41679 fixes it. Once that lands I will update this branch.