fix(security): ignore client-supplied LLM endpoint/credentials in llm_override

[EnjoyBacon7]

Issue (Critical)

metadata.llm_override let any authenticated user override base_url and api_key of the downstream LLM call (components/llm.py):

base_url = (llm_override.get("base_url") or self._base_url).rstrip("/")
api_key  = llm_override.get("api_key")  or self._api_key
response = await client.post(url=f"{base_url}/chat/completions",
                             headers={"Authorization": f"Bearer {api_key}"}, ...)

• SSRF: point base_url at http://169.254.169.254/..., http://localhost:8265 (Ray), internal services, etc. — the server makes the request from inside the trust boundary.
• Credential theft: set only base_url and omit api_key → the server's real LLM API key is sent in the Authorization header to the attacker's host.

No allowlist, scheme/host check, or role gate existed.

Fix

Only model may be overridden by the client; base_url and api_key always come from server configuration. Updated the field description and tests (added a regression test asserting a client base_url/api_key is ignored).

Breaking change

Clients can no longer redirect the LLM endpoint per-request. If a legitimate per-request endpoint feature is needed, it should be re-added behind admin auth + a server-side host allowlist.

Summary by CodeRabbit

• Bug Fixes

  • Model override behavior adjusted to accept only model name; client-supplied endpoint and credential values are no longer processed.

• Documentation

  • Updated LLM override documentation to clarify that only model name can be overridden.

• Tests

  • Updated test suite to validate model override functionality and confirm client-supplied endpoint and credential override values are properly rejected.

[coderabbitai]

📝 Walkthrough

Walkthrough

LLM._extract_llm_overrides is changed to ignore any base_url and api_key values supplied in llm_override, deriving those strictly from server configuration. Only model remains client-overridable. The OpenAIChatCompletionRequest.metadata field description and the override tests are updated to reflect this restriction.

Changes

SSRF/Key-Leak Fix: Restrict llm_override to model only

Layer / File(s)	Summary
Override restriction implementation and API contract update openrag/models/openai.py, openrag/components/llm.py	OpenAIChatCompletionRequest.metadata description is narrowed to document model-only override; _extract_llm_overrides forces base_url and Authorization from self._base_url/self.headers, discarding any client-supplied base_url/api_key with inline SSRF/key-exfiltration comments.
Updated override tests openrag/components/test_llm.py	test_model_override_applied verifies only payload["model"] changes while server base_url and Authorization are preserved; test_client_base_url_and_api_key_override_ignored asserts attacker-supplied values are silently dropped. Old test_override_all_fields and test_trailing_slash_stripped_from_base_url are removed.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 A bunny hopped in, saw a door left ajar,
"Those keys and URLs have traveled too far!"
She patched up the method, said "model's enough,"
No SSRF sneaking through, the server stays tough.
Now tests stand as guards at the override gate — 🔒
What the client may touch? Just the model. That's great!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main security fix: restricting client-controlled LLM overrides by ignoring supplied endpoint/credentials.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch security/llm-override-ssrf

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openrag/models/openai.py`:
- Line 35: The description string in the openray/models/openai.py file correctly
documents that llm_override only supports model-only overrides, but the external
API documentation in docs/content/docs/documentation/API.mdx still references
the old contract allowing base_url and api_key overrides. Update the API
documentation to remove all mentions of base_url and api_key override
capabilities for the llm_override parameter and align it with the current
model-only restriction to maintain consistency between the code contract and
public-facing documentation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b14c43f4-c413-4e80-9e62-6ee02b4ee363

📥 Commits

Reviewing files that changed from the base of the PR and between baf9fb1 and db92875.

📒 Files selected for processing (3)

• openrag/components/llm.py
• openrag/components/test_llm.py
• openrag/models/openai.py

[coderabbitai]

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Update external API docs to match the model-only override contract.

Line 35 correctly narrows llm_override to model-only, but the public docs snippet in docs/content/docs/documentation/API.mdx still advertises base_url/api_key overrides. Please align that doc to prevent client-side contract drift after this breaking change.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openrag/models/openai.py` at line 35, The description string in the
openray/models/openai.py file correctly documents that llm_override only
supports model-only overrides, but the external API documentation in
docs/content/docs/documentation/API.mdx still references the old contract
allowing base_url and api_key overrides. Update the API documentation to remove
all mentions of base_url and api_key override capabilities for the llm_override
parameter and align it with the current model-only restriction to maintain
consistency between the code contract and public-facing documentation.