feat: Add vLLM integration to the OpenAI plugin

[darshankparmar]

Summary by CodeRabbit

• New Features
  • Added vLLM endpoint support to the LLM class, enabling configuration with custom models, server URLs, authentication credentials, and advanced inference parameters such as temperature, tool calling, and response caching.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[darshankparmar]

Hi @Hormold 👋
This PR is ready for review.
Please let me know if you’d like any changes or additional tests.
Thanks!

[davidzhao]

@coderabbitai review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

📝 Walkthrough

Walkthrough

A new with_vllm static constructor method has been added to the LLM class in the OpenAI plugin to enable creating LLM instances backed by vLLM endpoints, accepting parameters such as model, base_url, client, api_key, and various LLM configuration options mirroring existing provider-specific constructors.

Changes

Cohort / File(s)	Summary
vLLM Static Constructor livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py	Added with_vllm static method to LLM class enabling vLLM endpoint integration. Method accepts model, base_url (default: http://localhost:8000/v1), client, api_key, temperature, parallel_tool_calls, tool_choice, reasoning_effort, safety_identifier, prompt_cache_key, and top_p parameters. Note: Method appears duplicated in the same class—requires verification.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 A vLLM path we hop and sow,
With with_vllm to make it go,
New endpoints light our burrow's way,
Plugins dance in brighter day!
Static methods, clean and bright—
Our AI models reach new height! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: adding vLLM integration to the OpenAI plugin, which matches the core objective of introducing a with_vllm static constructor method.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

🧹 Recent nitpick comments

livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py (1)

617-648: Consider using a dummy API key when none is provided to avoid environment variable conflicts.

The implementation correctly follows the pattern of with_ollama. However, when api_key is NOT_GIVEN, the LLM.__init__ passes None to the OpenAI client, which then falls back to the OPENAI_API_KEY environment variable if set. This could cause unexpected behavior for users running both OpenAI and vLLM simultaneously.

Consider using a dummy placeholder (similar to with_ollama using "ollama"):

Suggested improvement

     `@staticmethod`
     def with_vllm(
         *,
         model: str,
         base_url: str = "http://localhost:8000/v1",
         client: openai.AsyncClient | None = None,
         api_key: NotGivenOr[str] = NOT_GIVEN,
         temperature: NotGivenOr[float] = NOT_GIVEN,
         parallel_tool_calls: NotGivenOr[bool] = NOT_GIVEN,
         tool_choice: ToolChoice = "auto",
         reasoning_effort: NotGivenOr[ReasoningEffort] = NOT_GIVEN,
         safety_identifier: NotGivenOr[str] = NOT_GIVEN,
         prompt_cache_key: NotGivenOr[str] = NOT_GIVEN,
         top_p: NotGivenOr[float] = NOT_GIVEN,
     ) -> LLM:
         """
         Create a new instance of vLLM.
+
+        ``api_key`` can be set if your vLLM server is configured with authentication,
+        either using the argument or by setting the ``VLLM_API_KEY`` environment variable.
         """
+        api_key = api_key if is_given(api_key) else os.environ.get("VLLM_API_KEY", "vllm")
 
         return LLM(
             model=model,
             api_key=api_key,
             base_url=base_url,
             client=client,
             temperature=temperature,
             parallel_tool_calls=parallel_tool_calls,
             tool_choice=tool_choice,
             reasoning_effort=reasoning_effort,
             safety_identifier=safety_identifier,
             prompt_cache_key=prompt_cache_key,
             top_p=top_p,
         )

This provides a VLLM_API_KEY environment variable fallback for authenticated vLLM servers while using a dummy value ("vllm") when no key is needed, preventing accidental use of OPENAI_API_KEY.

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0cc7744 and 555f4e3.

📒 Files selected for processing (1)

• livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings

Files:

• livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py

🧬 Code graph analysis (1)

livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py (12)

livekit-plugins/livekit-plugins-assemblyai/livekit/plugins/assemblyai/stt.py (1)

• model (110-111)

livekit-plugins/livekit-plugins-google/livekit/plugins/google/llm.py (2)

• model (215-216)
• LLM (83-377)

livekit-agents/livekit/agents/llm/llm.py (2)

• model (99-108)
• LLM (85-149)

livekit-plugins/livekit-plugins-anthropic/livekit/plugins/anthropic/llm.py (2)

• model (118-119)
• LLM (54-223)

livekit-plugins/livekit-plugins-aws/livekit/plugins/aws/llm.py (2)

• model (118-119)
• LLM (51-201)

livekit-plugins/livekit-plugins-mistralai/livekit/plugins/mistralai/llm.py (2)

• model (61-62)
• LLM (42-104)

livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/realtime/realtime_model.py (1)

• model (363-364)

livekit-plugins/livekit-plugins-cartesia/livekit/plugins/cartesia/tts.py (1)

• model (198-199)

livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/stt.py (1)

• model (157-158)

livekit-plugins/livekit-plugins-elevenlabs/livekit/plugins/elevenlabs/tts.py (1)

• model (193-194)

livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/tts.py (1)

• model (104-105)

livekit-plugins/livekit-plugins-baseten/livekit/plugins/baseten/llm.py (1)

• LLM (20-76)

_{✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}