Add LLM provider routing for speaking bots

README.md

@@ -299,6 +299,122 @@ curl -X POST http://localhost:${PORT}/bots \
   }'
 ```
 
+You can attach external prompt context and MCP servers per bot. External
+context is loaded before the Pipecat process starts and capped by
+`prompt_data_token_limit` using an approximate token budget. URL sources block
+localhost and private-network targets by default; set
+`PROMPT_DATA_ALLOW_PRIVATE_URLS=true` only in trusted deployments.
+
+You can also select the bot LLM per request with `llm_provider` and
+`llm_model`. Supported providers are `openai`, `anthropic`, and `zai`.
+Provider credentials and base URLs are server-side environment variables only:
+`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `ZAI_API_KEY`, and optional
+`ZAI_BASE_URL`. OpenAI defaults to Pipecat's Responses API service for newest
+models; set `OPENAI_API_SURFACE=chat` to use the older Chat Completions bridge.
+The Z.ai integration uses the OpenAI-compatible Chat Completions bridge;
+Anthropic uses Pipecat's native Claude bridge. The API rejects unconfigured
+providers before creating the upstream MeetingBaaS bot.
+
+MCP servers are live-query capable only when their config is connectable.
+`http`, `streamable_http`, and `sse` servers require `transport`, `url`, and
+optional `headers`. Local process `stdio` MCP is intentionally not accepted by
+the public API because it would execute caller-supplied commands. Remote MCP
+URLs also block localhost and private-network targets by default; production
+deployments should use `MCP_ALLOWED_PRIVATE_URLS` with exact `http://host:port/path`
+entries for trusted loopback MCPs instead of the broad
+`MCP_ALLOW_PRIVATE_URLS=true` development bypass. If `transport` is omitted, the
+server is treated as metadata-only and MCP tools are not executed. Secrets are
+not required in the request, but `headers` are available for deployments that
+need them. Use `tool_allowlist` to constrain which server tools the bot may
+call, and set `enabled: false` to document a server without connecting to it.
+
+```bash
+curl -X POST http://localhost:${PORT}/bots \
+  -H "Content-Type: application/json" \
+  -H "x-meeting-baas-api-key: your-api-key" \
+  -d '{
+    "meeting_url": "https://meet.google.com/xxx-yyyy-zzz",
+    "personas": ["account_executive"],
+    "llm_provider": "anthropic",
+    "llm_model": "claude-opus-4-8",
+    "prompt_data_token_limit": 4000,
+    "prompt_data_sources": [
+      {
+        "name": "CRM account notes",
+        "type": "url",
+        "url": "https://example.com/account-notes.md"
+      },
+      {
+        "name": "Call objective",
+        "type": "text",
+        "text": "Confirm timeline, budget, and integration constraints."
+      }
+    ],
+    "mcp": {
+      "instructions": "Use Google Drive and CRM context only when relevant.",
+      "servers": [
+        {
+          "name": "drive-docs-work",
+          "enabled": true,
+          "transport": "streamable_http",
+          "url": "http://127.0.0.1:8123/mcp",
+          "tool_allowlist": ["search", "read"],
+          "timeout_seconds": 20,
+          "instructions": "Use only meeting-relevant Drive docs."
+        },
+        {
+          "name": "remote-crm",
+          "enabled": true,
+          "transport": "streamable_http",
+          "url": "https://mcp.example.com/mcp",
+          "headers": {
+            "Authorization": "Bearer optional-token"
+          },
+          "tools": ["get_account", "list_recent_calls"],
+          "tool_allowlist": ["get_account", "list_recent_calls"],
+          "timeout_seconds": 15
+        }
+      ]
+    },
+    "speech_speed": 1.25
+  }'
+```
+
+`speech_speed` overrides `CARTESIA_TTS_SPEED`, `TTS_SPEED`, or
+`SPEECH_SPEED`. The Cartesia runner clamps speed to `0.6..1.5`.
+
+LLM defaults are resolved as request value, then provider-specific env, then
+generic `LLM_MODEL`, then service default:
+
+- OpenAI: `OPENAI_MODEL`, default `gpt-5.5`. `OPENAI_API_SURFACE` defaults to
+  `responses`; set it to `chat` for compatibility with older OpenAI-compatible
+  paths. `OPENAI_SERVICE_TIER` is passed through when set.
+- Anthropic: `ANTHROPIC_MODEL`, default `claude-opus-4-8`. Low-latency example:
+  `claude-haiku-4-5`.
+- Z.ai: `ZAI_MODEL`, default `glm-5.2`; `ZAI_BASE_URL` defaults to
+  `https://api.z.ai/api/paas/v4/`.
+
+### OpenAPI Snapshots
+
+This repo contains three OpenAPI files with different roles:
+
+- `openapi.json` is this FastAPI service snapshot for generic tooling.
+- `speaking-bot-openapi.json` is the same service snapshot, named explicitly
+  for the speaking-bots MCP sync.
+- `meeting-baas-openapi-v1.json` is the upstream MeetingBaaS v1 API snapshot.
+- `openapi-v2.json` is the upstream MeetingBaaS v2 API snapshot.
+
+The service snapshots include `/bots`, `/bots/{bot_id}`,
+`/personas/generate-image`, `/health`, `/ready`, `/webhook`, and the current
+`BotRequest` fields for `prompt_data_sources`, `prompt_data_token_limit`, `mcp`,
+and `speech_speed`.
+
+Regenerate the service snapshot after API model changes:
+
+```bash
+poetry run python scripts/export_openapi.py
+```
+
 You can still manually specify a WebSocket URL if needed:
 
 ```bash
@@ -440,6 +556,7 @@ Once the server is running, you can access:
 
 - Interactive API docs: `http://localhost:${PORT}/docs`
 - OpenAPI specification: `http://localhost:${PORT}/openapi.json`
+- Committed service OpenAPI snapshots: `openapi.json`, `speaking-bot-openapi.json`
 - Health endpoint: `http://localhost:${PORT}/health`
 - Readiness endpoint: `http://localhost:${PORT}/ready`
 

app/models.py

@@ -1,9 +1,9 @@
 """Data models for the Speaking Meeting Bot API."""
 
 from datetime import datetime
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List, Literal, Optional
 
-from pydantic import BaseModel, ConfigDict, Field, field_validator
+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
 
 
 def _validate_meeting_url(value: str) -> str:
@@ -49,6 +49,153 @@ class TurnConfig(BaseModel):
     )
 
 
+class PromptDataSource(BaseModel):
+    """External context to append to the bot prompt under a token budget."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    name: str = Field(
+        "external_context",
+        min_length=1,
+        max_length=120,
+        description="Human-readable source name shown inside the prompt context block",
+    )
+    type: Literal["text", "url"] = Field(
+        ...,
+        description="Whether to load inline text or fetch an external HTTP(S) URL",
+    )
+    text: Optional[str] = Field(
+        None,
+        description="Inline context. Required when type is text.",
+    )
+    url: Optional[str] = Field(
+        None,
+        description="HTTP(S) URL to fetch. Required when type is url.",
+    )
+    headers: Optional[Dict[str, str]] = Field(
+        None,
+        description="Optional HTTP headers for URL sources. Avoid request-specific secrets unless needed.",
+    )
+    token_limit: Optional[int] = Field(
+        None,
+        ge=1,
+        le=50_000,
+        description="Optional per-source token cap before the request-level cap is applied",
+    )
+
+    @field_validator("url")
+    @classmethod
+    def validate_url(cls, value: Optional[str]) -> Optional[str]:
+        if value is None:
+            return value
+        normalized = value.strip()
+        if not normalized.startswith(("http://", "https://")):
+            raise ValueError("prompt data source url must start with http:// or https://")
+        return normalized
+
+    @model_validator(mode="after")
+    def validate_source_payload(self):
+        if self.type == "text" and not self.text:
+            raise ValueError("text is required when prompt data source type is text")
+        if self.type == "url" and not self.url:
+            raise ValueError("url is required when prompt data source type is url")
+        if self.type == "text" and self.url:
+            raise ValueError("url is not allowed when prompt data source type is text")
+        if self.type == "url" and self.text:
+            raise ValueError("text is not allowed when prompt data source type is url")
+        return self
+
+
+MCPTransport = Literal["http", "streamable_http", "sse"]
+LLMProvider = Literal["openai", "anthropic", "zai"]
+
+
+class MCPServerConfig(BaseModel):
+    """MCP server metadata and optional live connection details."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    name: str = Field(..., min_length=1, max_length=120)
+    enabled: bool = Field(
+        True,
+        description="Whether this server may be used. Disabled servers are documented but not connected.",
+    )
+    url: Optional[str] = Field(
+        None,
+        description="Remote MCP server URL. Required for http, streamable_http, and sse transports.",
+    )
+    headers: Optional[Dict[str, str]] = Field(
+        None,
+        description="Optional HTTP headers for remote MCP servers. Use only when a server requires them.",
+    )
+    transport: Optional[MCPTransport] = Field(
+        None,
+        description="Remote MCP transport. Omit for metadata-only servers that cannot execute tools.",
+    )
+    tools: Optional[List[str]] = Field(
+        None,
+        max_length=50,
+        description="Known tool names exposed by this MCP server",
+    )
+    tool_allowlist: Optional[List[str]] = Field(
+        None,
+        max_length=50,
+        description="Optional allowlist of MCP tool names this bot may call from this server.",
+    )
+    timeout_seconds: Optional[float] = Field(
+        None,
+        ge=0.1,
+        le=300.0,
+        description="Optional per-server connection/tool timeout in seconds.",
+    )
+    instructions: Optional[str] = Field(
+        None,
+        max_length=4_000,
+        description="Operator instructions or constraints for this MCP server",
+    )
+
+    @field_validator("url")
+    @classmethod
+    def validate_mcp_url(cls, value: Optional[str]) -> Optional[str]:
+        if value is None:
+            return value
+        normalized = value.strip()
+        if not normalized.startswith(("http://", "https://")):
+            raise ValueError("mcp server url must start with http:// or https://")
+        return normalized
+
+    @model_validator(mode="after")
+    def validate_connection_details(self):
+        if self.transport in {"http", "streamable_http", "sse"}:
+            if not self.url:
+                raise ValueError(
+                    f"url is required when MCP transport is {self.transport}"
+                )
+        else:
+            if self.url or self.headers:
+                raise ValueError(
+                    "transport is required when MCP connection details are supplied"
+                )
+        return self
+
+
+class MCPConfig(BaseModel):
+    """MCP server metadata and optional live connection details."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    servers: List[MCPServerConfig] = Field(
+        default_factory=list,
+        max_length=10,
+        description="MCP servers to document and optionally connect for tool calls",
+    )
+    instructions: Optional[str] = Field(
+        None,
+        max_length=4_000,
+        description="Global MCP usage instructions for the bot",
+    )
+
+
 class BotRequest(BaseModel):
     """Request model for creating a speaking bot in a meeting."""
 
@@ -64,6 +211,28 @@ class BotRequest(BaseModel):
                 "enable_tools": True,
                 "extra": {"company": "ACME Corp", "meeting_purpose": "Weekly sync"},
                 "websocket_url": "wss://bots.example.com",
+                "prompt_data_token_limit": 3000,
+                "llm_provider": "anthropic",
+                "llm_model": "claude-opus-4-8",
+                "prompt_data_sources": [
+                    {
+                        "name": "CRM account notes",
+                        "type": "url",
+                        "url": "https://example.com/account-notes.md",
+                    }
+                ],
+                "speech_speed": 1.15,
+                "mcp": {
+                    "servers": [
+                        {
+                            "name": "crm",
+                            "url": "https://mcp.example.com",
+                            "transport": "streamable_http",
+                            "tools": ["get_account", "list_recent_calls"],
+                            "tool_allowlist": ["get_account", "list_recent_calls"],
+                        }
+                    ]
+                },
                 "prompt": "You are Meeting Assistant, a concise and professional \
                 AI bot that helps summarize key points and keep the meeting on track. Speak clearly and stay on topic.",
             }
@@ -93,6 +262,37 @@ class BotRequest(BaseModel):
         None,
         description="Per-bot turn-taking tuning (VAD confidence/start_secs/stop_secs/min_volume)",
     )
+    prompt_data_sources: Optional[List[PromptDataSource]] = Field(
+        None,
+        max_length=10,
+        description="External text or URL data sources to append to the bot prompt",
+    )
+    prompt_data_token_limit: int = Field(
+        4_000,
+        ge=0,
+        le=50_000,
+        description="Approximate total token cap for loaded prompt_data_sources. 0 disables loading.",
+    )
+    mcp: Optional[MCPConfig] = Field(
+        None,
+        description="MCP server/tool metadata and optional live connection details",
+    )
+    llm_provider: Optional[LLMProvider] = Field(
+        None,
+        description="LLM provider for this bot. Defaults to LLM_PROVIDER, then openai.",
+    )
+    llm_model: Optional[str] = Field(
+        None,
+        min_length=1,
+        max_length=120,
+        description="Provider model for this bot. Defaults to provider-specific env vars.",
+    )
+    speech_speed: Optional[float] = Field(
+        None,
+        ge=0.5,
+        le=2.0,
+        description="TTS speaking speed multiplier. Defaults to CARTESIA_TTS_SPEED, TTS_SPEED, SPEECH_SPEED, or the runner default.",
+    )
 
     # NOTE: streaming_audio_frequency is intentionally excluded and handled internally