diff --git a/tools/server/README.md b/tools/server/README.md index 6927caddbb6c..1ced55021ca4 100644 --- a/tools/server/README.md +++ b/tools/server/README.md @@ -335,6 +335,58 @@ The server includes a set of built-in tools that enable the LLM to access the lo To use this feature, start the server with `--tools all`. You can also enable only specific tools by passing a comma-separated list: `--tools name1,name2,...`. Run `--help` for the full list of available tool names. +### MCP servers + +Besides the built-in tools, the server can expose tools coming from MCP servers, added in [#26062](https://github.com/ggml-org/llama.cpp/pull/26062). Only the stdio transport is supported: such a server is a child process reading JSON-RPC messages on its stdin and writing replies on its stdout, so nothing has to be started or maintained outside `llama-server`. + +Servers are declared in a Cursor-compatible JSON file: + +```json +{ + "mcpServers": { + "example": { "command": "/path/to/server", "args": [] } + } +} +``` + +```sh +llama-server -m model.gguf --mcp-servers-config mcp.json +``` + +The same JSON can be passed inline with `--mcp-servers-json`. Each entry under `mcpServers` accepts: + +| Key | Explanation | +| --- | ----------- | +| `command` | executable to spawn, required, entries without it are skipped | +| `args` | array of arguments | +| `env` | object merged over the parent environment | +| `cwd` | working directory of the child process | +| `timeout_ms` | per-tool-call timeout (default: 30000) | + +Every server is spawned once at startup to list its tools, then stopped, and respawned on demand when one of its tools is called. Tools are exposed as `_` alongside the built-in ones: they show up in the Web UI and in `GET /tools`, and the model calls them like any other tool. A name colliding with an already registered tool is skipped. This is independent of `--tools`, MCP servers can be the only tools available. + +The child process runs with the same privileges as the server, so only declare commands you trust. As with `--tools`, `--cors-origins` then defaults to `localhost`. + +Note: `--ui-mcp-proxy` is unrelated, it only lets the Web UI reach remote MCP servers from the browser. + +Any server written against the [MCP specification](https://modelcontextprotocol.io) works as is, whether it uses an official SDK or not: the transport is one JSON-RPC message per line on stdio, so a script wrapping an existing program is a valid server too. + +### CORS + +By default the server reflects any `Origin` header back with credentials allowed. This matches the old, always-on `*` behavior and is fine as long as the server only exposes stateless, read-only endpoints. + +Enabling `--tools` or `--agent` exposes file read/write over the API, so in that case `--cors-origins` defaults to `localhost` instead: only pages served from localhost can reach the server. Pass `--cors-origins` explicitly to override either default. + +Recommended `--cors-origins` setting, depending on where the server runs: + +| Deployment | Recommendation | +| ---------- | --------------- | +| Public | set an API key, put the server behind a reverse proxy, `--cors-origins` optional | +| Local network | set `--cors-origins` to your frontend's origin | +| Same machine | `--cors-origins localhost` (default once `--agent` is set) | + +Related flags: `--cors-origins`, `--cors-methods`, `--cors-headers`, `--cors-credentials` / `--no-cors-credentials`. Background and rationale: [#25655](https://github.com/ggml-org/llama.cpp/pull/25655). + ## Build `llama-server` is built alongside everything else from the root of the project