Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tools/server/README-dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ This endpoint is intended to be used internally by the Web UI and subject to cha
Get a list of tools, each tool has these fields:
- `tool` (string): the ID name of the tool, to be used in POST call. Example: `read_file`
- `display_name` (string): the name to be displayed on UI. Example: `Read file`
- `type` (string): `"builtin"` for a built-in tool, or `"mcp"` for a tool exposed by an MCP server
- `type` (string): `"server"` for a server tool, or `"mcp"` for a tool exposed by an MCP server
- `permissions` (object): a mapping string --> boolean that indicates the permission required by this tool. This is useful for the UI to ask the user before calling the tool. For now, the only permission supported is `"write"`
- `definition` (object): the OAI-compat definition of this tool

Expand Down
12 changes: 6 additions & 6 deletions tools/server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,11 +196,11 @@ For the full list of features, please refer to [server's changelog](https://gith
| `--ui-config, --webui-config JSON` | JSON that provides default UI settings (overrides UI defaults)<br/>(env: LLAMA_ARG_UI_CONFIG) |
| `--ui-config-file, --webui-config-file PATH` | JSON file that provides default UI settings (overrides UI defaults)<br/>(env: LLAMA_ARG_UI_CONFIG_FILE) |
| `--ui-mcp-proxy, --webui-mcp-proxy, --no-ui-mcp-proxy, --no-webui-mcp-proxy` | experimental: whether to enable MCP CORS proxy - do not enable in untrusted environments (default: disabled)<br/>(env: LLAMA_ARG_UI_MCP_PROXY) |
| `--tools TOOL1,TOOL2,...` | experimental: whether to enable built-in tools for AI agents - do not enable in untrusted environments (default: no tools)<br/>specify "all" to enable all tools<br/>available tools: read_file, file_glob_search, grep_search, exec_shell_command, write_file, edit_file, get_info<br/>note: for security reasons, this will limit --cors-origins to localhost by default<br/>(env: LLAMA_ARG_TOOLS) |
| `--tools TOOL1,TOOL2,...` | experimental: whether to enable server tools for AI agents - do not enable in untrusted environments (default: no tools)<br/>specify "all" to enable all tools<br/>available tools: read_file, file_glob_search, grep_search, exec_shell_command, write_file, edit_file, get_info<br/>note: for security reasons, this will limit --cors-origins to localhost by default<br/>(env: LLAMA_ARG_TOOLS) |
| `--tools-runtime OPTION` | experimental: run tools in a separate runtime environment (default: none, use host environment)<br/>available options:<br/> 'docker:<image>', 'podman:<image>': spin up a new container and reuse it for all invocations, clean up on server exit<br/> 'docker-container:<id>', 'podman-container:<id>': use an existing container by ID, won't stop on server exit<br/> 'ssh:<target>': run tools on a remote POSIX host over SSH, key-based auth and a trusted host key are required<br/><br/>(env: LLAMA_ARG_TOOLS_RUNTIME) |
| `--mcp-servers-config PATH` | experimental: path to JSON file with MCP server definitions (Cursor-compatible format) - do not enable in untrusted environments (default: none)<br/>note: for security reasons, this will limit --cors-origins to localhost by default<br/>(env: LLAMA_ARG_MCP_SERVERS_CONFIG) |
| `--mcp-servers-json JSON` | experimental: inline JSON with MCP server definitions (Cursor-compatible format) - do not enable in untrusted environments (default: none)<br/>note: for security reasons, this will limit --cors-origins to localhost by default<br/>(env: LLAMA_ARG_MCP_SERVERS_JSON) |
| `-ag, --agent, -no-ag, --no-agent` | whether to enable CORS proxy and all built-in tools - do not enable in untrusted environments (default: disabled)<br/>note: for security reasons, this will limit --cors-origins to localhost by default<br/>(env: LLAMA_ARG_AGENT) |
| `-ag, --agent, -no-ag, --no-agent` | whether to enable CORS proxy and all server tools - do not enable in untrusted environments (default: disabled)<br/>note: for security reasons, this will limit --cors-origins to localhost by default<br/>(env: LLAMA_ARG_AGENT) |
| `--ui, --webui, --no-ui, --no-webui` | whether to enable the Web UI (default: enabled)<br/>(env: LLAMA_ARG_UI) |
| `--embedding, --embeddings` | restrict to only support embedding use case; use only with dedicated embedding models (default: disabled)<br/>(env: LLAMA_ARG_EMBEDDINGS) |
| `--rerank, --reranking` | enable reranking endpoint on server (default: disabled)<br/>(env: LLAMA_ARG_RERANKING) |
Expand Down Expand Up @@ -337,9 +337,9 @@ It is currently available in the following endpoints:

For more details, please refer to [multimodal documentation](../../docs/multimodal.md)

### Built-in tools support
### Server tools support

The server includes a set of built-in tools that enable the LLM to access the local file system directly from the Web UI.
The server includes a set of server tools that enable the LLM to access the local file system directly from the Web UI.

To use this feature, start the server with `--tools all`. You can also enable only specific tools by passing a comma-separated list: `--tools name1,name2,...`. Run `--help` for the full list of available tool names.

Expand Down Expand Up @@ -1631,9 +1631,9 @@ curl http://localhost:8080/v1/messages/count_tokens \
{"input_tokens": 10}
```

## Server built-in tools
## Server tools

The server exposes a REST API under `/tools` that allows the Web UI to call built-in tools. This endpoint is intended to be used internally by the Web UI and subject to change or to be removed in the future.
The server exposes a REST API under `/tools` that allows the Web UI to call server tools. This endpoint is intended to be used internally by the Web UI and subject to change or to be removed in the future.

**Please do NOT use this endpoint in a downstream application**

Expand Down
2 changes: 1 addition & 1 deletion tools/server/server-tools.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2035,7 +2035,7 @@ void server_tools::setup(const std::vector<std::string> & enabled_tools,
}
}

// append MCP tools, skipping any that collide with a built-in or another MCP tool of the same "<server>_<tool>" name
// append MCP tools, skipping any that collide with a server tool or another MCP tool of the same "<server>_<tool>" name
if (!mcp_mgr.empty()) {
std::unordered_set<std::string> seen_names;
for (auto & t : tools) {
Expand Down
2 changes: 1 addition & 1 deletion tools/server/server-tools.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ struct server_tool {

virtual ~server_tool() = default;
virtual json get_definition() const = 0;
virtual std::string type() const { return "builtin"; }
virtual std::string type() const { return "server"; }

struct stream {
server_response & qr;
Expand Down
2 changes: 1 addition & 1 deletion tools/server/server.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ int llama_server(common_params & params, int argc, char ** argv) {
ctx_http.get ("/tools", ex_wrapper(tools.handle_get));
ctx_http.post("/tools", ex_wrapper(tools.handle_post));
if (!params.server_tools.empty()) {
warn_names.push_back("built-in tools (experimental)");
warn_names.push_back("server tools (experimental)");
}
if (!params.server_tools_runtime.empty()) {
warn_names.push_back("tools runtime (experimental)");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
<span>
Run llama-server with <code>{CLI_FLAGS.TOOLS}</code> flag to enable

<strong>Built-in Tools</strong>.
<strong>Server Tools</strong>.
</span>
</span>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
// it, the picker still opens for manual entry but explains why search is
// unavailable instead of firing searches that would only fail. Browse is
// hidden too: it resolves the picked folder name through the same tool.
const fileSearchKey = $derived(toolsStore.getPermissionKey(BuiltInTool.FILE_GLOB_SEARCH));
const fileSearchKey = $derived(toolsStore.getPermissionKey(BuiltInTool.SERVER_FILE_GLOB_SEARCH));
const fileSearchEnabled = $derived(
fileSearchKey !== null && toolsStore.isToolEnabled(fileSearchKey)
);
Expand Down Expand Up @@ -212,7 +212,7 @@
// so the caller fails visibly instead of committing a bare leaf name.
async function resolveNativeName(name: string): Promise<string | null> {
try {
const res = await ToolsService.executeToolRaw(BuiltInTool.FILE_GLOB_SEARCH, {
const res = await ToolsService.executeToolRaw(BuiltInTool.SERVER_FILE_GLOB_SEARCH, {
include: buildCaseInsensitiveGlob(name),
limit: SEARCH.NATIVE_LIMIT,
max_depth: SEARCH.NATIVE_MAX_DEPTH,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
// When the server does not expose file_glob_search (started without
// --tools) or the user disabled it, the picker still opens but explains
// why instead of firing searches that would only fail.
const fileSearchKey = $derived(toolsStore.getPermissionKey(BuiltInTool.FILE_GLOB_SEARCH));
const fileSearchKey = $derived(toolsStore.getPermissionKey(BuiltInTool.SERVER_FILE_GLOB_SEARCH));
const fileSearchEnabled = $derived(
fileSearchKey !== null && toolsStore.isToolEnabled(fileSearchKey)
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,19 +35,19 @@

{#if isSearchCall}
<ChatMessageToolCallBlockSearchResults {section} {open} {isStreaming} {onToggle} />
{:else if section.toolName === BuiltInTool.GET_DATETIME}
{:else if section.toolName === BuiltInTool.BROWSER_GET_DATETIME}
<ChatMessageToolCallBlockGetDatetime {section} {isStreaming} />
{:else if section.toolName === BuiltInTool.GET_INFO}
{:else if section.toolName === BuiltInTool.SERVER_GET_INFO}
<ChatMessageToolCallBlockGetInfo {section} {isStreaming} />
{:else if section.toolName === BuiltInTool.READ_FILE}
{:else if section.toolName === BuiltInTool.SERVER_READ_FILE}
<ChatMessageToolCallBlockReadFile {section} {open} {isStreaming} {onToggle} />
{:else if section.toolName === BuiltInTool.READ_MEDIA}
{:else if section.toolName === BuiltInTool.BROWSER_READ_MEDIA}
<ChatMessageToolCallBlockReadMedia {section} {open} {isStreaming} {onToggle} />
{:else if section.toolName === BuiltInTool.EDIT_FILE}
{:else if section.toolName === BuiltInTool.SERVER_EDIT_FILE}
<ChatMessageToolCallBlockEditFile {section} {open} {isStreaming} {onToggle} />
{:else if section.toolName === BuiltInTool.WRITE_FILE}
{:else if section.toolName === BuiltInTool.SERVER_WRITE_FILE}
<ChatMessageToolCallBlockWriteFile {section} {open} {isStreaming} {onToggle} />
{:else if section.toolName === BuiltInTool.EXEC_SHELL_COMMAND}
{:else if section.toolName === BuiltInTool.SERVER_EXEC_SHELL_COMMAND}
<ChatMessageToolCallBlockExecShellCommand
{section}
{open}
Expand All @@ -56,11 +56,11 @@
{attachments}
{onToggle}
/>
{:else if section.toolName === BuiltInTool.FILE_GLOB_SEARCH}
{:else if section.toolName === BuiltInTool.SERVER_FILE_GLOB_SEARCH}
<ChatMessageToolCallBlockFileGlobSearch {section} {open} {isStreaming} {onToggle} />
{:else if section.toolName === BuiltInTool.GREP_SEARCH}
{:else if section.toolName === BuiltInTool.SERVER_GREP_SEARCH}
<ChatMessageToolCallBlockGrepSearch {section} {open} {isStreaming} {onToggle} />
{:else if section.toolName === BuiltInTool.RUN_JAVASCRIPT}
{:else if section.toolName === BuiltInTool.BROWSER_RUN_JAVASCRIPT}
<ChatMessageToolCallBlockRunJavascript {section} {open} {isStreaming} {onToggle} />
{:else}
<ChatMessageToolCallBlockDefault {section} {open} {isStreaming} {attachments} {onToggle} />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
import {
classifyToolResult,
formatJsonPretty,
getBuiltinToolUi,
getToolUi,
parseToolResultWithMedia
} from '$lib/utils';
import { createBase64DataUrl } from '$lib/utils/data-url';
Expand All @@ -27,7 +27,7 @@

let { attachments, isStreaming, onToggle, open, section }: Props = $props();

const title = $derived(getBuiltinToolUi(section.toolName)?.label ?? section.toolName ?? '');
const title = $derived(getToolUi(section.toolName)?.label ?? section.toolName ?? '');
const outputKind = $derived(classifyToolResult(section.toolResult));
const parsedLines: ToolResultLine[] = $derived(
section.toolResult ? parseToolResultWithMedia(section.toolResult, attachments) : []
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import { MAX_HEIGHT_CODE_BLOCK } from '$lib/constants';
import { FileTypeText } from '$lib/enums';
import type { AgenticSection } from '$lib/types';
import { getBuiltinToolUi } from '$lib/utils';
import { getToolUi } from '$lib/utils';

interface Props {
section: AgenticSection;
Expand All @@ -18,7 +18,7 @@
let { isStreaming, onToggle, open, section }: Props = $props();

const runJsMeta = $derived(parseRunJavascriptMeta(section));
const title = $derived(getBuiltinToolUi(section.toolName)?.label ?? section.toolName ?? '');
const title = $derived(getToolUi(section.toolName)?.label ?? section.toolName ?? '');
</script>

<ToolCallBlock {section} {open} {isStreaming} meta={runJsMeta} {title} {onToggle}>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
import { ICON_CLASS_DEFAULT, ICON_CLASS_SPIN } from '$lib/constants';
import { AgenticSectionType } from '$lib/enums';
import { mcpStore } from '$lib/stores';
import type { AgenticSection, BuiltinToolUiEntry } from '$lib/types';
import { getBuiltinToolUi } from '$lib/utils';
import type { AgenticSection, ToolUiEntry } from '$lib/types';
import { getToolUi } from '$lib/utils';
import type { Component, Snippet } from 'svelte';

type ToolCallBlockMetaWithError = TMeta & { errorMessage?: string };
Expand Down Expand Up @@ -82,7 +82,7 @@
const showSpinner = $derived(isPending || (isStreamingCall && isStreaming) || extraLiveStreaming);
const isCodeStreaming = $derived(isStreaming && (isPending || isStreamingCall));

const toolUi: BuiltinToolUiEntry | null = $derived(getBuiltinToolUi(section.toolName));
const toolUi: ToolUiEntry | null = $derived(getToolUi(section.toolName));
const toolIcon: Component = $derived(
spinIconWhenActive && showSpinner ? Loader2 : (toolUi?.icon ?? Wrench)
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ export type EditFileMeta = {
};

export function parseEditFileMeta(section: AgenticSection): EditFileMeta | null {
const args = parseToolArgs(BuiltInTool.EDIT_FILE, section, { partial: true });
const args = parseToolArgs(BuiltInTool.SERVER_EDIT_FILE, section, { partial: true });

if (!args) return null;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ export type ExecShellCommandMeta = {
};

export function parseExecShellCommandMeta(section: AgenticSection): ExecShellCommandMeta | null {
const args = parseToolArgs(BuiltInTool.EXEC_SHELL_COMMAND, section);
const args = parseToolArgs(BuiltInTool.SERVER_EXEC_SHELL_COMMAND, section);

if (!args) return null;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ export type FileGlobSearchMeta = {
};

export function parseFileGlobSearchMeta(section: AgenticSection): FileGlobSearchMeta | null {
const args = parseToolArgs(BuiltInTool.FILE_GLOB_SEARCH, section);
const args = parseToolArgs(BuiltInTool.SERVER_FILE_GLOB_SEARCH, section);

if (!args) return null;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ export type GrepSearchMeta = {
};

export function parseGrepSearchMeta(section: AgenticSection): GrepSearchMeta | null {
const args = parseToolArgs(BuiltInTool.GREP_SEARCH, section);
const args = parseToolArgs(BuiltInTool.SERVER_GREP_SEARCH, section);

if (!args) return null;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ export type ReadFileMeta = {
};

export function parseReadFileMeta(section: AgenticSection): ReadFileMeta | null {
const args = parseToolArgs(BuiltInTool.READ_FILE, section, { partial: true });
const args = parseToolArgs(BuiltInTool.SERVER_READ_FILE, section, { partial: true });

if (!args) return null;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ export type RunJavascriptMeta = {
};

export function parseRunJavascriptMeta(section: AgenticSection): RunJavascriptMeta | null {
const args = parseToolArgs(BuiltInTool.RUN_JAVASCRIPT, section);
const args = parseToolArgs(BuiltInTool.BROWSER_RUN_JAVASCRIPT, section);

if (!args) return null;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ export type WriteFileMeta = {
};

export function parseWriteFileMeta(section: AgenticSection): WriteFileMeta | null {
const args = parseToolArgs(BuiltInTool.WRITE_FILE, section, { partial: true });
const args = parseToolArgs(BuiltInTool.SERVER_WRITE_FILE, section, { partial: true });

if (!args) return null;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@
{:else}
{@const source = toolsStore.getToolSource(toolName)}
{@const providerName =
source === ToolSource.BUILTIN
? TOOL_SERVER_LABELS[ToolSource.BUILTIN]
source === ToolSource.SERVER
? TOOL_SERVER_LABELS[ToolSource.SERVER]
: source === ToolSource.CUSTOM
? TOOL_SERVER_LABELS[ToolSource.CUSTOM]
: 'MCP Tools'}
Expand Down
4 changes: 2 additions & 2 deletions tools/ui/src/lib/components/app/chat/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ export { default as ChatFormInput } from './ChatForm/ChatFormInput/ChatFormInput
/**
* Working directory selector for agent mode. Renders a chip below the chat
* form; clicking it opens a popover with a directory picker backed by the
* server's `file_glob_search` built-in tool (POST /tools). The picked
* server's `file_glob_search` server tool (POST /tools). The picked
* directory is exposed via `bind:directory`; changing it records a
* synthetic "Set working directory to ..." user message into chat history
* and is enforced on tool calls via the `x-tool-cwd` request header.
Expand Down Expand Up @@ -380,7 +380,7 @@ export { default as ChatFormPickerListItemSkeleton } from './ChatForm/ChatFormPi

/**
* `@`-triggered file/folder mention picker. Resolves `@<query>` in the chat
* input to a filesystem match via the server's `file_glob_search` built-in
* input to a filesystem match via the server's `file_glob_search` server tool
* tool, scoped to the conversation cwd (or server home when unset).
* Selection splices a `[name](file:///<abs path>)` link into the input.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import { ICON_CLASS_DEFAULT } from '$lib/constants';
import { ToolSource } from '$lib/enums/tools.enums';
import { mcpStore, permissionsStore, toolsStore } from '$lib/stores';
import { getBuiltinToolUi } from '$lib/utils';
import { getToolUi } from '$lib/utils';
import { SvelteSet } from 'svelte/reactivity';

let expandedGroups = new SvelteSet<string>();
Expand Down Expand Up @@ -69,12 +69,12 @@

{#each group.tools as entry (entry.key)}
{@const toolName = entry.definition.function.name}
{@const builtinUi =
entry.source === ToolSource.BUILTIN || entry.source === ToolSource.FRONTEND
? getBuiltinToolUi(toolName)
{@const toolUi =
entry.source === ToolSource.SERVER || entry.source === ToolSource.BROWSER
? getToolUi(toolName)
: null}
{@const displayLabel = builtinUi?.label ?? toolName}
{@const IconComponent = builtinUi?.icon ?? null}
{@const displayLabel = toolUi?.label ?? toolName}
{@const IconComponent = toolUi?.icon ?? null}
{@const isEnabled = toolsStore.isToolEnabled(entry.key)}
{@const permissionKey = entry.key}
{@const isAlwaysAllowed = permissionsStore.hasTool(permissionKey)}
Expand Down
2 changes: 1 addition & 1 deletion tools/ui/src/lib/components/app/settings/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ export { default as SettingsChatFields } from './SettingsChat/SettingsChatFields
/**
* **SettingsChatToolsTab** - Tools configuration tab for chat settings
*
* Displays available tools grouped by source (built-in, MCP, custom) with
* Displays available tools grouped by source (server, browser, MCP, custom) with
* toggles to enable/disable individual tools and tool groups. Shows MCP
* server favicons and permission management controls.
*/
Expand Down
4 changes: 3 additions & 1 deletion tools/ui/src/lib/constants/browser-info.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ import { CLI_FLAGS } from './cli-flags.constants';
import { BuiltInTool, JsonSchemaType, ToolCallType } from '$lib/enums';
import type { OpenAIToolDefinition } from '$lib/types';

export const BROWSER_INFO_TOOL_NAME = BuiltInTool.GET_INFO;
// get_info is served by the server, but the browser falls back to this
// implementation when the server does not provide it - same wire name.
export const BROWSER_INFO_TOOL_NAME = BuiltInTool.SERVER_GET_INFO;

/** UA token to OS name, first match wins - Android and iOS UAs also carry the Linux / Mac OS X tokens */
export const BROWSER_INFO_OS_UA_PATTERNS: readonly [RegExp, string][] = [
Expand Down
Loading
Loading