Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
84f1736
MCP → SDK (vocab change only)
jlowin Dec 4, 2025
5fb9103
WIP: Sampling API with SamplingResult[T] and result_type
jlowin Dec 4, 2025
0d8679c
SEP-1577: Sampling with tools
jlowin Dec 4, 2025
ead4730
Fix tool result content handling in OpenAI handler
jlowin Dec 4, 2025
5c0f055
Remove @sampling_tool decorator - pass functions directly to sample()
jlowin Dec 5, 2025
b544cdd
Merge main into sampling-tools-sep-1577
jlowin Dec 5, 2025
1cf947b
Merge main into sampling-tools-sep-1577
github-actions[bot] Dec 10, 2025
4b5bfe6
Remove auto-conversion of MCP tools to sampling tools
jlowin Dec 12, 2025
1d7c1da
Merge branch 'main' into sampling-tools-sep-1577
jlowin Dec 14, 2025
7796179
Merge branch 'main' into sampling-tools-sep-1577
jlowin Dec 14, 2025
2cdc9b6
Refactor sampling API: replace sample_iter() with sample_step()
jlowin Dec 14, 2025
5e6c790
Address CodeRabbit nitpicks
jlowin Dec 14, 2025
1ef5319
Address CodeRabbit review feedback for sampling tools
jlowin Dec 14, 2025
b079fe5
Address additional CodeRabbit review feedback
jlowin Dec 14, 2025
07df7c6
Review fixes for sampling tools PR
jlowin Dec 14, 2025
1bba6e0
Fix OpenAI handler tests to use AsyncOpenAI
jlowin Dec 14, 2025
6bc676e
Merge branch 'main' into sampling-tools-sep-1577
jlowin Dec 14, 2025
5629f24
Address remaining CodeRabbit review comments
jlowin Dec 14, 2025
0d9272e
Add return type annotation to OpenAISamplingHandler.__init__
jlowin Dec 14, 2025
2c63948
Use explicit 'is not None' check for sampling_capabilities defaulting
jlowin Dec 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions docs/clients/sampling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,14 @@ The sampling handler receives three parameters:
<ResponseField name="metadata" type="dict[str, Any] | None">
Optional metadata to pass through to the LLM provider.
</ResponseField>

<ResponseField name="tools" type="list[Tool] | None">
Optional list of tools the LLM can use during sampling. See [Handling Tool Requests](#handling-tool-requests).
</ResponseField>

<ResponseField name="toolChoice" type="ToolChoice | None">
Optional control over tool usage behavior (`auto`, `required`, or `none`).
</ResponseField>
</Expandable>

</ResponseField>
Expand Down Expand Up @@ -153,4 +161,112 @@ client = Client(

<Note>
If the client doesn't provide a sampling handler, servers can optionally configure a fallback handler. See [Server Sampling](/servers/sampling#sampling-fallback-handler) for details.
</Note>

<Tip>
When you provide a `sampling_handler`, FastMCP automatically advertises full sampling capabilities including tool support to the server.
</Tip>

## Handling Tool Requests

<VersionBadge version="2.14.0" />

Servers may request sampling with tools, allowing the LLM to make tool calls during generation. When tools are provided in `params.tools`, your handler should return a `CreateMessageResultWithTools` object instead of a simple string.

### Checking for Tools

```python
from fastmcp import Client
from fastmcp.client.sampling import SamplingMessage, SamplingParams, RequestContext
from mcp.types import (
CreateMessageResultWithTools,
TextContent,
ToolUseContent,
ToolChoice,
)

async def sampling_handler_with_tools(
messages: list[SamplingMessage],
params: SamplingParams,
context: RequestContext
) -> str | CreateMessageResultWithTools:
# Check if tools were provided
if params.tools:
# Call your LLM with tool support
# Return CreateMessageResultWithTools with appropriate content
return CreateMessageResultWithTools(
role="assistant",
content=[TextContent(type="text", text="I'll help you with that.")],
model="gpt-4",
stopReason="endTurn",
)

# Standard text response when no tools
return "Generated response"

client = Client(
"my_mcp_server.py",
sampling_handler=sampling_handler_with_tools,
)
```
Comment on lines +166 to +178

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Clarify what “SamplingCapability()” disables (tools only vs sampling entirely).

The comment says “No tool support”, but SamplingCapability() reads like “no sampling features at all”. Consider showing SamplingCapability(tools=None) (or the exact “no-tools” shape) and stating what remains enabled.

🤖 Prompt for AI Agents
In docs/clients/sampling.mdx around lines 166 to 178, the example and text are
ambiguous about what SamplingCapability() disables; update the example to show
the explicit no-tools shape (e.g., SamplingCapability(tools=None) or the exact
parameter name used in the codebase) and change the surrounding copy to clearly
state that only tool support is being disabled while sampling itself (and other
capabilities) remains enabled; include the explicit shape and a short sentence
listing what still remains enabled.


### Returning Tool Use

When the LLM wants to call a tool, return a `CreateMessageResultWithTools` with `stopReason="toolUse"` and `ToolUseContent` in the content:

```python
from mcp.types import CreateMessageResultWithTools, ToolUseContent

async def sampling_handler_with_tools(
messages: list[SamplingMessage],
params: SamplingParams,
context: RequestContext
) -> str | CreateMessageResultWithTools:
if params.tools:
# Your LLM decided to call a tool
return CreateMessageResultWithTools(
role="assistant",
content=[
ToolUseContent(
type="toolUse",
id="call_123",
name="search",
input={"query": "Python tutorials"},
)
],
model="gpt-4",
stopReason="toolUse", # Indicates tool use
)

return "Response without tools"
```

### Tool Choice

The `params.toolChoice` field indicates how the server wants tools to be used:

- **`auto`**: The LLM decides whether to use tools
- **`required`**: The LLM must use at least one tool
- **`none`**: The LLM should not use any tools

```python
async def sampling_handler_with_tools(
messages: list[SamplingMessage],
params: SamplingParams,
context: RequestContext
) -> str | CreateMessageResultWithTools:
if params.tools and params.toolChoice:
if params.toolChoice.mode == "required":
# Must return a tool call
pass
elif params.toolChoice.mode == "none":
# Should not return tool calls even though tools are available
pass
# "auto" - let the LLM decide

return "Response"
```

<Note>
Tool execution happens on the server side. The client's role is to pass tools to the LLM and return the LLM's response (which may include tool use requests). The server then executes the tools and may send follow-up sampling requests with tool results.
</Note>
Loading
Loading