Run sync tools/resources/prompts in threadpool automatically#2865
Run sync tools/resources/prompts in threadpool automatically#2865
Conversation
Sync functions now automatically run in a threadpool via anyio.to_thread.run_sync(), preventing them from blocking the event loop during concurrent requests. This matches FastAPI/Starlette's behavior for sync endpoints. Also fixes callable object support for resources (normalization and name extraction).
marvin-context-protocol
Bot
added
enhancement
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 112e1e6d02
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if fn_is_async: | ||
| return await fn(**resolved_kwargs) | ||
| else: | ||
| # Run sync functions in threadpool to avoid blocking the event loop | ||
| return await call_sync_fn_in_threadpool(fn, **resolved_kwargs) |
There was a problem hiding this comment.
The new wrapper only awaits when inspect.iscoroutinefunction(fn) is true; otherwise it runs fn in a threadpool and returns whatever it produces. If a sync wrapper returns an awaitable (e.g., a functools.partial(async_fn, ...), or a decorator that hides coroutinefunction), this path now returns a coroutine object instead of awaiting it. Previously, any awaitable result was awaited, so these callables worked. With the change, prompts/resources/tools will try to serialize a coroutine or behave incorrectly. Consider awaiting result when it’s inspect.isawaitable(result) after the threadpool call to preserve the prior behavior for such wrappers.
Useful? React with 👍 / 👎.
WalkthroughThis pull request adds a new async helper Possibly related PRs
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
📜 Recent review detailsConfiguration used: Organization UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (4)
🧰 Additional context used📓 Path-based instructions (1)**/*.py📄 CodeRabbit inference engine (AGENTS.md)
Files:
🧠 Learnings (1)📚 Learning: 2026-01-13T03:11:40.907ZApplied to files:
🧬 Code graph analysis (4)src/fastmcp/tools/function_tool.py (3)
src/fastmcp/prompts/function_prompt.py (1)
src/fastmcp/server/dependencies.py (1)
src/fastmcp/resources/function_resource.py (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
🔇 Additional comments (10)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/servers/prompts.mdx (1)
364-375: Add missingaiohttpimport for runnable example.The async example uses
aiohttp.ClientSession()but the import statement is missing, making this code non-runnable. As per coding guidelines, documentation should include complete, runnable code examples.📝 Suggested fix
# Asynchronous prompt @mcp.prompt async def data_based_prompt(data_id: str) -> str: """Generates a prompt based on data that needs to be fetched.""" - # In a real scenario, you might fetch data from a database or API + import aiohttp # Or add to imports at top of your file async with aiohttp.ClientSession() as session: async with session.get(f"https://api.example.com/data/{data_id}") as response: data = await response.json() return f"Analyze this data: {data['content']}"Alternatively, add
import aiohttpalongside the other imports at line 31 in the initial code block, or use a comment indicating the required import.
🧹 Nitpick comments (1)
src/fastmcp/server/dependencies.py (1)
32-32: Consider whether re-exporting is intentional.
call_sync_fn_in_threadpoolis already the canonical export fromfastmcp.utilities.async_utils. Adding it to__all__here creates a secondary import path (from fastmcp.server.dependencies import call_sync_fn_in_threadpool).Based on learnings, the guidance is to "be intentional about module re-exports." If this re-export is for internal convenience within the
serversubpackage, that's fine. If it's meant as a public API, users should import fromutilities.async_utilsdirectly. Consider removing from__all__if it's not intended as a public API of the dependencies module.Also applies to: 50-50
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
tests/prompts/test_prompt.pyis excluded by none and included by nonetests/resources/test_function_resources.pyis excluded by none and included by nonetests/tools/test_tool.pyis excluded by none and included by none
📒 Files selected for processing (8)
docs/servers/prompts.mdxdocs/servers/resources.mdxdocs/servers/tools.mdxsrc/fastmcp/prompts/function_prompt.pysrc/fastmcp/resources/function_resource.pysrc/fastmcp/server/dependencies.pysrc/fastmcp/tools/function_tool.pysrc/fastmcp/utilities/async_utils.py
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Python ≥3.10 with full type annotations required for all code
Never use bareexcept- be specific with exception types in Python code
Files:
src/fastmcp/server/dependencies.pysrc/fastmcp/resources/function_resource.pysrc/fastmcp/tools/function_tool.pysrc/fastmcp/utilities/async_utils.pysrc/fastmcp/prompts/function_prompt.py
docs/**/*.mdx
📄 CodeRabbit inference engine (docs/.cursor/rules/mintlify.mdc)
docs/**/*.mdx: Use clear, direct language appropriate for technical audiences
Write in second person ('you') for instructions and procedures in MDX documentation
Use active voice over passive voice in MDX technical documentation
Employ present tense for current states and future tense for outcomes in MDX documentation
Maintain consistent terminology throughout all MDX documentation
Keep sentences concise while providing necessary context in MDX documentation
Use parallel structure in lists, headings, and procedures in MDX documentation
Lead with the most important information using inverted pyramid structure in MDX documentation
Use progressive disclosure in MDX documentation: present basic concepts before advanced ones
Break complex procedures into numbered steps in MDX documentation
Include prerequisites and context before instructions in MDX documentation
Provide expected outcomes for each major step in MDX documentation
End sections with next steps or related information in MDX documentation
Use descriptive, keyword-rich headings for navigation and SEO in MDX documentation
Focus on user goals and outcomes rather than system features in MDX documentation
Anticipate common questions and address them proactively in MDX documentation
Include troubleshooting for likely failure points in MDX documentation
Provide multiple pathways (beginner vs advanced) but offer an opinionated path to avoid overwhelming users in MDX documentation
Always include complete, runnable code examples that users can copy and execute in MDX documentation
Show proper error handling and edge case management in MDX code examples
Use realistic data instead of placeholder values in MDX code examples
Include expected outputs and results for verification in MDX code examples
Test all code examples thoroughly before publishing in MDX documentation
Specify language and include filename when relevant in MDX code examples
Add explanatory comments for complex logic in MDX code examples
Document all API...
Files:
docs/servers/prompts.mdxdocs/servers/resources.mdxdocs/servers/tools.mdx
🧠 Learnings (5)
📚 Learning: 2026-01-13T03:11:40.907Z
Learnt from: CR
Repo: jlowin/fastmcp PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-13T03:11:40.907Z
Learning: Applies to **/__init__.py : Be intentional about module re-exports - only re-export fundamental types to fastmcp.*; prefer users importing from specific submodules
Applied to files:
src/fastmcp/server/dependencies.py
📚 Learning: 2026-01-12T16:24:54.978Z
Learnt from: CR
Repo: jlowin/fastmcp PR: 0
File: .cursor/rules/core-mcp-objects.mdc:0-0
Timestamp: 2026-01-12T16:24:54.978Z
Learning: Applies to src/prompts/**/*.{ts,tsx,js,jsx} : Changes affecting MCP Prompts (like adding tags, importing, etc.) must be adopted, applied, and tested consistently
Applied to files:
docs/servers/prompts.mdx
📚 Learning: 2026-01-12T16:24:54.978Z
Learnt from: CR
Repo: jlowin/fastmcp PR: 0
File: .cursor/rules/core-mcp-objects.mdc:0-0
Timestamp: 2026-01-12T16:24:54.978Z
Learning: Applies to src/resources/**/*.{ts,tsx,js,jsx} : Changes affecting MCP Resources (like adding tags, importing, etc.) must be adopted, applied, and tested consistently
Applied to files:
docs/servers/resources.mdx
📚 Learning: 2026-01-12T16:24:54.978Z
Learnt from: CR
Repo: jlowin/fastmcp PR: 0
File: .cursor/rules/core-mcp-objects.mdc:0-0
Timestamp: 2026-01-12T16:24:54.978Z
Learning: Applies to src/tools/**/*.{ts,tsx,js,jsx} : Changes affecting MCP Tools (like adding tags, importing, etc.) must be adopted, applied, and tested consistently
Applied to files:
docs/servers/tools.mdx
📚 Learning: 2025-11-03T17:36:13.363Z
Learnt from: jlowin
Repo: jlowin/fastmcp PR: 2355
File: docs/clients/client.mdx:226-246
Timestamp: 2025-11-03T17:36:13.363Z
Learning: In FastMCP documentation, prefer showing the happy path in onboarding examples without over-explaining edge cases or adding defensive checks, as this reduces cognitive burden for new users learning the API.
Applied to files:
docs/servers/tools.mdx
🧬 Code graph analysis (4)
src/fastmcp/server/dependencies.py (1)
src/fastmcp/utilities/async_utils.py (1)
call_sync_fn_in_threadpool(13-21)
src/fastmcp/resources/function_resource.py (1)
src/fastmcp/utilities/async_utils.py (1)
call_sync_fn_in_threadpool(13-21)
src/fastmcp/tools/function_tool.py (2)
src/fastmcp/server/dependencies.py (1)
without_injected_parameters(435-496)src/fastmcp/utilities/async_utils.py (1)
call_sync_fn_in_threadpool(13-21)
src/fastmcp/prompts/function_prompt.py (1)
src/fastmcp/utilities/async_utils.py (1)
call_sync_fn_in_threadpool(13-21)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run tests: Python 3.13 on ubuntu-latest
- GitHub Check: Run tests: Python 3.10 on windows-latest
- GitHub Check: Run tests: Python 3.10 on ubuntu-latest
- GitHub Check: Run tests with lowest-direct dependencies
🔇 Additional comments (14)
src/fastmcp/utilities/async_utils.py (2)
3-8: LGTM on imports.The imports are correctly organized, and the alias
run_sync_in_threadpoolprovides clarity at the call site.
13-21: Implementation is correct.The use of
functools.partialto bundlefnwith*argsand**kwargsis the right approach sinceanyio.to_thread.run_syncexpects a zero-argument callable. The contextvar propagation claim is accurate—anyio'srun_synccopies contextvars from the calling task into the worker thread by default, making this safe for functions that depend on context (like dependency injection).The return type is
Any, which is acceptable for this utility pattern. ATypeVarcould provide better type inference for callers if desired in the future.src/fastmcp/resources/function_resource.py (4)
23-23: LGTM on import.
150-153: Good enhancement for callable object support.The fallback chain (
metadata.name→fn.__name__→fn.__class__.__name__) correctly handles callable classes that may not have__name__but do have a class name.
165-170: Consistent callable normalization.This matches the pattern used in
FunctionPrompt.from_functionandFunctionTool.from_function, ensuring uniform handling of callable classes and staticmethods across the codebase.
199-203: Sync branch reachability is correct and intentional.The analysis is accurate:
without_injected_parametersreturns the original function unchanged when there are no Context or Depends() parameters to exclude (line 466 in dependencies.py). When dependencies exist, it returns an async wrapper that always handles both sync and async functions internally.This design ensures:
- Simple sync resources without DI use the threadpool branch (lines 200-201)
- Resources with dependencies always use the async wrapper via the async branch (line 199)
The pattern is intentional and correct.
src/fastmcp/prompts/function_prompt.py (2)
33-33: LGTM on import.
297-301: Correct async/sync dispatch pattern.This mirrors the pattern in
FunctionResource.read(). Theiscoroutinefunctioncheck correctly identifies whetherself.fn(fromwithout_injected_parameters) is async or sync, routing sync prompts through the threadpool to avoid blocking.src/fastmcp/server/dependencies.py (1)
476-484: Clean async/sync dispatch in wrapper.Capturing
fn_is_async = inspect.iscoroutinefunction(fn)outside the wrapper avoids repeated introspection on each call. The branching logic correctly routes sync functions through the threadpool while preserving the resolved kwargs from dependency injection.src/fastmcp/tools/function_tool.py (2)
24-24: LGTM on imports.Import reorganization and addition of
call_sync_fn_in_threadpoolare clean.Also applies to: 33-33
239-247: Correct threadpool dispatch for sync tools.The
type_adapter.validate_python(arguments)call validates arguments against the function's schema and invokes the function. For syncwrapper_fn, this entire operation (validation + execution) runs in the threadpool, which is the correct behavior to avoid blocking.Note: The comment "Async function: validate_python returns a coroutine" is slightly imprecise—
validate_pythonitself is sync, but it returns an awaitable when the underlying function is async. The behavior is correct regardless.docs/servers/tools.mdx (1)
143-158: Clear and practical documentation update.The updated section correctly explains the new behavior: sync tools automatically run in a threadpool. The example with
time.sleep(2)effectively demonstrates a blocking operation that won't block the event loop. The guidance to prefer async for I/O-bound work while allowing sync for simpler cases provides a good mental model for users.docs/servers/resources.mdx (1)
273-292: Documentation aligns with implementation.The updated section mirrors the tools documentation pattern: sync functions automatically run in a threadpool, async is more efficient for I/O. The
aiofilesexample demonstrates proper async file reading with appropriate error handling, providing a good template for users.docs/servers/prompts.mdx (1)
355-362: LGTM!The updated documentation clearly explains the new threadpool behavior for synchronous prompts. The comment on line 358 helpfully indicates that the sync function runs in a threadpool, which aligns with the PR's implementation changes.
Address PR review comments: - Add inspect.isawaitable() check after threadpool execution to handle edge cases like functools.partial(async_fn) - Remove call_sync_fn_in_threadpool from __all__ (internal utility)
1 participant
Sync functions for tools, resources, and prompts now automatically run in a threadpool, preventing them from blocking the event loop during concurrent requests.
Previously, 3 concurrent calls to this tool would take 30 seconds (sequential). Now they take ~10 seconds (parallel).
Uses
anyio.to_thread.run_sync()which properly propagates contextvars, soContextandDependscontinue to work. This matches FastAPI/Starlette's behavior for sync endpoints.Also fixes callable object support for resources (was missing the
__call__normalization that tools/prompts had).