Fix MCP schemas for dynamically dispatched Hermes tools - #52493
Conversation
Related to #49800 (same root cause: FastMCP reflects the |
|
We independently reproduced this exact defect from recent Codex/gpt-5.6-sol sessions: FastMCP I opened #62762 before this older PR surfaced in the phrase search. I agree #52493 is the canonical earlier report for the core schema replacement. One concrete gap to preserve when landing a canonical fix: this patch replaces Happy for maintainers to choose #52493 as canonical and fold those validation/error tests in; I can close #62762 once the complete behavior is represented. |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for addressing the dynamic-tool schema defect. The premise is confirmed on current upstream main: agent/transports/hermes_tools_mcp_server.py:154-185 reads Hermes' params_schema but registers only a **kwargs callable, so the schema is not used by FastMCP.
Problems
agent/transports/hermes_tools_mcp_server.py:152-156replacescall_fn_with_arg_validationwith directfn(**kwargs)execution. This leaves the advertised Hermes schema unenforced before dispatch; currentmodel_tools.py:1065-1068performs coercion, not required/enum/nested/additional-properties validation. The July 11 discussion correctly calls out this protocol-contract gap.agent/transports/hermes_tools_mcp_server.py:236ignores a failed private-schema installation. The server can then continue with FastMCP's reflectedkwargscontract. The new test usesSimpleNamespace(tests/agent/transports/test_hermes_tools_mcp_server.py:120-156), so it does not prove compatibility with the pinned FastMCP implementation.
Suggested changes
- Preserve exact server-side schema validation and add real FastMCP negative tests showing invalid calls never dispatch.
- Fail closed if the required FastMCP internals cannot be replaced, and exercise a real tools/list plus structured-call round trip.
Automated hermes-sweeper review.
|
|
||
| object.__setattr__( | ||
| fn_metadata, | ||
| "call_fn_with_arg_validation", |
There was a problem hiding this comment.
This replaces FastMCP's validator with raw dispatch, so the newly advertised required, enum, nested, and additionalProperties constraints are never enforced before a Hermes tool runs. Please preserve equivalent validation and add protocol tests proving invalid calls do not reach handle_function_call.
| handler = _make_handler(name) | ||
| handler = mcp.tool(name=name, description=description)(handler) | ||
|
|
||
| _install_tool_schema_override(mcp, name, params_schema) |
There was a problem hiding this comment.
Please handle a false result here. _install_tool_schema_override() can fail when FastMCP's private shape differs; ignoring that failure silently restores the broken reflected-kwargs schema instead of surfacing an incompatible server.
Summary
**kwargsschema with each Hermes tool's authoritativeparametersschemahandle_function_callWhy
The hermes-tools MCP server registers dynamic trampoline functions (
**kwargs) for Hermes tools. FastMCP reflects that signature as akwargsparameter, so MCP clients see schemas like{"kwargs": "string"}instead of the real tool inputs such asimage_urlandquestionforvision_analyze. This makes standards-compliant MCP clients generate the wrong tool arguments even though Hermes already has the correct schema in its tool registry.Testing
uv run --extra dev python -m pytest tests/agent/transports/test_hermes_tools_mcp_server.py -qvision_analyzeexposesimage_url/questionand forwards structured args tohandle_function_call