Add outputSchema support #509
Conversation
✅ Docs preview readyThe preview is ready to be viewed. View the preview File Changes 0 new, 1 changed, 0 removedBuild ID: b369d45937ada96fdf90ef18 URL: https://www.apollographql.com/docs/deploy-preview/b369d45937ada96fdf90ef18 |
| ToolAnnotations::new() | ||
| .read_only(operation.operation_type != OperationType::Mutation), | ||
| ); | ||
| tool.output_schema = output_schema.map(std::sync::Arc::new); |
There was a problem hiding this comment.
Hey @swcollard, I considered using with_output_schema you suggested in the issue. However, I realized it needs a compile-time known type to generate the schema, which doesn't fit our case. We need the schema to be generated dynamically at runtime by walking through GraphQL selection sets. So, I'm setting output_schema directly here.
There was a problem hiding this comment.
Thanks for adding this! Definitely a much bigger lift than I expected.
I wonder if there is also an update needed to let character_count = tool_character_length(&tool);? The downside of adding the output schema is that for a very large graphql operation, I could see the overall tool definition taking up many more tokens than before.
There was a problem hiding this comment.
Good catch, @swcollard ! I've updated tool_character_length to include the output schema in the character count.
DaleSeo
force-pushed
the
AIR-69
branch
2 times, most recently
from
a1d4924 to
966635a
Compare
| let description = format!( | ||
| "{}\n\nValues:\n{}", | ||
| enum_def | ||
| .description | ||
| .as_ref() | ||
| .map(|n| n.to_string()) | ||
| .unwrap_or_default(), | ||
| enum_def | ||
| .values | ||
| .iter() | ||
| .map(|(name, value)| { | ||
| format!( | ||
| "{}: {}", | ||
| name, | ||
| value | ||
| .description | ||
| .as_ref() | ||
| .map(|d| d.to_string()) | ||
| .unwrap_or_default() | ||
| ) | ||
| }) | ||
| .collect::<Vec<_>>() | ||
| .join("\n") | ||
| ); |
There was a problem hiding this comment.
Could this possibly blow up token usage when large enums are being utilized?
There was a problem hiding this comment.
Good point! Simplified this to only include the enum's type-level description. The enum constraint already lists all valid values, so per-value descriptions are redundant for output schemas.
| let obj = output_schema.as_object().unwrap(); | ||
| assert!(obj.contains_key("properties")); | ||
|
|
||
| let props = obj.get("properties").unwrap().as_object().unwrap(); | ||
| assert!(props.contains_key("data")); | ||
| assert!(props.contains_key("errors")); |
There was a problem hiding this comment.
I wonder if snapshot tests would make more sense here
| let output_json = serde_json::to_string_pretty(&output_schema).unwrap(); | ||
| println!("{}", output_json); | ||
|
|
||
| // Schema should include nested profile structure | ||
| assert!(output_json.contains("profile")); | ||
| assert!(output_json.contains("bio")); | ||
| assert!(output_json.contains("avatar")); |
There was a problem hiding this comment.
Same goes for here. We can get rid of the print as well
There was a problem hiding this comment.
Done! Switched to snapshot tests and removed the println!.
DaleSeo
requested review from
andrewmcgivery,
esilverm and
gocamille
and removed request for
swcollard
|
Thanks for your feedback, @esilverm! After talking with @andrewmcgivery and @gocamille, I've made the output schema generation opt-in. The new overrides:
enable_output_schema: true # opt-inI hope this addresses your concerns about token usage. |
| @@ -0,0 +1,3 @@ | |||
| ### Add outputSchema support - @DaleSeo PR #509 | |||
|
|
|||
| This PR implements support for the MCP specification's [outputSchema](https://modelcontextprotocol.io/specification/2025-11-25/server/tools#output-schema) field on tools, which allows tools to declare the expected structure of their output. This helps LLMs better understand and reason about GraphQL response data. | |||
There was a problem hiding this comment.
Can we please include the part about needing to opt into it and include a config example? :)
There was a problem hiding this comment.
Good call! Updated the changeset.
Fixes #490
This PR implements support for the MCP specification's outputSchema field on tools, which allows tools to declare the expected structure of their output. This helps LLMs better understand and reason about GraphQL response data.
Testing
Test with an example server:
➜ apollo-mcp-server git:(AIR-69) ✗ cargo run -- graphql/TheSpaceDevs/config.yaml Compiling apollo-mcp-server v1.2.1 (/Users/dale.seo/work/apollo-mcp-server/crates/apollo-mcp-server) Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.57s Running `target/debug/apollo-mcp-server graphql/TheSpaceDevs/config.yaml` 2025-12-01T14:49:17.122003Z INFO Apollo MCP Server v1.2.1 // (c) Apollo Graph, Inc. // Licensed under MIT 2025-12-01T14:49:17.154306Z INFO load_tool: Tool SearchUpcomingLaunches loaded with a character count of 545. Estimated tokens: 136 2025-12-01T14:49:17.161384Z INFO load_tool: Tool ExploreCelestialBodies loaded with a character count of 759. Estimated tokens: 189 2025-12-01T14:49:17.168502Z INFO load_tool: Tool GetAstronautDetails loaded with a character count of 898. Estimated tokens: 224 2025-12-01T14:49:17.175388Z INFO load_tool: Tool GetAstronautsCurrentlyInSpace loaded with a character count of 501. Estimated tokens: 125 2025-12-01T14:49:17.193962Z INFO schema_index: Indexed 50 types in 18.15ms 2025-12-01T14:49:17.194264Z INFO Starting MCP server in Streamable HTTP mode port=8000 address=127.0.0.1The response for the
tools/listcall now includes theoutputSchemafield:The output schema also shows up before you try to call the tool:
After calling the tool, the actual output gets validated against the schema:
Also tested on Claude Desktop:
2025-12-01T15:07:21.636Z [info] [apollo] Message from client: {"method":"tools/list","params":{},"jsonrpc":"2.0","id":1} 2025-12-01T15:07:21.642Z [info] [apollo] Message from server: {"jsonrpc":"2.0","id":1,"result":{"tools":[{"name":"SearchUpcomingLaunches","description":"Fields searched - launch_designator, launch_service_provider__name, mission__name, name, pad__location__name, pad__name, rocket__configuration__manufacturer__abbrev, rocket__configuration__manufacturer__name, rocket__configuration__name, rocket__spacecraftflight__spacecraft__name. Codes are the best search terms to use. Single words are the next best alternative when you cannot use a code to search","inputSchema":{"type":"object","properties":{"query":{"type":"string"}},"required":["query"]},"outputSchema":{"type":"object","properties":{"data":{"type":"object","properties":{"upcomingLaunches":{"oneOf":[{"type":"object","properties":{"pageInfo":{"oneOf":[{"type":"object","properties":{"count":{"oneOf":[{"type":"integer"},{"type":"null"}]}}},{"type":"null"}]},"results":{"oneOf":[{"type":"array","items":{"oneOf":[{"type":"object","properties":{"id":{"oneOf":[{"type":"string"},{"type":"integer"}]},"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"weatherConcerns":{"oneOf":[{"type":"string"},{"type":"null"}]},"rocket":{"oneOf":[{"type":"object","properties":{"id":{"oneOf":[{"type":"string"},{"type":"integer"}]},"configuration":{"oneOf":[{"type":"object","properties":{"fullName":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]}},"required":["id"]},{"type":"null"}]},"mission":{"oneOf":[{"type":"object","properties":{"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"description":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]},"webcastLive":{"oneOf":[{"type":"boolean"},{"type":"null"}]},"provider":{"oneOf":[{"type":"object","properties":{"name":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]}},"required":["id"]},{"type":"null"}]}},{"type":"null"}]}}},{"type":"null"}]}}},"errors":{"type":"array","items":{"type":"object","properties":{"message":{"type":"string"},"locations":{"type":"array","items":{"type":"object","properties":{"line":{"type":"integer"},"column":{"type":"integer"}}}},"path":{"type":"array","items":{"oneOf":[{"type":"string"},{"type":"integer"}]}},"extensions":{"type":"object"}},"required":["message"]}},"extensions":{"type":"object"}}},"annotations":{"readOnlyHint":true}},{"name":"ExploreCelestialBodies","description":"The returned value is optional and has type `CelestialBodyConnection`\n---\ntype CelestialBody {\n id: ID!\n name: String\n type: CelestialType\n diameter: Float\n mass: Float\n gravity: Float\n lengthOfDay: String\n atmosphere: Boolean\n image: Image\n description: String\n wikiUrl: String\n}\n\ntype CelestialBodyConnection {\n pageInfo: PageInfo\n results: [CelestialBody]\n}\n\ntype CelestialType {\n id: ID!\n name: String\n}\n\ntype Image {\n url: String\n thumbnail: String\n credit: String\n}\n\ntype PageInfo {\n count: Int\n next: String\n previous: String\n}\n","inputSchema":{"type":"object","properties":{"search":{"type":"string"},"limit":{"type":"number"},"offset":{"type":"number"}}},"outputSchema":{"type":"object","properties":{"data":{"type":"object","properties":{"celestialBodies":{"oneOf":[{"type":"object","properties":{"pageInfo":{"oneOf":[{"type":"object","properties":{"count":{"oneOf":[{"type":"integer"},{"type":"null"}]},"next":{"oneOf":[{"type":"string"},{"type":"null"}]},"previous":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]},"results":{"oneOf":[{"type":"array","items":{"oneOf":[{"type":"object","properties":{"id":{"oneOf":[{"type":"string"},{"type":"integer"}]},"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"diameter":{"oneOf":[{"type":"number"},{"type":"null"}]},"mass":{"oneOf":[{"type":"number"},{"type":"null"}]},"gravity":{"oneOf":[{"type":"number"},{"type":"null"}]},"lengthOfDay":{"oneOf":[{"type":"string"},{"type":"null"}]},"atmosphere":{"oneOf":[{"type":"boolean"},{"type":"null"}]},"type":{"oneOf":[{"type":"object","properties":{"id":{"oneOf":[{"type":"string"},{"type":"integer"}]},"name":{"oneOf":[{"type":"string"},{"type":"null"}]}},"required":["id"]},{"type":"null"}]},"image":{"oneOf":[{"type":"object","properties":{"url":{"oneOf":[{"type":"string"},{"type":"null"}]},"thumbnail":{"oneOf":[{"type":"string"},{"type":"null"}]},"credit":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]},"description":{"oneOf":[{"type":"string"},{"type":"null"}]},"wikiUrl":{"oneOf":[{"type":"string"},{"type":"null"}]}},"required":["id"]},{"type":"null"}]}},{"type":"null"}]}}},{"type":"null"}]}}},"errors":{"type":"array","items":{"type":"object","properties":{"message":{"type":"string"},"locations":{"type":"array","items":{"type":"object","properties":{"line":{"type":"integer"},"column":{"type":"integer"}}}},"path":{"type":"array","items":{"oneOf":[{"type":"string"},{"type":"integer"}]}},"extensions":{"type":"object"}},"required":["message"]}},"extensions":{"type":"object"}}},"annotations":{"readOnlyHint":true}},{"name":"GetAstronautDetails","description":"The returned value is optional and has type `Astronaut`\n---\ntype Agency {\n id: ID!\n name: String\n abbrev: String\n country: [Country]\n}\n\ntype Astronaut {\n id: ID!\n name: String\n status: String\n agency: Agency\n image: Image\n inSpace: Boolean\n timeInSpace: String\n evaTime: String\n age: Int\n dateOfBirth: String\n dateOfDeath: String\n nationality: Country\n bio: String\n wiki: String\n lastFlight: String\n firstFlight: String\n socialMediaLinks: [SocialMediaLink]\n}\n\ntype Country {\n name: String\n alpha2Code: String\n nationalityName: String\n}\n\ntype Image {\n url: String\n thumbnail: String\n credit: String\n}\n\ntype SocialMedia {\n name: String\n url: String\n}\n\ntype SocialMediaLink {\n url: String\n socialMedia: SocialMedia\n}\n","inputSchema":{"type":"object","properties":{"astronautId":{"type":"string"}},"required":["astronautId"]},"outputSchema":{"type":"object","properties":{"data":{"type":"object","properties":{"astronaut":{"oneOf":[{"type":"object","properties":{"id":{"oneOf":[{"type":"string"},{"type":"integer"}]},"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"status":{"oneOf":[{"type":"string"},{"type":"null"}]},"inSpace":{"oneOf":[{"type":"boolean"},{"type":"null"}]},"age":{"oneOf":[{"type":"integer"},{"type":"null"}]},"dateOfBirth":{"oneOf":[{"type":"string"},{"type":"null"}]},"dateOfDeath":{"oneOf":[{"type":"string"},{"type":"null"}]},"firstFlight":{"oneOf":[{"type":"string"},{"type":"null"}]},"lastFlight":{"oneOf":[{"type":"string"},{"type":"null"}]},"timeInSpace":{"oneOf":[{"type":"string"},{"type":"null"}]},"evaTime":{"oneOf":[{"type":"string"},{"type":"null"}]},"agency":{"oneOf":[{"type":"object","properties":{"id":{"oneOf":[{"type":"string"},{"type":"integer"}]},"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"abbrev":{"oneOf":[{"type":"string"},{"type":"null"}]},"country":{"oneOf":[{"type":"array","items":{"oneOf":[{"type":"object","properties":{"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"nationalityName":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]}},{"type":"null"}]}},"required":["id"]},{"type":"null"}]},"nationality":{"oneOf":[{"type":"object","properties":{"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"nationalityName":{"oneOf":[{"type":"string"},{"type":"null"}]},"alpha2Code":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]},"image":{"oneOf":[{"type":"object","properties":{"url":{"oneOf":[{"type":"string"},{"type":"null"}]},"thumbnail":{"oneOf":[{"type":"string"},{"type":"null"}]},"credit":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]},"bio":{"oneOf":[{"type":"string"},{"type":"null"}]},"wiki":{"oneOf":[{"type":"string"},{"type":"null"}]},"socialMediaLinks":{"oneOf":[{"type":"array","items":{"oneOf":[{"type":"object","properties":{"url":{"oneOf":[{"type":"string"},{"type":"null"}]},"socialMedia":{"oneOf":[{"type":"object","properties":{"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"url":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]}}},{"type":"null"}]}},{"type":"null"}]}},"required":["id"]},{"type":"null"}]}}},"errors":{"type":"array","items":{"type":"object","properties":{"message":{"type":"string"},"locations":{"type":"array","items":{"type":"object","properties":{"line":{"type":"integer"},"column":{"type":"integer"}}}},"path":{"type":"array","items":{"oneOf":[{"type":"string"},{"type":"integer"}]}},"extensions":{"type":"object"}},"required":["message"]}},"extensions":{"type":"object"}}},"annotations":{"readOnlyHint":true}},{"name":"GetAstronautsCurrentlyInSpace","description":"The returned value is optional and has type `AstronautConnection`\n---\ntype Agency {\n name: String\n abbrev: String\n country: [Country]\n}\n\ntype Astronaut {\n id: ID!\n name: String\n agency: Agency\n image: Image\n timeInSpace: String\n nationality: Country\n lastFlight: String\n}\n\ntype AstronautConnection {\n results: [Astronaut]\n}\n\ntype Country {\n name: String\n nationalityName: String\n}\n\ntype Image {\n thumbnail: String\n}\n","inputSchema":{"type":"object","properties":{}},"outputSchema":{"type":"object","properties":{"data":{"type":"object","properties":{"astronauts":{"oneOf":[{"type":"object","properties":{"results":{"oneOf":[{"type":"array","items":{"oneOf":[{"type":"object","properties":{"id":{"oneOf":[{"type":"string"},{"type":"integer"}]},"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"timeInSpace":{"oneOf":[{"type":"string"},{"type":"null"}]},"lastFlight":{"oneOf":[{"type":"string"},{"type":"null"}]},"agency":{"oneOf":[{"type":"object","properties":{"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"abbrev":{"oneOf":[{"type":"string"},{"type":"null"}]},"country":{"oneOf":[{"type":"array","items":{"oneOf":[{"type":"object","properties":{"name":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]}},{"type":"null"}]}}},{"type":"null"}]},"nationality":{"oneOf":[{"type":"object","properties":{"name":{"oneOf":[{"type":"string"},{"type":"null"}]},"nationalityName":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]},"image":{"oneOf":[{"type":"object","properties":{"thumbnail":{"oneOf":[{"type":"string"},{"type":"null"}]}}},{"type":"null"}]}},"required":["id"]},{"type":"null"}]}},{"type":"null"}]}}},{"type":"null"}]}}},"errors":{"type":"array","items":{"type":"object","properties":{"message":{"type":"string"},"locations":{"type":"array","items":{"type":"object","properties":{"line":{"type":"integer"},"column":{"type":"integer"}}}},"path":{"type":"array","items":{"oneOf":[{"type":"string"},{"type":"integer"}]}},"extensions":{"type":"object"}},"required":["message"]}},"extensions":{"type":"object"}}},"annotations":{"readOnlyHint":true}},{"name":"execute","description":"Execute a GraphQL operation. Use the `introspect` tool to get information about the GraphQL schema. Always use the schema to create operations - do not try arbitrary operations. If available, first use the `validate` tool to validate operations. DO NOT try to execute introspection queries.","inputSchema":{"$schema":"http://json-schema.org/draft-07/schema#","title":"Input","description":"Input for the execute tool.","type":"object","properties":{"query":{"description":"The GraphQL operation","type":"string"},"variables":{"description":"The variable values represented as JSON","type":"string","default":null}},"required":["query"]}},{"name":"introspect","description":"Get information about a given GraphQL type defined in the schema. Instructions: Use this tool to explore the schema by providing specific type names. Start with the root query (Query) or mutation (Mutation) types to discover available fields. If the search tool is also available, use this tool first to get the fields, then use the search tool with relevant field return types and argument input types (ignore default GraphQL scalars) as search terms.","inputSchema":{"$schema":"http://json-schema.org/draft-07/schema#","title":"Input","description":"Input for the introspect tool.","type":"object","properties":{"type_name":{"description":"The name of the type to get information about.","type":"string"},"depth":{"description":"How far to recurse the type hierarchy. Use 0 for no limit. Defaults to 1.","type":"integer","format":"uint","minimum":0,"default":1}},"required":["type_name"]}},{"name":"search","description":"Search a GraphQL schema for types matching the provided search terms. Returns complete type definitions including all related types needed to construct GraphQL operations. Instructions: If the introspect tool is also available, you can discover type names by using the introspect tool starting from the root Query or Mutation types. Avoid reusing previously searched terms for more efficient exploration.","inputSchema":{"$schema":"http://json-schema.org/draft-07/schema#","title":"Input","description":"Input for the search tool.","type":"object","properties":{"terms":{"description":"The search terms","type":"array","items":{"type":"string"}}},"required":["terms"]}},{"name":"validate","description":"Validates a GraphQL operation against the schema. Use the `introspect` tool first to get information about the GraphQL schema. Operations should be validated prior to calling the `execute` tool.","inputSchema":{"$schema":"http://json-schema.org/draft-07/schema#","title":"Input","description":"Input for the validate tool","type":"object","properties":{"operation":{"description":"The GraphQL operation","type":"string"}},"required":["operation"]}}]}}