perf(toolshim): compact tool schema JSON - #10409
Conversation
ToolShim includes schemas in text prompts, so pretty-print whitespace scales with every exposed tool. Compact serialization preserves schema semantics while reducing prompt bytes. Related to aaif-goose#1303.
|
This serializes ToolShim input schemas as compact JSON instead of pretty-printed JSON, so schema indentation and line breaks stop consuming prompt space on the path used for models without native tool calling; tool names, descriptions, order, separators, and the surrounding instruction text are unchanged. On the fixture the prompt goes from 50,445 to 34,605 bytes at 128 tools (about 31% less), deterministic and hardware-independent — the change itself is one file, +20/-1, with a regression test that the emitted schema is single-line and parses back to the original value. It deliberately does not claim a measured change in tool-call success rate. Checks are green and there are no conflicts, though the branch is behind |
Brings the branch up to date with main (106 commits). No conflicts; crates/goose/src/providers/toolshim.rs is untouched upstream, so the compact-schema change and its regression test apply unchanged.
Summary
ToolShim places tool definitions in a text prompt for models without native tool calling. The previous formatting repeated indentation and line breaks for every property of every tool schema. Those bytes reach the model but do not change the JSON schema.
The original ToolShim PR recommends a larger context window for this prompt path (#1448), and #1303 is prior evidence of tool definitions contributing substantial context pressure. This change is deliberately narrower: it only removes schema whitespace and leaves tool names, descriptions, order, labels, separators, and the surrounding instructions unchanged.
Measured impact
The strict fixture covers Unicode descriptions, nested and empty objects, required fields, arrays, and JSON-valid but schema-invalid shapes.
toolshim_prompt_bytes_128is deterministic and hardware-independent. The evaluator also checks byte determinism, exact tool names and descriptions, tool order, semantic schema equality after parsing, empty tools, and the unchanged ToolShim instruction text.This PR claims prompt-byte reduction and schema/prompt contract preservation. It does not claim a measured change in model tool-call success rate; JSON whitespace can still affect model behavior even when the parsed schema is identical.
This optimization was explored and validated through a 10-step autoresearch run with Weco: public run. The submitted patch keeps only the measured input-schema change from the best valid step, plus the focused regression test.
Testing
The external strict evaluator also passed repeatedly with
toolshim_prompt_bytes_128=34605and all correctness gates enabled.Related Issues
Context-pressure background: #1303
ToolShim introduction: #1448
Screenshots/Demos (for UX changes)
N/A