Skip to content

perf(toolshim): compact tool schema JSON - #10409

Merged
lifeizhou-ap merged 2 commits into
aaif-goose:mainfrom
dexhunter:perf/toolshim-compact-schema
Jul 31, 2026
Merged

perf(toolshim): compact tool schema JSON#10409
lifeizhou-ap merged 2 commits into
aaif-goose:mainfrom
dexhunter:perf/toolshim-compact-schema

Conversation

@dexhunter

Copy link
Copy Markdown
Contributor

Summary

  • Serialize ToolShim input schemas as compact JSON instead of pretty-printed JSON.
  • Add a regression test that checks the emitted schema is single-line JSON and parses back to the original schema value.

ToolShim places tool definitions in a text prompt for models without native tool calling. The previous formatting repeated indentation and line breaks for every property of every tool schema. Those bytes reach the model but do not change the JSON schema.

The original ToolShim PR recommends a larger context window for this prompt path (#1448), and #1303 is prior evidence of tool definitions contributing substantial context pressure. This change is deliberately narrower: it only removes schema whitespace and leaves tool names, descriptions, order, labels, separators, and the surrounding instructions unchanged.

Measured impact

The strict fixture covers Unicode descriptions, nested and empty objects, required fields, arrays, and JSON-valid but schema-invalid shapes.

Tools Before After Reduction
1 814 bytes 740 bytes 9.1%
8 3,593 bytes 2,603 bytes 27.6%
32 12,949 bytes 8,989 bytes 30.6%
128 50,445 bytes 34,605 bytes 31.4%

toolshim_prompt_bytes_128 is deterministic and hardware-independent. The evaluator also checks byte determinism, exact tool names and descriptions, tool order, semantic schema equality after parsing, empty tools, and the unchanged ToolShim instruction text.

This PR claims prompt-byte reduction and schema/prompt contract preservation. It does not claim a measured change in model tool-call success rate; JSON whitespace can still affect model behavior even when the parsed schema is identical.

This optimization was explored and validated through a 10-step autoresearch run with Weco: public run. The submitted patch keeps only the measured input-schema change from the best valid step, plus the focused regression test.

Testing

cargo test -p goose toolshim
cargo fmt --all -- --check
cargo clippy -p goose --all-targets -- -D warnings

The external strict evaluator also passed repeatedly with toolshim_prompt_bytes_128=34605 and all correctness gates enabled.

Related Issues

Context-pressure background: #1303
ToolShim introduction: #1448

Screenshots/Demos (for UX changes)

N/A

ToolShim includes schemas in text prompts, so pretty-print whitespace scales with every exposed tool. Compact serialization preserves schema semantics while reducing prompt bytes.

Related to aaif-goose#1303.
@dexhunter

Copy link
Copy Markdown
Contributor Author

This serializes ToolShim input schemas as compact JSON instead of pretty-printed JSON, so schema indentation and line breaks stop consuming prompt space on the path used for models without native tool calling; tool names, descriptions, order, separators, and the surrounding instruction text are unchanged. On the fixture the prompt goes from 50,445 to 34,605 bytes at 128 tools (about 31% less), deterministic and hardware-independent — the change itself is one file, +20/-1, with a regression test that the emitted schema is single-line and parses back to the original value. It deliberately does not claim a measured change in tool-call success rate. Checks are green and there are no conflicts, though the branch is behind main by 106 commits now, so I'm happy to rebase it. Glad to narrow the scope further if that would help.

Brings the branch up to date with main (106 commits). No conflicts;
crates/goose/src/providers/toolshim.rs is untouched upstream, so the
compact-schema change and its regression test apply unchanged.
@lifeizhou-ap
lifeizhou-ap added this pull request to the merge queue Jul 31, 2026
Merged via the queue into aaif-goose:main with commit 950575b Jul 31, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants