-
Notifications
You must be signed in to change notification settings - Fork 2.4k
fix(code-mode): improve tool signatures for LLM discovery #6177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR aims to improve LLM tool discovery in code mode by fixing schema/handler mismatches, adding proper JSON Schema $ref dereferencing for enums, and clarifying tool descriptions. The changes target the read_module parameter naming, enum type detection via oneOf + const patterns, and removing JavaScript-like syntax from documentation.
Key changes:
- Added
unbinderdependency (v0.1.7) for JSON Schema $ref dereferencing - Implemented enum handling via
oneOf+constpattern detection in tool parameter types - Updated tool descriptions to remove JavaScript-like syntax and improve clarity for LLMs
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| crates/goose/src/agents/code_execution_extension.rs | Renamed schema parameter module_path to path, added $ref dereferencing logic, improved enum type detection, updated tool descriptions, and added test coverage |
| crates/goose/Cargo.toml | Added unbinder crate dependency for schema dereferencing |
| crates/goose-mcp/src/computercontroller/mod.rs | Simplified and clarified web_scrape parameter descriptions |
| Cargo.lock | Added unbinder dependency entry with version 0.1.7 |
|
race condition.. I'll fix this back to module_path everywhere |
michaelneale
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
conditional approve - once happy with comments (and copilot thing about path vs module_path to clear up - but seems to work fine spot testing it). nice one.
if we are keen on using the schema dep, I can add more cases to the handling besides oneOf so no bombs like other things coming up as 'any' making the llm guess what the params might be. |
|
@codefromthecrypt sure - also that Live test failure - is that legit that it is tripping up qwen? (or can't be, as this is only code mode now?) |
fd7c043 to
64cea6e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.
|
will take out of draft when done |
- Parse enum types from JSON Schema (oneOf+const, enum arrays) - Dereference $ref using unbinder for schemars-generated schemas - Extract return types from output_schema (default: string) - Remove JS-like syntax from descriptions that confused LLMs Fixes web_scrape showing save_as as "any" instead of "text" | "json" | "binary" Signed-off-by: Adrian Cole <[email protected]>
Signed-off-by: Adrian Cole <[email protected]>
b19e835 to
8812286
Compare
|
@michaelneale I think we're good. I looked at the live tests and think that qwen fail earlier was just a flake. I've pushed several times to re-trigger and it hasn't failed again. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 4 out of 5 changed files in this pull request and generated no new comments.
|
@michaelneale gonna mergeroo this one. The live provider tests are flakey this hasn't anything to do with it. In the last week ~1 in 5 times across any branch flaked. |
Summary
Allows the following prompt to work occasionally with code mode in dumber LLM like gpt-5-nano
Fixes web_scrape showing save_as as "any" instead of "text" | "json" | "binary"
Type of Change
AI Assistance
Testing
Unified test_mcp_tool_signature with real MCP server data
Added a test and used the mentioned prompt successfully
$ cargo run -p goose-cli --release -- session --with-builtin code_execution,developer,computercontroller Compiling goose v1.17.0 (/Users/codefromthecrypt/oss/goose/crates/goose) Compiling goose-bench v1.17.0 (/Users/codefromthecrypt/oss/goose/crates/goose-bench) Compiling goose-cli v1.17.0 (/Users/codefromthecrypt/oss/goose/crates/goose-cli) Finished `release` profile [optimized] target(s) in 47.95s Running `target/release/goose session --with-builtin code_execution,developer,computercontroller` starting session | provider: tetrate model: gpt-5-nano session id: 20251219_3 working directory: /Users/codefromthecrypt/oss/goose goose is running! Enter your instructions, or try asking what goose can do. Context: ○○○○○○○○○○ 0% (0/272000 tokens) ( O)> Search for web_scrape and text_editor tools. Use them to save https://httpbingo.org/get to /tmp/result.txt. ─── search_modules | code_execution ────────────────────────── terms: ["web_scrape", "text_editor"] ─── read_module | code_execution ────────────────────────── module_path: computercontroller ─── read_module | code_execution ────────────────────────── module_path: developer ─── read_module | code_execution ────────────────────────── module_path: extensionmanager ─── read_module | code_execution ────────────────────────── module_path: github ─── execute_code | code_execution ────────────────────────── code: import { web_scrape } from "computercontroller"; import { text_editor } from "developer"; const content = web_scrape({ save_as: "text", url: "https://httpbingo.org/get" }); text_editor({ path: "/tmp/result.tx... ## Result - Action completed: Saved the contents of https://httpbingo.org/get to /tmp/result.txt in a single batched operation. - Internal note: The content was also cached locally at the tool’s temp location: - /Users/codefromthecrypt/.cache/goose/computer_controller/web_20251219_104714.txt ## Verification commands (choose one) - View the full file: - cat /tmp/result.txt - Preview the first 200 lines: - head -n 200 /tmp/result.txt - If you expect JSON and want a quick check: - head -n 50 /tmp/result.txt ## What’s in the file - The page at httpbingo.org/get typically returns a simple JSON response with details about the request (args, headers, URL). If you’d like, I can parse and display specific fields (e.g., URL, headers) or convert it to another format (JSON, pretty-printed, etc.). ## Next steps - Save to a different path or filename? - Fetch another URL and save similarly? - Parse and extract specific fields from the saved content? If yes, tell me which fields you want. ⏱️ Elapsed time: 54.07s Context: ○○○○○○○○○○ 2% (6168/272000 tokens)Related Issues
Fixes #6147