Add analyze tool to Developer extension

[tlongwell-block]

Adds an analyze tool to the developer MCP extension that provides code structure analysis and symbol tracking capabilities across multiple programming languages.

Changes

• Add tree-sitter based code analysis for Python, Rust, JavaScript/TypeScript, Go, Swift, and Java
• Support three analysis modes:
  • Structure: Quick overview of files, functions, and classes
  • Semantic: Detailed analysis including imports and call relationships
  • Focused: Track specific symbols and their call chains across files
• Include LRU caching to improve performance for repeated analyses
• Respect .gooseignore and .gitignore patterns for file traversal

Usage

# Analyze a file or directory structure
goose> analyze path/to/code

# Track a specific function and its relationships
goose> analyze path/to/project --focus main --follow-depth 2

# Limit directory traversal depth
goose> analyze large/project --max-depth 2

Implementation Notes

• Modular architecture with separate modules for parsing, caching, traversal, and formatting
• Comprehensive test coverage (44 tests)

Testing

Tested with various codebases including Python packages, Rust crates, and mixed-language projects. The tool correctly identifies code structure and tracks cross-file dependencies.

[michaelneale]

I think this approach has promise, have closed #3389 in favour of this one in the developer MCP which is the right place. Will want to make sure can deal with a tree compute that takes a few minutes without it timing out (so the tool can reply and run in background possibly).

[michaelneale]

is this best here or closer to where rg is mentioned elsewhere in the prompt (or was that system prompt?)

[tlongwell-block]

It is already in the developer extension prompt in the editor description for some reason. Not sure why there in particular, but it seems like general developer instructions might fit better here in the general developer mcp description

[michaelneale]

oh I mean in the common shell instructions. it says to use ripgrep - but it doesn't mention -C N at all, which seems to be the thing that hurts it (lacking that), but probably doesn't hurt here.

[michaelneale]

this seems to work nicely when I try it, a very nice addition. Any measuable improvement on benches @tlongwell-block ?

[tlongwell-block]

this seems to work nicely when I try it, a very nice addition. Any measuable improvement on benches @tlongwell-block ?

Yes, a fairly consistent 3-6% boost with this change. It gets Sonnet up to our current Opus average.

[michaelneale]

I think this is ok, makes goose smarter with code specifically, helps performance (I also like that it brings sonnet up with opus). Doesn't seem a huge overhead either.

but would love a tick from @jamadeo or @DOsinga to be sure before we ship it - as brings in a few libs, tree sitter etc.

[tlongwell-block]

The tool itself is feature complete. The instructions/prompt need a little tweaking around helping the llm understand the max depth parameter and how it effects analysis

[michaelneale]

@tlongwell-block I would think a response first/last line (probably both) saying "you will only see X, there is still Y" if possible in the response?

[tlongwell-block]

@tlongwell-block I would think a response first/last line (probably both) saying "you will only see X, there is still Y" if possible in the response?

The tool intentionally does lazy parsing of the codebase (because monorepos are a thing), so we can't really tell it how deep the call graph actually is without the agent electing to parse the entire repo to max_depth=0 (which it totally can do)

It can also surgically search for the calls in subdirectories or individual files to avoid having to parse everything. This works well, too