Add unified diff support to text editor

[tlongwell-block]

Adds support for applying unified diffs (both single-file and multi-file) through the text editor's str_replace command. This enhancement reduces token usage when making multi-line edits by up to 90%.

Why This Matters

Previously, multi-line edits required sending the entire old and new content as parameters, consuming substantial tokens. With diff support, only the changes are transmitted, dramatically reducing token usage for large edits.

What's New

Core Features

• Unified diff support: Apply standard git-style unified diffs to files
• Multi-file operations: Handle multiple file changes in a single operation
• Fuzzy matching: Intelligently applies patches even with minor context differences
• Token efficiency: Reduces token consumption by up to 90% for multi-line edits

Implementation

Text Editor Enhancement

• New diff parameter: Optional parameter added to TextEditorParams for the str_replace command
• Diff parsing: Uses mpatch crate (v0.2.0) for robust diff parsing and application
• Markdown support: Automatically handles diffs wrapped in markdown code blocks
• Atomic operations: Multi-file changes are all-or-nothing with automatic rollback on failure

Safety & Validation

• Path traversal protection: Validates all file paths to prevent directory traversal attacks
• Symlink detection: Blocks operations on symbolic links for security
• Size limits:
  • Maximum diff size: 1MB
  • Maximum files per diff: 100
• File history tracking: Maintains history for undo operations

Architecture

The feature is built with a modular, functional approach:

Key Components

• apply_diff(): Main entry point that coordinates the diff application process
• validate_path_safety(): Ensures all file operations are secure
• apply_single_patch(): Handles individual file patches with fuzzy matching
• DiffResults: Tracks statistics about the applied changes

Processing Flow

1. Validate diff size and content
2. Parse patches from the unified diff format
3. Apply each patch with fuzzy matching for resilience
4. Track all changes in file history
5. Generate a summary of modifications

Benefits

• Massive token savings: Up to 90% reduction for multi-line edits
• Git compatibility: Works seamlessly with git-generated diffs
• Robust error handling: Clear feedback when patches cannot be applied
• Undo support: All diff operations can be rolled back
• Atomic multi-file changes: Ensures consistency across multiple files

Usage Example

Instead of:

{
  "command": "str_replace",
  "old_str": "// 50 lines of old code...",
  "new_str": "// 50 lines of new code..."
}

Now you can use:

{
  "command": "str_replace",
  "diff": "--- a/file.rs\n+++ b/file.rs\n@@ -10,3 +10,3 @@\n-old line\n+new line"
}

Testing

Comprehensive test suite includes:

• Single and multi-file diff application
• File creation, modification, and deletion operations
• Context validation with fuzzy matching
• Undo functionality after diff operations
• Security validations (path traversal, symlinks)
• Edge cases (empty files, wrong line numbers, context mismatches)

All tests pass successfully.

[michaelneale]

seems nice - not sure if we have this AND replace_all (maybe....). curious how performance is

[michaelneale]

I think this is ok, I wasn't able to trigger it to use a diff, it would happily use str_replace when it made sense, but that is probably ok. I did try to tell it to make a diff, it did, but it didn't work (probably as it was contrived). Any tips on how to see this out? just has to be day to day?

Seems a nicely focussed change and improvement. Main concern is how it works with lower end models, but the prompt doesn't really change much so should be ok.

[tlongwell-block]

@michaelneale yeah, it is reluctant to use it. The old/new str is more intuitive for it, I think.

But, in my testing, sonnet handled the diff format for multiple files (going up to 10) really well. I'm not sure if I should use stronger language here to encourage using diff editing more, because it is pretty aggressive already.

[michaelneale]

@tlongwell-block I would leave it as is, and consider (while cracked open) adding in replace_all in this one omnibus PR and then we can get it in? covers all bases then.

[understood-the-assignment]

Claude 3.7 Sonnet seems to be having a lot of trouble with this new tool for me.

It tries to:

path: ~/a/b/c/d/e/f/g/s/i/transmogrification/transmogrify.py
command: str_replace
diff: --- a/src/initech/transmogrification/transmogrify.py
+++ b/src/initech/transmogrification/transmogrify.py
@@ -127,7 +127,7 @@ ....
 ...patch follows ...

-32603: Mcp error: -32603: File '/home/user/a/b/c/d/e/f/g/src/initech/transmogrification/src/initech/transmogrification/transmogrify.py' not found and patch doesn't create it

i.e. it puts the full path in the path parameter and then a relative path in the diff header.

IMHO Goose should stop trying to invent clever new variations of tools that are completely different from what the common models out there are trained on. We should try to mimic the prompts and tools that Claude Code, Gemini CLI, and OpenAI Codex are providing to their respective models.