Strengthen guidance for git commands in the terminal tool - #55787
Merged
Conversation
Move guidance about pty-blocking commands from the struct-level doc comment to the `command` field's doc comment so it's surfaced more prominently to the model, and call out `--no-pager` and `GIT_EDITOR=true` explicitly for git operations.
cla-bot
Bot
added
the
cla-signed
zed-community-bot
Bot
added
the
staff
Two new evals under `crates/agent/src/tools/evals/terminal_tool.rs`, gated behind the `unit-eval` feature flag (same as the existing `edit_file` evals): * `eval_git_log_uses_no_pager`: prompts the model to show recent commits and asserts the resulting terminal command contains `git`, `log`, and `--no-pager`. * `eval_git_rebase_sets_git_editor`: prompts the model to rebase the current branch onto `origin/main` and asserts the resulting terminal command contains `git`, `rebase`, and `GIT_EDITOR=true`. Each eval runs 100 iterations and requires a 95% pass rate, matching the pattern in `edit_file.rs`. The eval harness streams the model completion, pulls out the first complete `terminal` tool use, parses it as `TerminalToolInput`, and asserts substrings against the `command` field.
Move the pty-blocking guidance back to the TerminalToolInput struct-level doc comment (so it lands in the top-level tool description rather than buried in a per-field schema entry) and rewrite it with concrete MUST/NOT examples for both --no-pager and GIT_EDITOR=true. Also add a shorter REMINDER on the command field itself. In a 10-iteration unit-eval, the previous wording produced 0/10 correct invocations on both the git-log and git-rebase prompts; this wording produces 10/10 on both. Also add eval_git_rebase_implied_sets_git_editor as a stress test where 'rebase' is implied (combine 3 commits) but not literally requested.
Replace the per-eval substring check (`command_contains_all`) with a single `git_pty_safe` assertion that's intentionally permissive about which git subcommand the model picks. It only fails if a known pager-piping subcommand (`log`, `diff`, `show`, `blame`) is missing `--no-pager` (or `PAGER=cat` / `GIT_PAGER=cat`), or a known editor-invoking subcommand (`rebase`, `commit`, `merge`, `tag`) is missing `GIT_EDITOR=true` (or equivalent). Motivation: on the indirect-rebase prompt the model correctly investigates the commit history with `git log` first before deciding how to combine the commits. The previous assertion (which required `git`, `rebase`, and `GIT_EDITOR=true` substrings) treated this as a failure even though the model produced a perfectly pty-safe `git --no-pager log` invocation. The new assertion correctly accepts either path.
rtfeldman
marked this pull request as ready for review
May 5, 2026 18:03
Satisfy clippy::manual_contains.
ebaah46
pushed a commit
to ebaah46/zed
that referenced
this pull request
May 6, 2026
…ries#55787) Move guidance about pty-blocking commands from the `TerminalToolInput` struct-level doc comment to the `command` field's doc comment so it's surfaced more prominently to the model in the tool's JSON schema, and call out `--no-pager` and `GIT_EDITOR=true` explicitly for git operations. In practice, the previous wording about `git --no-pager diff` was easy for agents to overlook, leading to terminal calls that block waiting on `less` (for `git log`/`git diff`/`git show`) or on an interactive editor (for `git rebase`/`git commit`/`git merge`). Unit eval change before/after wording change: <img width="1662" height="740" alt="image" src="https://github.com/user-attachments/assets/5f7921f3-87fa-4f88-9839-193ca4522572" /> Closes AI-154 Release Notes: - Zed Agent's terminal tool now much more consistently uses `--no-pager` and `GIT_EDITOR` with `git` commands
This was referenced May 13, 2026
jonx
pushed a commit
to jonx/zed-aros
that referenced
this pull request
Jul 17, 2026
…ries#55787) Move guidance about pty-blocking commands from the `TerminalToolInput` struct-level doc comment to the `command` field's doc comment so it's surfaced more prominently to the model in the tool's JSON schema, and call out `--no-pager` and `GIT_EDITOR=true` explicitly for git operations. In practice, the previous wording about `git --no-pager diff` was easy for agents to overlook, leading to terminal calls that block waiting on `less` (for `git log`/`git diff`/`git show`) or on an interactive editor (for `git rebase`/`git commit`/`git merge`). Unit eval change before/after wording change: <img width="1662" height="740" alt="image" src="https://github.com/user-attachments/assets/5f7921f3-87fa-4f88-9839-193ca4522572" /> Closes AI-154 Release Notes: - Zed Agent's terminal tool now much more consistently uses `--no-pager` and `GIT_EDITOR` with `git` commands
jolutz
pushed a commit
to jolutz/zed
that referenced
this pull request
Aug 8, 2026
…ries#55787) Move guidance about pty-blocking commands from the `TerminalToolInput` struct-level doc comment to the `command` field's doc comment so it's surfaced more prominently to the model in the tool's JSON schema, and call out `--no-pager` and `GIT_EDITOR=true` explicitly for git operations. In practice, the previous wording about `git --no-pager diff` was easy for agents to overlook, leading to terminal calls that block waiting on `less` (for `git log`/`git diff`/`git show`) or on an interactive editor (for `git rebase`/`git commit`/`git merge`). Unit eval change before/after wording change: <img width="1662" height="740" alt="image" src="https://github.com/user-attachments/assets/5f7921f3-87fa-4f88-9839-193ca4522572" /> Closes AI-154 Release Notes: - Zed Agent's terminal tool now much more consistently uses `--no-pager` and `GIT_EDITOR` with `git` commands
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Move guidance about pty-blocking commands from the
TerminalToolInputstruct-level doc comment to thecommandfield's doc comment so it's surfaced more prominently to the model in the tool's JSON schema, and call out--no-pagerandGIT_EDITOR=trueexplicitly for git operations.In practice, the previous wording about
git --no-pager diffwas easy for agents to overlook, leading to terminal calls that block waiting onless(forgit log/git diff/git show) or on an interactive editor (forgit rebase/git commit/git merge).Unit eval change before/after wording change:
Closes AI-154
Release Notes:
--no-pagerandGIT_EDITORwithgitcommands