Skip to content

refactor(prompt): simplify Buzz agent guidance - #6340

Merged
salman1993 merged 1 commit into
mainfrom
smohammed/buzz-native-prompt-candidate
Aug 20, 2026
Merged

refactor(prompt): simplify Buzz agent guidance#6340
salman1993 merged 1 commit into
mainfrom
smohammed/buzz-native-prompt-candidate

Conversation

@salman1993

@salman1993 salman1993 commented Aug 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Why

The Buzz agent prompt carries verbose agent-draft instructions that add prompt weight without improving benchmark correctness. Buzz-native channel trials also showed agents creating permanent channels while claiming they had applied a one-hour TTL. The static CLI inventory remains for efficient command discovery by smaller agents.

What

  • Retain the static Buzz CLI command table and direct agents to use --help for full usage
  • Compact the owner-reviewed agent-draft guidance without changing its workflow
  • Clarify that buzz channels create is permanent unless --ttl is provided

Benchmark Results

  • Terminal-Bench retained 22/22 correctness in the initial no-table candidate while reducing prompt size, input tokens, tool calls, and cost; total active time was effectively flat.
  • Across two Buzz-native runs, the non-channel cases tied at 26/30; the CLI clarification directly addresses the observed TTL omission.

Update — 2026-08-20

  • Restored the CLI command table following review feedback about multistep discovery with smaller agents. The restored-table candidate has not been re-benchmarked.

@salman1993
salman1993 marked this pull request as ready for review August 19, 2026 18:53
@salman1993
salman1993 requested a review from a team as a code owner August 19, 2026 18:53
@salman1993
salman1993 changed the base branch from smohammed/buzz-native-reply-thread to main August 19, 2026 19:02
@salman1993
salman1993 force-pushed the smohammed/buzz-native-prompt-candidate branch from d180a9b to ec3dece Compare August 19, 2026 19:02
philazar
philazar previously approved these changes Aug 20, 2026
View reviewed changes

@philazar philazar left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

atishpatel
atishpatel reviewed Aug 20, 2026
View reviewed changes
Comment thread crates/buzz-acp/src/base_prompt.md

The `buzz` CLI is your primary interface. Auth env vars: `BUZZ_RELAY_URL`, `BUZZ_PRIVATE_KEY`, `BUZZ_AUTH_TAG`. Exit codes: 0 ok, 1 user error, 2 network, 3 auth, 4 other. Output is structured JSON.

| Group | Key commands |

@atishpatel atishpatel Aug 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't removing this mean the agent will always have to make multiple tool calls to see available buzz cli commands?

Discovery with smaller agents will become a real issue if it's multistep

@atishpatel atishpatel Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe need to pair this with buzz --help output improvement?
In the help output you can include subcommands so it's not multiple calls

@salman1993 salman1993 Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think for most turns, the agent doesn't need to call buzz CLI commands outside of buzz messages which is already mentioned in the prompt in the section below. Its better to progressively disclose. In the buzz-native benchmark, i didn't see much of a change when this table was removed.

Run buzz --help or buzz <group> --help for full usage. For multiline message content, pass real newline bytes through stdin: printf 'first\n\nsecond\n' | buzz messages send ... --content -. Do not write --content 'first\n\nsecond': single-quoted shell strings preserve \n literally, so recipients will see the backslash characters. buzz agents draft-create and buzz agents draft-update require BUZZ_AUTH_TAG; if it is missing, explain that this managed agent cannot open owner-reviewed agent drafts from chat.

atishpatel
atishpatel requested changes Aug 20, 2026
View reviewed changes

@atishpatel atishpatel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improve the --help output so subcommands are visible

@salman1993
refactor(prompt): simplify Buzz agent guidance
752efe6 
Compact the owner-reviewed agent-draft guidance while retaining the static CLI inventory for efficient discovery.

Clarify in channels create help that omitting --ttl creates a permanent channel.

Signed-off-by: Salman Mohammed <smohammed@squareup.com>
@salman1993
salman1993 force-pushed the smohammed/buzz-native-prompt-candidate branch from ec3dece to 752efe6 Compare August 20, 2026 14:36
@salman1993
salman1993 requested a review from atishpatel August 20, 2026 14:36
@salman1993
salman1993 enabled auto-merge (squash) August 20, 2026 14:39
@salman1993
salman1993 merged commit 2a236e4 into main Aug 20, 2026
31 checks passed
@salman1993
salman1993 deleted the smohammed/buzz-native-prompt-candidate branch August 20, 2026 14:59
wpfleger96 added a commit that referenced this pull request Aug 20, 2026
@wpfleger96
Merge remote-tracking branch 'origin/main' into duncan/guard-public-r…
419cb99 
…elay-skills

* origin/main:
  feat(desktop): close Buzz window with Cmd+W (#6314)
  refactor(prompt): simplify Buzz agent guidance (#6340)

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
tellaho added a commit that referenced this pull request Aug 20, 2026
@tellaho
Merge remote-tracking branch 'origin/main' into tho/composer-auto-add…
60e3abe 
…ress

* origin/main:
  fix(desktop): distinguish duplicate agent devices (#6337)
  feat(desktop): close Buzz window with Cmd+W (#6314)
  refactor(prompt): simplify Buzz agent guidance (#6340)
  feat(desktop): make Projects workspaces selectable (#6368)
  Add Buzz-native collaboration benchmarks (#6264)
  Polish mobile timeline and emoji interactions (#6297)
  feat(desktop): make the Projects overview follow the selected section (#6335)
  refactor(desktop): coordinate TTS playback (#6341)
  fix(desktop): show complete repository trees (#5102)
  Add appearance preference previews (#6193)
  fix(desktop): restore emoji recents (#6263)
  chore: serialize mobile pre-push checks (#6322)
  fix(buzz-acp): loosen workspace-scan guardrail to allow named paths (#6261)
  fix(buzz-dev-mcp): expand leading ~ in read_file/str_replace paths (#6271)
  perf(desktop): move five hot renderer paths from JS into Rust (#6024)
  fix(media): accept portrait video resolutions (#6058)
  fix(desktop): hide archived channels from #/Tab autocomplete (#6156)
  Unify mobile channel details (#6113)
  Revert "fix(acp): gate relay-signed workflow messages on their attributed author" (#6311)

Signed-off-by: Taylor Ho <taylorkmho@gmail.com>

# Conflicts:
#	desktop/src/features/messages/ui/MentionAutocomplete.test.mjs
morgmart added a commit that referenced this pull request Aug 20, 2026
@morgmart
Merge remote-tracking branch 'origin/main' into investigate-buzz-cli-…
ba41398 
…handoff

* origin/main:
  feat(desktop): refine context-aware Projects collaboration (#6396)
  fix(desktop): distinguish duplicate agent devices (#6337)
  feat(desktop): close Buzz window with Cmd+W (#6314)
  refactor(prompt): simplify Buzz agent guidance (#6340)
  feat(desktop): make Projects workspaces selectable (#6368)
  Add Buzz-native collaboration benchmarks (#6264)
  Polish mobile timeline and emoji interactions (#6297)
  feat(desktop): make the Projects overview follow the selected section (#6335)
  refactor(desktop): coordinate TTS playback (#6341)
  fix(desktop): show complete repository trees (#5102)
  Add appearance preference previews (#6193)
  fix(desktop): restore emoji recents (#6263)
  chore: serialize mobile pre-push checks (#6322)

Signed-off-by: morgmart <98432065+morgmart@users.noreply.github.com>
wpfleger96 pushed a commit that referenced this pull request Aug 20, 2026
Hayt
Merge remote-tracking branch 'origin/main' into hayt/split-open-chann…
c883fcb 
…el-directory

* origin/main:
  feat(desktop): refine context-aware Projects collaboration (#6396)
  fix(desktop): distinguish duplicate agent devices (#6337)
  feat(desktop): close Buzz window with Cmd+W (#6314)
  refactor(prompt): simplify Buzz agent guidance (#6340)
  feat(desktop): make Projects workspaces selectable (#6368)
  Add Buzz-native collaboration benchmarks (#6264)
  Polish mobile timeline and emoji interactions (#6297)
  feat(desktop): make the Projects overview follow the selected section (#6335)
  refactor(desktop): coordinate TTS playback (#6341)

Signed-off-by: Hayt <9e1c23a3fd83f61da34420e4e88ff1b16e45cafcc0cd9019eb07d4ecfa8ca9b0@buzz.block.builderlab.xyz>
wpfleger96 added a commit that referenced this pull request Aug 20, 2026
Hayt @wpfleger96
Merge remote-tracking branch 'origin/main' into hayt/split-open-chann…
bdc1873 
…el-directory

* origin/main:
  feat(desktop): refine context-aware Projects collaboration (#6396)
  fix(desktop): distinguish duplicate agent devices (#6337)
  feat(desktop): close Buzz window with Cmd+W (#6314)
  refactor(prompt): simplify Buzz agent guidance (#6340)
  feat(desktop): make Projects workspaces selectable (#6368)
  Add Buzz-native collaboration benchmarks (#6264)
  Polish mobile timeline and emoji interactions (#6297)
  feat(desktop): make the Projects overview follow the selected section (#6335)
  refactor(desktop): coordinate TTS playback (#6341)

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Hayt <9e1c23a3fd83f61da34420e4e88ff1b16e45cafcc0cd9019eb07d4ecfa8ca9b0@buzz.block.builderlab.xyz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@atishpatel atishpatel atishpatel approved these changes
@philazar philazar philazar left review comments
@wesbillman wesbillman Awaiting requested review from wesbillman
@wpfleger96 wpfleger96 Awaiting requested review from wpfleger96

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@salman1993 @atishpatel @philazar