Add filtering for agentVisible: false messages on streaming providers #4847

katzdave · 2025-09-26T16:33:58Z

This is the root cause of the input too long issues we're seeing.

I also saw some quirks with auto-compact triggering right after compact; this is from the provider metadata but I updated the token estimation approach to also filter.

perform_compaction doesn't need filtering since it falls thru to complete_fast().

…g-impl * 'main' of github.com:block/goose: Docs: Add link to Plug & Play video for Reddit MCP (#4852) remove only-pr-labels (#4842) Update video link in README.md to lowercase goose (#4846)

DOsinga

looking at this again, I think you want to do this on a lower level.

we have a number of format_messages(..) - this feels like the right place to put this, where we convert the messages to what the client should see, not on a per provider basis. we currently already filter on two places, one time I'm pretty sure after we already have, so this is just going to be a whack a mole game I think

what do you think?

DOsinga · 2025-09-26T21:11:21Z

crates/goose/src/conversation/mod.rs

+    /// // Get messages visible ONLY to agent (not visible to user)
+    /// let agent_only = conversation.filtered_messages(|meta| meta.agent_visible && !meta.user_visible);
+    /// ```
+    pub fn filtered_messages<F>(&self, filter: F) -> Vec<Message>


ah, this is not really what I meant - meant just pass it a visibility level for the messages and then return those

visibility level kind of needs some kind of mask though on MessageMetadata? Don't quite follow.

i meant something like

pub fn visible_messages(for: User|Agent) but maybe that's not the API you imagined.

Can do that but I was imagining adding more tags to message metadata like summary marker, notification so wanted to keep them synced.

ok, some other time, let's just drop this for now. or keep it but remove the comment

ack. leaving for now but will follow up and use it + might update it a bit.

katzdave · 2025-09-26T22:31:04Z

pushed some changes refactoring the provider side impl. I also noticed it did some filtering of thinking messages in anthropic provider.

Unsure about those, seems to have been modified to match what google is doing. Could these be related to goose repeating itself?

DOsinga · 2025-09-27T16:25:40Z

crates/goose/src/providers/formats/anthropic.rs

-    // Convert messages to Anthropic format
-    for message in messages {
+    // Filter messages to only include agent_visible ones and convert to Anthropic format
+    for message in messages.iter().filter(|m| m.is_agent_visible()) {


to be honest, if we go through the loop here anyway, we might as well skip them in place rather than prefilter

Goose claims:

The current implementation with .iter().filter() is already optimal - it combines filtering and processing in a single, lazy iteration without any unnecessary allocations. There's no performance benefit to pre-filtering in this case.

DOsinga · 2025-09-27T16:26:24Z

crates/goose/src/context_mgmt/auto_compact.rs

+                .iter()
+                .filter(|m| m.is_agent_visible())
+                .cloned()
+                .collect();


shouldn't we move this inside get_messages_token_counts_async?

done, nice call.

DOsinga · 2025-09-27T16:26:45Z

crates/goose/src/conversation/mod.rs

+    /// // Get messages visible ONLY to agent (not visible to user)
+    /// let agent_only = conversation.filtered_messages(|meta| meta.agent_visible && !meta.user_visible);
+    /// ```
+    pub fn filtered_messages<F>(&self, filter: F) -> Vec<Message>


ok, some other time, let's just drop this for now. or keep it but remove the comment

DOsinga · 2025-09-27T16:29:32Z

crates/goose/src/providers/formats/anthropic.rs

-                    }));
+                MessageContent::RedactedThinking(_) => {
+                    // Skip redacted thinking - internal use only
+                    continue;


I'm not sure about this. it sounds reasonable, but do we want to make a change we don't really really understand for this if we want to cherry-pick?

Agreed, reverted but wanted to just call this out as somewhere to investigate.

DOsinga · 2025-09-27T16:33:53Z

crates/goose/tests/streaming_agent_visible_test.rs

+    captured_messages: std::sync::Arc<std::sync::Mutex<Vec<Message>>>,
+}
+
+impl MockStreamingProvider {


this feels very LLM generated. did you read the code? delete

DOsinga · 2025-09-27T16:34:59Z

crates/goose/src/providers/base.rs

+                    self.complete_with_model(&model_config, system, messages, tools)
                        .await
                } else {
                    Err(e)


nice. there's also snowflake & anthropic that come with their own message formats

DOsinga · 2025-09-27T17:11:14Z

crates/goose/src/conversation/mod.rs

    }
+
+    #[test]
+    fn test_filtered_messages() {


these tests don't add anything

…#4847)

…-unification * 'main' of github.com:block/goose: (24 commits) feat(cli): add `path` & `limit` to `session list` command (#4878) Allow better concurrent access (#4896) fix: Windows prompt cursor positioning issue with ANSI escape sequences (#4464) Fix: LiteLLM API key field not showing in UI configuration (#4105) fix: path is duplicated on tool calls causing them to fail (#4658) (#4859) add new prompt to get all available tutorials (#4802) Add filtering for agentVisible: false messages on streaming providers (#4847) alexhancock/mcp-crate-cleanup (#4885) docs: rename sub-recipe to subrecipe (#4886) docs: new multi-model section with autopilot topic (#4864) make agent manager singleton (#4880) Cli web auth token (#4456) fix(token_counter): fix panic with GitHub Copilot (#4632) Revert "Internal MCP Crate Cleanup (#4800)" (#4883) remove 2 redundant comments and one that lies (#4866) Internal MCP Crate Cleanup (#4800) Fix #4612: Return non-zero exit code when CLI session ends with error (#4621) Dead code cleanup (#4873) fix: restoring test data and correcting name (#4875) Add .goosehints file to enforce lowercase branding in documentation (#4870) ...

* remove only-pr-labels (block#4842) Signed-off-by: Angela Ning <[email protected]> * Docs: Add link to Plug & Play video for Reddit MCP (block#4852) * Fix: Token count UI doesn't re-render if it's open. (block#4822) * Update databricks flash model (block#4836) * Session manager (block#4648) Co-authored-by: Douwe Osinga <[email protected]> * Add Hacktoberfest Guides (block#4830) Co-authored-by: taniashiba <[email protected]> * docs: goose x Hacktoberfest 2025 Blog (block#4855) Co-authored-by: Tania Chakraborty <[email protected]> Co-authored-by: Angie Jones <[email protected]> * fix: delete some flaky and not-useful tests (block#4861) * can tell the system what shell it is using (block#4807) * new subrecipe blog post banner (block#4862) * docs: remove recipe generator link from next to extension search (block#4858) * lowercase g in goose (block#4832) * doc: file parameter recipe update (block#4594) * Fiie input recipe ref doc (block#4869) * Add .goosehints file to enforce lowercase branding in documentation (block#4870) Co-authored-by: Angie Jones <[email protected]> * fix: restoring test data and correcting name (block#4875) * Dead code cleanup (block#4873) Co-authored-by: Douwe Osinga <[email protected]> Co-authored-by: Michael Neale <[email protected]> * Fix block#4612: Return non-zero exit code when CLI session ends with error (block#4621) Signed-off-by: jalateras <[email protected]> * Internal MCP Crate Cleanup (block#4800) * remove 2 redundant comments and one that lies (block#4866) Co-authored-by: Douwe Osinga <[email protected]> * Revert "Internal MCP Crate Cleanup (block#4800)" (block#4883) * fix(token_counter): fix panic with GitHub Copilot (block#4632) Signed-off-by: sings-to-bees-on-wednesdays <[email protected]> main was broken. this seems important * Cli web auth token (block#4456) Signed-off-by: Evanfeenstra <[email protected]> * make agent manager singleton (block#4880) * docs: new multi-model section with autopilot topic (block#4864) * docs: rename sub-recipe to subrecipe (block#4886) * alexhancock/mcp-crate-cleanup (block#4885) * Temporary workaround for mcp server * Add filtering for agentVisible: false messages on streaming providers (block#4847) * add new prompt to get all available tutorials (block#4802) Signed-off-by: AdemolaAri <[email protected]> * Fix for auth in extension, fix for stale keychain * fix: path is duplicated on tool calls causing them to fail (block#4658) (block#4859) Signed-off-by: demetrio108 <[email protected]> * Fix: LiteLLM API key field not showing in UI configuration (block#4105) Signed-off-by: jalateras <[email protected]> Co-authored-by: Ebony Louis <[email protected]> Co-authored-by: Claude <[email protected]> Co-authored-by: Jack Amadeo <[email protected]> * fix: Windows prompt cursor positioning issue with ANSI escape sequences (block#4464) Signed-off-by: Matt Donovan <[email protected]> Co-authored-by: Matt Donovan <[email protected]> * Allow better concurrent access (block#4896) Co-authored-by: Douwe Osinga <[email protected]> * feat(cli): add `path` & `limit` to `session list` command (block#4878) * Added CMD+T keyboard shortcut that takes you to the Home tab (block#4541) Signed-off-by: Guillaume Simard <[email protected]> --------- Signed-off-by: Angela Ning <[email protected]> Signed-off-by: jalateras <[email protected]> Signed-off-by: Evanfeenstra <[email protected]> Signed-off-by: AdemolaAri <[email protected]> Signed-off-by: demetrio108 <[email protected]> Signed-off-by: Matt Donovan <[email protected]> Signed-off-by: Guillaume Simard <[email protected]> Co-authored-by: Angela Ning <[email protected]> Co-authored-by: Emma Youndtsmith <[email protected]> Co-authored-by: David Katz <[email protected]> Co-authored-by: Douwe Osinga <[email protected]> Co-authored-by: Douwe Osinga <[email protected]> Co-authored-by: Ebony Louis <[email protected]> Co-authored-by: taniashiba <[email protected]> Co-authored-by: taniandjerry <[email protected]> Co-authored-by: Tania Chakraborty <[email protected]> Co-authored-by: Angie Jones <[email protected]> Co-authored-by: Jack Amadeo <[email protected]> Co-authored-by: Michael Neale <[email protected]> Co-authored-by: w. ian douglas <[email protected]> Co-authored-by: Alex Hancock <[email protected]> Co-authored-by: Jarrod Sibbison <[email protected]> Co-authored-by: Rizel Scarlett <[email protected]> Co-authored-by: Jim Alateras <[email protected]> Co-authored-by: sings-to-bees-on-wednesdays <[email protected]> Co-authored-by: Evan Feenstra <[email protected]> Co-authored-by: Yingjie He <[email protected]> Co-authored-by: dianed-square <[email protected]> Co-authored-by: Ademola Arigbabuwo <[email protected]> Co-authored-by: Demetrio ⚡️ <[email protected]> Co-authored-by: Claude <[email protected]> Co-authored-by: Jack Amadeo <[email protected]> Co-authored-by: Matt Donovan <[email protected]> Co-authored-by: Matt Donovan <[email protected]> Co-authored-by: Robert Mcgregor <[email protected]> Co-authored-by: Guillaume Simard <[email protected]>

* main: (206 commits) Tiny: fix github casing (block#4903) remove anyOf from create_task tool (block#4897) chore(deps): bump tracing-subscriber from 0.3.19 to 0.3.20 (block#4442) fix optional recipe schema zod validation (block#4900) Added CMD+T keyboard shortcut that takes you to the Home tab (block#4541) feat(cli): add `path` & `limit` to `session list` command (block#4878) Allow better concurrent access (block#4896) fix: Windows prompt cursor positioning issue with ANSI escape sequences (block#4464) Fix: LiteLLM API key field not showing in UI configuration (block#4105) fix: path is duplicated on tool calls causing them to fail (block#4658) (block#4859) add new prompt to get all available tutorials (block#4802) Add filtering for agentVisible: false messages on streaming providers (block#4847) alexhancock/mcp-crate-cleanup (block#4885) docs: rename sub-recipe to subrecipe (block#4886) docs: new multi-model section with autopilot topic (block#4864) make agent manager singleton (block#4880) Cli web auth token (block#4456) fix(token_counter): fix panic with GitHub Copilot (block#4632) Revert "Internal MCP Crate Cleanup (block#4800)" (block#4883) remove 2 redundant comments and one that lies (block#4866) ...

…block#4847) Signed-off-by: HikaruEgashira <[email protected]>

katzdave added 4 commits September 26, 2025 12:24

Streaming filtering

45da1a2

filter in autocompact

3f2262f

fmt

ea2af98

message filtering

dfff91e

katzdave requested a review from DOsinga September 26, 2025 17:25

katzdave added 5 commits September 26, 2025 13:32

fix docs

d2decd2

clean up docs

e5d9980

fix ci

a9ae41d

fix doctest again

25ee0c4

DOsinga reviewed Sep 26, 2025

View reviewed changes

refactor provider

8aeae4e

DOsinga approved these changes Sep 27, 2025

View reviewed changes

DOsinga self-assigned this Sep 27, 2025

DOsinga reviewed Sep 27, 2025

View reviewed changes

katzdave added 8 commits September 29, 2025 12:18

move async token counter

0619a9d

change thinking back

c4d167a

rm bad test

891ffb1

rm test filters

f9318d7

full filtering coverage

61d83e0

rm excessive comments

e324b87

nl

65a118d

final couple providers

b98dd63

katzdave merged commit 1a92111 into main Sep 29, 2025
11 checks passed

katzdave deleted the dkatz/fix-streaming-impl branch September 29, 2025 21:20

katzdave added a commit that referenced this pull request Sep 29, 2025

Add filtering for agentVisible: false messages on streaming providers (…

ef0b745

…#4847)

katzdave mentioned this pull request Sep 29, 2025

Release/1.9.1 #4889

Closed

HikaruEgashira pushed a commit to HikaruEgashira/goose that referenced this pull request Oct 3, 2025

Add filtering for agentVisible: false messages on streaming providers (…

849e89b

…block#4847) Signed-off-by: HikaruEgashira <[email protected]>

This was referenced Oct 8, 2025

chore(release): release version 1.10.0 #5060

Closed

release/1.10.0 #5101

Closed

Add filtering for agentVisible: false messages on streaming providers #4847

Add filtering for agentVisible: false messages on streaming providers #4847

Uh oh!

Conversation

katzdave commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DOsinga left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

katzdave commented Sep 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

katzdave Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

katzdave commented Sep 26, 2025 •

edited

Loading

katzdave Sep 29, 2025 •

edited

Loading