Token counting reliability + summarization integration. #3721

katzdave · 2025-07-29T21:48:27Z

Use the token counter as a fallback for when providers don't provide tokens; Ensures we always have some token counts.
Remove old summarization algorithm completely. Only have the single shot. Simplify the contract since it only returns 1 message.
Update metadata with summarization results for immediate visibility in context window display.
Remove some standalone codepaths testing tokenizer.

TODO (future PR):
Audit desktop more closely and make sure we have full parity + clean up the UI around context display.

* 'main' of github.com:block/goose: (23 commits) fix: add fallback id to messages if none provided (#3584) feat: migrate ErrorData from internal mcp crates to rmcp version (#3586) fix: adjust subrecipe description to allow running tests (#3585) Scenario tests (#3430) feat: migrate JsonRpcMessage/Request/Response/Error/Notification from internal mcp crates to rmcp versions (#3564) Restore recipe parameters functionality (#3530) feat: Enhanced loading states with thinking icons and flying bird animation (#3478) Agent loop defensive (#3554) chore: remove needless clone() in goose/providers (#2528) Add recipe install warning (#3537) Replace mcp_core::resource::* with rmcp types (#3563) Add YouTube video embed to using-goosehints.md (#3560) fix: ensure retry-config and success-criteria are populated in openapi spec (#3575) fix: use sequential when sub recipe task is 1. (#3573) fix: track message id to keep like with like (#3572) Replace mcp_core::prompt with rmcp::model types (#3561) feat (ui): close recipe modals with esc key (#3568) feat: recipes can retry with success criteria (#3474) Env var to set Ollama request timeout (#3516) Updating docs to match new UI (#3552) ...

* 'main' of github.com:block/goose: Remove mcp_macros and unused types (#3581)

* 'main' of github.com:block/goose: docs: use installer component for tutorial, add updating extensions section (#3608) fix: show token alert popover during agent responses and agent failure cases (#3536) reuse the cancellation token in the agent level (#3599) Docs: Move MongoDB tutorial to MCP section (#3602) docs: desktop recipe format (#3594) Fix model display name not being updated immediately after leaving settings (#3587) Added option to summarize the chat when an error is triggered (#3598)

Co-authored-by: Zane <[email protected]> Co-authored-by: dianed-square <[email protected]> Co-authored-by: Rizel Scarlett <[email protected]> Co-authored-by: Lifei Zhou <[email protected]> Co-authored-by: David Katz <[email protected]>

* 'main' of github.com:block/goose: Adds the `WaitingForUserInput` state (#3620) docs: update extensions library (#3612) Fixing grants path (#3632) docs: June 2024 Community All-Stars Spotlight (#3631) grant program (#3630) Lifei/sub recipe desktop temp (#3576) docs: local LLMs context size tip (#3454) fix: Handle non-default base path for OpenAI compatible model fetching (#3566) Goose security updates (#3579) fix: multi-tool calls in streamed openai-compatible responses (#3609) feat: subagent turn count, simple agent loop (#3597) feat: subagent independent extension manager (#3596) Improve session history loading resiliency (#3588) Added logging and changed default route case to not redirect to home when there is an invalid route (#3610) fix: chat sidebar layout overlapping content occasionally (#3590) fix: loading shared sessions (#3607)

* 'main' of github.com:block/goose: Make the window title reflect what we are doing (#3883) additional metrics + Ui implementation (#3871) feat: Add session description editing functionality (#3819) Update filename in contributing docs (#3866) Fix voice dictation provider selection bug (#3862) doc: Update supported container runtimes (#3874) feat: add OAuth provider abstraction for CLI configuration (#3157) Don't ignore lockfiles on linux/windows builds (#3859) Use RMCP for StreamableHTTP OAuth support (#3845) Try to keep key order for Databricks (#3876) Fix OpenAI Provider with GitHub Models (#3875) Cmd click open finder (#3807) fix: recipe parameter form max height and not scrolling (#3879) fix: optimise reading large file content (#3767) fix: replace glob/grep tool with shell (#3834) docs: Add Youtube Link to dev.to tutorial (#3869)

michaelneale · 2025-08-06T23:46:43Z

crates/goose/src/prompts/summarize_oneshot.md

 Generate detailed summary of conversation to date.  
 Include user requests, your responses, and all technical content.  

+**Conversation History:**


how did this work before?

Trying to understand that one too; I think we are falling back to the old summarization algorithm 😱.

why do we still have the old algorithm

This deletes it for good

was the session passed some other way to the compact call?

Ugh, nope just did a full investigation. We are calling summarize_messages_async instead of summarize_messages; All the tests swap over but didn't make the swap in agent.rs + delete summarize_messages_async.

Can potentially break this up and swap the summarization algorithm first, but the numbers that show up in the auto-compact message + context window aren't the best atm.

* 'main' of github.com:block/goose: remove fallback routing to hub/home for unknown routes (#3954) Use cross in linux bundle workflow (#3950) fix: disable signing for release branches until we figure out keys for this flow (#3951) Sanitize Tags Unicode Block (#3920) Add a message about DCO to CONTRIBUTING.md (#3741) Move hardcoded LLM prompts to template files (#3934) docs: migrate streamable config to consolidated component (#3936) feat: streamline list args on cli (#3937) mcp/developer: Refactor to use tokio SplitStream (#3894) feat: first time automated ollama install experience and openrouter (#3881) chore: rmcp 0.5.0 (#3935) add gpt-5 to openai provider format (#3924) added gpt5 context limit (#3927) show status of osx codesigning and increase timeout (#3926) Bump auto-compact threshold to 80% (#3925) FIX: gemini tool call hanging (#3898) feat(deps): upgrade rmcp to 0.4.1 (#3918) Fix dark mode rendering of config form and centered providers grid for wider screens. (#3837) fix: extension list not refreshing after installing from deeplink (#3878)

DOsinga · 2025-08-08T18:31:30Z

crates/goose-server/src/routes/context.rs

            .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
    } else if request.manage_action == "summarize" {
-        (processed_messages, token_counts) = agent
+        (processed_messages, token_counts, _) = agent


Does this still use the old algorithm? any reason not to switch this over?

also, the truncate above is not used i think

I'm not clear on truncate. Will talk to @michaelneale about it but I think makes sense to not touch that until the next wave.

This uses the same call as the old algorithm made, but summarize.rs is completely gutted to just have the single shot summarizer.

I think truncate may have veen used on CLI for people using recipes, but as long as summarize works out of the box, it is ok to clean it out I think (as ideally we don't want people having to specify it, in the past it would just break if you didn't specify a strategy). so I think ok to get rid of it if it helps?

it can always be done with a later release

Yeah I was just going to pick it apart on the next wave since this PR is already large.

* 'main' of github.com:block/goose: docs: update release docs with an additional step needed + adjust list formatting (#4005) chore(release): release version 1.3.0 (#3921) docs: MCP-ui blog content (#3996) feat: Add `GOOSE_TERMINAL` env variable to spawned terminals (#3911) add missing dependencies for developer setup (#3930) blog: subagents workshop (#3993) Adding better deeplink support for streamable-http (#3986) Docs: Add Kiwi Flight Search Extension Tutorial (#3985) Revert "Fix file extension on signed osx release (#3974)" Fix file extension on signed osx release (#3974) Dagger container-use MCP (#3932) Add Sunno Song Generator To Recipe Cookbook (#3969) docs: update jetbrains tutorial (#3971) chore: update linux build - don't run on every PR (#3961)

michaelneale · 2025-08-12T02:02:11Z

crates/goose/src/context_mgmt/auto_compact.rs

-///
-/// # Returns
-/// * Tuple of (compacted_messages, tokens_before, tokens_after)
-pub async fn perform_compaction(


@katzdave where did this functionality go, this was dead code?

It was just a thin wrapper around agent.summarize with some logging so got rid of it.

michaelneale · 2025-08-12T02:10:59Z

saw it kick in at what I thought was 10%:

(that is different message to before and seemed fast, and was after some tool calls I think)

not sure if that is still a bug or not, things seem to work well though.

michaelneale

a big one so a little hard to review, but seemed to work well (modulo surprising me with compaction) and stable.

* 'main' of github.com:block/goose: Move summarize button inside of context view (#4015) blog: post on lead/worker model (#3994) Actually send cancellation to MCP servers (#3865) fix: enable 'goose://' handler for debian systems (#3952) fit: default ollama port (#4001) Remove cognitive complexity clippy lint (#4010) feat: add @-mention file reference expansion to .goosehints (#3873) feat(cli): Add --name/-n to session remove and --id/-i alias for session export (#3941) Docs: provider and model run options (#4013) To-Do Tools (#3902) ci: correctly match doc only changes (#4009) Remove PR trigger for Linux build workflow (#4008)

* 'main' of github.com:block/goose: feat: ToolError migration to ErrorData (#4051) docs: rename sessions (#4053) Add mcp automated testing blog (#4004) MCP session replay integration test (#3939) Docs: Cost tracking in CLI (#4043) sanitize message content on deserialization (#3966)

* 'main' of github.com:block/goose: docs: mcp-ui support (#4049) fix: delete dialog layout (#4037) ci: fix markdown file pattern to skip builds for all .md files (#4061) docs: add window title (#4059) blog: cleaning up some posts (#4050) fix: this should be a debug message not a warn (#4024) Better provider logging (#4052)

* 'main' of github.com:block/goose: fix(build): feed electronforge the icon explicitly for linux (#4045) Docs: Troubleshooting tip - Nodejs path on windows (#4065) fix: flag out uncompilable bit in windows (#4068) ci: fix docs-only filter to properly skip tests for documentation changes (#4066) fix: ctrl-C interruption in the CLI (#4057)

* main: (21 commits) docs: fix experimental warning (#4102) fix not being able to add spaces to activity message area (#4055) remove and cleanup unused code (#4074) docs: Add missing references to smart_approve mode. (#4094) quick typo fix pulse blog (#4095) fix: update dictation settings handling and improve user feedback (#4093) chore: add info tooltip to share session button (#4038) Reroute routes (#4088) fix(cli): fix compiling issue on windows system by adding a missing param (#4019) fix: hide console window for stdio extensions on Windows (#4085) Do not fetch pricing information if GOOSE_CLI_SHOW_COST is false (#3901) blog: fixing links (#4089) Token counting reliability + summarization integration. (#3721) Skip ci builds for docs (#4087) blog: agent coordination patterns (#4086) docs: Blog - How I Used Goose to Rebuild My Website (#4076) docs: custom context file names (#4077) Blog: How Pulse MCP Used Goose to Automate Their Newsletter (#4075) Load recipe deeplinks in single window when app is closed (#4048) docs: make accurate the comments with links to unsigned builds of the app (#4070) ...

Signed-off-by: Jack Wright <[email protected]>

katzdave and others added 30 commits July 18, 2025 12:47

comiples

563fbe2

tests

aa43f09

context usage fix

9ea702b

change overhead

6bbba20

shrink works

ff5c9a2

merge

75c074b

new prompt

52099ce

move token counter

f8b37d0

reset changes to mod.rs and agent.rs to focus on summarization algorithm

e83cadc

rm old summarizer

324c511

shrink token counting

5546bef

reset token counting

dc668bb

fmt

7b6f17e

Merge branch 'main' of github.com:block/goose into dkatz/goose-compact2

e55dc40

* 'main' of github.com:block/goose: Remove mcp_macros and unused types (#3581)

fix test

f71fe31

rm tool call removal fns

0f37623

one more unused symbol

ef432b5

fmt

624e3eb

split compaction into check fn

caf4557

refactor into agent reply

1c27b65

add logging

ad7ca3d

fix token reduction

94dc7a0

merge

46a8ec7

fmt tidy up

0aa079a

import the right tool, don't rely on wildcard

23162e4

rm debug logs

8342d35

katzdave added 2 commits August 6, 2025 13:49

rm tokens before

8ebc10c

katzdave requested review from DOsinga and michaelneale August 6, 2025 21:20

michaelneale reviewed Aug 6, 2025

View reviewed changes

katzdave added 3 commits August 7, 2025 13:05

messy merge resolved

f2da61c

fmt

abde666

DOsinga reviewed Aug 8, 2025

View reviewed changes

michaelneale reviewed Aug 12, 2025

View reviewed changes

michaelneale approved these changes Aug 12, 2025

View reviewed changes

katzdave added 8 commits August 12, 2025 12:57

summarization usage fix

9689670

update message

601ee4b

rm warning

bac76c8

fmt

b8c82b7

katzdave merged commit 80826c2 into main Aug 14, 2025
10 checks passed

katzdave deleted the dkatz/token-counting branch August 14, 2025 15:23

katzdave added a commit that referenced this pull request Aug 15, 2025

Token counting reliability + summarization integration. (#3721)

636ed6e

alexhancock pushed a commit that referenced this pull request Aug 16, 2025

Token counting reliability + summarization integration. (#3721)

7d25d38

alexhancock pushed a commit that referenced this pull request Aug 16, 2025

Token counting reliability + summarization integration. (#3721)

07bf325

This was referenced Aug 18, 2025

chore(release): release version 1.5.0 #4154

Closed

chore(release): release version 1.5.0 #4169

Merged

ayax79 pushed a commit to ayax79/goose that referenced this pull request Aug 21, 2025

Token counting reliability + summarization integration. (block#3721)

8b4e04b

Signed-off-by: Jack Wright <[email protected]>

Token counting reliability + summarization integration. #3721

Token counting reliability + summarization integration. #3721

Uh oh!

Conversation

katzdave commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

katzdave Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelneale commented Aug 12, 2025

Uh oh!

michaelneale left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

katzdave commented Jul 29, 2025 •

edited

Loading

katzdave Aug 11, 2025 •

edited

Loading