Autocompact + One Shot Summarization algorithm

[katzdave]

Replaces the summarization algorithm with a single prompt variant.

Auto-compact uses GOOSE_AUTO_COMPACT_THRESHOLD to determine if we should compact the session. Triggers this check after every user message.

[michaelneale]

nice - taking for a spin

[michaelneale]

nice @katzdave - seems to work well, I will use it a bit more as a daily thing and see if I can abuse it more! lets get this in!

[michaelneale]

did hit a snag, I told it to look at ~/.local/share/goose/sessions for jsonl to extract themes, and it ran up quite the context:
(a prompt like " can you take a look in ~/.local/share/goose/sessions at the jsonl files to extract some common themes")

Now let me examine the session file to understand the specific issue:
─── text_editor | developer ──────────────────────────
path: ~/.local/share/goose/sessions/20250722_151520.jsonl
command: view
view_range:
    -
        1
    -
        20

Error: Request failed: Request failed with status: 400 Bad Request. Message: {"external_model_provider":"databricks-model-serving","external_model_error":{"error_code":"BAD_REQUEST","message":"Bad Request"}}
The tool calling loop was interrupted. How would you like to proceed?
  error: The error above was an exception we were not able to handle.
These errors are often related to connection or authentication
We've removed the conversation up to the most recent user message
- depending on the error you may be able to continue


Context: ●●●●●●●○○○ 68% (136646/200000 tokens)

might be able to reproduce it with things like that (jsonl is good as there is lots of stuff buried in there in single lines to mess it up), but then again, may not be related? I did ask it a few things and a few times to get it to run up a big session)

[katzdave]

did hit a snag, I told it to look at ~/.local/share/goose/sessions for jsonl to extract themes, and it ran up quite the context: (a prompt like " can you take a look in ~/.local/share/goose/sessions at the jsonl files to extract some common themes")

Now let me examine the session file to understand the specific issue:
─── text_editor | developer ──────────────────────────
path: ~/.local/share/goose/sessions/20250722_151520.jsonl
command: view
view_range:
    -
        1
    -
        20

Error: Request failed: Request failed with status: 400 Bad Request. Message: {"external_model_provider":"databricks-model-serving","external_model_error":{"error_code":"BAD_REQUEST","message":"Bad Request"}}
The tool calling loop was interrupted. How would you like to proceed?
  error: The error above was an exception we were not able to handle.
These errors are often related to connection or authentication
We've removed the conversation up to the most recent user message
- depending on the error you may be able to continue


Context: ●●●●●●●○○○ 68% (136646/200000 tokens)

might be able to reproduce it with things like that (jsonl is good as there is lots of stuff buried in there in single lines to mess it up), but then again, may not be related? I did ask it a few things and a few times to get it to run up a big session)

I changed the context counting a bit, as it was annoying me that the context didn't update until the next message. I ripped that out into another branch though, and am trying to get things in a row there.

[michaelneale]

more testing, mostly summarize was spot on, but I did hit this roadblock - no idea if related, but it just will never respond/do anything now:

not sure if related to session or provider

this was after a compaction, and the next prompt after that (ie once it had summarized/compacted, returned, I gave it one more instruction and that worked, but then got into this state). Probably not related?

edit:

config:

GOOSE_MODEL=claude-opus-4-20250514
GOOSE_PROVIDER=anthropic

[michaelneale]

@katzdave this is my take on auto compact: #3600

still trying it out - so not sure how good, but thought you may find it interesting where to put the calls to compact - client end still seems easier (vs moving to agent and changing how api looks) - but take a look if it helps. I have seen it autocompact once (but it seemed to stall before it hit it - so may not be quite right) - it did autocompact down to 1K tokens (should it be that low?)

eg:

[michaelneale]

@katzdave I did have a quick look (but failed) to see if it would be easy to add it to the main agent loop - but I wasn't sure if we still had the summarizer that runs when it fills up (if we don't, we may need to add something like it back in on a higher threshold but not waiting for user input to trigger) to avoid running up against the limit when it is in the agent loop

[katzdave]

@katzdave hrm just saw this:

Error: Request failed: Streaming request failed with status: 400 Bad Request. Error: {"type":"error","error":{"type":"invalid_request_error","message":"prompt is too long: 210052 tokens > 200000 maximum"}}
The tool calling loop was interrupted. How would you like to proceed?
  error: The error above was an exception we were not able to handle.
These errors are often related to connection or authentication
We've removed the conversation up to the most recent user message
- depending on the error you may be able to continue


Context: ●●●●●●●●○○ 83% (165147/200000 tokens)
( O)> Press Enter to send, Ctrl-J for new line

which is first time - was a long run with anthropic provider + opus. Not sure if this is up to date with main but looks like token calculating is quite right, but even then - the problem is I think we do need to compact during the agent loop for complex changes unfortunately

One idea I chatted with Douwe about is to potentially have a tool for the agent to decide itself to autocompact + combined with task tracking that should help resume after compaction; I worry if we compact in the middle of an agent loop it might get pretty lost whereas with those signals it can start fresh a bit more cleanly.

I also think that marking a task as done provides a strong signal that the agent might be in a decent resting point to potentially compact or feed in new user context.

Working on some token counting fixes, but I think adding to the agent loop makes sense as a next wave effort.

[katzdave]

I also swapped in a 50% more summarize prompt. I don't notice much of a difference.

[michaelneale]

@katzdave in that case do you think we should we have task tracking in this PR (if we can make it work) - or anther PR (could also move it to developer MCP if that made it easier) - or is over loading it?

"tool for the agent to decide itself to autocompact + combined with task tracking" yes - so you think compact is itself a tool to call? (or just part of agent loop + heuristics so it isn't an extra tool call?)

[michaelneale]

not sure about how many tokens an emoji uses?

[michaelneale]

this intended to be here or should be in existing test code dir?

[katzdave]

Nope, deleted