Skip to content

Conversation

@tlongwell-block
Copy link
Collaborator

Implements ephemeral context injection at the provider boundary to automatically include timestamp and TODO content in LLM context without modifying conversation history.

Problem

Previously, accessing TODO content required explicit todo__read tool calls, cluttering conversation history and consuming tokens. Timestamps were injected via template variables in system prompts, creating inconsistency across different prompt variants and preventing effective prompt caching.

Solution

MOIM (Minus One Info Message) injects a synthetic user message at approximately position -1 during provider calls. This message contains:

  • Current timestamp (always)
  • TODO content from session metadata (when available)

The injection happens in reply_parts.rs at the provider boundary, ensuring the message is never persisted to storage or returned in conversation history.

Architecture Benefits

  • Zero changes to core data structures (Message, Conversation)
  • No persistence layer modifications
  • Clean module boundaries
  • No impact on existing sessions or conversation replay

Testing

Comprehensive test coverage in crates/goose/tests/moim_tests.rs with proper test isolation using serial_test for tests that modify environment variables.

Migration

  • Removed todo__read tool (breaking change for recipes using it)
  • Updated todo__write description to reflect automatic availability
  • System prompts now receive timestamp via MOIM instead of template variables

@tlongwell-block tlongwell-block marked this pull request as ready for review September 26, 2025 18:08
Copy link
Collaborator

@DOsinga DOsinga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some preliminary comments. I think we should do something like this, but we need a structure that allows any platform tool this sort of thing. see: #4868 and the relevant discussion

#[serial]
async fn test_what_is_your_name() -> Result<()> {
run_scenario(
std::env::set_var("GOOSE_MOIM_ENABLED", "false");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't set environment variables in tests like this; if they fail you now changed the settings for the next test. also if you are going to add this to all run_scenario, maybe handle this there?

Copy link
Collaborator Author

@tlongwell-block tlongwell-block Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the tests in serial now. I set the env var at the top of the test and unset at the bottom. It won't effect other tests that way

I think you're right that I could get the same effect by un/setting it in run_scenario(), but still making sure that everything that calls run_scenario() is serial

The reason I didn't is just because I can't internally guarantee that run_scenario will be in serial, hence it would be possible for an environment race condition to occur if called by a non-serial test


// Inject MOIM (timestamp + TODO content) if enabled
let messages_for_provider =
super::moim::inject_moim_if_enabled(messages_for_provider, session).await;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should do this in agent.reply. there you can stick it into the block that already checks for session and make session non optional

Copy link
Collaborator Author

@tlongwell-block tlongwell-block Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put it in stream_response_from_provider because I wanted it to be at the very, very edge of the shared inference execution path. This is the last point on that execution path that doesn't split between different providers

This makes reasoning about it much easier. There's no chance to be written to a session file, be compacted, etc, etc

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as session optionality goes: it should 100% be required, but we can't do it yet. Once recipes/subagents/scheduler are on Agent Manager, they will have real sessions.

But, until then, stream_response_from_provider will be called without a session when dealing with recipes/subagents

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the dependency is there from how the TODO stool stores its state over actually needing the session.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Instead of passing the session to inject_moim_if_enabled, I could pass TodoState::from_extension_data directly? It just ended up looking cleaner this way

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, no, lets keep passing the session in because we'll probably want to add more to moim from the session metadata later at some point. And its neater

@tlongwell-block
Copy link
Collaborator Author

tlongwell-block commented Sep 29, 2025

left some preliminary comments. I think we should do something like this, but we need a structure that allows any platform tool this sort of thing. see: #4868 and the relevant discussion

Agree, yes. I think adding to build_moim_content() is the way to go here since there will probably be formatting and helpers involved when moving platform tool content from its storage into the MOIM (ie an array of sources is probably not sufficient, so using the build function is probably the best way forward)

@tlongwell-block tlongwell-block force-pushed the moim branch 2 times, most recently from 117dee3 to af70b3b Compare September 29, 2025 18:26
use crate::session::SessionManager;
use chrono::Local;

async fn build_moim_content(session: &Option<SessionConfig>) -> Option<String> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really like to see this as something the tool itself can define rather than a specific global place.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I talked about my thinking in a reply above:

"... I think adding to build_moim_content() is the way to go here since there will probably be formatting and helpers involved when moving platform tool content from its storage into the MOIM (ie an array of sources is probably not sufficient, so using the build function is probably the best way forward)"

Having each tool register itself to a moim system would work, too, but we might still want to format how they fit together, which requires a global function like this one

also, this global function allows us to put data in the moim that isn't related to a tool. Like the timestamp here. This could also be environment information or other non-tool context that we find useful for the agent

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in light of the latest infra changes here's my proposal:

  • add a method to the platform extension type get_moim() and implement that for todo
  • add a method to extension_manager get_moim that goes through its platform extensions and collects them and then adds the current time
  • call this method from reply_internal (yeah, I know that is scary)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working in #5027


// Inject MOIM (timestamp + TODO content) if enabled
let messages_for_provider =
super::moim::inject_moim_if_enabled(messages_for_provider, session).await;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the dependency is there from how the TODO stool stores its state over actually needing the session.

@tlongwell-block
Copy link
Collaborator Author

#5027

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants