Calling `chat.to_llm` keeps appending messages to the message array

[bborn]

Using the acts_as to_llm adds messages to the chat array on every call (because @chat is memoized).

  it 'persists chat history' do
    chat = Chat.create!(model_id: 'gpt-4.1-nano')
    chat.ask("What's your favorite Ruby feature?")

    expect(chat.messages.count).to eq(2)
    expect(chat.messages.first.role).to eq('user')
    expect(chat.messages.last.role).to eq('assistant')
    expect(chat.messages.last.content).to be_present
    expect(chat.messages.last.input_tokens).to be_positive
    expect(chat.messages.last.output_tokens).to be_positive

    chat.to_llm

    expect(chat.to_llm.messages.count).to eq(2)
  end

  1) RubyLLM::ActiveRecord::ActsAs persists chat history
     Failure/Error: expect(chat.to_llm.messages.count).to eq(2)
     
       expected: 2
            got: 6

It seems like just removing the memoization here would fix it, but I'm not sure if there's a reason it was done that way.

[jayelkaake]

Want to throw up a PR for this? I ran into it as well.

Here's my code change in RubyLLM::ActsAs

def to_llm
  return @chat if defined?(@chat)

  @chat = RubyLLM.chat(model: model_id)

  # .... rest is the same.
end

[bborn]

I think you need the memoization, otherwise things like with_tools don't work. Let's see what @crmne says ...

[jayelkaake]

I think you need the memoization, otherwise things like with_tools don't work. Let's see what @crmne says ...

Yeah you do need it.

What I posted keeps it in-tact, just fixes it so it's not running the code after it instantiates it that re-adds the messages from the model to the LLM::Chat instance again.

[rhys117]

I've been troubleshooting the same issue today after seeing repetitive instruction prompts appear when combining them with tool usage.

I suspect we need to refresh the memoization during the with_instructions call. Which also raises another question, should we also consider previously loaded tools when adding instructions and ensure they persist on the @chat object?

As a temporary workaround, I'm using the quick fix below, although it reloads all messages every time to_llm is called, which is less than ideal.

acts_as.rb

def to_llm
  @chat ||= RubyLLM.chat(model: model_id)
  @chat.reset_messages
  # ... remaining implementation

chat.rb

def reset_messages
  @messages = []
end

[timaro]

Also hitting this. It feels like the change @jayelkaake posted is correct -- I can't see why you'd need to copy over the messages again or re-set the on_new_message and on_end_message callbacks every time to_llm is called?