FEAT: Add Chat History File Functionality for Efficient Auto-GPT Resumption

[dreadful-dev]

Background

To improve the overall user experience, this PR focuses on adding the functionality to maintain a chat history file for Auto-GPT. This feature will enable Auto-GPT to resume work more efficiently when stopped, by allowing it to load, update, and clear the chat history file.

Changes

• Implemented chat_history utility functions to handle chat history file operations (load, update, and clear).
• Modified the main chat loop to save chat history to a file.
• Modified initialization to load chat_history from file.
• Added a command-line option to customize the chat history file used.

Documentation

The new functionality has been documented through in-code comments.

Test Plan

• Tested the functionality with various chat scenarios, including normal conversations and edge cases.
• Ensured proper file creation and deletion when user chooses not to continue with the last created agent
• Tested the chat history loading and updating process to verify correctness.
• Validated the performance impact of the new functionality, confirming that it does not introduce significant overhead.

PR Quality Checklist

• My pull request is atomic and focuses on a single change.
• I have thoroughly tested my changes with multiple different prompts.
• I have considered potential risks and mitigations for my changes.
• I have documented my changes clearly and comprehensively.
• I have not snuck in any "extra" small tweaks changes.

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[ntindle]

Add Tests

[dreadful-dev]

Will do

[p-i-]

This is a mass message from the AutoGPT core team.
Our apologies for the ongoing delay in processing PRs.
This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to:
https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

[Boostrix]

Sorry, just wondering: is this now the de facto method of implementing workspace recovery/resumption ?

[Pwuts]

Resumption is a much-wanted feature. Before reviewing further this PR would need rebasing though. @dreadful-dev would you be willing to take a shot at that? The merge conflicts are quite extensive so it might be more efficient to re-apply the changes to latest master by hand.

[dreadful-dev]

@Pwuts No problem at all, I'll get this branch in sync with master for review. Any recommendations for tests you'd like to see are also welcome.

[lc0rp]

Thoughts for discussion:

• If we enable debug mode (--debug), the complete message history will be saved to FULL_MESSAGE_HISTORY_FILE_NAME.
• Also, resuming Auto-GPT may also include preserving memory, skipping certain ai_settings initialization steps, and examining its workspace (which could already be included in the message history)

That makes me wonder whether this feature can include:

• Enabling a --resume option
• Making FULL_MESSAGE_HISTORY_FILE_NAME configurable in .env
• Initializing full_message_history from FULL_MESSAGE_HISTORY_FILE_NAME if --resume is set
• Skipping memory truncation if --resume is set
• Optional: Enabling --debug if --resume is set (then we get the writing for free) - we could instead add logic to activate writing for both --resume and --debug

Additional thoughts:

• Should we require that the file is located in the workspace_directory?

Tests could include

• Demonstration of successful read/write file
• Error handling when read/write/json.loads fails e.g. read-only file, wrong JSON format, etc)

[Pwuts]

@dreadful-dev bump ;)

[dreadful-dev]

Assumptions:

• MessageHistory(ChatSequence) and Memory serve two different purposes.
• MessageHistory(ChatSequence) is not currently saved to a file, although the entire history is saved every cycle, but never cleared, so the entire message history is continuously resaved to FULL_MESSAGE_HISTORY_FILE_NAME
• self.memory within the Agent is not currently used. Full project search for self.memory shows no methods being called.

For this to work as originally intended, the N most recent items in the ChatSequence should be loaded from a history file without duplicate entries, and those should then be fed into the get_relevant method of the now apparently unused VectorMemoryProvider to collect the relevant llm thoughts that would have been present at the time of agent shutdown.

As far as I can tell, this would involve adding a flag around clearing the memory here so that previous memories in the index were not deleted unless a flag was present (or not present). The VectorMemoryProvider would then need to be utilized to save all LLM "thoughts" similarly to how they were saved in April. Then the code here would need to be updated to get the last N most recent entries from the ChatSequence which would then be fed into get_relevant to get older, relevant memories from the vector store. On first run, the relevant_memories should not be shuffled for reproducible agent initialization.

Forgive me if I've missed something or made incorrect assumptions, the codebase has evolved quite rapidly! I'm happy to resume work on this now that I've got my head around the updated codebase, given none of my assumptions are wildly incorrect. The one requirement is that the VectorMemoryProvider is ready to be used to save and retrieve memories by relevance.

[Pwuts]

• MessageHistory(ChatSequence) and VectorMemory serve two different purposes.

FTFY, and correct.

• MessageHistory(ChatSequence) is not currently saved to a file, although the entire history is saved every cycle, but never cleared, so the entire message history is continuously resaved to FULL_MESSAGE_HISTORY_FILE_NAME

That LogCycle thing is for debug purposes, continuously recording a timeline of the program state.

• self.memory within the Agent is not currently used. Full project search for self.memory shows no methods being called.

Yup, see #3536 and #4799. It's work-in-progress, and VectorMemory will soon be taken into use.

For this to work as originally intended, the N most recent items in the ChatSequence should be loaded from a history file without duplicate entries, and those should then be fed into the get_relevant method of the now apparently unused VectorMemoryProvider to collect the relevant llm thoughts that would have been present at the time of agent shutdown.

The full message history is stored in agent.history.messages. The list of messages that is added to the prompt every cycle is truncated, but agent.history.messages itself is not.

In order to resume the program, I think loading something into agent.history.messages before the Agent is started (agent.start_interaction_loop()) should do it for now.

Furthermore, I feel that we might be served with a graceful shutdown handler. It could save the program state (including at the very least the current message history), which can then be used later to resume operations. A less refined solution (saving on every cycle) could work, too.

Forgive me if I've missed something or made incorrect assumptions, the codebase has evolved quite rapidly! I'm happy to resume work on this now that I've got my head around the updated codebase, given none of my assumptions are wildly incorrect. The one requirement is that the VectorMemoryProvider is ready to be used to save and retrieve memories by relevance.

I'm not sure why the VectorMemoryProvider should be needed to implement state-saving and resumption functionality. That being said, it is ready to use, although the exact implementation could still change when we start reshaping the agent loop around retrieval augmentation / memory integration.