Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add options to filter conversation history messages used in sequential LLM and Agent nodes #3653

Merged
merged 8 commits into from
Dec 12, 2024
Merged

feat: Add options to filter conversation history messages used in sequential LLM and Agent nodes #3653

merged 8 commits into from
Dec 12, 2024
+147 −29

Conversation

jeanibarz
Copy link
Contributor

@jeanibarz jeanibarz commented Dec 8, 2024 •
edited
Loading

  • Add new disableConversationHistory boolean parameter under Additional Parameters in Agent and LLMNode to optionally skip including conversation history in prompts, improving sequential agent workflow customization capabilities
  • Fix potential error in Agent.ts when messages array is empty by adding null safety checks

@jeanibarz
feat: Add option to disable conversation history
be048eb 
- Add new `disableConversationHistory` boolean parameter in LLMNodes.ts and Agent.ts to optionally skip including conversation history in prompts
- Fix potential error in Agent.ts when messages array is empty by adding null safety checks
- Improve memory efficiency by allowing stateless interactions when history isn't needed
@jeanibarz
Copy link
Contributor Author

jeanibarz commented Dec 8, 2024 •
edited
Loading

When testing locally, I encountered an error with the agent tool when the "Require Approval" toggle is set to true. However, this issue occurs regardless of whether "Disable Conversation History" is set to false or true, so I suspect the problem is unrelated to the changes in this PR.
image

Feel free to check with the joint "Test flow Agents.json" agent workflow.
Test flow Agents.json

Additionally, my existing LLM nodes/agent nodes are not updated automatically. I had to create new nodes for the "Disable Conversation History" parameter to appear under additional parameters. I'm unsure if this behavior is expected, but it happens regardless of whether I bump the component versions.

@jeanibarz jeanibarz mentioned this pull request Dec 8, 2024
Open
@saatchi-david
Copy link
Contributor

@jeanibarz
This is great! So for example, if you only need to give a node the last message, you could disable including the full history, and manually pass just the last message right?

@jeanibarz
Copy link
Contributor Author

Yes, exactly! The feature disables the inclusion of historical messages in the prompt but still allows you to manually add specific messages—whether from the prompt or from the script that allow to create custom conversation messages. In theory, this means you could include the initial user query or even the last two messages from the state messages history, for example. Additionally, it’s worth noting that the node will still append its output to the state messages as usual.

@saatchi-david
Copy link
Contributor

Awesome, I had my fingers crossed that this would come soon. I think a lot of people don't realize the amount tokens they are burning behind the scenes. Hopefully just seeing the option will help increase awareness (on top of control of course).

@jeanibarz
Copy link
Contributor Author

jeanibarz commented Dec 9, 2024 •
edited
Loading

I attempted to add a history message when the boolean disableConversationHistory is set to true. Here's what I observed:

  • Success with Human Prompt and Format Values: Using a placeholder and the value from $flow.state.messages[-1].content, the placeholder was correctly replaced with the latest message. This approach works perfectly. Alternatively, the value can also be retrieved directly from the state or the output of another node.

  • Issue with Script Code in the "Messages History" Field: (probably because I don't know how to use it properly though...) When I tried using state.messages[-1].content, flow.state.messages[-1].content, or $flow.state.messages[-1].content within the script code, I encountered an error stating that the key messages does not exist.
    I also attempted to use a placeholder in the format prompt (similar to the human prompt), but the placeholder was not replaced with the expected value. You can see this issue in the screenshot below:
    but probably because I don't know how to use it properly: when I use state.messages[-1].content or flow.state.messages[-1].content or $flow.state.messages[-1].content I get an error saying that key messages doesn't exist. I tried also using a placeholder and setting the value in format prompt, as for the human prompt, but the placeholder is not replaced with the value, see the screenshot below. The issue is present whether the boolean is set to true or false, so I suspect this issue doesn't come from the proposed changes in this PR: is it possible to retrieve state messages/values from the Messages History code script ?

image

@HenryHengZJ
Copy link
Contributor

When testing locally, I encountered an error with the agent tool when the "Require Approval" toggle is set to true. However, this issue occurs regardless of whether "Disable Conversation History" is set to false or true, so I suspect the problem is unrelated to the changes in this PR. image

Feel free to check with the joint "Test flow Agents.json" agent workflow. Test flow Agents.json

Additionally, my existing LLM nodes/agent nodes are not updated automatically. I had to create new nodes for the "Disable Conversation History" parameter to appear under additional parameters. I'm unsure if this behavior is expected, but it happens regardless of whether I bump the component versions.

When testing locally, I encountered an error with the agent tool when the "Require Approval" toggle is set to true. However, this issue occurs regardless of whether "Disable Conversation History" is set to false or true, so I suspect the problem is unrelated to the changes in this PR. image

Feel free to check with the joint "Test flow Agents.json" agent workflow. Test flow Agents.json

Additionally, my existing LLM nodes/agent nodes are not updated automatically. I had to create new nodes for the "Disable Conversation History" parameter to appear under additional parameters. I'm unsure if this behavior is expected, but it happens regardless of whether I bump the component versions.

For approval tools, you need an agent memory attached

@jeanibarz
Copy link
Contributor Author

jeanibarz commented Dec 10, 2024 •
edited
Loading

When testing locally, I encountered an error with the agent tool when the "Require Approval" toggle is set to true. However, this issue occurs regardless of whether "Disable Conversation History" is set to false or true, so I suspect the problem is unrelated to the changes in this PR. image
Feel free to check with the joint "Test flow Agents.json" agent workflow. Test flow Agents.json
Additionally, my existing LLM nodes/agent nodes are not updated automatically. I had to create new nodes for the "Disable Conversation History" parameter to appear under additional parameters. I'm unsure if this behavior is expected, but it happens regardless of whether I bump the component versions.

When testing locally, I encountered an error with the agent tool when the "Require Approval" toggle is set to true. However, this issue occurs regardless of whether "Disable Conversation History" is set to false or true, so I suspect the problem is unrelated to the changes in this PR. image
Feel free to check with the joint "Test flow Agents.json" agent workflow. Test flow Agents.json
Additionally, my existing LLM nodes/agent nodes are not updated automatically. I had to create new nodes for the "Disable Conversation History" parameter to appear under additional parameters. I'm unsure if this behavior is expected, but it happens regardless of whether I bump the component versions.

For approval tools, you need an agent memory attached

I just tested with AgentMemory and it works when the required approval is true: thank you !

@jeanibarz
feat: add conversation history filtering options
98abc09 
Replace the disable conversation history feature with a more flexible filtering system that allows selecting:
- User question only
- Last message only
- All messages (default)
- No messages

This provides more granular control over conversation context management.
@jeanibarz
chore: break lines
c03107a
@jeanibarz
chore: removed ending semi columns
f629397
@jeanibarz jeanibarz changed the title feat: Add option to disable conversation history feat: Add options to filter conversation history messages used in sequential LLM and Agent nodes Dec 11, 2024
@jeanibarz
fix(sequentialagents): improve conversation history filtering logic
192c728 
- Remove unnecessary state.messages check for user_question case
- Add proper null handling for last_message and all_messages cases
- Remove @ts-ignore comments with proper typing
@HenryHengZJ
Copy link
Contributor

rearranging order of the parameters to correctly reflect how sys prompt, history, human prompt are being sent:

image

HenryHengZJ
HenryHengZJ approved these changes Dec 12, 2024
View reviewed changes
Copy link
Contributor

@HenryHengZJ HenryHengZJ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you so much @jeanibarz ! really good work

@jeanibarz
Copy link
Contributor Author

Thank you very much to you for your time and your patience ;-)

@HenryHengZJ HenryHengZJ merged commit 26b78ad into FlowiseAI:main Dec 12, 2024
2 checks passed
@jeanibarz jeanibarz mentioned this pull request Dec 16, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HenryHengZJ HenryHengZJ HenryHengZJ approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jeanibarz @saatchi-david @HenryHengZJ