Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GroupChat handle is_termination_msg #804

Merged
merged 2 commits into from
Nov 29, 2023
Merged

Conversation

afourney
Copy link
Member

Why are these changes needed?

Setting the is_termination_msg function handler in the GroupChatManager has no effect. This is because the GroupChatManager simply forwards the message to the next agent, and it falls on those agents to stop the conversation. This is problematic as it wastes orchestration LLM calls when the conversation is over, and because it prevents the ability to use the GroupChatManager as a central place to control group chat termination.

This PR fixes the issue by checking the termination message before rebroadcasting.

Related issue number

Closes #802

Checks

@codecov-commenter
Copy link

codecov-commenter commented Nov 29, 2023

Codecov Report

Attention: Patch coverage is 50.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 46.29%. Comparing base (f96963e) to head (1272a69).
Report is 1574 commits behind head on main.

Files with missing lines Patch % Lines
autogen/agentchat/groupchat.py 50.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #804       +/-   ##
===========================================
+ Coverage   27.76%   46.29%   +18.52%     
===========================================
  Files          27       27               
  Lines        3493     3497        +4     
  Branches      791      835       +44     
===========================================
+ Hits          970     1619      +649     
+ Misses       2452     1713      -739     
- Partials       71      165       +94     
Flag Coverage Δ
unittests 46.06% <50.00%> (+18.35%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@victordibia victordibia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this @afourney .
The justifications here makes sense

  • enabling better conversation control via the GroupChatManager
  • better mental model on GroupChat behavior .. its easier to reason about setting termination logic via the central groupchatmanager than a specific agent or all agents
  • also improves efficiency when the groupchatmanger can terminate conversations early/ correctly.

@afourney Any chance may be side effects here e.g. GroupChat notebooks where some results might benefit from multiple back and forths?

Approving in the mean time.

@afourney
Copy link
Member Author

afourney commented Nov 29, 2023

There should not be any side effects. The superclass default for is_termination_msg is None. In that case, this new PR acts identically to the published version. It's only when setting is_termination_msg to something other than None that a change in behavior occurs. If other notebooks were setting is_termination_msg for the GroupChatManager then this is actually a problem for those notebooks, because they were propagating the false belief that the is_terminiation_msg would be considered. In those cases, the revised behavior will indeed differ (and will likely more closely align to the author's intent). However, I'd argue than any such notebooks are already broken without this PR.

@sonichi sonichi added this pull request to the merge queue Nov 29, 2023
Merged via the queue into main with commit 8d5f176 Nov 29, 2023
67 of 71 checks passed
@sonichi sonichi deleted the group_chat_termination_msg branch December 3, 2023 18:13
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
* Have GroupChatManager check is_termination_msg

* Added test cases.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
group chat/teams group-chat-related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GroupChatManager ignores is_termination_msg
4 participants