Added an agent description field distinct from the system_message.

[afourney]

Why are these changes needed?

GroupChat and other orchestrators currently rely on the system_message to determine what role each agent serves. However, system_messages are occasionally long and detailed (AssistantAgent), or are completely missing (UserProxyAgent). The messages are often also written in the wrong perspective #319. This can lead to all kinds of orchestration problems, with the GorupChatManager being no better than Round Robin or even Random in some cases #688.

This PR addresses these issues by adding a "description" field that defaults to the system_message (for backward compatibility), but can diverge. This description message is then used for GroupChat.

Related issue number

#319 and general orchestration issues.

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[codecov-commenter]

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (8ea6377) 26.60% compared to head (48c8196) 48.39%.

Files	Patch %	Lines
autogen/agentchat/assistant_agent.py	75.00%	0 Missing and 1 partial ⚠️
autogen/agentchat/groupchat.py	83.33%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #736       +/-   ##
===========================================
+ Coverage   26.60%   48.39%   +21.79%     
===========================================
  Files          28       28               
  Lines        3733     3742        +9     
  Branches      847      891       +44     
===========================================
+ Hits          993     1811      +818     
+ Misses       2667     1747      -920     
- Partials       73      184      +111     

Flag	Coverage Δ
unittests	48.20% <83.33%> (+21.66%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[afourney]

Figured I'd add the advice that we talked about offline here. Certainly the agent description will help a lot here since it gives a more accurate representation of what the selector agent cares about for groupchat.

However in order to really have it make sensical decisions, the best approach that we've found is to actually force the selector to output a json with 2 fields: selection and reasoning. Selection being the agent to speak next and reasoning being a reasoning phrase that justifies the selection. Adding this to one of our custom group chat classes has made the group chat speaker selection much more logical.

I 100% agree, and I had/will have another PR to introduce changes to the GroupChatManager that do just that. But again, implementing that (and all other variants), depends on having a description field that can be used. So let's get this in, and then we can propose additional or modified Managers in another PR.

For what it's worth, this is a prompt that I found to be very useful (but again, it needs the description field):

autogen/autogen/agentchat/groupchat2.py

Lines 96 to 116 in 4094367

	def select_speaker_msg(self, agents: List[Agent]):
	"""Return the system message for selecting the next speaker. This is always the *first* message in the context."""
	return f"""You are moderating a conversation between {len(self.agents)} participants who are working together to answer questions and perform tasks. Your role is as a moderator. DON'T DIRECTLY ANSWER THE QUESTIONS OR PERFORM ANY OF THE WORK YOURSELF. IN PARTICULAR, DO NOT WRITE ANY CODE YOURSELF. INSTEAD, DIRECT THE PARTICIPANTS TO DO SO, AS APPROPRIATE. In attendance are the following participants:
	{self._participant_roles(agents)}
	Read the following conversation, then carefully consider who you should speak to next, and what you should ask of them, so as to make the most progress on the task). Speakers do not need equal speaking time. You may even ignore non-relevant participants. Your focus is on efficiently driving progress toward task completion.
	After each participant response, decide the following:
	- WHO should speak next? (A valid participant name, selected from this list: {[agent.name for agent in agents]})
	- WHAT should you ask of them? (phrased the way you would actually ask them in conversation)
	- WHY it makes sense to ask them at this moment (your internal reasoning)
	Your output should be a perfect JSON object as per below:
	{{
	"why": your_reasoning,
	"who": participant_name,
	"what": your_question_or_request
	}}
	DO NOT OUTPUT ANYTHING OTHER THAN THIS JSON OBJECT. YOUR OUTPUT MUST BE PARSABLE AS JSON.
	"""

And,

autogen/autogen/agentchat/groupchat2.py

Lines 118 to 141 in 4094367

	def select_speaker_prompt(self, agents: List[Agent], excluded_agent: Optional[Union[Agent, None]] = None):
	"""Return the floating system prompt selecting the next speaker. This is always the *last* message in the context."""
	exclude_speaker_msg = ""
	if excluded_agent is not None:
	exclude_speaker_msg = f"\nNote: Don't ask {excluded_agent.name} again, since they just spoke. Instead ask {' or '.join([agent.name for agent in agents])}."
	return f"""Remember, YOUR role is to serve as a moderator. DON'T ANSWER QUESTIONS, CODE, OR PERFORM OTHER WORK YOURSELF. Instead, read the above conversation, then carefully decide the following, with a focus on making progress on the task:
	- WHO should speak next? (A valid participant name, selected from this list: {[agent.name for agent in agents]})
	- WHAT should you ask of them? (phrased the way you would actually ask them in conversation)
	- WHY it makes sense to ask them at this moment (your internal reasoning)
	{exclude_speaker_msg}
	Your output should be a perfect JSON object as per below:
	{{
	"why": your_reasoning,
	"who": participant_name,
	"what": your_question_or_request
	}}
	DO NOT OUTPUT ANYTHING OTHER THAN THIS JSON OBJECT. YOUR OUTPUT MUST BE PARSABLE AS JSON.
	"""

[radman-x]

Figured I'd add the advice that we talked about offline here. Certainly the agent description will help a lot here since it gives a more accurate representation of what the selector agent cares about for groupchat.
However in order to really have it make sensical decisions, the best approach that we've found is to actually force the selector to output a json with 2 fields: selection and reasoning. Selection being the agent to speak next and reasoning being a reasoning phrase that justifies the selection. Adding this to one of our custom group chat classes has made the group chat speaker selection much more logical.

I 100% agree, and I had/will have another PR to introduce changes to the GroupChatManager that do just that. But again, implementing that (and all other variants), depends on having a description field that can be used. So let's get this in, and then we can propose additional or modified Managers in another PR.

For what it's worth, this is a prompt that I found to be very useful (but again, it needs the description field):

autogen/autogen/agentchat/groupchat2.py

Lines 96 to 116 in 4094367

	def select_speaker_msg(self, agents: List[Agent]):
	"""Return the system message for selecting the next speaker. This is always the *first* message in the context."""
	return f"""You are moderating a conversation between {len(self.agents)} participants who are working together to answer questions and perform tasks. Your role is as a moderator. DON'T DIRECTLY ANSWER THE QUESTIONS OR PERFORM ANY OF THE WORK YOURSELF. IN PARTICULAR, DO NOT WRITE ANY CODE YOURSELF. INSTEAD, DIRECT THE PARTICIPANTS TO DO SO, AS APPROPRIATE. In attendance are the following participants:
	{self._participant_roles(agents)}
	Read the following conversation, then carefully consider who you should speak to next, and what you should ask of them, so as to make the most progress on the task). Speakers do not need equal speaking time. You may even ignore non-relevant participants. Your focus is on efficiently driving progress toward task completion.
	After each participant response, decide the following:
	- WHO should speak next? (A valid participant name, selected from this list: {[agent.name for agent in agents]})
	- WHAT should you ask of them? (phrased the way you would actually ask them in conversation)
	- WHY it makes sense to ask them at this moment (your internal reasoning)
	Your output should be a perfect JSON object as per below:
	{{
	"why": your_reasoning,
	"who": participant_name,
	"what": your_question_or_request
	}}
	DO NOT OUTPUT ANYTHING OTHER THAN THIS JSON OBJECT. YOUR OUTPUT MUST BE PARSABLE AS JSON.
	"""

And,

autogen/autogen/agentchat/groupchat2.py

Lines 118 to 141 in 4094367

	def select_speaker_prompt(self, agents: List[Agent], excluded_agent: Optional[Union[Agent, None]] = None):
	"""Return the floating system prompt selecting the next speaker. This is always the *last* message in the context."""
	exclude_speaker_msg = ""
	if excluded_agent is not None:
	exclude_speaker_msg = f"\nNote: Don't ask {excluded_agent.name} again, since they just spoke. Instead ask {' or '.join([agent.name for agent in agents])}."
	return f"""Remember, YOUR role is to serve as a moderator. DON'T ANSWER QUESTIONS, CODE, OR PERFORM OTHER WORK YOURSELF. Instead, read the above conversation, then carefully decide the following, with a focus on making progress on the task:
	- WHO should speak next? (A valid participant name, selected from this list: {[agent.name for agent in agents]})
	- WHAT should you ask of them? (phrased the way you would actually ask them in conversation)
	- WHY it makes sense to ask them at this moment (your internal reasoning)
	{exclude_speaker_msg}
	Your output should be a perfect JSON object as per below:
	{{
	"why": your_reasoning,
	"who": participant_name,
	"what": your_question_or_request
	}}
	DO NOT OUTPUT ANYTHING OTHER THAN THIS JSON OBJECT. YOUR OUTPUT MUST BE PARSABLE AS JSON.
	"""

@afourney Have you started a PR to merge in your groupchat2 into the latest codebase? If so, what #? I have used it a bit and it works much better than the current groupchat so looking forward to merge.