Controlling select_speaker using Graph Modelling Language

[joshkyh]

I suggest @afourney and @thinkall review this work because of their recent contributions to incorporate modes of select_speaker in #688.

Why are these changes needed?

This notebook shows how we can use an expressive graph to control select_speaker. Instead of hard-coding complex logic to override select_speaker, this example shows how we can input a directed graph that dictates the transition paths between agents. If desired, we can also extend GroupChat select_speaker mode to "graph" in the near future (extending the work in #688) in another PR. I believe this is a necessary step towards having more agents working together, since the number of transition pairs increases rapidly (N Choose 2).

Related issue number

#485
#438 in terms of dividing the problem.

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 47.08%. Comparing base (f6498dc) to head (e3f0af0).
Report is 1604 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #720       +/-   ##
===========================================
+ Coverage   27.89%   47.08%   +19.18%     
===========================================
  Files          27       27               
  Lines        3466     3466               
  Branches      784      826       +42     
===========================================
+ Hits          967     1632      +665     
+ Misses       2428     1674      -754     
- Partials       71      160       +89     

Flag	Coverage Δ
unittests	46.85% <ø> (+19.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[afourney]

Thanks. This looks really compelling. I need to dig into it more.

One thing I noticed is that it selects the next speaker randomly from among the candidate outgoing edges (or did I misread that?). I would love to see this instead use some variation of the current GroupChat prompt (or similar) to select the next speaker based on the conversations.

[joshkyh]

Thanks. This looks really compelling. I need to dig into it more.

One thing I noticed is that it selects the next speaker randomly from among the candidate outgoing edges (or did I misread that?). I would love to see this instead use some variation of the current GroupChat prompt (or similar) to select the next speaker based on the conversations.

Hi @afourney,

Glad you find it useful. A partial yes, if suggested_next by the previous speaker is a legal transition, then that transition would happen. suggested_next is actually not random, it is suggested by the previous speaker.

However, if it is not a legal transition, we would pick a random agent from the legal transitions eligible_speakers as a catch-all. Yes, we can potentially insert the use of an LLM to pick an agent from the eligible_speakers before the catch-all.

Please feel free to review when you get some time. I'll act on the feedback tomorrow. Thanks for the review! :)

[BeibinLi]

Fantastic idea! I have thought about something similar earlier, but your implementation is much cleaner. I will take a deeper look later.

[thinkall]

Thanks @joshkyh , I love it. Allowing selecting speaker based on the complex network would push the group chat to a higher level, enabling a more meaningful large group chat.

Do you only want to show it as a notebook example? I believe it worth to add into the code.

[joshkyh]

Would you like to add a link to this notebook in the website? There are two pages to update.

Thanks for the review, @sonichi. I have acted on the four feedback points and added the link to this notebook in two pages.

[Friedensfreak]

Hi @joshkyh what is secret for each agent in the graph modelling. And how is this useful?

[joshkyh]

Hey @Friedensfreak, thanks for the question.

In the notebook, the task was to find the total number of chocolates every agent have, which is denoted by secret. The example illustrates every agent contributing towards the problem but inter-team transitions only happen via the team leaders. Outside of this example, there is no need to use the secret attribute.

[Morriz]

will this be codified? Sharing code override examples is not the way forward imo.

[sonichi]

See #743 and related PRs.