Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

print exit feedback for user #882

Closed
wants to merge 2 commits into from

Conversation

PyroGenesis
Copy link
Contributor

@PyroGenesis PyroGenesis commented Dec 5, 2023

Why are these changes needed?

This would help inform the user when a conversation has ended and for what reason.

Related issue number

Closes #706

Termination scenarios

One-to-one conversation - agent returns termination msg

image

One-to-one conversation - agent returns None

image

One-to-one conversation - max consecutive auto-reply limit

image

One-to-one conversation - terminated by human

image

GroupChat conversation - agent returns termination msg

image

GroupChat conversation - agent returns None

image

GroupChat conversation - max rounds limit

image

GroupChat conversation - terminated by human

image

Checks

@codecov-commenter
Copy link

codecov-commenter commented Dec 5, 2023

Codecov Report

Attention: Patch coverage is 46.87500% with 17 lines in your changes missing coverage. Please review.

Project coverage is 38.14%. Comparing base (31c1664) to head (be975c4).
Report is 1996 commits behind head on main.

Files with missing lines Patch % Lines
autogen/agentchat/conversable_agent.py 52.38% 10 Missing ⚠️
autogen/agentchat/groupchat.py 36.36% 7 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #882       +/-   ##
===========================================
+ Coverage   25.24%   38.14%   +12.90%     
===========================================
  Files          28       28               
  Lines        3728     3751       +23     
  Branches      848      850        +2     
===========================================
+ Hits          941     1431      +490     
+ Misses       2710     2197      -513     
- Partials       77      123       +46     
Flag Coverage Δ
unittests 38.09% <46.87%> (+12.85%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@PyroGenesis
Copy link
Contributor Author

@microsoft-github-policy-service agree company="Personable Inc."

this would help inform the user when a conversation has ended and for what reason
handle the case where the default auto reply exits the conversation
@PyroGenesis PyroGenesis marked this pull request as ready for review December 7, 2023 18:03
@PyroGenesis
Copy link
Contributor Author

Is this a good way to implement this?

Another possible way I was thinking of was to have the exit reason propagate from the last agent. through the send / receive calls, back to the agent who initiated the conversation which would then print the exit reason. However, that would lead to changes in the send / receive method signatures of ConversableAgent (and Agent).

@sonichi
Copy link
Contributor

sonichi commented Dec 9, 2023

I think the termination reason message is only necessary when the termination is not triggered by a human input. But I'm not a HCI expert. So trying to get help from @afourney @gagb

@afourney
Copy link
Member

afourney commented Dec 13, 2023

This is an interesting PR. I think if we are providing the feedback for the benefit of the user/developer, then we don't necessarily need the manual termination message. BUT if we want to use this signal to manage any other automation, we likely do want it.

That is a good segue to my question: Is there simple way of adding a programmatic means of accessing this signal? E.g., can I subscribe to it somehow? Or, can I read from some status variable? I think this would be very useful for some orchestration scenarios.

Ideally initiate_chat would somehow return this almost like an exit code.

@gagb gagb removed their request for review March 8, 2024 23:55
@jtoy
Copy link
Contributor

jtoy commented May 17, 2024

@afourney I love the idea of having having signals or hooks to be able to subscribe to events, it would definitely make for a more powerful and robust system.

@PyroGenesis
Copy link
Contributor Author

PyroGenesis commented May 17, 2024

I see that there is an implementation of adding Termination reason to the chat history for groupchat in the experimental branch. That seems like a better approach and would satisfy the idea of programmatically processing the termination reason. Is that approach planned to be merged into the main branch, specifically for ConversableAgent? @jackgerrits

Copy link
Collaborator

@ekzhu ekzhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This is useful. As a next step we can put the exit reason in the chat result object returned by initiate_chat as well.

@ekzhu
Copy link
Collaborator

ekzhu commented May 22, 2024

Pls resolve the conflicts.

Copy link

gitguardian bot commented Jul 20, 2024

⚠️ GitGuardian has uncovered 96 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

Since your pull request originates from a forked repository, GitGuardian is not able to associate the secrets uncovered with secret incidents on your GitGuardian dashboard.
Skipping this check run and merging your pull request will create secret incidents on your GitGuardian dashboard.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
12853598 Triggered Generic High Entropy Secret 79dbb7b test/oai/test_utils.py View secret
10404693 Triggered Generic High Entropy Secret e43a86c test/oai/test_utils.py View secret
10404693 Triggered Generic High Entropy Secret bdb40d7 test/oai/test_utils.py View secret
10404693 Triggered Generic High Entropy Secret 954ca45 test/oai/test_utils.py View secret
10404693 Triggered Generic High Entropy Secret 79dbb7b test/oai/test_utils.py View secret
10404662 Triggered Generic CLI Secret eff19ac .github/workflows/dotnet-release.yml View secret
10404662 Triggered Generic CLI Secret 06a0a5d .github/workflows/dotnet-release.yml View secret
10404662 Triggered Generic CLI Secret 0524c77 .github/workflows/dotnet-release.yml View secret
10404662 Triggered Generic CLI Secret d7ea410 .github/workflows/dotnet-release.yml View secret
10404662 Triggered Generic CLI Secret e43a86c .github/workflows/dotnet-build.yml View secret
10404662 Triggered Generic CLI Secret 841ed31 .github/workflows/dotnet-release.yml View secret
10404662 Triggered Generic CLI Secret 802f099 .github/workflows/dotnet-release.yml View secret
10404662 Triggered Generic CLI Secret 9a484d8 .github/workflows/dotnet-build.yml View secret
10404662 Triggered Generic CLI Secret e973ac3 .github/workflows/dotnet-release.yml View secret
10404662 Triggered Generic CLI Secret 89650e7 .github/workflows/dotnet-release.yml View secret
10404662 Triggered Generic CLI Secret e07b06b .github/workflows/dotnet-release.yml View secret
10404662 Triggered Generic CLI Secret abe4c41 .github/workflows/dotnet-build.yml View secret
10404662 Triggered Generic CLI Secret 7362fb9 .github/workflows/dotnet-release.yml View secret
12853599 Triggered Generic High Entropy Secret 79dbb7b test/oai/test_utils.py View secret
10404694 Triggered Generic High Entropy Secret e43a86c test/oai/test_utils.py View secret
10404694 Triggered Generic High Entropy Secret 954ca45 test/oai/test_utils.py View secret
10404694 Triggered Generic High Entropy Secret bdb40d7 test/oai/test_utils.py View secret
10404695 Triggered Generic High Entropy Secret abad9ff test/oai/test_utils.py View secret
10404695 Triggered Generic High Entropy Secret 954ca45 test/oai/test_utils.py View secret
10404695 Triggered Generic High Entropy Secret c7bb588 test/oai/test_utils.py View secret
10404695 Triggered Generic High Entropy Secret b97b99d test/oai/test_utils.py View secret
10404695 Triggered Generic High Entropy Secret e43a86c test/oai/test_utils.py View secret
12853600 Triggered Generic High Entropy Secret 79dbb7b test/oai/test_utils.py View secret
12853601 Triggered Generic High Entropy Secret 79dbb7b test/oai/test_utils.py View secret
10493810 Triggered Generic Password 49e8053 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 501610b notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 49e8053 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 501610b notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password d422c63 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 97fa339 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 49e8053 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password d422c63 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 97fa339 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password d422c63 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 97fa339 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 501610b notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10404696 Triggered Generic High Entropy Secret 954ca45 test/oai/test_utils.py View secret
10404696 Triggered Generic High Entropy Secret bdb40d7 test/oai/test_utils.py View secret
10404696 Triggered Generic High Entropy Secret 79dbb7b test/oai/test_utils.py View secret
10404696 Triggered Generic High Entropy Secret e43a86c test/oai/test_utils.py View secret
10422482 Triggered Generic High Entropy Secret 79dbb7b test/oai/test_utils.py View secret
10422482 Triggered Generic High Entropy Secret bdb40d7 test/oai/test_utils.py View secret
12853602 Triggered Generic High Entropy Secret 79dbb7b test/oai/test_utils.py View secret
11616921 Triggered Generic High Entropy Secret a86d0fd notebook/agentchat_agentops.ipynb View secret
11616921 Triggered Generic High Entropy Secret 394561b notebook/agentchat_agentops.ipynb View secret
11616921 Triggered Generic High Entropy Secret 3eac646 notebook/agentchat_agentops.ipynb View secret
11616921 Triggered Generic High Entropy Secret f45b553 notebook/agentchat_agentops.ipynb View secret
11616921 Triggered Generic High Entropy Secret 6563248 notebook/agentchat_agentops.ipynb View secret
12853598 Triggered Generic High Entropy Secret 2b3a9ae test/oai/test_utils.py View secret
12853598 Triggered Generic High Entropy Secret c03558f test/oai/test_utils.py View secret
10404693 Triggered Generic High Entropy Secret c03558f test/oai/test_utils.py View secret
10404693 Triggered Generic High Entropy Secret 2b3a9ae test/oai/test_utils.py View secret
10404693 Triggered Generic High Entropy Secret 0a3c6c4 test/oai/test_utils.py View secret
10404693 Triggered Generic High Entropy Secret 76f5f5a test/oai/test_utils.py View secret
10404662 Triggered Generic CLI Secret 954ca45 .github/workflows/dotnet-build.yml View secret
12853599 Triggered Generic High Entropy Secret 2b3a9ae test/oai/test_utils.py View secret
12853599 Triggered Generic High Entropy Secret c03558f test/oai/test_utils.py View secret
10404694 Triggered Generic High Entropy Secret 76f5f5a test/oai/test_utils.py View secret
10404694 Triggered Generic High Entropy Secret 0a3c6c4 test/oai/test_utils.py View secret
10404695 Triggered Generic High Entropy Secret 3b79cc6 test/oai/test_utils.py View secret
10404695 Triggered Generic High Entropy Secret 11baa52 test/oai/test_utils.py View secret
12853600 Triggered Generic High Entropy Secret c03558f test/oai/test_utils.py View secret
12853600 Triggered Generic High Entropy Secret 2b3a9ae test/oai/test_utils.py View secret
12853601 Triggered Generic High Entropy Secret c03558f test/oai/test_utils.py View secret
12853601 Triggered Generic High Entropy Secret 2b3a9ae test/oai/test_utils.py View secret
10493810 Triggered Generic Password 3b79cc6 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 11baa52 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 11baa52 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10493810 Triggered Generic Password 3b79cc6 notebook/agentchat_pgvector_RetrieveChat.ipynb View secret
10404696 Triggered Generic High Entropy Secret 0a3c6c4 test/oai/test_utils.py View secret
10404696 Triggered Generic High Entropy Secret 76f5f5a test/oai/test_utils.py View secret
10404696 Triggered Generic High Entropy Secret c03558f test/oai/test_utils.py View secret
10404696 Triggered Generic High Entropy Secret 2b3a9ae test/oai/test_utils.py View secret
10422482 Triggered Generic High Entropy Secret 2b3a9ae test/oai/test_utils.py View secret
10422482 Triggered Generic High Entropy Secret c03558f test/oai/test_utils.py View secret

and 16 others.

🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@gagb
Copy link
Collaborator

gagb commented Aug 26, 2024

closing because of inactivity.

@gagb gagb closed this Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GroupChat: Feedback to the user when it finishes early
7 participants