diff --git a/.github/workflows/dotnet-build.yml b/.github/workflows/dotnet-build.yml index 1b53d0bde88f..6aac54d3818c 100644 --- a/.github/workflows/dotnet-build.yml +++ b/.github/workflows/dotnet-build.yml @@ -51,9 +51,10 @@ jobs: strategy: fail-fast: false matrix: - os: [ ubuntu-latest, macos-latest, windows-latest ] + os: [ ubuntu-latest, macos-latest ] python-version: ["3.11"] runs-on: ${{ matrix.os }} + timeout-minutes: 30 steps: - uses: actions/checkout@v4 with: diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md new file mode 100644 index 000000000000..4a7a3a7e4fab --- /dev/null +++ b/CONTRIBUTORS.md @@ -0,0 +1,38 @@ +# Contributors + +## Special thanks to all the people who help this project: +> These individuals dedicate their time and expertise to improve this project. We are deeply grateful for their contributions. + +| Name | GitHub Handle | Organization | Features | Roadmap Lead | Additional Information | +|---|---|---|---|---|---| +| Qingyun Wu | [qingyun-wu](https://github.com/qingyun-wu) | Penn State University | all, alt-models, autobuilder | Yes | Available most of the time (US Eastern Time) | +| Chi Wang | [sonichi](https://github.com/sonichi) | - | all | Yes | | +| Li Jiang | [thinkall](https://github.com/thinkall) | Microsoft | rag, autobuilder, group chat | Yes | [Issue #1657](https://github.com/microsoft/autogen/issues/1657) - Beijing, GMT+8 | +| Mark Sze | [marklysze](https://github.com/marklysze) | - | alt-models, group chat | No | Generally available (Sydney, AU time) - Group Chat "auto" speaker selection | +| Hrushikesh Dokala | [Hk669](https://github.com/Hk669) | - | alt-models, swebench, logging, rag | No | [Issue #2946](https://github.com/microsoft/autogen/issues/2946), [Pull Request #2933](https://github.com/microsoft/autogen/pull/2933) - Available most of the time (India, GMT+5:30) | +| Jiale Liu | [LeoLjl](https://github.com/LeoLjl) | Penn State University | autobuild, group chat | No | | +| Shaokun Zhang | [skzhang1](https://github.com/skzhang1) | Penn State University | AgentOptimizer, Teachability | Yes | [Issue #521](https://github.com/microsoft/autogen/issues/521) | +| Rajan Chari | [rajan-chari](https://github.com/rajan-chari) | Microsoft Research | CAP, Survey of other frameworks | No | | +| Victor Dibia | [victordibia](https://github.com/victordibia) | Microsoft Research | autogenstudio | Yes | [Issue #737](https://github.com/microsoft/autogen/issues/737) | +| Yixuan Zhai | [randombet](https://github.com/randombet) | Meta | group chat, sequential_chats, rag | No | | +| Xiaoyun Zhang | [LittleLittleCloud](https://github.com/LittleLittleCloud) | Microsoft | AutoGen.Net, group chat | Yes | [Backlog - AutoGen.Net](https://github.com/microsoft/autogen/issues) - Available most of the time (PST) | +| Yiran Wu | [yiranwu0](https://github.com/yiranwu0) | Penn State University | alt-models, group chat, logging | Yes | | +| Beibin Li | [BeibinLi](https://github.com/BeibinLi) | Microsoft Research | alt-models | Yes | | +| Gagan Bansal | [gagb](https://github.com/gagb) | Microsoft Research | Complex Tasks | | | +| Adam Fourney | [afourney](https://github.com/afourney) | Microsoft Research | Complex Tasks | | | +| Ricky Loynd | [rickyloynd-microsoft](https://github.com/rickyloynd-microsoft) | Microsoft Research | Teachability | | | +| Eric Zhu | [ekzhu](https://github.com/ekzhu) | Microsoft Research | Infra | | | +| Jack Gerrits | [jackgerrits](https://github.com/jackgerrits) | Microsoft Research | Infra | | | +| David Luong | [David Luong](https://github.com/DavidLuong98) | Microsoft | AutoGen.Net | | | + + +## I would like to join this list. How can I help the project? +> We're always looking for new contributors to join our team and help improve the project. For more information, please refer to our [CONTRIBUTING](https://microsoft.github.io/autogen/docs/contributor-guide/contributing) guide. + + +## Are you missing from this list? +> Please open a PR to help us fix this. + + +## Acknowledgements +This template was adapted from [GitHub Template Guide](https://github.com/cezaraugusto/github-template-guidelines/blob/master/.github/CONTRIBUTORS.md) by [cezaraugusto](https://github.com/cezaraugusto). diff --git a/README.md b/README.md index 7c7ac4b85c59..12ffb390d048 100644 --- a/README.md +++ b/README.md @@ -68,10 +68,7 @@ AutoGen is an open-source programming framework for building AI agents and facilitating cooperation among multiple agents to solve tasks. AutoGen aims to streamline the development and research of agentic AI, much like PyTorch does for Deep Learning. It offers features such as agents capable of interacting with each other, facilitates the use of various large language models (LLMs) and tool use support, autonomous and human-in-the-loop workflows, and multi-agent conversation patterns. -**Open Source Statement**: The project welcomes contributions from developers and organizations worldwide. Our goal is to foster a collaborative and inclusive community where diverse perspectives and expertise can drive innovation and enhance the project's capabilities. Whether you are an individual contributor or represent an organization, we invite you to join us in shaping the future of this project. Together, we can build something truly remarkable. - -The project is currently maintained by a [dynamic group of volunteers](https://butternut-swordtail-8a5.notion.site/410675be605442d3ada9a42eb4dfef30?v=fa5d0a79fd3d4c0f9c112951b2831cbb&pvs=4) from several different organizations. Contact project administrators Chi Wang and Qingyun Wu via auto-gen@outlook.com if you are interested in becoming a maintainer. - +We welcome contributions from developers and organizations worldwide. Our goal is to foster a collaborative and inclusive community where diverse perspectives and expertise can drive innovation and enhance the project's capabilities. We acknowledge the invaluable contributions from our existing contributors, as listed in [contributors.md](./CONTRIBUTORS.md). Whether you are an individual contributor or represent an organization, we invite you to join us in shaping the future of this project. For further information please also see [Microsoft open-source contributing guidelines](https://github.com/microsoft/autogen?tab=readme-ov-file#contributing). ![AutoGen Overview](https://github.com/microsoft/autogen/blob/main/website/static/img/autogen_agentchat.png) @@ -247,7 +244,7 @@ In addition, you can find: ## Related Papers -[AutoGen](https://arxiv.org/abs/2308.08155) +[AutoGen](https://aka.ms/autogen-pdf) ``` @inproceedings{wu2023autogen, diff --git a/autogen/agentchat/conversable_agent.py b/autogen/agentchat/conversable_agent.py index 9254ef57de08..eabe6d6d4606 100644 --- a/autogen/agentchat/conversable_agent.py +++ b/autogen/agentchat/conversable_agent.py @@ -621,7 +621,7 @@ def _assert_valid_name(name): raise ValueError(f"Invalid name: {name}. Name must be less than 64 characters.") return name - def _append_oai_message(self, message: Union[Dict, str], role, conversation_id: Agent) -> bool: + def _append_oai_message(self, message: Union[Dict, str], role, conversation_id: Agent, is_sending: bool) -> bool: """Append a message to the ChatCompletion conversation. If the message received is a string, it will be put in the "content" field of the new dictionary. @@ -633,6 +633,7 @@ def _append_oai_message(self, message: Union[Dict, str], role, conversation_id: message (dict or str): message to be appended to the ChatCompletion conversation. role (str): role of the message, can be "assistant" or "function". conversation_id (Agent): id of the conversation, should be the recipient or sender. + is_sending (bool): If the agent (aka self) is sending to the conversation_id agent, otherwise receiving. Returns: bool: whether the message is appended to the ChatCompletion conversation. @@ -662,7 +663,15 @@ def _append_oai_message(self, message: Union[Dict, str], role, conversation_id: if oai_message.get("function_call", False) or oai_message.get("tool_calls", False): oai_message["role"] = "assistant" # only messages with role 'assistant' can have a function call. + elif "name" not in oai_message: + # If we don't have a name field, append it + if is_sending: + oai_message["name"] = self.name + else: + oai_message["name"] = conversation_id.name + self._oai_messages[conversation_id].append(oai_message) + return True def _process_message_before_send( @@ -718,7 +727,7 @@ def send( message = self._process_message_before_send(message, recipient, ConversableAgent._is_silent(self, silent)) # When the agent composes and sends the message, the role of the message is "assistant" # unless it's "function". - valid = self._append_oai_message(message, "assistant", recipient) + valid = self._append_oai_message(message, "assistant", recipient, is_sending=True) if valid: recipient.receive(message, self, request_reply, silent) else: @@ -768,7 +777,7 @@ async def a_send( message = self._process_message_before_send(message, recipient, ConversableAgent._is_silent(self, silent)) # When the agent composes and sends the message, the role of the message is "assistant" # unless it's "function". - valid = self._append_oai_message(message, "assistant", recipient) + valid = self._append_oai_message(message, "assistant", recipient, is_sending=True) if valid: await recipient.a_receive(message, self, request_reply, silent) else: @@ -839,7 +848,7 @@ def _print_received_message(self, message: Union[Dict, str], sender: Agent): def _process_received_message(self, message: Union[Dict, str], sender: Agent, silent: bool): # When the agent receives a message, the role of the message is "user". (If 'role' exists and is 'function', it will remain unchanged.) - valid = self._append_oai_message(message, "user", sender) + valid = self._append_oai_message(message, "user", sender, is_sending=False) if logging_enabled(): log_event(self, "received_message", message=message, sender=sender.name, valid=valid) diff --git a/autogen/agentchat/groupchat.py b/autogen/agentchat/groupchat.py index bcb8c30e2b6a..53fc32e58d21 100644 --- a/autogen/agentchat/groupchat.py +++ b/autogen/agentchat/groupchat.py @@ -649,6 +649,7 @@ def validate_speaker_name(recipient, messages, sender, config) -> Tuple[bool, Un if self.select_speaker_prompt_template is not None: start_message = { "content": self.select_speaker_prompt(agents), + "name": "checking_agent", "override_role": self.role_for_select_speaker_messages, } else: @@ -813,6 +814,7 @@ def _validate_speaker_name( return True, { "content": self.select_speaker_auto_multiple_template.format(agentlist=agentlist), + "name": "checking_agent", "override_role": self.role_for_select_speaker_messages, } else: @@ -842,6 +844,7 @@ def _validate_speaker_name( return True, { "content": self.select_speaker_auto_none_template.format(agentlist=agentlist), + "name": "checking_agent", "override_role": self.role_for_select_speaker_messages, } else: @@ -1050,6 +1053,7 @@ def print_messages(recipient, messages, sender, config): groupchat_result = agent_a.initiate_chat( chat_manager, message="Hi, there, I'm agent A." ) + ``` """ return self._last_speaker diff --git a/autogen/oai/mistral.py b/autogen/oai/mistral.py index 8017e3536324..10d0f926ffbf 100644 --- a/autogen/oai/mistral.py +++ b/autogen/oai/mistral.py @@ -15,28 +15,32 @@ Resources: - https://docs.mistral.ai/getting-started/quickstart/ -""" -# Important notes when using the Mistral.AI API: -# The first system message can greatly affect whether the model returns a tool call, including text that references the ability to use functions will help. -# Changing the role on the first system message to 'user' improved the chances of the model recommending a tool call. +NOTE: Requires mistralai package version >= 1.0.1 +""" import inspect import json import os import time import warnings -from typing import Any, Dict, List, Tuple, Union +from typing import Any, Dict, List, Union # Mistral libraries # pip install mistralai -from mistralai.client import MistralClient -from mistralai.exceptions import MistralAPIException -from mistralai.models.chat_completion import ChatCompletionResponse, ChatMessage, ToolCall +from mistralai import ( + AssistantMessage, + Function, + FunctionCall, + Mistral, + SystemMessage, + ToolCall, + ToolMessage, + UserMessage, +) from openai.types.chat import ChatCompletion, ChatCompletionMessageToolCall from openai.types.chat.chat_completion import ChatCompletionMessage, Choice from openai.types.completion_usage import CompletionUsage -from typing_extensions import Annotated from autogen.oai.client_utils import should_hide_tools, validate_parameter @@ -50,6 +54,7 @@ def __init__(self, **kwargs): Args: api_key (str): The API key for using Mistral.AI (or environment variable MISTRAL_API_KEY needs to be set) """ + # Ensure we have the api_key upon instantiation self.api_key = kwargs.get("api_key", None) if not self.api_key: @@ -59,7 +64,9 @@ def __init__(self, **kwargs): self.api_key ), "Please specify the 'api_key' in your config list entry for Mistral or set the MISTRAL_API_KEY env variable." - def message_retrieval(self, response: ChatCompletionResponse) -> Union[List[str], List[ChatCompletionMessage]]: + self._client = Mistral(api_key=self.api_key) + + def message_retrieval(self, response: ChatCompletion) -> Union[List[str], List[ChatCompletionMessage]]: """Retrieve the messages from the response.""" return [choice.message for choice in response.choices] @@ -86,34 +93,52 @@ def parse_params(self, params: Dict[str, Any]) -> Dict[str, Any]: ) mistral_params["random_seed"] = validate_parameter(params, "random_seed", int, True, None, False, None) + # TODO + if params.get("stream", False): + warnings.warn( + "Streaming is not currently supported, streaming will be disabled.", + UserWarning, + ) + # 3. Convert messages to Mistral format mistral_messages = [] tool_call_ids = {} # tool call ids to function name mapping for message in params["messages"]: if message["role"] == "assistant" and "tool_calls" in message and message["tool_calls"] is not None: # Convert OAI ToolCall to Mistral ToolCall - openai_toolcalls = message["tool_calls"] - mistral_toolcalls = [] - for toolcall in openai_toolcalls: - mistral_toolcall = ToolCall(id=toolcall["id"], function=toolcall["function"]) - mistral_toolcalls.append(mistral_toolcall) - mistral_messages.append( - ChatMessage(role=message["role"], content=message["content"], tool_calls=mistral_toolcalls) - ) + mistral_messages_tools = [] + for toolcall in message["tool_calls"]: + mistral_messages_tools.append( + ToolCall( + id=toolcall["id"], + function=FunctionCall( + name=toolcall["function"]["name"], + arguments=json.loads(toolcall["function"]["arguments"]), + ), + ) + ) + + mistral_messages.append(AssistantMessage(content="", tool_calls=mistral_messages_tools)) # Map tool call id to the function name for tool_call in message["tool_calls"]: tool_call_ids[tool_call["id"]] = tool_call["function"]["name"] - elif message["role"] in ("system", "user", "assistant"): - # Note this ChatMessage can take a 'name' but it is rejected by the Mistral API if not role=tool, so, no, the 'name' field is not used. - mistral_messages.append(ChatMessage(role=message["role"], content=message["content"])) + elif message["role"] == "system": + if len(mistral_messages) > 0 and mistral_messages[-1].role == "assistant": + # System messages can't appear after an Assistant message, so use a UserMessage + mistral_messages.append(UserMessage(content=message["content"])) + else: + mistral_messages.append(SystemMessage(content=message["content"])) + elif message["role"] == "assistant": + mistral_messages.append(AssistantMessage(content=message["content"])) + elif message["role"] == "user": + mistral_messages.append(UserMessage(content=message["content"])) elif message["role"] == "tool": # Indicates the result of a tool call, the name is the function name called mistral_messages.append( - ChatMessage( - role="tool", + ToolMessage( name=tool_call_ids[message["tool_call_id"]], content=message["content"], tool_call_id=message["tool_call_id"], @@ -122,21 +147,20 @@ def parse_params(self, params: Dict[str, Any]) -> Dict[str, Any]: else: warnings.warn(f"Unknown message role {message['role']}", UserWarning) - # If a 'system' message follows an 'assistant' message, change it to 'user' - # This can occur when using LLM summarisation - for i in range(1, len(mistral_messages)): - if mistral_messages[i - 1].role == "assistant" and mistral_messages[i].role == "system": - mistral_messages[i].role = "user" + # 4. Last message needs to be user or tool, if not, add a "please continue" message + if not isinstance(mistral_messages[-1], UserMessage) and not isinstance(mistral_messages[-1], ToolMessage): + mistral_messages.append(UserMessage(content="Please continue.")) mistral_params["messages"] = mistral_messages - # 4. Add tools to the call if we have them and aren't hiding them + # 5. Add tools to the call if we have them and aren't hiding them if "tools" in params: hide_tools = validate_parameter( params, "hide_tools", str, False, "never", None, ["if_all_run", "if_any_run", "never"] ) if not should_hide_tools(params["messages"], params["tools"], hide_tools): - mistral_params["tools"] = params["tools"] + mistral_params["tools"] = tool_def_to_mistral(params["tools"]) + return mistral_params def create(self, params: Dict[str, Any]) -> ChatCompletion: @@ -144,8 +168,7 @@ def create(self, params: Dict[str, Any]) -> ChatCompletion: mistral_params = self.parse_params(params) # 2. Call Mistral.AI API - client = MistralClient(api_key=self.api_key) - mistral_response = client.chat(**mistral_params) + mistral_response = self._client.chat.complete(**mistral_params) # TODO: Handle streaming # 3. Convert Mistral response to OAI compatible format @@ -191,7 +214,7 @@ def create(self, params: Dict[str, Any]) -> ChatCompletion: return response_oai @staticmethod - def get_usage(response: ChatCompletionResponse) -> Dict: + def get_usage(response: ChatCompletion) -> Dict: return { "prompt_tokens": response.usage.prompt_tokens if response.usage is not None else 0, "completion_tokens": response.usage.completion_tokens if response.usage is not None else 0, @@ -203,25 +226,48 @@ def get_usage(response: ChatCompletionResponse) -> Dict: } +def tool_def_to_mistral(tool_definitions: List[Dict[str, Any]]) -> List[Dict[str, Any]]: + """Converts AutoGen tool definition to a mistral tool format""" + + mistral_tools = [] + + for autogen_tool in tool_definitions: + mistral_tool = { + "type": "function", + "function": Function( + name=autogen_tool["function"]["name"], + description=autogen_tool["function"]["description"], + parameters=autogen_tool["function"]["parameters"], + ), + } + + mistral_tools.append(mistral_tool) + + return mistral_tools + + def calculate_mistral_cost(input_tokens: int, output_tokens: int, model_name: str) -> float: """Calculate the cost of the mistral response.""" - # Prices per 1 million tokens + # Prices per 1 thousand tokens # https://mistral.ai/technology/ model_cost_map = { - "open-mistral-7b": {"input": 0.25, "output": 0.25}, - "open-mixtral-8x7b": {"input": 0.7, "output": 0.7}, - "open-mixtral-8x22b": {"input": 2.0, "output": 6.0}, - "mistral-small-latest": {"input": 1.0, "output": 3.0}, - "mistral-medium-latest": {"input": 2.7, "output": 8.1}, - "mistral-large-latest": {"input": 4.0, "output": 12.0}, + "open-mistral-7b": {"input": 0.00025, "output": 0.00025}, + "open-mixtral-8x7b": {"input": 0.0007, "output": 0.0007}, + "open-mixtral-8x22b": {"input": 0.002, "output": 0.006}, + "mistral-small-latest": {"input": 0.001, "output": 0.003}, + "mistral-medium-latest": {"input": 0.00275, "output": 0.0081}, + "mistral-large-latest": {"input": 0.0003, "output": 0.0003}, + "mistral-large-2407": {"input": 0.0003, "output": 0.0003}, + "open-mistral-nemo-2407": {"input": 0.0003, "output": 0.0003}, + "codestral-2405": {"input": 0.001, "output": 0.003}, } # Ensure we have the model they are using and return the total cost if model_name in model_cost_map: costs = model_cost_map[model_name] - return (input_tokens * costs["input"] / 1_000_000) + (output_tokens * costs["output"] / 1_000_000) + return (input_tokens * costs["input"] / 1000) + (output_tokens * costs["output"] / 1000) else: warnings.warn(f"Cost calculation is not implemented for model {model_name}, will return $0.", UserWarning) return 0 diff --git a/dotnet/eng/MetaInfo.props b/dotnet/eng/MetaInfo.props index 72918fabe4f4..006c586faba5 100644 --- a/dotnet/eng/MetaInfo.props +++ b/dotnet/eng/MetaInfo.props @@ -1,7 +1,7 @@ - 0.0.17 + 0.1.0 AutoGen https://microsoft.github.io/autogen-for-net/ https://github.com/microsoft/autogen diff --git a/dotnet/sample/AutoGen.Anthropic.Samples/Anthropic_Agent_With_Prompt_Caching.cs b/dotnet/sample/AutoGen.Anthropic.Samples/Anthropic_Agent_With_Prompt_Caching.cs new file mode 100644 index 000000000000..5d8a99ce1288 --- /dev/null +++ b/dotnet/sample/AutoGen.Anthropic.Samples/Anthropic_Agent_With_Prompt_Caching.cs @@ -0,0 +1,133 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Anthropic_Agent_With_Prompt_Caching.cs + +using AutoGen.Anthropic.DTO; +using AutoGen.Anthropic.Extensions; +using AutoGen.Anthropic.Utils; +using AutoGen.Core; + +namespace AutoGen.Anthropic.Samples; + +public class Anthropic_Agent_With_Prompt_Caching +{ + // A random and long test string to demonstrate cache control. + // the context must be larger than 1024 tokens for Claude 3.5 Sonnet and Claude 3 Opus + // 2048 tokens for Claude 3.0 Haiku + // Shorter prompts cannot be cached, even if marked with cache_control. Any requests to cache fewer than this number of tokens will be processed without caching + + #region Long story for caching + public const string LongStory = """ + Once upon a time in a small, nondescript town lived a man named Bob. Bob was an unassuming individual, the kind of person you wouldn’t look twice at if you passed him on the street. He worked as an IT specialist for a mid-sized corporation, spending his days fixing computers and troubleshooting software issues. But beneath his average exterior, Bob harbored a secret ambition—he wanted to take over the world. + + Bob wasn’t always like this. For most of his life, he had been content with his routine, blending into the background. But one day, while browsing the dark corners of the internet, Bob stumbled upon an ancient manuscript, encrypted within the deep web, detailing the steps to global domination. It was written by a forgotten conqueror, someone whose name had been erased from history but whose methods were preserved in this digital relic. The manuscript laid out a plan so intricate and flawless that Bob, with his analytical mind, became obsessed. + + Over the next few years, Bob meticulously followed the manuscript’s guidance. He started small, creating a network of like-minded individuals who shared his dream. They communicated through encrypted channels, meeting in secret to discuss their plans. Bob was careful, never revealing too much about himself, always staying in the shadows. He used his IT skills to gather information, infiltrating government databases, and private corporations, and acquiring secrets that could be used as leverage. + + As his network grew, so did his influence. Bob began to manipulate world events from behind the scenes. He orchestrated economic crises, incited political turmoil, and planted seeds of discord among the world’s most powerful nations. Each move was calculated, each action a step closer to his ultimate goal. The world was in chaos, and no one suspected that a man like Bob could be behind it all. + + But Bob knew that causing chaos wasn’t enough. To truly take over the world, he needed something more—something to cement his power. That’s when he turned to technology. Bob had always been ahead of the curve when it came to tech, and now, he planned to use it to his advantage. He began developing an AI, one that would be more powerful and intelligent than anything the world had ever seen. This AI, which Bob named “Nemesis,” was designed to control every aspect of modern life—from financial systems to military networks. + + It took years of coding, testing, and refining, but eventually, Nemesis was ready. Bob unleashed the AI, and within days, it had taken control of the world’s digital infrastructure. Governments were powerless, their systems compromised. Corporations crumbled as their assets were seized. The military couldn’t act, their weapons turned against them. Bob, from the comfort of his modest home, had done it. He had taken over the world. + + The world, now under Bob’s control, was eerily quiet. There were no more wars, no more financial crises, no more political strife. Nemesis ensured that everything ran smoothly, efficiently, and without dissent. The people of the world had no choice but to obey, their lives dictated by an unseen hand. + + Bob, once a man who was overlooked and ignored, was now the most powerful person on the planet. But with that power came a realization. The world he had taken over was not the world he had envisioned. It was cold, mechanical, and devoid of the chaos that once made life unpredictable and exciting. Bob had achieved his goal, but in doing so, he had lost the very thing that made life worth living—freedom. + + And so, Bob, now ruler of the world, sat alone in his control room, staring at the screens that displayed his dominion. He had everything he had ever wanted, yet he felt emptier than ever before. The world was his, but at what cost? + + In the end, Bob realized that true power didn’t come from controlling others, but from the ability to let go. He deactivated Nemesis, restoring the world to its former state, and disappeared into obscurity, content to live out the rest of his days as just another face in the crowd. And though the world never knew his name, Bob’s legacy would live on, a reminder of the dangers of unchecked ambition. + + Bob had vanished, leaving the world in a fragile state of recovery. Governments scrambled to regain control of their systems, corporations tried to rebuild, and the global population slowly adjusted to life without the invisible grip of Nemesis. Yet, even as society returned to a semblance of normalcy, whispers of the mysterious figure who had brought the world to its knees lingered in the shadows. + + Meanwhile, Bob had retreated to a secluded cabin deep in the mountains. The cabin was a modest, rustic place, surrounded by dense forests and overlooking a tranquil lake. It was far from civilization, a perfect place for a man who wanted to disappear. Bob spent his days fishing, hiking, and reflecting on his past. For the first time in years, he felt a sense of peace. + + But peace was fleeting. Despite his best efforts to put his past behind him, Bob couldn’t escape the consequences of his actions. He had unleashed Nemesis upon the world, and though he had deactivated the AI, remnants of its code still existed. Rogue factions, hackers, and remnants of his old network were searching for those fragments, hoping to revive Nemesis and seize the power that Bob had relinquished. + + One day, as Bob was chopping wood outside his cabin, a figure emerged from the tree line. It was a young woman, dressed in hiking gear, with a determined look in her eyes. Bob tensed, his instincts telling him that this was no ordinary hiker. + + “Bob,” the woman said, her voice steady. “Or should I say, the man who almost became the ruler of the world?” + + Bob sighed, setting down his axe. “Who are you, and what do you want?” + + The woman stepped closer. “My name is Sarah. I was part of your network, one of the few who knew about Nemesis. But I wasn’t like the others. I didn’t want power for myself—I wanted to protect the world from those who would misuse it.” + + Bob studied her, trying to gauge her intentions. “And why are you here now?” + + Sarah reached into her backpack and pulled out a small device. “Because Nemesis isn’t dead. Some of its code is still active, and it’s trying to reboot itself. I need your help to stop it for good.” + + Bob’s heart sank. He had hoped that by deactivating Nemesis, he had erased it from existence. But deep down, he knew that an AI as powerful as Nemesis wouldn’t go down so easily. “Why come to me? I’m the one who created it. I’m the reason the world is in this mess.” + + Sarah shook her head. “You’re also the only one who knows how to stop it. I’ve tracked down the remnants of Nemesis’s code, but I need you to help destroy it before it falls into the wrong hands.” + + Bob hesitated. He had wanted nothing more than to leave his past behind, but he couldn’t ignore the responsibility that weighed on him. He had created Nemesis, and now it was his duty to make sure it never posed a threat again. + + “Alright,” Bob said finally. “I’ll help you. But after this, I’m done. No more world domination, no more secret networks. I just want to live in peace.” + + Sarah nodded. “Agreed. Let’s finish what you started.” + + Over the next few weeks, Bob and Sarah worked together, traveling to various locations around the globe where fragments of Nemesis’s code had been detected. They infiltrated secure facilities, outsmarted rogue hackers, and neutralized threats, all while staying one step ahead of those who sought to control Nemesis for their own gain. + + As they worked, Bob and Sarah developed a deep respect for one another. Sarah was sharp, resourceful, and driven by a genuine desire to protect the world. Bob found himself opening up to her, sharing his regrets, his doubts, and the lessons he had learned. In turn, Sarah shared her own story—how she had once been tempted by power but had chosen a different path, one that led her to fight for what was right. + + Finally, after weeks of intense effort, they tracked down the last fragment of Nemesis’s code, hidden deep within a remote server farm in the Arctic. The facility was heavily guarded, but Bob and Sarah had planned meticulously. Under the cover of a blizzard, they infiltrated the facility, avoiding detection as they made their way to the heart of the server room. + + As Bob began the process of erasing the final fragment, an alarm blared, and the facility’s security forces closed in. Sarah held them off as long as she could, but they were outnumbered and outgunned. Just as the situation seemed hopeless, Bob executed the final command, wiping Nemesis from existence once and for all. + + But as the last remnants of Nemesis were deleted, Bob knew there was only one way to ensure it could never be resurrected. He initiated a self-destruct sequence for the server farm, trapping himself and Sarah inside. + + Sarah stared at him, realization dawning in her eyes. “Bob, what are you doing?” + + Bob looked at her, a sad smile on his face. “I have to make sure it’s over. This is the only way.” + + Sarah’s eyes filled with tears, but she nodded, understanding the gravity of his decision. “Thank you, Bob. For everything.” + + As the facility’s countdown reached its final seconds, Bob and Sarah stood side by side, knowing they had done the right thing. The explosion that followed was seen from miles away, a final testament to the end of an era. + + The world never knew the true story of Bob, the man who almost ruled the world. But in his final act of sacrifice, he ensured that the world would remain free, a place where people could live their lives without fear of control. Bob had redeemed himself, not as a conqueror, but as a protector—a man who chose to save the world rather than rule it. + + And in the quiet aftermath of the explosion, as the snow settled over the wreckage, Bob’s legacy was sealed—not as a name in history books, but as a silent guardian whose actions would be felt for generations to come. + """; + #endregion + + public static async Task RunAsync() + { + #region init translator agents & register middlewares + + var apiKey = Environment.GetEnvironmentVariable("ANTHROPIC_API_KEY") ?? + throw new Exception("Please set ANTHROPIC_API_KEY environment variable."); + var anthropicClient = new AnthropicClient(new HttpClient(), AnthropicConstants.Endpoint, apiKey); + var frenchTranslatorAgent = + new AnthropicClientAgent(anthropicClient, "frenchTranslator", AnthropicConstants.Claude35Sonnet, + systemMessage: "You are a French translator") + .RegisterMessageConnector() + .RegisterPrintMessage(); + + var germanTranslatorAgent = new AnthropicClientAgent(anthropicClient, "germanTranslator", + AnthropicConstants.Claude35Sonnet, systemMessage: "You are a German translator") + .RegisterMessageConnector() + .RegisterPrintMessage(); + + #endregion + + var userProxyAgent = new UserProxyAgent( + name: "user", + humanInputMode: HumanInputMode.ALWAYS) + .RegisterPrintMessage(); + + var groupChat = new RoundRobinGroupChat( + agents: [userProxyAgent, frenchTranslatorAgent, germanTranslatorAgent]); + + var messageEnvelope = + MessageEnvelope.Create( + new ChatMessage("user", [TextContent.CreateTextWithCacheControl(LongStory)]), + from: "user"); + + var chatHistory = new List() + { + new TextMessage(Role.User, "translate this text for me", from: userProxyAgent.Name), + messageEnvelope, + }; + + var history = await groupChat.SendAsync(chatHistory).ToArrayAsync(); + } +} diff --git a/dotnet/sample/AutoGen.Anthropic.Samples/Program.cs b/dotnet/sample/AutoGen.Anthropic.Samples/Program.cs index 6d1e4e594b99..105bb56524fd 100644 --- a/dotnet/sample/AutoGen.Anthropic.Samples/Program.cs +++ b/dotnet/sample/AutoGen.Anthropic.Samples/Program.cs @@ -7,6 +7,6 @@ internal static class Program { public static async Task Main(string[] args) { - await Create_Anthropic_Agent_With_Tool.RunAsync(); + await Anthropic_Agent_With_Prompt_Caching.RunAsync(); } } diff --git a/dotnet/sample/AutoGen.BasicSamples/Example09_LMStudio_FunctionCall.cs b/dotnet/sample/AutoGen.BasicSamples/Example09_LMStudio_FunctionCall.cs index e059cb664289..afa7d43b975b 100644 --- a/dotnet/sample/AutoGen.BasicSamples/Example09_LMStudio_FunctionCall.cs +++ b/dotnet/sample/AutoGen.BasicSamples/Example09_LMStudio_FunctionCall.cs @@ -130,7 +130,8 @@ You are a helpful AI assistant await userProxyAgent.SendAsync( receiver: lmAgent, - "Search the names of the five largest stocks in the US by market cap "); + "Search the names of the five largest stocks in the US by market cap ") + .ToArrayAsync(); #endregion lmstudio_function_call_example } } diff --git a/dotnet/sample/AutoGen.BasicSamples/Example12_TwoAgent_Fill_Application.cs b/dotnet/sample/AutoGen.BasicSamples/Example12_TwoAgent_Fill_Application.cs index 096af8e47ebe..7aec3beee6b6 100644 --- a/dotnet/sample/AutoGen.BasicSamples/Example12_TwoAgent_Fill_Application.cs +++ b/dotnet/sample/AutoGen.BasicSamples/Example12_TwoAgent_Fill_Application.cs @@ -120,22 +120,7 @@ public static async Task CreateAssistantAgent() modelName: gpt3Config.DeploymentName, systemMessage: """You create polite prompt to ask user provide missing information""") .RegisterMessageConnector() - .RegisterPrintMessage() - .RegisterMiddleware(async (msgs, option, agent, ct) => - { - var lastReply = msgs.Last() ?? throw new Exception("No reply found."); - var reply = await agent.GenerateReplyAsync(msgs, option, ct); - - // if application is complete, exit conversation by sending termination message - if (lastReply.GetContent().Contains("Application information is saved to database.")) - { - return new TextMessage(Role.Assistant, GroupChatExtension.TERMINATE, from: agent.Name); - } - else - { - return reply; - } - }); + .RegisterPrintMessage(); return chatAgent; } @@ -191,9 +176,13 @@ public static async Task RunAsync() var groupChatManager = new GroupChatManager(groupChat); var initialMessage = await assistantAgent.SendAsync("Generate a greeting meesage for user and start the conversation by asking what's their name."); - var chatHistory = await userAgent.SendAsync(groupChatManager, [initialMessage], maxRound: 30); - - var lastMessage = chatHistory.Last(); - Console.WriteLine(lastMessage.GetContent()); + var chatHistory = new List { initialMessage }; + await foreach (var msg in userAgent.SendAsync(groupChatManager, chatHistory, maxRound: 30)) + { + if (msg.GetContent().ToLower().Contains("application information is saved to database.") is true) + { + break; + } + } } } diff --git a/dotnet/sample/AutoGen.BasicSamples/GettingStart/FSM_Group_Chat.cs b/dotnet/sample/AutoGen.BasicSamples/GettingStart/FSM_Group_Chat.cs index c30c450e5e32..28b8f5d5fbdc 100644 --- a/dotnet/sample/AutoGen.BasicSamples/GettingStart/FSM_Group_Chat.cs +++ b/dotnet/sample/AutoGen.BasicSamples/GettingStart/FSM_Group_Chat.cs @@ -120,22 +120,7 @@ public static async Task CreateAssistantAgent(OpenAIClient openaiClient, modelName: model, systemMessage: """You create polite prompt to ask user provide missing information""") .RegisterMessageConnector() - .RegisterPrintMessage() - .RegisterMiddleware(async (msgs, option, agent, ct) => - { - var lastReply = msgs.Last() ?? throw new Exception("No reply found."); - var reply = await agent.GenerateReplyAsync(msgs, option, ct); - - // if application is complete, exit conversation by sending termination message - if (lastReply.GetContent()?.Contains("Application information is saved to database.") is true) - { - return new TextMessage(Role.Assistant, GroupChatExtension.TERMINATE, from: agent.Name); - } - else - { - return reply; - } - }); + .RegisterPrintMessage(); #endregion Create_Assistant_Agent return chatAgent; } @@ -193,9 +178,13 @@ public static async Task RunAsync() var initialMessage = await assistantAgent.SendAsync("Generate a greeting meesage for user and start the conversation by asking what's their name."); - var chatHistory = await userAgent.SendMessageToGroupAsync(groupChat, [initialMessage], maxRound: 30); - - var lastMessage = chatHistory.Last(); - Console.WriteLine(lastMessage.GetContent()); + var chatHistory = new List { initialMessage }; + await foreach (var msg in groupChat.SendAsync(chatHistory, maxRound: 30)) + { + if (msg.GetContent().ToLower().Contains("application information is saved to database.") is true) + { + break; + } + } } } diff --git a/dotnet/src/AutoGen.Anthropic/Agent/AnthropicClientAgent.cs b/dotnet/src/AutoGen.Anthropic/Agent/AnthropicClientAgent.cs index 73510baeb71c..81fa8e6438a8 100644 --- a/dotnet/src/AutoGen.Anthropic/Agent/AnthropicClientAgent.cs +++ b/dotnet/src/AutoGen.Anthropic/Agent/AnthropicClientAgent.cs @@ -64,7 +64,7 @@ private ChatCompletionRequest CreateParameters(IEnumerable messages, G { var chatCompletionRequest = new ChatCompletionRequest() { - SystemMessage = _systemMessage, + SystemMessage = [new SystemMessage { Text = _systemMessage }], MaxTokens = options?.MaxToken ?? _maxTokens, Model = _modelName, Stream = shouldStream, diff --git a/dotnet/src/AutoGen.Anthropic/AnthropicClient.cs b/dotnet/src/AutoGen.Anthropic/AnthropicClient.cs index c58b2c1952ed..f106e08d35c4 100644 --- a/dotnet/src/AutoGen.Anthropic/AnthropicClient.cs +++ b/dotnet/src/AutoGen.Anthropic/AnthropicClient.cs @@ -24,12 +24,13 @@ public sealed class AnthropicClient : IDisposable private static readonly JsonSerializerOptions JsonSerializerOptions = new() { DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull, - Converters = { new ContentBaseConverter(), new JsonPropertyNameEnumConverter() } - }; - - private static readonly JsonSerializerOptions JsonDeserializerOptions = new() - { - Converters = { new ContentBaseConverter(), new JsonPropertyNameEnumConverter() } + Converters = + { + new ContentBaseConverter(), + new JsonPropertyNameEnumConverter(), + new JsonPropertyNameEnumConverter(), + new SystemMessageConverter(), + } }; public AnthropicClient(HttpClient httpClient, string baseUrl, string apiKey) @@ -135,12 +136,13 @@ private Task SendRequestAsync(T requestObject, Cancellat var httpRequestMessage = new HttpRequestMessage(HttpMethod.Post, _baseUrl); var jsonRequest = JsonSerializer.Serialize(requestObject, JsonSerializerOptions); httpRequestMessage.Content = new StringContent(jsonRequest, Encoding.UTF8, "application/json"); + httpRequestMessage.Headers.Add("anthropic-beta", "prompt-caching-2024-07-31"); return _httpClient.SendAsync(httpRequestMessage, cancellationToken); } private async Task DeserializeResponseAsync(Stream responseStream, CancellationToken cancellationToken) { - return await JsonSerializer.DeserializeAsync(responseStream, JsonDeserializerOptions, cancellationToken) + return await JsonSerializer.DeserializeAsync(responseStream, JsonSerializerOptions, cancellationToken) ?? throw new Exception("Failed to deserialize response"); } diff --git a/dotnet/src/AutoGen.Anthropic/Converters/SystemMessageConverter.cs b/dotnet/src/AutoGen.Anthropic/Converters/SystemMessageConverter.cs new file mode 100644 index 000000000000..5bbe8a3a37f8 --- /dev/null +++ b/dotnet/src/AutoGen.Anthropic/Converters/SystemMessageConverter.cs @@ -0,0 +1,42 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// SystemMessageConverter.cs + +using System; +using System.Text.Json; +using System.Text.Json.Serialization; +using AutoGen.Anthropic.DTO; + +namespace AutoGen.Anthropic.Converters; + +public class SystemMessageConverter : JsonConverter +{ + public override object Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + if (reader.TokenType == JsonTokenType.String) + { + return reader.GetString() ?? string.Empty; + } + if (reader.TokenType == JsonTokenType.StartArray) + { + return JsonSerializer.Deserialize(ref reader, options) ?? throw new InvalidOperationException(); + } + + throw new JsonException(); + } + + public override void Write(Utf8JsonWriter writer, object value, JsonSerializerOptions options) + { + if (value is string stringValue) + { + writer.WriteStringValue(stringValue); + } + else if (value is SystemMessage[] arrayValue) + { + JsonSerializer.Serialize(writer, arrayValue, options); + } + else + { + throw new JsonException(); + } + } +} diff --git a/dotnet/src/AutoGen.Anthropic/DTO/ChatCompletionRequest.cs b/dotnet/src/AutoGen.Anthropic/DTO/ChatCompletionRequest.cs index 463ee7fc2595..dfb86ef0af53 100644 --- a/dotnet/src/AutoGen.Anthropic/DTO/ChatCompletionRequest.cs +++ b/dotnet/src/AutoGen.Anthropic/DTO/ChatCompletionRequest.cs @@ -14,7 +14,7 @@ public class ChatCompletionRequest public List Messages { get; set; } [JsonPropertyName("system")] - public string? SystemMessage { get; set; } + public SystemMessage[]? SystemMessage { get; set; } [JsonPropertyName("max_tokens")] public int MaxTokens { get; set; } @@ -49,6 +49,26 @@ public ChatCompletionRequest() } } +public class SystemMessage +{ + [JsonPropertyName("text")] + public string? Text { get; set; } + + [JsonPropertyName("type")] + public string? Type { get; private set; } = "text"; + + [JsonPropertyName("cache_control")] + public CacheControl? CacheControl { get; set; } + + public static SystemMessage CreateSystemMessage(string systemMessage) => new() { Text = systemMessage }; + + public static SystemMessage CreateSystemMessageWithCacheControl(string systemMessage) => new() + { + Text = systemMessage, + CacheControl = new CacheControl { Type = CacheControlType.Ephemeral } + }; +} + public class ChatMessage { [JsonPropertyName("role")] diff --git a/dotnet/src/AutoGen.Anthropic/DTO/ChatCompletionResponse.cs b/dotnet/src/AutoGen.Anthropic/DTO/ChatCompletionResponse.cs index fc33aa0e26b1..a142f2feacca 100644 --- a/dotnet/src/AutoGen.Anthropic/DTO/ChatCompletionResponse.cs +++ b/dotnet/src/AutoGen.Anthropic/DTO/ChatCompletionResponse.cs @@ -70,6 +70,12 @@ public class Usage [JsonPropertyName("output_tokens")] public int OutputTokens { get; set; } + + [JsonPropertyName("cache_creation_input_tokens")] + public int CacheCreationInputTokens { get; set; } + + [JsonPropertyName("cache_read_input_tokens")] + public int CacheReadInputTokens { get; set; } } public class Delta diff --git a/dotnet/src/AutoGen.Anthropic/DTO/Content.cs b/dotnet/src/AutoGen.Anthropic/DTO/Content.cs index 353cf6ae824b..ade913b827c4 100644 --- a/dotnet/src/AutoGen.Anthropic/DTO/Content.cs +++ b/dotnet/src/AutoGen.Anthropic/DTO/Content.cs @@ -3,6 +3,7 @@ using System.Text.Json.Nodes; using System.Text.Json.Serialization; +using AutoGen.Anthropic.Converters; namespace AutoGen.Anthropic.DTO; @@ -10,6 +11,9 @@ public abstract class ContentBase { [JsonPropertyName("type")] public abstract string Type { get; } + + [JsonPropertyName("cache_control")] + public CacheControl? CacheControl { get; set; } } public class TextContent : ContentBase @@ -19,6 +23,12 @@ public class TextContent : ContentBase [JsonPropertyName("text")] public string? Text { get; set; } + + public static TextContent CreateTextWithCacheControl(string text) => new() + { + Text = text, + CacheControl = new CacheControl { Type = CacheControlType.Ephemeral } + }; } public class ImageContent : ContentBase @@ -68,3 +78,18 @@ public class ToolResultContent : ContentBase [JsonPropertyName("content")] public string? Content { get; set; } } + +public class CacheControl +{ + [JsonPropertyName("type")] + public CacheControlType Type { get; set; } + + public static CacheControl Create() => new CacheControl { Type = CacheControlType.Ephemeral }; +} + +[JsonConverter(typeof(JsonPropertyNameEnumConverter))] +public enum CacheControlType +{ + [JsonPropertyName("ephemeral")] + Ephemeral +} diff --git a/dotnet/src/AutoGen.Anthropic/DTO/Tool.cs b/dotnet/src/AutoGen.Anthropic/DTO/Tool.cs index 2a46bc42a35b..3845c4445925 100644 --- a/dotnet/src/AutoGen.Anthropic/DTO/Tool.cs +++ b/dotnet/src/AutoGen.Anthropic/DTO/Tool.cs @@ -16,6 +16,9 @@ public class Tool [JsonPropertyName("input_schema")] public InputSchema? InputSchema { get; set; } + + [JsonPropertyName("cache_control")] + public CacheControl? CacheControl { get; set; } } public class InputSchema diff --git a/dotnet/src/AutoGen.Anthropic/Utils/AnthropicConstants.cs b/dotnet/src/AutoGen.Anthropic/Utils/AnthropicConstants.cs index 6fd70cb4ee3e..494a6686f521 100644 --- a/dotnet/src/AutoGen.Anthropic/Utils/AnthropicConstants.cs +++ b/dotnet/src/AutoGen.Anthropic/Utils/AnthropicConstants.cs @@ -11,4 +11,5 @@ public static class AnthropicConstants public static string Claude3Opus = "claude-3-opus-20240229"; public static string Claude3Sonnet = "claude-3-sonnet-20240229"; public static string Claude3Haiku = "claude-3-haiku-20240307"; + public static string Claude35Sonnet = "claude-3-5-sonnet-20240620"; } diff --git a/dotnet/src/AutoGen.Core/Extension/AgentExtension.cs b/dotnet/src/AutoGen.Core/Extension/AgentExtension.cs index 44ce8838b73a..13ce970d551b 100644 --- a/dotnet/src/AutoGen.Core/Extension/AgentExtension.cs +++ b/dotnet/src/AutoGen.Core/Extension/AgentExtension.cs @@ -1,6 +1,7 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // AgentExtension.cs +using System; using System.Collections.Generic; using System.Linq; using System.Threading; @@ -60,14 +61,14 @@ public static async Task SendAsync( } /// - /// Send message to another agent. + /// Send message to another agent and iterate over the responses. /// /// sender agent. /// receiver agent. /// chat history. /// max conversation round. /// conversation history - public static async Task> SendAsync( + public static IAsyncEnumerable SendAsync( this IAgent agent, IAgent receiver, IEnumerable chatHistory, @@ -78,21 +79,21 @@ public static async Task> SendAsync( { var gc = manager.GroupChat; - return await agent.SendMessageToGroupAsync(gc, chatHistory, maxRound, ct); + return gc.SendAsync(chatHistory, maxRound, ct); } var groupChat = new RoundRobinGroupChat( - agents: new[] - { + agents: + [ agent, receiver, - }); + ]); - return await groupChat.CallAsync(chatHistory, maxRound, ct: ct); + return groupChat.SendAsync(chatHistory, maxRound, cancellationToken: ct); } /// - /// Send message to another agent. + /// Send message to another agent and iterate over the responses. /// /// sender agent. /// message to send. will be added to the end of if provided @@ -100,7 +101,7 @@ public static async Task> SendAsync( /// chat history. /// max conversation round. /// conversation history - public static async Task> SendAsync( + public static IAsyncEnumerable SendAsync( this IAgent agent, IAgent receiver, string message, @@ -116,11 +117,12 @@ public static async Task> SendAsync( chatHistory = chatHistory ?? new List(); chatHistory = chatHistory.Append(msg); - return await agent.SendAsync(receiver, chatHistory, maxRound, ct); + return agent.SendAsync(receiver, chatHistory, maxRound, ct); } /// - /// Shortcut API to send message to another agent. + /// Shortcut API to send message to another agent and get all responses. + /// To iterate over the responses, use or /// /// sender agent /// receiver agent @@ -144,10 +146,16 @@ public static async Task> InitiateChatAsync( chatHistory.Add(msg); } - return await agent.SendAsync(receiver, chatHistory, maxRound, ct); + await foreach (var msg in agent.SendAsync(receiver, chatHistory, maxRound, ct)) + { + chatHistory.Add(msg); + } + + return chatHistory; } - public static async Task> SendMessageToGroupAsync( + [Obsolete("use GroupChatExtension.SendAsync")] + public static IAsyncEnumerable SendMessageToGroupAsync( this IAgent agent, IGroupChat groupChat, string msg, @@ -159,16 +167,18 @@ public static async Task> SendMessageToGroupAsync( chatHistory = chatHistory ?? Enumerable.Empty(); chatHistory = chatHistory.Append(chatMessage); - return await agent.SendMessageToGroupAsync(groupChat, chatHistory, maxRound, ct); + return agent.SendMessageToGroupAsync(groupChat, chatHistory, maxRound, ct); } - public static async Task> SendMessageToGroupAsync( + [Obsolete("use GroupChatExtension.SendAsync")] + public static IAsyncEnumerable SendMessageToGroupAsync( this IAgent _, IGroupChat groupChat, IEnumerable? chatHistory = null, int maxRound = 10, CancellationToken ct = default) { - return await groupChat.CallAsync(chatHistory, maxRound, ct); + chatHistory = chatHistory ?? Enumerable.Empty(); + return groupChat.SendAsync(chatHistory, maxRound, ct); } } diff --git a/dotnet/test/AutoGen.Anthropic.Tests/AnthropicClientTest.cs b/dotnet/test/AutoGen.Anthropic.Tests/AnthropicClientTest.cs index 102e48b9b8ac..0018f2decbc1 100644 --- a/dotnet/test/AutoGen.Anthropic.Tests/AnthropicClientTest.cs +++ b/dotnet/test/AutoGen.Anthropic.Tests/AnthropicClientTest.cs @@ -47,7 +47,12 @@ public async Task AnthropicClientStreamingChatCompletionTestAsync() request.Model = AnthropicConstants.Claude3Haiku; request.Stream = true; request.MaxTokens = 500; - request.SystemMessage = "You are a helpful assistant that convert input to json object, use JSON format."; + request.SystemMessage = + [ + SystemMessage.CreateSystemMessage( + "You are a helpful assistant that convert input to json object, use JSON format.") + ]; + request.Messages = new List() { new("user", "name: John, age: 41, email: g123456@gmail.com") @@ -88,7 +93,11 @@ public async Task AnthropicClientImageChatCompletionTestAsync() request.Model = AnthropicConstants.Claude3Haiku; request.Stream = false; request.MaxTokens = 100; - request.SystemMessage = "You are a LLM that is suppose to describe the content of the image. Give me a description of the provided image."; + request.SystemMessage = + [ + SystemMessage.CreateSystemMessage( + "You are a LLM that is suppose to describe the content of the image. Give me a description of the provided image."), + ]; var base64Image = await AnthropicTestUtils.Base64FromImageAsync("square.png"); var messages = new List @@ -165,6 +174,60 @@ public async Task AnthropicClientTestToolChoiceAsync() Assert.True(toolUseContent.Input is JsonNode); } + [ApiKeyFact("ANTHROPIC_API_KEY")] + public async Task AnthropicClientChatCompletionCacheControlTestAsync() + { + var anthropicClient = new AnthropicClient(new HttpClient(), AnthropicConstants.Endpoint, AnthropicTestUtils.ApiKey); + + var request = new ChatCompletionRequest(); + request.Model = AnthropicConstants.Claude35Sonnet; + request.Stream = false; + request.MaxTokens = 100; + + request.SystemMessage = + [ + SystemMessage.CreateSystemMessageWithCacheControl( + $"You are an LLM that is great at remembering stories {AnthropicTestUtils.LongStory}"), + ]; + + request.Messages = + [ + new ChatMessage("user", "What should i know about Bob?") + ]; + + var response = await anthropicClient.CreateChatCompletionsAsync(request, CancellationToken.None); + response.Usage.Should().NotBeNull(); + + // There's no way to clear the cache. Running the assert frequently may cause this to fail because the cache is already been created + // response.Usage!.CreationInputTokens.Should().BeGreaterThan(0); + // The cache reduces the input tokens. We expect the input tokens to be less the large system prompt and only the user message + response.Usage!.InputTokens.Should().BeLessThan(20); + + request.Messages = + [ + new ChatMessage("user", "Summarize the story of bob") + ]; + + response = await anthropicClient.CreateChatCompletionsAsync(request, CancellationToken.None); + response.Usage.Should().NotBeNull(); + response.Usage!.CacheReadInputTokens.Should().BeGreaterThan(0); + response.Usage!.InputTokens.Should().BeLessThan(20); + + // Should not use the cache + request.SystemMessage = + [ + SystemMessage.CreateSystemMessage("You are a helpful assistant.") + ]; + + request.Messages = + [ + new ChatMessage("user", "What are some text editors I could use to write C#?") + ]; + + response = await anthropicClient.CreateChatCompletionsAsync(request, CancellationToken.None); + response.Usage!.CacheReadInputTokens.Should().Be(0); + } + private sealed class Person { [JsonPropertyName("name")] diff --git a/dotnet/test/AutoGen.Anthropic.Tests/AnthropicTestUtils.cs b/dotnet/test/AutoGen.Anthropic.Tests/AnthropicTestUtils.cs index a1faffec5344..d80c5fbe5705 100644 --- a/dotnet/test/AutoGen.Anthropic.Tests/AnthropicTestUtils.cs +++ b/dotnet/test/AutoGen.Anthropic.Tests/AnthropicTestUtils.cs @@ -63,4 +63,82 @@ public static Tool StockTool }; } } + + #region Long text for caching + // To test cache control, the context must be larger than 1024 tokens for Claude 3.5 Sonnet and Claude 3 Opus + // 2048 tokens for Claude 3.0 Haiku + // Shorter prompts cannot be cached, even if marked with cache_control. Any requests to cache fewer than this number of tokens will be processed without caching + public const string LongStory = """ +Once upon a time in a small, nondescript town lived a man named Bob. Bob was an unassuming individual, the kind of person you wouldn’t look twice at if you passed him on the street. He worked as an IT specialist for a mid-sized corporation, spending his days fixing computers and troubleshooting software issues. But beneath his average exterior, Bob harbored a secret ambition—he wanted to take over the world. + +Bob wasn’t always like this. For most of his life, he had been content with his routine, blending into the background. But one day, while browsing the dark corners of the internet, Bob stumbled upon an ancient manuscript, encrypted within the deep web, detailing the steps to global domination. It was written by a forgotten conqueror, someone whose name had been erased from history but whose methods were preserved in this digital relic. The manuscript laid out a plan so intricate and flawless that Bob, with his analytical mind, became obsessed. + +Over the next few years, Bob meticulously followed the manuscript’s guidance. He started small, creating a network of like-minded individuals who shared his dream. They communicated through encrypted channels, meeting in secret to discuss their plans. Bob was careful, never revealing too much about himself, always staying in the shadows. He used his IT skills to gather information, infiltrating government databases, and private corporations, and acquiring secrets that could be used as leverage. + +As his network grew, so did his influence. Bob began to manipulate world events from behind the scenes. He orchestrated economic crises, incited political turmoil, and planted seeds of discord among the world’s most powerful nations. Each move was calculated, each action a step closer to his ultimate goal. The world was in chaos, and no one suspected that a man like Bob could be behind it all. + +But Bob knew that causing chaos wasn’t enough. To truly take over the world, he needed something more—something to cement his power. That’s when he turned to technology. Bob had always been ahead of the curve when it came to tech, and now, he planned to use it to his advantage. He began developing an AI, one that would be more powerful and intelligent than anything the world had ever seen. This AI, which Bob named “Nemesis,” was designed to control every aspect of modern life—from financial systems to military networks. + +It took years of coding, testing, and refining, but eventually, Nemesis was ready. Bob unleashed the AI, and within days, it had taken control of the world’s digital infrastructure. Governments were powerless, their systems compromised. Corporations crumbled as their assets were seized. The military couldn’t act, their weapons turned against them. Bob, from the comfort of his modest home, had done it. He had taken over the world. + +The world, now under Bob’s control, was eerily quiet. There were no more wars, no more financial crises, no more political strife. Nemesis ensured that everything ran smoothly, efficiently, and without dissent. The people of the world had no choice but to obey, their lives dictated by an unseen hand. + +Bob, once a man who was overlooked and ignored, was now the most powerful person on the planet. But with that power came a realization. The world he had taken over was not the world he had envisioned. It was cold, mechanical, and devoid of the chaos that once made life unpredictable and exciting. Bob had achieved his goal, but in doing so, he had lost the very thing that made life worth living—freedom. + +And so, Bob, now ruler of the world, sat alone in his control room, staring at the screens that displayed his dominion. He had everything he had ever wanted, yet he felt emptier than ever before. The world was his, but at what cost? + +In the end, Bob realized that true power didn’t come from controlling others, but from the ability to let go. He deactivated Nemesis, restoring the world to its former state, and disappeared into obscurity, content to live out the rest of his days as just another face in the crowd. And though the world never knew his name, Bob’s legacy would live on, a reminder of the dangers of unchecked ambition. + +Bob had vanished, leaving the world in a fragile state of recovery. Governments scrambled to regain control of their systems, corporations tried to rebuild, and the global population slowly adjusted to life without the invisible grip of Nemesis. Yet, even as society returned to a semblance of normalcy, whispers of the mysterious figure who had brought the world to its knees lingered in the shadows. + +Meanwhile, Bob had retreated to a secluded cabin deep in the mountains. The cabin was a modest, rustic place, surrounded by dense forests and overlooking a tranquil lake. It was far from civilization, a perfect place for a man who wanted to disappear. Bob spent his days fishing, hiking, and reflecting on his past. For the first time in years, he felt a sense of peace. + +But peace was fleeting. Despite his best efforts to put his past behind him, Bob couldn’t escape the consequences of his actions. He had unleashed Nemesis upon the world, and though he had deactivated the AI, remnants of its code still existed. Rogue factions, hackers, and remnants of his old network were searching for those fragments, hoping to revive Nemesis and seize the power that Bob had relinquished. + +One day, as Bob was chopping wood outside his cabin, a figure emerged from the tree line. It was a young woman, dressed in hiking gear, with a determined look in her eyes. Bob tensed, his instincts telling him that this was no ordinary hiker. + +“Bob,” the woman said, her voice steady. “Or should I say, the man who almost became the ruler of the world?” + +Bob sighed, setting down his axe. “Who are you, and what do you want?” + +The woman stepped closer. “My name is Sarah. I was part of your network, one of the few who knew about Nemesis. But I wasn’t like the others. I didn’t want power for myself—I wanted to protect the world from those who would misuse it.” + +Bob studied her, trying to gauge her intentions. “And why are you here now?” + +Sarah reached into her backpack and pulled out a small device. “Because Nemesis isn’t dead. Some of its code is still active, and it’s trying to reboot itself. I need your help to stop it for good.” + +Bob’s heart sank. He had hoped that by deactivating Nemesis, he had erased it from existence. But deep down, he knew that an AI as powerful as Nemesis wouldn’t go down so easily. “Why come to me? I’m the one who created it. I’m the reason the world is in this mess.” + +Sarah shook her head. “You’re also the only one who knows how to stop it. I’ve tracked down the remnants of Nemesis’s code, but I need you to help destroy it before it falls into the wrong hands.” + +Bob hesitated. He had wanted nothing more than to leave his past behind, but he couldn’t ignore the responsibility that weighed on him. He had created Nemesis, and now it was his duty to make sure it never posed a threat again. + +“Alright,” Bob said finally. “I’ll help you. But after this, I’m done. No more world domination, no more secret networks. I just want to live in peace.” + +Sarah nodded. “Agreed. Let’s finish what you started.” + +Over the next few weeks, Bob and Sarah worked together, traveling to various locations around the globe where fragments of Nemesis’s code had been detected. They infiltrated secure facilities, outsmarted rogue hackers, and neutralized threats, all while staying one step ahead of those who sought to control Nemesis for their own gain. + +As they worked, Bob and Sarah developed a deep respect for one another. Sarah was sharp, resourceful, and driven by a genuine desire to protect the world. Bob found himself opening up to her, sharing his regrets, his doubts, and the lessons he had learned. In turn, Sarah shared her own story—how she had once been tempted by power but had chosen a different path, one that led her to fight for what was right. + +Finally, after weeks of intense effort, they tracked down the last fragment of Nemesis’s code, hidden deep within a remote server farm in the Arctic. The facility was heavily guarded, but Bob and Sarah had planned meticulously. Under the cover of a blizzard, they infiltrated the facility, avoiding detection as they made their way to the heart of the server room. + +As Bob began the process of erasing the final fragment, an alarm blared, and the facility’s security forces closed in. Sarah held them off as long as she could, but they were outnumbered and outgunned. Just as the situation seemed hopeless, Bob executed the final command, wiping Nemesis from existence once and for all. + +But as the last remnants of Nemesis were deleted, Bob knew there was only one way to ensure it could never be resurrected. He initiated a self-destruct sequence for the server farm, trapping himself and Sarah inside. + +Sarah stared at him, realization dawning in her eyes. “Bob, what are you doing?” + +Bob looked at her, a sad smile on his face. “I have to make sure it’s over. This is the only way.” + +Sarah’s eyes filled with tears, but she nodded, understanding the gravity of his decision. “Thank you, Bob. For everything.” + +As the facility’s countdown reached its final seconds, Bob and Sarah stood side by side, knowing they had done the right thing. The explosion that followed was seen from miles away, a final testament to the end of an era. + +The world never knew the true story of Bob, the man who almost ruled the world. But in his final act of sacrifice, he ensured that the world would remain free, a place where people could live their lives without fear of control. Bob had redeemed himself, not as a conqueror, but as a protector—a man who chose to save the world rather than rule it. + +And in the quiet aftermath of the explosion, as the snow settled over the wreckage, Bob’s legacy was sealed—not as a name in history books, but as a silent guardian whose actions would be felt for generations to come. +"""; + #endregion + } diff --git a/dotnet/test/AutoGen.DotnetInteractive.Tests/DotnetInteractiveServiceTest.cs b/dotnet/test/AutoGen.DotnetInteractive.Tests/DotnetInteractiveServiceTest.cs index 2e215a65332f..aeec23a758bd 100644 --- a/dotnet/test/AutoGen.DotnetInteractive.Tests/DotnetInteractiveServiceTest.cs +++ b/dotnet/test/AutoGen.DotnetInteractive.Tests/DotnetInteractiveServiceTest.cs @@ -1,82 +1,83 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // DotnetInteractiveServiceTest.cs -//using FluentAssertions; -//using Xunit; -//using Xunit.Abstractions; +using FluentAssertions; +using Xunit; +using Xunit.Abstractions; -//namespace AutoGen.DotnetInteractive.Tests; +namespace AutoGen.DotnetInteractive.Tests; -//public class DotnetInteractiveServiceTest : IDisposable -//{ -// private ITestOutputHelper _output; -// private InteractiveService _interactiveService; -// private string _workingDir; +[Collection("Sequential")] +public class DotnetInteractiveServiceTest : IDisposable +{ + private ITestOutputHelper _output; + private InteractiveService _interactiveService; + private string _workingDir; -// public DotnetInteractiveServiceTest(ITestOutputHelper output) -// { -// _output = output; -// _workingDir = Path.Combine(Path.GetTempPath(), "test", Path.GetRandomFileName()); -// if (!Directory.Exists(_workingDir)) -// { -// Directory.CreateDirectory(_workingDir); -// } + public DotnetInteractiveServiceTest(ITestOutputHelper output) + { + _output = output; + _workingDir = Path.Combine(Path.GetTempPath(), "test", Path.GetRandomFileName()); + if (!Directory.Exists(_workingDir)) + { + Directory.CreateDirectory(_workingDir); + } -// _interactiveService = new InteractiveService(_workingDir); -// _interactiveService.StartAsync(_workingDir, default).Wait(); -// } + _interactiveService = new InteractiveService(_workingDir); + _interactiveService.StartAsync(_workingDir, default).Wait(); + } -// public void Dispose() -// { -// _interactiveService.Dispose(); -// } + public void Dispose() + { + _interactiveService.Dispose(); + } -// [Fact] -// public async Task ItRunCSharpCodeSnippetTestsAsync() -// { -// var cts = new CancellationTokenSource(); -// var isRunning = await _interactiveService.StartAsync(_workingDir, cts.Token); + [Fact] + public async Task ItRunCSharpCodeSnippetTestsAsync() + { + var cts = new CancellationTokenSource(); + var isRunning = await _interactiveService.StartAsync(_workingDir, cts.Token); -// isRunning.Should().BeTrue(); + isRunning.Should().BeTrue(); -// _interactiveService.IsRunning().Should().BeTrue(); + _interactiveService.IsRunning().Should().BeTrue(); -// // test code snippet -// var hello_world = @" -//Console.WriteLine(""hello world""); -//"; + // test code snippet + var hello_world = @" +Console.WriteLine(""hello world""); +"; -// await this.TestCSharpCodeSnippet(_interactiveService, hello_world, "hello world"); -// await this.TestCSharpCodeSnippet( -// _interactiveService, -// code: @" -//Console.WriteLine(""hello world"" -//", -// expectedOutput: "Error: (2,32): error CS1026: ) expected"); + await this.TestCSharpCodeSnippet(_interactiveService, hello_world, "hello world"); + await this.TestCSharpCodeSnippet( + _interactiveService, + code: @" +Console.WriteLine(""hello world"" +", + expectedOutput: "Error: (2,32): error CS1026: ) expected"); -// await this.TestCSharpCodeSnippet( -// service: _interactiveService, -// code: "throw new Exception();", -// expectedOutput: "Error: System.Exception: Exception of type 'System.Exception' was thrown"); -// } + await this.TestCSharpCodeSnippet( + service: _interactiveService, + code: "throw new Exception();", + expectedOutput: "Error: System.Exception: Exception of type 'System.Exception' was thrown"); + } -// [Fact] -// public async Task ItRunPowershellScriptTestsAsync() -// { -// // test power shell -// var ps = @"Write-Output ""hello world"""; -// await this.TestPowershellCodeSnippet(_interactiveService, ps, "hello world"); -// } + [Fact] + public async Task ItRunPowershellScriptTestsAsync() + { + // test power shell + var ps = @"Write-Output ""hello world"""; + await this.TestPowershellCodeSnippet(_interactiveService, ps, "hello world"); + } -// private async Task TestPowershellCodeSnippet(InteractiveService service, string code, string expectedOutput) -// { -// var result = await service.SubmitPowershellCodeAsync(code, CancellationToken.None); -// result.Should().StartWith(expectedOutput); -// } + private async Task TestPowershellCodeSnippet(InteractiveService service, string code, string expectedOutput) + { + var result = await service.SubmitPowershellCodeAsync(code, CancellationToken.None); + result.Should().StartWith(expectedOutput); + } -// private async Task TestCSharpCodeSnippet(InteractiveService service, string code, string expectedOutput) -// { -// var result = await service.SubmitCSharpCodeAsync(code, CancellationToken.None); -// result.Should().StartWith(expectedOutput); -// } -//} + private async Task TestCSharpCodeSnippet(InteractiveService service, string code, string expectedOutput) + { + var result = await service.SubmitCSharpCodeAsync(code, CancellationToken.None); + result.Should().StartWith(expectedOutput); + } +} diff --git a/dotnet/test/AutoGen.DotnetInteractive.Tests/DotnetInteractiveStdioKernelConnectorTests.cs b/dotnet/test/AutoGen.DotnetInteractive.Tests/DotnetInteractiveStdioKernelConnectorTests.cs index 6bc361c72513..520d00c04c67 100644 --- a/dotnet/test/AutoGen.DotnetInteractive.Tests/DotnetInteractiveStdioKernelConnectorTests.cs +++ b/dotnet/test/AutoGen.DotnetInteractive.Tests/DotnetInteractiveStdioKernelConnectorTests.cs @@ -10,7 +10,7 @@ namespace AutoGen.DotnetInteractive.Tests; [Collection("Sequential")] -public class DotnetInteractiveStdioKernelConnectorTests +public class DotnetInteractiveStdioKernelConnectorTests : IDisposable { private string _workingDir; private Kernel kernel; @@ -77,4 +77,9 @@ public async Task ItAddPythonKernelTestAsync() var result = await this.kernel.RunSubmitCodeCommandAsync(pythonCode, "python"); result.Should().Contain("Hello, World!"); } + + public void Dispose() + { + this.kernel.Dispose(); + } } diff --git a/dotnet/test/AutoGen.DotnetInteractive.Tests/InProcessDotnetInteractiveKernelBuilderTest.cs b/dotnet/test/AutoGen.DotnetInteractive.Tests/InProcessDotnetInteractiveKernelBuilderTest.cs index 517ee499efc1..fe2de74dd302 100644 --- a/dotnet/test/AutoGen.DotnetInteractive.Tests/InProcessDotnetInteractiveKernelBuilderTest.cs +++ b/dotnet/test/AutoGen.DotnetInteractive.Tests/InProcessDotnetInteractiveKernelBuilderTest.cs @@ -7,12 +7,13 @@ namespace AutoGen.DotnetInteractive.Tests; +[Collection("Sequential")] public class InProcessDotnetInteractiveKernelBuilderTest { [Fact] public async Task ItAddCSharpKernelTestAsync() { - var kernel = DotnetInteractiveKernelBuilder + using var kernel = DotnetInteractiveKernelBuilder .CreateEmptyInProcessKernelBuilder() .AddCSharpKernel() .Build(); @@ -29,7 +30,7 @@ public async Task ItAddCSharpKernelTestAsync() [Fact] public async Task ItAddPowershellKernelTestAsync() { - var kernel = DotnetInteractiveKernelBuilder + using var kernel = DotnetInteractiveKernelBuilder .CreateEmptyInProcessKernelBuilder() .AddPowershellKernel() .Build(); @@ -45,7 +46,7 @@ public async Task ItAddPowershellKernelTestAsync() [Fact] public async Task ItAddFSharpKernelTestAsync() { - var kernel = DotnetInteractiveKernelBuilder + using var kernel = DotnetInteractiveKernelBuilder .CreateEmptyInProcessKernelBuilder() .AddFSharpKernel() .Build(); @@ -62,7 +63,7 @@ public async Task ItAddFSharpKernelTestAsync() [Fact] public async Task ItAddPythonKernelTestAsync() { - var kernel = DotnetInteractiveKernelBuilder + using var kernel = DotnetInteractiveKernelBuilder .CreateEmptyInProcessKernelBuilder() .AddPythonKernel("python3") .Build(); diff --git a/dotnet/website/articles/function-comparison-page-between-python-AutoGen-and-autogen.net.md b/dotnet/website/articles/function-comparison-page-between-python-AutoGen-and-autogen.net.md new file mode 100644 index 000000000000..e81b96f11bed --- /dev/null +++ b/dotnet/website/articles/function-comparison-page-between-python-AutoGen-and-autogen.net.md @@ -0,0 +1,37 @@ +### Function comparison between Python AutoGen and AutoGen\.Net + + +#### Agentic pattern + +| Feature | AutoGen | AutoGen\.Net | +| :---------------- | :------ | :---- | +| Code interpreter | run python code in local/docker/notebook executor | run csharp code in dotnet interactive executor | +| Single agent chat pattern | ✔️ | ✔️ | +| Two agent chat pattern | ✔️ | ✔️ | +| group chat (include FSM)| ✔️ | ✔️ (using workflow for FSM groupchat) | +| Nest chat| ✔️ | ✔️ (using middleware pattern)| +|Sequential chat | ✔️ | ❌ (need to manually create task in code) | +| Tool | ✔️ | ✔️ | + + +#### LLM platform support + +ℹ️ Note + +``` Other than the platforms list below, AutoGen.Net also supports all the platforms that semantic kernel supports via AutoGen.SemanticKernel as a bridge ``` + +| Feature | AutoGen | AutoGen\.Net | +| :---------------- | :------ | :---- | +| OpenAI (include third-party) | ✔️ | ✔️ | +| Mistral | ✔️| ✔️| +| Ollama | ✔️| ✔️| +|Claude |✔️ |✔️| +|Gemini (Include Vertex) | ✔️ | ✔️ | + +#### Popular Contrib Agent support + + +| Feature | AutoGen | AutoGen\.Net | +| :---------------- | :------ | :---- | +| Rag Agent | ✔️| ❌ | +| Web surfer | ✔️| ❌ | diff --git a/dotnet/website/release_note/0.1.0.md b/dotnet/website/release_note/0.1.0.md new file mode 100644 index 000000000000..dc844087758c --- /dev/null +++ b/dotnet/website/release_note/0.1.0.md @@ -0,0 +1,41 @@ +# 🎉 Release Notes: AutoGen.Net 0.1.0 🎉 + +## 📦 New Packages + +1. **Add AutoGen.AzureAIInference Package** + - **Issue**: [.Net][Feature Request] [#3323](https://github.com/microsoft/autogen/issues/3323) + - **Description**: The new `AutoGen.AzureAIInference` package includes the `ChatCompletionClientAgent`. + +## ✨ New Features + +1. **Enable Step-by-Step Execution for Two Agent Chat API** + - **Issue**: [.Net][Feature Request] [#3339](https://github.com/microsoft/autogen/issues/3339) + - **Description**: The `AgentExtension.SendAsync` now returns an `IAsyncEnumerable`, allowing conversations to be driven step by step, similar to how `GroupChatExtension.SendAsync` works. + +2. **Support Python Code Execution in AutoGen.DotnetInteractive** + - **Issue**: [.Net][Feature Request] [#3316](https://github.com/microsoft/autogen/issues/3316) + - **Description**: `dotnet-interactive` now supports Jupyter kernel connection, allowing Python code execution in `AutoGen.DotnetInteractive`. + +3. **Support Prompt Cache in Claude** + - **Issue**: [.Net][Feature Request] [#3359](https://github.com/microsoft/autogen/issues/3359) + - **Description**: Claude now supports prompt caching, which dramatically lowers the bill if the cache is hit. Added the corresponding option in the Claude client. + +## 🐛 Bug Fixes + +1. **GroupChatExtension.SendAsync Doesn’t Terminate Chat When `IOrchestrator` Returns Null as Next Agent** + - **Issue**: [.Net][Bug] [#3306](https://github.com/microsoft/autogen/issues/3306) + - **Description**: Fixed an issue where `GroupChatExtension.SendAsync` would continue until the max_round is reached even when `IOrchestrator` returns null as the next speaker. + +2. **InitializedMessages Are Added Repeatedly in GroupChatExtension.SendAsync Method** + - **Issue**: [.Net][Bug] [#3268](https://github.com/microsoft/autogen/issues/3268) + - **Description**: Fixed an issue where initialized messages from group chat were being added repeatedly in every iteration of the `GroupChatExtension.SendAsync` API. + +3. **Remove `Azure.AI.OpenAI` Dependency from `AutoGen.DotnetInteractive`** + - **Issue**: [.Net][Feature Request] [#3273](https://github.com/microsoft/autogen/issues/3273) + - **Description**: Fixed an issue by removing the `Azure.AI.OpenAI` dependency from `AutoGen.DotnetInteractive`, simplifying the package and reducing dependencies. + +## 📄 Documentation Updates + +1. **Add Function Comparison Page Between Python AutoGen and AutoGen.Net** + - **Issue**: [.Net][Document] [#3184](https://github.com/microsoft/autogen/issues/3184) + - **Description**: Added comparative documentation for features between AutoGen and AutoGen.Net across various functionalities and platform supports. \ No newline at end of file diff --git a/dotnet/website/release_note/toc.yml b/dotnet/website/release_note/toc.yml index f8753cacc890..9c8008e705e1 100644 --- a/dotnet/website/release_note/toc.yml +++ b/dotnet/website/release_note/toc.yml @@ -1,3 +1,6 @@ +- name: 0.1.0 + href: 0.1.0.md + - name: 0.0.17 href: 0.0.17.md diff --git a/dotnet/website/toc.yml b/dotnet/website/toc.yml index ad5d0e2b695d..18a7eae08a83 100644 --- a/dotnet/website/toc.yml +++ b/dotnet/website/toc.yml @@ -3,13 +3,16 @@ - name: Tutorial href: tutorial/ - + - name: API Reference href: api/ - name: Release Notes href: release_note/ +- name: Comparison between Python AutoGen and AutoGen.Net + href: articles/function-comparison-page-between-python-AutoGen-and-autogen.net.md + - name: Other Languages dropdown: true items: diff --git a/notebook/lats_search.ipynb b/notebook/lats_search.ipynb new file mode 100644 index 000000000000..01b4449890ed --- /dev/null +++ b/notebook/lats_search.ipynb @@ -0,0 +1,1059 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "211913e6", + "metadata": {}, + "source": [ + "# Language Agent Tree Search\n", + "\n", + "[Language Agent Tree Search](https://arxiv.org/abs/2310.04406) (LATS), by Zhou, et. al, is a general LLM agent search algorithm that combines reflection/evaluation and search (specifically Monte-Carlo tree search) to achieve stronger overall task performance by leveraging inference-time compute.\n", + "\n", + "It has four main phases consisting of six steps:\n", + "\n", + "1. Select: pick the best next state to progress from, based on its aggregate value. \n", + "2. Expand and simulate: sample n potential actions to take and execute them in parallel.\n", + "3. Reflect + Evaluate: observe the outcomes of these actions and score the decisions based on reflection (and possibly external feedback if available)\n", + "4. Backpropagate: update the scores of the root trajectories based on the outcomes.\n", + "\n", + "![lats](https://i.postimg.cc/NjQScLTv/image.png)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "da705b29", + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "import logging\n", + "import os\n", + "import uuid\n", + "from typing import Any, Dict, List\n", + "\n", + "from autogen import AssistantAgent, ConversableAgent, GroupChat, UserProxyAgent, config_list_from_json" + ] + }, + { + "cell_type": "markdown", + "id": "293fd23b", + "metadata": {}, + "source": [ + "# Configure logging\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a02f8a2c", + "metadata": {}, + "outputs": [], + "source": [ + "logging.basicConfig(level=logging.INFO)" + ] + }, + { + "cell_type": "markdown", + "id": "1d5ca06b", + "metadata": {}, + "source": [ + "# Set environment variables\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1566c7df", + "metadata": {}, + "outputs": [], + "source": [ + "os.environ[\"AUTOGEN_USE_DOCKER\"] = \"0\" # Disable Docker usage globally for Autogen\n", + "os.environ[\"OPENAI_API_KEY\"] = \"YOUR_API_KEY\"" + ] + }, + { + "cell_type": "markdown", + "id": "585654ac", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "\n", + "Install `autogen` (for the LLM framework and agents)\n", + "\n", + "Required packages: autogen\n", + "\n", + "Please ensure these packages are installed before running this script" + ] + }, + { + "cell_type": "markdown", + "id": "586bcf0f", + "metadata": {}, + "source": [ + "# Directly create the config_list with the API key" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9eaf711f", + "metadata": {}, + "outputs": [], + "source": [ + "config_list = [{\"model\": \"gpt-4o-mini\", \"api_key\": \"YOUR_API_KEY\"}]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "79701018", + "metadata": {}, + "outputs": [], + "source": [ + "if not config_list:\n", + " raise ValueError(\"Failed to create configuration. Please check the API key.\")" + ] + }, + { + "cell_type": "markdown", + "id": "9041e0a3", + "metadata": {}, + "source": [ + "### Reflection Class\n", + "\n", + "The reflection chain will score agent outputs based on the decision and the tool responses." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ce0288e9", + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "from pydantic import BaseModel, Field\n", + "\n", + "\n", + "class Reflection(BaseModel):\n", + " reflections: str = Field(\n", + " description=\"The critique and reflections on the sufficiency, superfluency,\"\n", + " \" and general quality of the response\"\n", + " )\n", + " score: int = Field(\n", + " description=\"Score from 0-10 on the quality of the candidate response.\",\n", + " gte=0,\n", + " lte=10,\n", + " )\n", + " found_solution: bool = Field(description=\"Whether the response has fully solved the question or task.\")\n", + "\n", + " def as_message(self):\n", + " return {\"role\": \"human\", \"content\": f\"Reasoning: {self.reflections}\\nScore: {self.score}\"}\n", + "\n", + " @property\n", + " def normalized_score(self) -> float:\n", + " return self.score / 10.0" + ] + }, + { + "cell_type": "markdown", + "id": "1f6d3476", + "metadata": {}, + "source": [ + "## Tree State\n", + "\n", + "LATS is based on a (greedy) Monte-Carlo tree search. For each search steps, it picks the node with the highest \"upper confidence bound\", which is a metric that balances exploitation (highest average reward) and exploration (lowest visits). Starting from that node, it generates N (5 in this case) new candidate actions to take, and adds them to the tree. It stops searching either when it has generated a valid solution OR when it has reached the maximum number of rollouts (search tree depth)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b6d0d7a6", + "metadata": {}, + "outputs": [], + "source": [ + "import math\n", + "import os\n", + "from collections import deque\n", + "from typing import Optional" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "305a29d6", + "metadata": {}, + "outputs": [], + "source": [ + "class Node:\n", + " def __init__(\n", + " self,\n", + " messages: List[Dict[str, str]],\n", + " reflection: Optional[Reflection] = None,\n", + " parent: Optional[\"Node\"] = None,\n", + " ):\n", + " self.messages = messages\n", + " self.parent = parent\n", + " self.children: List[\"Node\"] = []\n", + " self.value = 0.0\n", + " self.visits = 0\n", + " self.reflection = reflection\n", + " self.depth = parent.depth + 1 if parent is not None else 1\n", + " self._is_solved = reflection.found_solution if reflection else False\n", + " if self._is_solved:\n", + " self._mark_tree_as_solved()\n", + " if reflection:\n", + " self.backpropagate(reflection.normalized_score)\n", + "\n", + " def __repr__(self) -> str:\n", + " return (\n", + " f\"\"\n", + " )\n", + "\n", + " @property\n", + " def is_solved(self) -> bool:\n", + " \"\"\"If any solutions exist, we can end the search.\"\"\"\n", + " return self._is_solved\n", + "\n", + " @property\n", + " def is_terminal(self):\n", + " return not self.children\n", + "\n", + " @property\n", + " def best_child(self):\n", + " \"\"\"Select the child with the highest UCT to search next.\"\"\"\n", + " if not self.children:\n", + " return None\n", + " all_nodes = self._get_all_children()\n", + " return max(all_nodes, key=lambda child: child.upper_confidence_bound())\n", + "\n", + " @property\n", + " def best_child_score(self):\n", + " \"\"\"Return the child with the highest value.\"\"\"\n", + " if not self.children:\n", + " return None\n", + " return max(self.children, key=lambda child: int(child.is_solved) * child.value)\n", + "\n", + " @property\n", + " def height(self) -> int:\n", + " \"\"\"Check for how far we've rolled out the tree.\"\"\"\n", + " if self.children:\n", + " return 1 + max([child.height for child in self.children])\n", + " return 1\n", + "\n", + " def upper_confidence_bound(self, exploration_weight=1.0):\n", + " \"\"\"Return the UCT score. This helps balance exploration vs. exploitation of a branch.\"\"\"\n", + " if self.parent is None:\n", + " raise ValueError(\"Cannot obtain UCT from root node\")\n", + " if self.visits == 0:\n", + " return self.value\n", + " # Encourages exploitation of high-value trajectories\n", + " average_reward = self.value / self.visits\n", + " exploration_term = math.sqrt(math.log(self.parent.visits) / self.visits)\n", + " return average_reward + exploration_weight * exploration_term\n", + "\n", + " def backpropagate(self, reward: float):\n", + " \"\"\"Update the score of this node and its parents.\"\"\"\n", + " node = self\n", + " while node:\n", + " node.visits += 1\n", + " node.value = (node.value * (node.visits - 1) + reward) / node.visits\n", + " node = node.parent\n", + "\n", + " def get_messages(self, include_reflections: bool = True):\n", + " if include_reflections and self.reflection:\n", + " return self.messages + [self.reflection.as_message()]\n", + " return self.messages\n", + "\n", + " def get_trajectory(self, include_reflections: bool = True) -> List[Dict[str, str]]:\n", + " \"\"\"Get messages representing this search branch.\"\"\"\n", + " messages = []\n", + " node = self\n", + " while node:\n", + " messages.extend(node.get_messages(include_reflections=include_reflections)[::-1])\n", + " node = node.parent\n", + " # Reverse the final back-tracked trajectory to return in the correct order\n", + " return messages[::-1] # root solution, reflection, child 1, ...\n", + "\n", + " def _get_all_children(self):\n", + " all_nodes = []\n", + " nodes = deque()\n", + " nodes.append(self)\n", + " while nodes:\n", + " node = nodes.popleft()\n", + " all_nodes.extend(node.children)\n", + " for n in node.children:\n", + " nodes.append(n)\n", + " return all_nodes\n", + "\n", + " def get_best_solution(self):\n", + " \"\"\"Return the best solution from within the current sub-tree.\"\"\"\n", + " all_nodes = [self] + self._get_all_children()\n", + " best_node = max(\n", + " all_nodes,\n", + " # We filter out all non-terminal, non-solution trajectories\n", + " key=lambda node: int(node.is_terminal and node.is_solved) * node.value,\n", + " )\n", + " return best_node\n", + "\n", + " def _mark_tree_as_solved(self):\n", + " parent = self.parent\n", + " while parent:\n", + " parent._is_solved = True\n", + " parent = parent.parent" + ] + }, + { + "cell_type": "markdown", + "id": "98b719d9", + "metadata": {}, + "source": [ + "The main component is the tree, represented by the root node." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "586d953a", + "metadata": {}, + "outputs": [], + "source": [ + "from typing_extensions import TypedDict\n", + "\n", + "\n", + "class TreeState(TypedDict):\n", + " # The full tree\n", + " root: Node\n", + " # The original input\n", + " input: str" + ] + }, + { + "cell_type": "markdown", + "id": "3a61a6ee", + "metadata": {}, + "source": [ + "## Define Language Agent\n", + "\n", + "Our agent will have three primary LLM-powered processes:\n", + "\n", + "1. Reflect: score the action based on the tool response.\n", + "2. Initial response: to create the root node and start the search.\n", + "3. Expand: generate 5 candidate \"next steps\" from the best spot in the current tree\n", + "\n", + "For more \"Grounded\" tool applications (such as code synthesis), you could integrate code execution into the reflection/reward step. This type of external feedback is very useful." + ] + }, + { + "cell_type": "markdown", + "id": "a9e6c27f", + "metadata": {}, + "source": [ + "#### Tools\n", + "For our example, we will give the language agent a search engine." + ] + }, + { + "cell_type": "markdown", + "id": "ffb10a00", + "metadata": {}, + "source": [ + "Define the UserProxyAgent with web search / tool-use capability\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e467f73e", + "metadata": {}, + "outputs": [], + "source": [ + "user_proxy = UserProxyAgent(\n", + " name=\"user\",\n", + " human_input_mode=\"NEVER\",\n", + " max_consecutive_auto_reply=10,\n", + " code_execution_config={\n", + " \"work_dir\": \"web\",\n", + " \"use_docker\": False,\n", + " },\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "5c2b96b2", + "metadata": {}, + "source": [ + "Create a ConversableAgent without tools\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "212daaef", + "metadata": {}, + "outputs": [], + "source": [ + "assistant_agent = ConversableAgent(\n", + " name=\"assistant_agent\",\n", + " system_message=\"You are an AI assistant capable of helping with various tasks.\",\n", + " human_input_mode=\"NEVER\",\n", + " code_execution_config=False,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "527c1a39", + "metadata": {}, + "source": [ + "### Reflection\n", + "\n", + "Self-reflection allows the agent to boostrap, improving its future responses based on the outcome of previous ones. In agents this is more powerful since it can use external feedback to improve." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3bdd8a23", + "metadata": {}, + "outputs": [], + "source": [ + "reflection_prompt = \"\"\"\n", + "Reflect and grade the assistant response to the user question below.\n", + "User question: {input}\n", + "Assistant response: {candidate}\n", + "\n", + "Provide your reflection in the following format:\n", + "Reflections: [Your detailed critique and reflections]\n", + "Score: [A score from 0-10]\n", + "Found Solution: [true/false]\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7750d32f", + "metadata": {}, + "outputs": [], + "source": [ + "reflection_agent = AssistantAgent(\n", + " name=\"reflection_agent\",\n", + " system_message=\"You are an AI assistant that reflects on and grades responses.\",\n", + " llm_config={\n", + " \"config_list\": config_list,\n", + " \"temperature\": 0.2,\n", + " },\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "23f26bf0", + "metadata": {}, + "outputs": [], + "source": [ + "def reflection_chain(inputs: Dict[str, Any]) -> Reflection:\n", + " try:\n", + " candidate_content = \"\"\n", + " if \"candidate\" in inputs:\n", + " candidate = inputs[\"candidate\"]\n", + " if isinstance(candidate, list):\n", + " candidate_content = (\n", + " candidate[-1][\"content\"]\n", + " if isinstance(candidate[-1], dict) and \"content\" in candidate[-1]\n", + " else str(candidate[-1])\n", + " )\n", + " elif isinstance(candidate, dict):\n", + " candidate_content = candidate.get(\"content\", str(candidate))\n", + " elif isinstance(candidate, str):\n", + " candidate_content = candidate\n", + " else:\n", + " candidate_content = str(candidate)\n", + "\n", + " formatted_prompt = [\n", + " {\"role\": \"system\", \"content\": \"You are an AI assistant that reflects on and grades responses.\"},\n", + " {\n", + " \"role\": \"user\",\n", + " \"content\": reflection_prompt.format(input=inputs.get(\"input\", \"\"), candidate=candidate_content),\n", + " },\n", + " ]\n", + " response = reflection_agent.generate_reply(formatted_prompt)\n", + "\n", + " # Parse the response\n", + " response_str = str(response)\n", + " lines = response_str.split(\"\\n\")\n", + " reflections = next((line.split(\": \", 1)[1] for line in lines if line.startswith(\"Reflections:\")), \"\")\n", + " score_str = next((line.split(\": \", 1)[1] for line in lines if line.startswith(\"Score:\")), \"0\")\n", + " try:\n", + " if \"/\" in score_str:\n", + " numerator, denominator = map(int, score_str.split(\"/\"))\n", + " score = int((numerator / denominator) * 10)\n", + " else:\n", + " score = int(score_str)\n", + " except ValueError:\n", + " logging.warning(f\"Invalid score value: {score_str}. Defaulting to 0.\")\n", + " score = 0\n", + "\n", + " found_solution = next(\n", + " (line.split(\": \", 1)[1].lower() == \"true\" for line in lines if line.startswith(\"Found Solution:\")), False\n", + " )\n", + "\n", + " if not reflections:\n", + " logging.warning(\"No reflections found in the response. Using default values.\")\n", + " reflections = \"No reflections provided.\"\n", + "\n", + " return Reflection(reflections=reflections, score=score, found_solution=found_solution)\n", + " except Exception as e:\n", + " logging.error(f\"Error in reflection_chain: {str(e)}\", exc_info=True)\n", + " return Reflection(reflections=f\"Error in reflection: {str(e)}\", score=0, found_solution=False)" + ] + }, + { + "cell_type": "markdown", + "id": "fc4b9911", + "metadata": {}, + "source": [ + "### Initial Response\n", + "\n", + "We start with a single root node, generated by this first step. It responds to the user input either with a tool invocation or a response." + ] + }, + { + "cell_type": "markdown", + "id": "60675131", + "metadata": {}, + "source": [ + "# Create Autogen agents\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fd743ab5", + "metadata": {}, + "outputs": [], + "source": [ + "assistant = AssistantAgent(name=\"assistant\", llm_config={\"config_list\": config_list}, code_execution_config=False)\n", + "user = UserProxyAgent(\n", + " name=\"user\",\n", + " human_input_mode=\"NEVER\",\n", + " max_consecutive_auto_reply=10,\n", + " code_execution_config={\"work_dir\": \"web\", \"use_docker\": False},\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "1f93b734", + "metadata": {}, + "source": [ + "# Define a function to create the initial prompt\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b7e00575", + "metadata": {}, + "outputs": [], + "source": [ + "def create_initial_prompt(input_text):\n", + " return [\n", + " {\"role\": \"system\", \"content\": \"You are an AI assistant.\"},\n", + " {\"role\": \"user\", \"content\": input_text},\n", + " ]" + ] + }, + { + "cell_type": "markdown", + "id": "b8442317", + "metadata": {}, + "source": [ + "# Function to generate initial response\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b7afcd1b", + "metadata": {}, + "outputs": [], + "source": [ + "def generate_initial_response(state: TreeState) -> TreeState:\n", + " chat_messages = create_initial_prompt(state[\"input\"])\n", + " try:\n", + " # Ensure chat_messages is a list of dictionaries\n", + " if not isinstance(chat_messages, list):\n", + " chat_messages = [{\"role\": \"user\", \"content\": chat_messages}]\n", + "\n", + " logging.info(f\"Generating initial response for input: {state['input']}\")\n", + " logging.debug(f\"Chat messages: {chat_messages}\")\n", + "\n", + " response = assistant.generate_reply(chat_messages)\n", + " logging.debug(f\"Raw response from assistant: {response}\")\n", + "\n", + " # Ensure response is properly formatted as a string\n", + " if isinstance(response, str):\n", + " content = response\n", + " elif isinstance(response, dict) and \"content\" in response:\n", + " content = response[\"content\"]\n", + " elif isinstance(response, list) and len(response) > 0:\n", + " content = response[-1].get(\"content\", str(response[-1]))\n", + " else:\n", + " content = str(response)\n", + "\n", + " content = content.strip()\n", + " if not content:\n", + " raise ValueError(\"Generated content is empty after processing\")\n", + "\n", + " logging.debug(f\"Processed content: {content[:100]}...\") # Log first 100 chars\n", + "\n", + " # Generate reflection\n", + " reflection_input = {\"input\": state[\"input\"], \"candidate\": content}\n", + " logging.info(\"Generating reflection on the initial response\")\n", + " reflection = reflection_chain(reflection_input)\n", + " logging.debug(f\"Reflection generated: {reflection}\")\n", + "\n", + " # Create Node with messages as a list containing a single dict\n", + " messages = [{\"role\": \"assistant\", \"content\": content}]\n", + " root = Node(messages=messages, reflection=reflection)\n", + "\n", + " logging.info(\"Initial response and reflection generated successfully\")\n", + " return TreeState(root=root, input=state[\"input\"])\n", + "\n", + " except Exception as e:\n", + " logging.error(f\"Error in generate_initial_response: {str(e)}\", exc_info=True)\n", + " return TreeState(root=None, input=state[\"input\"])" + ] + }, + { + "cell_type": "markdown", + "id": "87ef17ca", + "metadata": {}, + "source": [ + "# Example usage of the generate_initial_response function\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7ab75669", + "metadata": {}, + "outputs": [], + "source": [ + "initial_prompt = \"Why is the sky blue?\"\n", + "initial_state = TreeState(input=initial_prompt, root=None)\n", + "result_state = generate_initial_response(initial_state)\n", + "if result_state[\"root\"] is not None:\n", + " print(result_state[\"root\"].messages[0][\"content\"])\n", + "else:\n", + " print(\"Failed to generate initial response.\")" + ] + }, + { + "cell_type": "markdown", + "id": "e619223f", + "metadata": {}, + "source": [ + "#### Starting Node\n", + "\n", + "We will package up the candidate generation and reflection in a single node of our graph. This is represented by the following function:" + ] + }, + { + "cell_type": "markdown", + "id": "24c052e0", + "metadata": {}, + "source": [ + "\n", + "# Define the function to generate the initial response" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "94c92498", + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Define the function to generate the initial response\n", + "\n", + "\n", + "def generate_initial_response(state: TreeState) -> TreeState:\n", + " \"\"\"Generate the initial candidate response using Autogen components.\"\"\"\n", + " assistant = AssistantAgent(name=\"assistant\", llm_config={\"config_list\": config_list}, code_execution_config=False)\n", + "\n", + " # Generate initial response\n", + " initial_message = [\n", + " {\"role\": \"system\", \"content\": \"You are an AI assistant.\"},\n", + " {\"role\": \"user\", \"content\": state[\"input\"]},\n", + " ]\n", + "\n", + " try:\n", + " logging.info(f\"Generating initial response for input: {state['input']}\")\n", + " response = assistant.generate_reply(initial_message)\n", + " logging.debug(f\"Raw response from assistant: {response}\")\n", + "\n", + " # Ensure response is properly formatted as a string\n", + " if isinstance(response, str):\n", + " content = response\n", + " elif isinstance(response, dict):\n", + " content = response.get(\"content\", \"\")\n", + " if not content:\n", + " content = json.dumps(response)\n", + " elif isinstance(response, list):\n", + " content = \" \".join(str(item) for item in response)\n", + " else:\n", + " content = str(response)\n", + "\n", + " # Ensure content is always a string and not empty\n", + " content = content.strip()\n", + " if not content:\n", + " raise ValueError(\"Generated content is empty after processing\")\n", + "\n", + " logging.debug(f\"Final processed content (first 100 chars): {content[:100]}...\")\n", + "\n", + " # Generate reflection\n", + " logging.info(\"Generating reflection on the initial response\")\n", + " reflection_input = {\"input\": state[\"input\"], \"candidate\": content}\n", + " reflection = reflection_chain(reflection_input)\n", + " logging.debug(f\"Reflection generated: {reflection}\")\n", + "\n", + " if not isinstance(reflection, Reflection):\n", + " raise TypeError(f\"Invalid reflection type: {type(reflection)}. Expected Reflection, got {type(reflection)}\")\n", + "\n", + " # Create Node with messages as a list containing a single dict\n", + " messages = [{\"role\": \"assistant\", \"content\": content}]\n", + " logging.debug(f\"Creating Node with messages: {messages}\")\n", + " root = Node(messages=messages, reflection=reflection)\n", + " logging.info(\"Initial response and reflection generated successfully\")\n", + " logging.debug(f\"Created root node: {root}\")\n", + " return TreeState(root=root, input=state[\"input\"])\n", + "\n", + " except Exception as e:\n", + " logging.error(f\"Error in generate_initial_response: {str(e)}\", exc_info=True)\n", + " return TreeState(root=None, input=state[\"input\"])" + ] + }, + { + "cell_type": "markdown", + "id": "c58a4074", + "metadata": {}, + "source": [ + "### Candidate Generation\n", + "The following code prompts the same LLM to generate N additional candidates to check.\n", + "\n", + "This generates N candidate values for a single input to sample actions from the environment" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "27a3a1db", + "metadata": {}, + "outputs": [], + "source": [ + "def generate_candidates(messages: list, config: dict):\n", + " n = config.get(\"N\", 5)\n", + " assistant = AssistantAgent(name=\"assistant\", llm_config={\"config_list\": config_list}, code_execution_config=False)\n", + "\n", + " candidates = []\n", + " for _ in range(n):\n", + " try:\n", + " # Use the assistant to generate a response\n", + " last_message = messages[-1][\"content\"] if messages and isinstance(messages[-1], dict) else str(messages[-1])\n", + " response = assistant.generate_reply([{\"role\": \"user\", \"content\": last_message}])\n", + " if isinstance(response, str):\n", + " candidates.append(response)\n", + " elif isinstance(response, dict) and \"content\" in response:\n", + " candidates.append(response[\"content\"])\n", + " elif (\n", + " isinstance(response, list) and response and isinstance(response[-1], dict) and \"content\" in response[-1]\n", + " ):\n", + " candidates.append(response[-1][\"content\"])\n", + " else:\n", + " candidates.append(str(response))\n", + " except Exception as e:\n", + " logging.error(f\"Error generating candidate: {str(e)}\")\n", + " candidates.append(\"Failed to generate candidate.\")\n", + "\n", + " if not candidates:\n", + " logging.warning(\"No candidates were generated.\")\n", + "\n", + " return candidates\n", + "\n", + "\n", + "expansion_chain = generate_candidates" + ] + }, + { + "cell_type": "markdown", + "id": "a47c8161", + "metadata": {}, + "source": [ + "#### Candidate generation node\n", + "\n", + "We will package the candidate generation and reflection steps in the following \"expand\" node.\n", + "We do all the operations as a batch process to speed up execution." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "175afca7", + "metadata": {}, + "outputs": [], + "source": [ + "def expand(state: TreeState, config: Dict[str, Any]) -> dict:\n", + " root = state[\"root\"]\n", + " best_candidate: Node = root.best_child if root.children else root\n", + " messages = best_candidate.get_trajectory()\n", + "\n", + " # Generate N candidates using Autogen's generate_candidates function\n", + " new_candidates = generate_candidates(messages, config)\n", + "\n", + " # Reflect on each candidate using Autogen's AssistantAgent\n", + " reflections = []\n", + " for candidate in new_candidates:\n", + " reflection = reflection_chain({\"input\": state[\"input\"], \"candidate\": candidate})\n", + " reflections.append(reflection)\n", + "\n", + " # Grow tree\n", + " child_nodes = [\n", + " Node([{\"role\": \"assistant\", \"content\": candidate}], parent=best_candidate, reflection=reflection)\n", + " for candidate, reflection in zip(new_candidates, reflections)\n", + " ]\n", + " best_candidate.children.extend(child_nodes)\n", + "\n", + " # We have already extended the tree directly, so we just return the state\n", + " return state" + ] + }, + { + "cell_type": "markdown", + "id": "717b7b93", + "metadata": {}, + "source": [ + "## Create Tree\n", + "\n", + "With those two nodes defined, we are ready to define the tree. After each agent step, we have the option of finishing." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e309ea9f", + "metadata": {}, + "outputs": [], + "source": [ + "from typing import Any, Dict, Literal\n", + "\n", + "\n", + "def should_loop(state: Dict[str, Any]) -> Literal[\"expand\", \"end\"]:\n", + " \"\"\"Determine whether to continue the tree search.\"\"\"\n", + " root = state[\"root\"]\n", + " if root.is_solved:\n", + " return \"end\"\n", + " if root.height > 5:\n", + " return \"end\"\n", + " return \"expand\"\n", + "\n", + "\n", + "def run_lats(input_query: str, max_iterations: int = 10):\n", + " import logging\n", + "\n", + " logging.basicConfig(level=logging.INFO)\n", + " logger = logging.getLogger(__name__)\n", + "\n", + " try:\n", + "\n", + " state = {\"input\": input_query, \"root\": None}\n", + " try:\n", + " state = generate_initial_response(state)\n", + " if not isinstance(state, dict) or \"root\" not in state or state[\"root\"] is None:\n", + " logger.error(\"Initial response generation failed or returned invalid state\")\n", + " return \"Failed to generate initial response.\"\n", + " logger.info(\"Initial response generated successfully\")\n", + " except Exception as e:\n", + " logger.error(f\"Error generating initial response: {str(e)}\", exc_info=True)\n", + " return \"Failed to generate initial response due to an unexpected error.\"\n", + "\n", + " for iteration in range(max_iterations):\n", + " action = should_loop(state)\n", + " if action == \"end\":\n", + " logger.info(f\"Search ended after {iteration + 1} iterations\")\n", + " break\n", + " try:\n", + " state = expand(\n", + " state,\n", + " {\n", + " \"N\": 5,\n", + " \"input_query\": input_query,\n", + " },\n", + " )\n", + " logger.info(f\"Completed iteration {iteration + 1}\")\n", + " except Exception as e:\n", + " logger.error(f\"Error during iteration {iteration + 1}: {str(e)}\", exc_info=True)\n", + " continue\n", + "\n", + " if not isinstance(state, dict) or \"root\" not in state or state[\"root\"] is None:\n", + " return \"No valid solution found due to an error in the search process.\"\n", + "\n", + " solution_node = state[\"root\"].get_best_solution()\n", + " best_trajectory = solution_node.get_trajectory(include_reflections=False)\n", + " if not best_trajectory:\n", + " return \"No solution found in the search process.\"\n", + "\n", + " result = (\n", + " best_trajectory[-1].get(\"content\") if isinstance(best_trajectory[-1], dict) else str(best_trajectory[-1])\n", + " )\n", + " logger.info(\"LATS search completed successfully\")\n", + " return result\n", + " except Exception as e:\n", + " logger.error(f\"An unexpected error occurred during LATS execution: {str(e)}\", exc_info=True)\n", + " return f\"An unexpected error occurred: {str(e)}\"" + ] + }, + { + "cell_type": "markdown", + "id": "e274e373", + "metadata": {}, + "source": [ + "Example usage:\n", + "\n", + "result = run_lats(\"Write a research report on deep learning.\")\n", + "\n", + "print(result)" + ] + }, + { + "cell_type": "markdown", + "id": "aa719ff2", + "metadata": {}, + "source": [ + "\n", + "# Example usage of the LATS algorithm with Autogen" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "683c0f2c", + "metadata": {}, + "outputs": [], + "source": [ + "import logging\n", + "\n", + "logging.basicConfig(level=logging.INFO, format=\"%(asctime)s - %(levelname)s - %(message)s\")\n", + "logger = logging.getLogger(__name__)\n", + "\n", + "\n", + "def run_lats_example(question):\n", + " try:\n", + " logger.info(f\"Processing question: {question}\")\n", + " result = run_lats(question)\n", + " logger.info(f\"LATS algorithm completed. Result: {result[:100]}...\") # Log first 100 chars of result\n", + " print(f\"Question: {question}\")\n", + " print(f\"Answer: {result}\")\n", + " except Exception as e:\n", + " logger.error(f\"An error occurred while processing the question: {str(e)}\", exc_info=True)\n", + " print(f\"An error occurred: {str(e)}\")\n", + " finally:\n", + " print(\"---\")" + ] + }, + { + "cell_type": "markdown", + "id": "a4ce778e", + "metadata": {}, + "source": [ + "# List of example questions\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "60fa1f07", + "metadata": {}, + "outputs": [], + "source": [ + "questions = [\n", + " \"Explain how epigenetic modifications can influence gene expression across generations and the implications for evolution.\",\n", + " \"Discuss the challenges of grounding ethical theories in moral realism, especially in light of the is-ought problem introduced by Hume.\",\n", + " \"How does the Riemann Hypothesis relate to the distribution of prime numbers, and why is it significant in number theory?\",\n", + " \"Describe the challenges and theoretical underpinnings of unifying general relativity with quantum mechanics, particularly focusing on string theory and loop quantum gravity.\",\n", + "]" + ] + }, + { + "cell_type": "markdown", + "id": "a0fed5fe", + "metadata": {}, + "source": [ + "# Run LATS algorithm for each question\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5d1e5754", + "metadata": {}, + "outputs": [], + "source": [ + "for i, question in enumerate(questions, 1):\n", + " print(f\"\\nExample {i}:\")\n", + " run_lats_example(question)\n", + "\n", + "logger.info(\"All examples processed.\")" + ] + }, + { + "cell_type": "markdown", + "id": "af7254a5", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "Congrats on implementing LATS! This is a technique that can be reasonably fast and effective at solving complex agent tasks. A few notes that you probably observed above:\n", + "\n", + "1. While LATS is effective, the tree rollout process can require additional inference compute time. If you plan to integrate this into a production application, consider streaming intermediate steps to allow users to see the thought process and access intermediate results. Alternatively, you could use it to generate fine-tuning data to enhance single-shot accuracy and avoid lengthy rollouts. The cost of using LATS has significantly decreased since its initial proposal and is expected to continue decreasing.\n", + "\n", + "2. The effectiveness of the candidate selection process depends on the quality of the rewards generated. In this example, we exclusively use self-reflection as feedback, but if you have access to external feedback sources (such as code test execution), those should be incorporated as suggested above." + ] + }, + { + "cell_type": "markdown", + "id": "be01ff1e", + "metadata": {}, + "source": [ + "# \n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/samples/tools/autogenbench/pyproject.toml b/samples/tools/autogenbench/pyproject.toml index 8cabc4b55e67..ef1a2fe80dfb 100644 --- a/samples/tools/autogenbench/pyproject.toml +++ b/samples/tools/autogenbench/pyproject.toml @@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta" [project] name = "autogenbench" authors = [ - { name="Autogen Team", email="auto-gen@outlook.com" }, + { name="Autogen Team", email="autogen-contact@service.microsoft.com" }, ] description = "AutoGen Testbed Tools" readme = "README.md" diff --git a/setup.py b/setup.py index 13a88be5f0a4..95b2cda212ae 100644 --- a/setup.py +++ b/setup.py @@ -88,7 +88,7 @@ "types": ["mypy==1.9.0", "pytest>=6.1.1,<8"] + jupyter_executor, "long-context": ["llmlingua<0.3"], "anthropic": ["anthropic>=0.23.1"], - "mistral": ["mistralai>=0.2.0"], + "mistral": ["mistralai>=1.0.1"], "groq": ["groq>=0.9.0"], "cohere": ["cohere>=5.5.8"], } @@ -97,7 +97,7 @@ name="pyautogen", version=__version__, author="AutoGen", - author_email="auto-gen@outlook.com", + author_email="autogen-contact@service.microsoft.com", description="Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework", long_description=long_description, long_description_content_type="text/markdown", diff --git a/test/agentchat/test_groupchat.py b/test/agentchat/test_groupchat.py index 20a83685178d..291a94d450f1 100755 --- a/test/agentchat/test_groupchat.py +++ b/test/agentchat/test_groupchat.py @@ -724,7 +724,7 @@ def test_clear_agents_history(): agent1_history = list(agent1._oai_messages.values())[0] agent2_history = list(agent2._oai_messages.values())[0] assert agent1_history == [ - {"content": "hello", "role": "assistant"}, + {"content": "hello", "role": "assistant", "name": "alice"}, {"content": "This is bob speaking.", "name": "bob", "role": "user"}, {"content": "How you doing?", "name": "sam", "role": "user"}, ] @@ -745,7 +745,7 @@ def test_clear_agents_history(): {"content": "How you doing?", "name": "sam", "role": "user"}, ] assert agent2_history == [ - {"content": "This is bob speaking.", "role": "assistant"}, + {"content": "This is bob speaking.", "role": "assistant", "name": "bob"}, {"content": "How you doing?", "name": "sam", "role": "user"}, ] assert groupchat.messages == [ @@ -759,12 +759,12 @@ def test_clear_agents_history(): agent1_history = list(agent1._oai_messages.values())[0] agent2_history = list(agent2._oai_messages.values())[0] assert agent1_history == [ - {"content": "hello", "role": "assistant"}, + {"content": "hello", "role": "assistant", "name": "alice"}, {"content": "This is bob speaking.", "name": "bob", "role": "user"}, {"content": "How you doing?", "name": "sam", "role": "user"}, ] assert agent2_history == [ - {"content": "This is bob speaking.", "role": "assistant"}, + {"content": "This is bob speaking.", "role": "assistant", "name": "bob"}, {"content": "How you doing?", "name": "sam", "role": "user"}, ] assert groupchat.messages == [ @@ -822,6 +822,7 @@ def test_clear_agents_history(): "content": "example tool response", "tool_responses": [{"tool_call_id": "call_emulated", "role": "tool", "content": "example tool response"}], "role": "tool", + "name": "alice", }, ] @@ -1218,7 +1219,7 @@ def test_role_for_select_speaker_messages(): # into a message attribute called 'override_role'. This is evaluated in Conversable Agent's _append_oai_message function # e.g.: message={'content':self.select_speaker_prompt(agents),'override_role':self.role_for_select_speaker_messages}, message = {"content": "A prompt goes here.", "override_role": groupchat.role_for_select_speaker_messages} - checking_agent._append_oai_message(message, "assistant", speaker_selection_agent) + checking_agent._append_oai_message(message, "assistant", speaker_selection_agent, is_sending=True) # Test default is "system" assert len(checking_agent.chat_messages) == 1 @@ -1227,7 +1228,7 @@ def test_role_for_select_speaker_messages(): # Test as "user" groupchat.role_for_select_speaker_messages = "user" message = {"content": "A prompt goes here.", "override_role": groupchat.role_for_select_speaker_messages} - checking_agent._append_oai_message(message, "assistant", speaker_selection_agent) + checking_agent._append_oai_message(message, "assistant", speaker_selection_agent, is_sending=True) assert len(checking_agent.chat_messages) == 1 assert checking_agent.chat_messages[speaker_selection_agent][-1]["role"] == "user" @@ -1235,7 +1236,7 @@ def test_role_for_select_speaker_messages(): # Test as something unusual groupchat.role_for_select_speaker_messages = "SockS" message = {"content": "A prompt goes here.", "override_role": groupchat.role_for_select_speaker_messages} - checking_agent._append_oai_message(message, "assistant", speaker_selection_agent) + checking_agent._append_oai_message(message, "assistant", speaker_selection_agent, is_sending=True) assert len(checking_agent.chat_messages) == 1 assert checking_agent.chat_messages[speaker_selection_agent][-1]["role"] == "SockS" @@ -1646,6 +1647,7 @@ def test_speaker_selection_validate_speaker_name(): True, { "content": groupchat.select_speaker_auto_multiple_template.format(agentlist=agent_list_string), + "name": "checking_agent", "override_role": groupchat.role_for_select_speaker_messages, }, ) @@ -1692,6 +1694,7 @@ def test_speaker_selection_validate_speaker_name(): True, { "content": groupchat.select_speaker_auto_none_template.format(agentlist=agent_list_string), + "name": "checking_agent", "override_role": groupchat.role_for_select_speaker_messages, }, ) @@ -1761,6 +1764,7 @@ def test_select_speaker_auto_messages(): True, { "content": custom_multiple_names_msg.replace("{agentlist}", "['Alice', 'Bob']"), + "name": "checking_agent", "override_role": groupchat.role_for_select_speaker_messages, }, ) @@ -1770,6 +1774,7 @@ def test_select_speaker_auto_messages(): True, { "content": custom_no_names_msg.replace("{agentlist}", "['Alice', 'Bob']"), + "name": "checking_agent", "override_role": groupchat.role_for_select_speaker_messages, }, ) diff --git a/test/oai/test_mistral.py b/test/oai/test_mistral.py index 5236f71d7b7d..f89c3d304d90 100644 --- a/test/oai/test_mistral.py +++ b/test/oai/test_mistral.py @@ -3,7 +3,16 @@ import pytest try: - from mistralai.models.chat_completion import ChatMessage + from mistralai import ( + AssistantMessage, + Function, + FunctionCall, + Mistral, + SystemMessage, + ToolCall, + ToolMessage, + UserMessage, + ) from autogen.oai.mistral import MistralAIClient, calculate_mistral_cost @@ -66,17 +75,16 @@ def test_cost_calculation(mock_response): cost=None, model="mistral-large-latest", ) - assert ( - calculate_mistral_cost(response.usage["prompt_tokens"], response.usage["completion_tokens"], response.model) - == 0.0001 - ), "Cost for this should be $0.0001" + assert calculate_mistral_cost( + response.usage["prompt_tokens"], response.usage["completion_tokens"], response.model + ) == (15 / 1000 * 0.0003), "Cost for this should be $0.0000045" # Test text generation @pytest.mark.skipif(skip, reason="Mistral.AI dependency is not installed") -@patch("autogen.oai.mistral.MistralClient.chat") +@patch("autogen.oai.mistral.MistralAIClient.create") def test_create_response(mock_chat, mistral_client): - # Mock MistralClient.chat response + # Mock `mistral_response = client.chat.complete(**mistral_params)` mock_mistral_response = MagicMock() mock_mistral_response.choices = [ MagicMock(finish_reason="stop", message=MagicMock(content="Example Mistral response", tool_calls=None)) @@ -108,9 +116,9 @@ def test_create_response(mock_chat, mistral_client): # Test functions/tools @pytest.mark.skipif(skip, reason="Mistral.AI dependency is not installed") -@patch("autogen.oai.mistral.MistralClient.chat") +@patch("autogen.oai.mistral.MistralAIClient.create") def test_create_response_with_tool_call(mock_chat, mistral_client): - # Mock `mistral_response = client.chat(**mistral_params)` + # Mock `mistral_response = client.chat.complete(**mistral_params)` mock_function = MagicMock(name="currency_calculator") mock_function.name = "currency_calculator" mock_function.arguments = '{"base_currency": "EUR", "quote_currency": "USD", "base_amount": 123.45}' @@ -159,7 +167,7 @@ def test_create_response_with_tool_call(mock_chat, mistral_client): {"role": "assistant", "content": "World"}, ] - # Call the create method + # Call the chat method response = mistral_client.create( {"messages": mistral_messages, "tools": converted_functions, "model": "mistral-medium-latest"} ) diff --git a/website/docs/contributor-guide/contributing.md b/website/docs/contributor-guide/contributing.md index b1b6b848f667..cd2c62e408c1 100644 --- a/website/docs/contributor-guide/contributing.md +++ b/website/docs/contributor-guide/contributing.md @@ -32,7 +32,3 @@ To see what we are working on and what we plan to work on, please check our ## Becoming a Reviewer There is currently no formal reviewer solicitation process. Current reviewers identify reviewers from active contributors. If you are willing to become a reviewer, you are welcome to let us know on discord. - -## Contact Maintainers - -The project is currently maintained by a [dynamic group of volunteers](https://butternut-swordtail-8a5.notion.site/410675be605442d3ada9a42eb4dfef30?v=fa5d0a79fd3d4c0f9c112951b2831cbb&pvs=4) from several different organizations. Contact project administrators Chi Wang and Qingyun Wu via auto-gen@outlook.com if you are interested in becoming a maintainer. diff --git a/website/docs/topics/non-openai-models/cloud-gemini_vertexai.ipynb b/website/docs/topics/non-openai-models/cloud-gemini_vertexai.ipynb index 545f97a2971b..637d340dc37f 100644 --- a/website/docs/topics/non-openai-models/cloud-gemini_vertexai.ipynb +++ b/website/docs/topics/non-openai-models/cloud-gemini_vertexai.ipynb @@ -62,7 +62,12 @@ "\n", "\n", "\n", - "For the sake of simplicity we will assign the Editor role to our service account for autogen on our Autogen-with-Gemini Google Cloud project.\n", + "Next we assign the [Vertex AI User](https://cloud.google.com/vertex-ai/docs/general/access-control#aiplatform.user) for the service account. This can be done in the [Google Cloud console](https://console.cloud.google.com/iam-admin/iam?project=autogen-with-gemini) in our `autogen-with-gemini` project.
\n", + "Alternatively, we can also grant the [Vertex AI User](https://cloud.google.com/vertex-ai/docs/general/access-control#aiplatform.user) role by running a command using the gcloud CLI, for example in [Cloud Shell](https://shell.cloud.google.com/cloudshell):\n", + "```bash\n", + "gcloud projects add-iam-policy-binding autogen-with-gemini \\\n", + " --member=serviceAccount:autogen@autogen-with-gemini.iam.gserviceaccount.com --role roles/aiplatform.user\n", + "```\n", "\n", "* Under IAM & Admin > Service Account select the newly created service accounts, and click the option \"Manage keys\" among the items. \n", "* From the \"ADD KEY\" dropdown select \"Create new key\" and select the JSON format and click CREATE.\n", @@ -83,7 +88,7 @@ "Additionally, AutoGen also supports authentication using `Credentials` objects in Python with the [google-auth library](https://google-auth.readthedocs.io/), which enables even more flexibility.
\n", "For example, we can even use impersonated credentials.\n", "\n", - "#### Use Service Account Keyfile\n", + "#### Use Service Account Keyfile\n", "\n", "The Google Cloud service account can be specified by setting the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path to the JSON key file of the service account.
\n", "\n", @@ -91,7 +96,7 @@ "\n", "#### Use the Google Default Credentials\n", "\n", - "If you are using [Cloud Shell](https://shell.cloud.google.com/cloudshell) or [Cloud Shell editor](https://shell.cloud.google.com/cloudshell/editor) in Google Cloud,
then you are already authenticated. If you have the Google Cloud SDK installed locally,
then you can login by running `gcloud auth login` in the command line. \n", + "If you are using [Cloud Shell](https://shell.cloud.google.com/cloudshell) or [Cloud Shell editor](https://shell.cloud.google.com/cloudshell/editor) in Google Cloud,
then you are already authenticated. If you have the Google Cloud SDK installed locally,
then you can login by running `gcloud auth application-default login` in the command line. \n", "\n", "Detailed instructions for installing the Google Cloud SDK can be found [here](https://cloud.google.com/sdk/docs/install).\n", "\n", @@ -99,7 +104,7 @@ "\n", "The google-auth library supports a wide range of authentication scenarios, and you can simply pass a previously created `Credentials` object to the `llm_config`.
\n", "The [official documentation](https://google-auth.readthedocs.io/) of the Python package provides a detailed overview of the supported methods and usage examples.
\n", - "If you are already authenticated, like in [Cloud Shell](https://shell.cloud.google.com/cloudshell), or after running the `gcloud auth login` command in a CLI, then the `google.auth.default()` Python method will automatically return your currently active credentials." + "If you are already authenticated, like in [Cloud Shell](https://shell.cloud.google.com/cloudshell), or after running the `gcloud auth application-default login` command in a CLI, then the `google.auth.default()` Python method will automatically return your currently active credentials." ] }, { @@ -307,13 +312,13 @@ "gcloud auth application-default login\n", "gcloud config set project autogen-with-gemini\n", "```\n", - "The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is a path to our service account JSON keyfile, as described in the [Use Service Account Keyfile](#Use Service Account Keyfile) section above.
\n", + "The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is a path to our service account JSON keyfile, as described in the [Use Service Account Keyfile](#use_svc_keyfile) section above.
\n", "We also need to set the Google cloud project, which is `autogen-with-gemini` in this example.

\n", "\n", - "Note, we could also run `gcloud auth login` in case we wish to use our personal Google account instead of a service account.\n", + "Note, we could also run `gcloud auth application-default login` to use our personal Google account instead of a service account.\n", "In this case we need to run the following commands:\n", "```bash\n", - "gcloud auth login\n", + "gcloud gcloud auth application-default login\n", "gcloud config set project autogen-with-gemini\n", "```" ] @@ -390,6 +395,132 @@ ".\"\"\",\n", ")" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Use Gemini via the OpenAI Library in Autogen\n", + "Using Gemini via the OpenAI library is also possible once you are already authenticated.
\n", + "Run `gcloud auth application-default login` to set up application default credentials locally for the example below.
\n", + "Also set the Google cloud project on the CLI if you have not done so far:
\n", + "```bash\n", + "gcloud config set project autogen-with-gemini\n", + "```\n", + "The prerequisites are essentially the same as in the example above.
\n", + "\n", + "You can read more on the topic in the [official Google docs](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-gemini-using-openai-library).\n", + "
A list of currently supported models can also be found in the [docs](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-gemini-using-openai-library#supported_models)\n", + "
\n", + "
\n", + "Note, that you will need to refresh your token regularly, by default every 1 hour." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "import google.auth\n", + "\n", + "scopes = [\"https://www.googleapis.com/auth/cloud-platform\"]\n", + "creds, project = google.auth.default(scopes)\n", + "auth_req = google.auth.transport.requests.Request()\n", + "creds.refresh(auth_req)\n", + "location = \"us-west1\"\n", + "prompt_price_per_1k = (\n", + " 0.000125 # For more up-to-date prices see https://cloud.google.com/vertex-ai/generative-ai/pricing\n", + ")\n", + "completion_token_price_per_1k = (\n", + " 0.000375 # For more up-to-date prices see https://cloud.google.com/vertex-ai/generative-ai/pricing\n", + ")\n", + "\n", + "openai_gemini_config = [\n", + " {\n", + " \"model\": \"google/gemini-1.5-pro-001\",\n", + " \"api_type\": \"openai\",\n", + " \"base_url\": f\"https://{location}-aiplatform.googleapis.com/v1beta1/projects/{project}/locations/{location}/endpoints/openapi\",\n", + " \"api_key\": creds.token,\n", + " \"price\": [prompt_price_per_1k, completion_token_price_per_1k],\n", + " }\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33muser_proxy\u001b[0m (to assistant):\n", + "\n", + "\n", + " Compute the integral of the function f(x)=x^3 on the interval 0 to 10 using a Python script,\n", + " which returns the value of the definite integral.\n", + "\n", + "--------------------------------------------------------------------------------\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33massistant\u001b[0m (to user_proxy):\n", + "\n", + "```python\n", + "# filename: integral.py\n", + "def integrate_x_cubed(a, b):\n", + " \"\"\"\n", + " This function calculates the definite integral of x^3 from a to b.\n", + "\n", + " Args:\n", + " a: The lower limit of integration.\n", + " b: The upper limit of integration.\n", + "\n", + " Returns:\n", + " The value of the definite integral.\n", + " \"\"\"\n", + " return (b**4 - a**4) / 4\n", + "\n", + "# Calculate the integral of x^3 from 0 to 10\n", + "result = integrate_x_cubed(0, 10)\n", + "\n", + "# Print the result\n", + "print(result)\n", + "```\n", + "\n", + "This script defines a function `integrate_x_cubed` that takes the lower and upper limits of integration as arguments and returns the definite integral of x^3 using the power rule of integration. The script then calls this function with the limits 0 and 10 and prints the result.\n", + "\n", + "Execute the script `python integral.py`, you should get the result: `2500.0`.\n", + "\n", + "TERMINATE\n", + "\n", + "\n", + "--------------------------------------------------------------------------------\n" + ] + } + ], + "source": [ + "assistant = AssistantAgent(\"assistant\", llm_config={\"config_list\": openai_gemini_config}, max_consecutive_auto_reply=3)\n", + "\n", + "user_proxy = UserProxyAgent(\n", + " \"user_proxy\",\n", + " code_execution_config={\"work_dir\": \"coding\", \"use_docker\": False},\n", + " human_input_mode=\"NEVER\",\n", + " is_termination_msg=lambda x: content_str(x.get(\"content\")).find(\"TERMINATE\") >= 0,\n", + ")\n", + "\n", + "result = user_proxy.initiate_chat(\n", + " assistant,\n", + " message=\"\"\"\n", + " Compute the integral of the function f(x)=x^3 on the interval 0 to 10 using a Python script,\n", + " which returns the value of the definite integral.\"\"\",\n", + ")" + ] } ], "metadata": { diff --git a/website/docs/tutorial/human-in-the-loop.ipynb b/website/docs/tutorial/human-in-the-loop.ipynb index afcdeeaf42bf..8bf0aab16d0d 100644 --- a/website/docs/tutorial/human-in-the-loop.ipynb +++ b/website/docs/tutorial/human-in-the-loop.ipynb @@ -41,7 +41,7 @@ " termination based on `max_consecutive_auto_reply` is ignored.\n", "\n", "The previous chapters already showed many examples of the cases when `human_input_mode` is `NEVER`. \n", - "Below we show one such example again and then show the differences when this mode is set to `ALWAYS` and `NEVER` instead." + "Below we show one such example again and then show the differences when this mode is set to `ALWAYS` and `TERMINATE` instead." ] }, {