Implement Overwrite Tools Functionality in GPTAssistantAgent

[jtrugman]

Summary:

This pull request introduces a new feature to the GPTAssistantAgent class within autogen, specifically focusing on enhancing the flexibility and control over the assistant's tools configuration. The key addition is the overwrite_tools logic, which allows users to explicitly decide whether they want to overwrite the existing tools of an OpenAI Assistant with a new set specified in the llm_config.

Why are these changes needed?

Prior to this update, the GPTAssistantAgent class did not provide a straightforward mechanism to update or overwrite the tools of an existing assistant. This limitation posed challenges in dynamic environments where the needs or capabilities associated with an assistant might change over time. With the introduction of the overwrite_tools flag, users now have the ability to update the toolset of an assistant easily, enhancing the assistant's adaptability and usefulness in various scenarios.

Details of the Change:

• Added a boolean flag overwrite_tools to the GPTAssistantAgent class constructor. This flag determines whether the tools of an existing assistant should be overwritten by the tools specified in the llm_config (similar to the current overwrite_instructions boolean).
• Implemented logic to check if the overwrite_tools flag is set to True. If so, and if tools are specified in llm_config, the assistant's tools are updated accordingly.
• Included checks to handle scenarios where no tools are specified or the assistant currently has no tools set
• Add relevant unit test to cover the new functionality, ensuring the feature works as expected

This enhancement aligns with the goal of making the GPTAssistantAgent more flexible and capable of adapting to changing requirements, thereby improving its utility in a broader range of applications.

Related issue number

N/A

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[IANTHEREAL]

Thanks for your contributor @jtrugman .

I'm inspired by your contribution! It sparked a new idea—what if we added specific functions like updateTools/getTools and updateInstruction/Instruction to handle these configurations? This could potentially:

• Allow more flexible dynamic updates
• Provide a clear interface for users to interact with the assistant's settings.

What are your thoughts on this suggestion?

[gagb]

Thank you! I will take a look tomorrow morning.

[jtrugman]

Thanks for your contributor @jtrugman .

I'm inspired by your contribution! It sparked a new idea—what if we added specific functions like updateTools/getTools and updateInstruction/Instruction to handle these configurations? This could potentially:

• Allow more flexible dynamic updates
• Provide a clear interface for users to interact with the assistant's settings.

What are your thoughts on this suggestion?

I think the overwrite instructions (and now overwrite tools) is good enough and straightforward enough in the short term. But I can see that being helpful in the long term.

For example, this is how I am using both overwrites locally:

    researcher = GPTAssistantAgent(
        name=create_valid_agent_name("Researcher", recordStatus),
        instructions= researcher_instructions[recordStatus],
        overwrite_instructions = True, # overwrite any existing instructions with the ones provided
        overwrite_tools = True,        
        llm_config = {
            "config_list": config_list,
            "tools": [{
                "type": "function",
                "function": {
                    "name": "google_search",
                    "description": "search on google for more information",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "search_keyword": {
                                "type": "string",
                                "description": "Keyword optimized to return the results you are looking for"
                            }
                        },
                        "required": ["search_keyword"]
                    }
                }
            }, {
                "type": "function",
                "function": {
                    "name": "web_scraping",
                    "description": "Scrape the webpage",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "url": {
                                "type": "string",
                                "description": "the address of the website to scrape"
                            },
                            "objective": {
                                "type": "string",
                                "description": "scrape the webpage for the information you are looking"
                            }
                        },
                        "required": ["url", "objective"]
                    }
                }
            }] + researcher_tools.get(recordStatus, []),
            "assistant_id": researcher_assistant_id[recordStatus]
        }
    )

I think this is straightforward enough for now.

What would be more helpful (at least for me) though is more robust documentation with examples showing how to use the overwrite instructions and pass in the tools in the llm_config. Happy to contribute to this to by the way.

[gagb]

@jtrugman , when I run the tests, I get

"FAILED test/agentchat/contrib/test_gpt_assistant.py::test_gpt_assistant_tools_overwrite - AttributeError: 'GPTAssistantAgent' object has no attribute 'get_assistant_tools'"

[IANTHEREAL]

@jtrugman After looking at your test cases, I realized that getTools might be quite useful. It seems necessary to check the current state before updating tools, which is where my idea comes from.

Feel free to proceed with the current tasks, and we can consider my suggestions at a later stage. It would be really helpful if we could include a comprehensive example to demonstrate the use of the 'overwrite' feature. Could you contribute such an example?

[jtrugman]

@jtrugman could you please fix the code format error? https://github.com/microsoft/autogen/actions/runs/7715536058/job/21068429961?pr=1434 You can use pre-commit to help.

Thanks @sonichi, just fixed the code formatting.

Please let me know if there are any additional changes you would like me to make

[jtrugman]

@gagb I agree with you. Now this pr is LGTM

Sounds good to me, will submit a new PR with the notebook for it

[gagb]

@gagb I agree with you. Now this pr is LGTM

@IANTHEREAL , @jtrugman Are all the tests passing for you? All tests pass for current main.

For this branch, I am still getting this error in dev container, when I run pytest.

self = Assistant(id='asst_choZ4Fp9s2AMP7Fs16dRgVuM', created_at=1706738604, description=None, file_ids=[], instructions='This...a={}, model='gpt-4-1106-preview', name='For test_gpt_assistant_existing_no_instructions', object='assistant', tools=[])
item = 'get'

    def __getattr__(self, item: str) -> Any:
        private_attributes = object.__getattribute__(self, '__private_attributes__')
        if item in private_attributes:
            attribute = private_attributes[item]
            if hasattr(attribute, '__get__'):
                return attribute.__get__(self, type(self))  # type: ignore
    
            try:
                # Note: self.__pydantic_private__ cannot be None if self.__private_attributes__ has items
                return self.__pydantic_private__[item]  # type: ignore
            except KeyError as exc:
                raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}') from exc
        else:
            # `__pydantic_extra__` can fail to be set if the model is not yet fully initialized.
            # See `BaseModel.__repr_args__` for more details
            try:
                pydantic_extra = object.__getattribute__(self, '__pydantic_extra__')
            except AttributeError:
                pydantic_extra = None
    
            if pydantic_extra is not None:
                try:
>                   return pydantic_extra[item]
E                   KeyError: 'get'

/home/vscode/.local/lib/python3.10/site-packages/pydantic/main.py:759: KeyError

The above exception was the direct cause of the following exception:

    @pytest.mark.skipif(
        sys.platform in ["darwin", "win32"] or skip,
        reason="do not run on MacOS or windows OR dependency is not installed OR requested to skip",
    )
    def test_gpt_assistant_existing_no_instructions():
        """
        Test function to check if the GPTAssistantAgent can retrieve instructions for an existing assistant
        even if the assistant was created with no instructions initially.
        """
        name = "For test_gpt_assistant_existing_no_instructions"
        instructions = "This is a test #1"
    
        assistant = GPTAssistantAgent(
            name,
            instructions=instructions,
            llm_config={
                "config_list": config_list,
            },
        )
    
        assistant_id = assistant.assistant_id
    
        # create a new assistant with the same ID but no instructions
>       assistant = GPTAssistantAgent(
            name,
            llm_config={
                "config_list": config_list,
                "assistant_id": assistant_id,
            },
        )

test/agentchat/contrib/test_gpt_assistant.py:172: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
autogen/agentchat/contrib/gpt_assistant_agent.py:118: in __init__
    if self._openai_assistant.get("tools"):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Assistant(id='asst_choZ4Fp9s2AMP7Fs16dRgVuM', created_at=1706738604, description=None, file_ids=[], instructions='This...a={}, model='gpt-4-1106-preview', name='For test_gpt_assistant_existing_no_instructions', object='assistant', tools=[])
item = 'get'

    def __getattr__(self, item: str) -> Any:
        private_attributes = object.__getattribute__(self, '__private_attributes__')
        if item in private_attributes:
            attribute = private_attributes[item]
            if hasattr(attribute, '__get__'):
                return attribute.__get__(self, type(self))  # type: ignore
    
            try:
                # Note: self.__pydantic_private__ cannot be None if self.__private_attributes__ has items
                return self.__pydantic_private__[item]  # type: ignore
            except KeyError as exc:
                raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}') from exc
        else:
            # `__pydantic_extra__` can fail to be set if the model is not yet fully initialized.
            # See `BaseModel.__repr_args__` for more details
            try:
                pydantic_extra = object.__getattribute__(self, '__pydantic_extra__')
            except AttributeError:
                pydantic_extra = None
    
            if pydantic_extra is not None:
                try:
                    return pydantic_extra[item]
                except KeyError as exc:
>                   raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}') from exc
E                   AttributeError: 'Assistant' object has no attribute 'get'

/home/vscode/.local/lib/python3.10/site-packages/pydantic/main.py:761: AttributeError
--------------------------------------------------------------------------------------------------- Captured log call ----------------------------------------------------------------------------------------------------
WARNING  autogen.agentchat.contrib.gpt_assistant_agent:gpt_assistant_agent.py:58 GPT Assistant only supports one OpenAI client. Using the first client in the list.
WARNING  autogen.agentchat.contrib.gpt_assistant_agent:gpt_assistant_agent.py:71 No matching assistant found, creating a new assistant
WARNING  autogen.agentchat.contrib.gpt_assistant_agent:gpt_assistant_agent.py:58 GPT Assistant only supports one OpenAI client. Using the first client in the list.
WARNING  autogen.agentchat.contrib.gpt_assistant_agent:gpt_assistant_agent.py:96 No instructions were provided for given assistant. Using existing instructions from assistant API.
==================================================================================================== warnings summary ====================================================================================================
test/agentchat/contrib/test_gpt_assistant.py::test_gpt_assistant_chat
  /home/vscode/.local/lib/python3.10/site-packages/pydantic/main.py:1024: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.6/migration/
    warnings.warn('The `dict` method is deprecated; use `model_dump` instead.', category=PydanticDeprecatedSince20)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================================ short test summary info =================================================================================================
FAILED test/agentchat/contrib/test_gpt_assistant.py::test_gpt_assistant_instructions_overwrite - AttributeError: 'Assistant' object has no attribute 'get'
FAILED test/agentchat/contrib/test_gpt_assistant.py::test_gpt_assistant_existing_no_instructions - AttributeError: 'Assistant' object has no attribute 'get'
=================================================================================== 2 failed, 6 passed, 1 warning in 70.23s (0:01:10) 

[jtrugman]

Made a fix, should resolve the issue causing the tests to fail

[IANTHEREAL]

@IANTHEREAL , @jtrugman Are all the tests passing for you? All tests pass for current main.

@gagb my local tests can pass now

[sonichi]

@skzhang1 @JieyuZ2 @qingyun-wu FYI