[Issue]: Unable to register custom model client for Assistant Agent

[kyokohunter]

Describe the issue

I'd like to use the custom model feature as described in AutoGen with Custom Models: Empowering Users to Use Their Own Inference Mechanism to create a dummy model with no LLM connection, so I can develop my script (e.g. build out front-end UI) without needing to interact with a remote or local LLM - a random quote is to be returned for prompts.

In my simple and naive script to test the concept, at the point that I register the dummy model client with an AssistantAgent I get an error that my dummy model is not found in the CONFIG_LIST. However immediately prior to the error, AutoGen reports the detection of the custom model in the config and informs us that register_model_client needs to be called for it. This seems like a contradiction, but I don't know if the problem is my script or if there is a bug.

Steps to reproduce

Step 1: Run following Python script

import random
from autogen import AssistantAgent, UserProxyAgent, GroupChat, Agent, GroupChatManager

class DummyModelClient:
    """
    A non-interactive dummy client to use with AutoGen.
    Use when testing features that don't require a live LLM connection e.g. UI changes.
    Based on code from https://microsoft.github.io/autogen/blog/2024/01/26/Custom-Models/
    """
    def __init__(self, config, **kwargs):
        print(f"DummyModelClient config: {config}")
        
    def create(self, params):
        num_of_responses = params.get("n", 1)
        
        # Create a dummy data response class
        # adhering to AutoGen's ModelClientResponseProtocol
        response = SimpleNamespace()
        response.choices = []
        response.model = "dummy-model-client"
        
        for _ in range(num_of_responses):
            text = random.choice(
                [
                    "What's your favorite thing about space? Mine is space.",
                    "Space space wanna go to space yes please space. Space space. Go to space.",
                    "Atmosphere. Black holes. Astronauts. Nebulas. Jupiter. The Big Dipper.",
                    "Ohhh, the Sun. I'm gonna meet the Sun. Oh no! What'll I say? 'Hi! Hi, Sun!' Oh, boy!",
                    "Look, an eclipse! No. Don't look."
                ]
            )
            choice = SimpleNamespace()
            choice.message = SimpleNamespace()
            choice.message.content = text
            choice.message.function_call = None
            response.choices.append(choice)
        return response
        
    def message_retrieval(self, response):
        choices = response.choices
        return [choice.message.content for choice in choices]
        
    def cost(self, response) -> float:
        response.cost = 0
        return 0
        
    @staticmethod
    def get_usage(response):
        return {}
            
llm_config = {
    "config_list": [
        {"model_client_cls":"DummyModelClient", "model":"dummy-model-client"}
    ],
    "seed": 42,  # seed for reproducibility
    "temperature": 0  # temperature of 0 means deterministic output
}

agents = []
agents.append(
    AssistantAgent(
        name="assistantComedian",
        system_message="You are a comedian. You specialise in jokes about toast.",
        llm_config=llm_config
    )
)
agents[-1].register_model_client(model_client_cls=DummyModelClient)
        
agents.append(
    TrackableUserProxyAgent(
        name="user",
        is_termination_msg=lambda x: x.get("content", "").strip().endswith("TERMINATE"),
        code_execution_config=False
    )
)
        
group_chat = GroupChat(
    agents=agents,
    messages=[],
    max_round=5
)
        
manager = GroupChatManager(
    groupchat=group_chat,
    llm_config=llm_config,
    code_execution_config=False,
    is_termination_msg=lambda x: x.get("content", "").strip().endswith("TERMINATE")
)

user_proxy.a_initiate_chat(
    manager,
    message="Tell me a joke.",
    max_consecutive_auto_reply=5,
    is_termination_msg=lambda x: x.get("content", "").strip().endswith("TERMINATE"),
)

See error in Screenshots and logs below

Screenshots and logs

[autogen.oai.client: 02-13 14:35:04] {403} INFO - Detected custom model client in config: DummyModelClient, model client can not be used until register_model_client is called.
Traceback (most recent call last):
  File "/home/autogen/autogen/myapp/personality-core-space.py", line 67, in <module>
    agents[-1].register_model_client(model_client_cls=DummyModelClient)
  File "/home/autogen/autogen/autogen/agentchat/conversable_agent.py", line 2206, in register_model_client
    self.client.register_model_client(model_client_cls, **kwargs)
  File "/home/autogen/autogen/autogen/oai/client.py", line 437, in register_model_client
    raise ValueError(
ValueError: Model client "DummyModelClient" is being registered but was not found in the config_list. Please make sure to include an entry in the config_list with "model_client_cls": "DummyModelClient"

Additional Information

AutoGen Version: v0.2.11
Operating System: Windows 10
Python Version: 3.11
Docker Version: 4.26.1

[ekzhu]

@olgavrou has a PR that is fixing this exact issue: #1653. @kyokohunter you can take a look. Otherwise please check if this is fixed once that PR gets into the next release.

[ekzhu]

@kyokohunter BTW, I see you forgot to register custom client for the group chat manager in your code -- you might want to use a standard model for group chat manager, e.g., GPT-4.

[woodswift]

@ekzhu I got a similar issue when using custom model client. Based on {the groupchat notebook}(https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat.ipynb), there was not a step to register custom client. My best guess is that the registration step was hidden somewhere or missed in the notebook. Could you please double check this?

[olgavrou]

@woodswift

If you are using a custom model you need to register it. The group chat notebook was made as an example for group chat and not for custom models specifically, so if you are modifying it with custom models you need to add additional steps

[woodswift]

@woodswift

If you are using a custom model you need to register it. The group chat notebook was made as an example for group chat and not for custom models specifically, so if you are modifying it with custom models you need to add additional steps

Thank you @olgavrou for replying to my comment! Could you please point me to the instruction of those additional steps you mentioned above?

I have created a new issue (#1738) to describe my issue in detail. In short, I was trying to combine loading custom model with configuring agents with tools provided. However, I got stuck in how to register the custom model.

[ekzhu]

@olgavrou could you follow up with @woodswift here?

[olgavrou]

@woodswift the additional steps are

1. defining the model in the OAI_CONFIG_FILE with at least:

{
        "model_client_cls": "DummyModelClient",
        "model": "<modelname>"
    }

2. after you have created any agent to use this specific config, but before initiating any chat with that agent, you need to call

agents.register_model_client(model_client_cls=DummyModelClient)

is that still not working?

[olgavrou]

Reading the other thread this specific issue seems to be solved, so we can close this issue @woodswift ?

[woodswift]

Reading the other thread this specific issue seems to be solved, so we can close this issue @woodswift ?

Hi @olgavrou, please feel free to do so. I am no longer blocked. Thank you so much for your time and help!