BasicPlanner doesn't work with 'from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion'

[yuichiromukaiyama]

This issue is related to #4923. I am posting this as a question as well, as I have been struggling with how to handle this issue. Here is a minimal code that reproduces the problem.

kernel = sk.Kernel()

kernel.add_chat_service(
    "chat_completion",
    AzureChatCompletion(
        deployment_name=AZURE_OPENAI_DEPLOYMENT_NAME,
        endpoint=AZURE_OPENAI_ENDPOINT,
        api_key=AZURE_OPENAI_API_KEY,
    ),
)

planner = BasicPlanner()
plan = planner.create_plan(goal=goal, kernel=kernel)

When I run it, I get the error 'Choice' object has no attribute 'logprobs' as in Issue.
Investigating the cause, I found that BasicPlanner is calling create_semantic_function internally.

semantic-kernel/python/semantic_kernel/planning/basic_planner.py

Line 186 in 1490eb6

planner = kernel.create_semantic_function(prompt, max_tokens=1000, temperature=0.8)

Further tracing inside reveals the following line, which uses PromptTemplate.

semantic-kernel/python/semantic_kernel/kernel.py

Line 870 in 1490eb6

template = PromptTemplate(prompt_template, self.prompt_template_engine, config)

This causes harm in the following places has_chat_prompt is always False for PromptTemplate.

semantic-kernel/python/semantic_kernel/kernel.py

Line 727 in 1490eb6

if function_config.has_chat_prompt:

@dataclass
class SemanticFunctionConfig:
    prompt_template_config: "PromptTemplateConfig"
    prompt_template: "PromptTemplate"

    @property
    def has_chat_prompt(self) -> bool:
        return isinstance(self.prompt_template, ChatPromptTemplate)

Therefore, AzureChatCompletion is not used when creating a plan in BasicPlanner. As proof, I change PromptTemplate to ChatPromptTemplate and run it and it works fine.

Is there a workable solution to this? I am thinking as follows

1. Python: AttributeError: 'Choice' object has no attribute 'logprobs' #4923 made 'logprobs' nullable for ChatCompletion only. Publish a PR that modifies TextCompletion as well.

2. Publish a PR to change 'PromptTemplate' to 'ChatPromptTemplate' according to chat-service.

3. BasicPlanner uses the principle TextCompletion or AzureTextCompletion

※ In case 3, I think it's better to change the Quick Start.
https://github.com/microsoft/semantic-kernel/blob/main/python/notebooks/05-using-the-planner.ipynb

[yuichiromukaiyama]

same problem

#4924 (comment)

[ymuichiro]

PR

https://github.com/microsoft/semantic-kernel/pull/5071/files

[ymuichiro]

Merged

[moonbox3]

Hi @yuichiromukaiyama, please, in the future, open this as a GitHub issue instead of a Discussion. We'll be able to get to this much faster. Thank you!

[yuichiromukaiyama]

From now on, I will write directly to the issue. Thank you!