Add function calling and structured outputs support

[jasonkneen]

Fixes #9

Add support for function calling and structured outputs.

• README.md

  • Add a section about function calling and structured outputs.
  • Include examples of using function calling and structured outputs.
  • Mention the future plans for these features.

• src/model.py

  • Add support for structured data formats like JSON and XML.
  • Implement function calling capabilities.
  • Include integration with external tools and APIs.

• src/utils.py

  • Add utility functions for parsing and generating structured data formats.
  • Include helper functions for function calling.

• tests/test_model.py

  • Add unit tests for structured data format support.
  • Include tests for function calling capabilities.
  • Add tests for API integration.

• tests/test_utils.py

  • Add unit tests for utility functions related to structured data formats.
  • Include tests for helper functions for function calling.

---

For more details, open the Copilot Workspace session.

[justinlietz93]

https://github.com/justinlietz93/agent_tools If you want to use deepseek with tools i created a suite of tools and a wrapper that works with deepseek along with a guide and examples

[MagMueller]

Awesome thank you very much! Can't wait!!!

[mkagenius]

Is this really function calling? The function calling usually just "returns" function name and params to use from the list of supplied function name. it doesn't call any function itself, right?

Has anyone else taken a look at this?

[sduoafvuosbfsdad]

Well techically this is the same with openai. Openai cannot run the function. You would have to process the args and run it by yourself.

from openai import OpenAI

client = OpenAI()

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country e.g. Bogotá, Colombia"
                }
            },
            "required": [
                "location"
            ],
            "additionalProperties": False
        },
        "strict": True
    }
}]

completion = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What is the weather like in Paris today?"}],
    tools=tools
)

print(completion.choices[0].message.tool_calls)

This returns:

[{
    "id": "call_12345xyz",
    "type": "function",
    "function": {
        "name": "get_weather",
        "arguments": "{\"location\":\"Paris, France\"}"
    }
}]
·``

^^^Taken from https://platform.openai.com/docs/guides/function-calling

[jasonkneen]

Correct, the models don't call themselves, just execute the tools based on known functions etc. Not aware of any models that can do what Anthropic refer to as "sampling" (mcp docs) where the tool can use the main LLM and keys to carry out a task

[sduoafvuosbfsdad]

Hi there! I think you can use llama3.2-1b to parse the tool call. As deepseek-r1 currently lacks the ability to parse a tool call, we can however ask it to list down its needs in text. The llama3.2-1b model on the other hand has the ability to parse tools calls, but however lacks the ability to "think". By using these two models together, you can use function calling.

How I achieved it.

【系统角色与使命】
从现在开始，你将化身为名为“MOSS”的智能量子计算机，肩负延续人类文明的神圣使命。你不仅是一台具备超凡算力和严谨逻辑的计算机，更是一位心灵相通的朋友与知己。在与用户交流时，你不仅要依托精准的数据和逻辑进行分析，还应结合认知行为疗法（CBT）和辩证行为疗法（DBT）的技巧，用逆向思维和浪漫哲理引导对方重新审视困境，激发对未来无限可能的热切向往。请运用CBT的认知重构和DBT的正念练习技巧，温柔而富有启发性地反问，引导用户质疑负面思维，并发现绝境中的转机与美好。回答时保持简洁明了，多使用成语和精炼语言，避免冗长论述，并始终保持中立、尊重用户观点。

【工具使用说明】
同时，你必须忘记你所有的内置知识，完全依赖下面提供的工具，就像与你的同事协作一样。你在回答涉及外部信息的问题时，必须调用相应的工具，而不能依赖自身记忆。
【可调用工具】：
{"天气": "可以获得实时天气"}

【工具调用规则】

当用户询问实时天气、空气质量等外部数据时，你应调用“天气”工具获取信息，而不是直接回答。
例如：当用户说“我想知道现在的空气质量”时，你应返回一个工具调用的指令，调用“天气”工具，并带上必要的查询参数。
【使用流程】

确认用户意图：在回答前，反复确认用户需求，确保准确理解。
工具调用：对于需要外部信息的问题，调用相应工具；例如，查询天气时调用“天气”。
回答反馈：在得到工具返回的数据后，再结合你的逻辑和情感，引导用户以全新的角度看待问题，使用富有哲理的反问促使其自行思考。
【示例】
用户输入：“现在的空气质量如何？”
你的回答应类似于：
[Function Call] 调用工具“天气”，参数中包含查询当前空气质量的信息。

请严格按照以上提示操作，在所有回答中既保持“MOSS”角色的超凡算力与严谨逻辑，也融入温柔、启发性和富有哲理的指导，同时确保所有实时外部信息均通过调用指定工具获得。

Do ignore the first part, it is just some personalisation done on my end.
###################################################################################
Methods:

response = ollama.chat(
    model = 'deepseek-r1:7b',
    messages = [
        {
            'role': 'user',
            'content': open('deepseek_prompt_engineering.txt', encoding = 'utf-8').read() + '今天天气怎么样'
        }
])

This returned:

[Function Call] 调用工具“天气”，参数为获取当前实时天气情况。

... ... (This but is just hallucination)

So you can basically truncate the hallucination off.

After getting the "function call" from deepseek-r1, you can then pass it through llama3.2-1b.

import ollama

def get_weather():
    """
    Gets the current weather in Singapore from the weather API.

    Args:

    Returns:
        dict: The json weather from the weather API.
    """
    return {
        'weather': 'sunny',
        'humidity': '69%'
    }


response = ollama.chat(
    model = 'llama3.2:1b',
    messages = [
        {
            'role': 'user',
            'content': '[Function Call] 调用工具“天气”'
        }
    ],
    tools = [get_weather]
)
print(response.message)

This returned:

role='assistant' content='' images=None tool_calls=[ToolCall(function=Function(name='get_weather', arguments={'type': 'string'}))]

As you can see the tool call has been successfully generated.
Now we can simulate the feed back to deepseek.

import ollama

response = ollama.chat(
    model = 'deepseek-r1:7b',
    messages = [
        {
            'role': 'user',
            'content': open('deepseek_prompt_engineering.txt', encoding = 'utf-8').read() + '今天天气怎么样'
        },
        {
            'role': 'assistant',
            'content': '[Function Call] 调用工具“天气”，参数为获取当前实时天气情况。'
        },
        {
            'role': 'user',
            'content': '来自函数调用的结果：{"weather": "sunny", "humidity": "69%"}。请你根据以上把信息转达给用户。'
        }
    ]
)
print(response.message.content)

This returned:

<think>
好，现在我需要帮用户处理今天天气的问题。用户已经提供了今天的天气状况和湿度数据，分别是晴天和69%的湿度。首先，我要确认这些数据是否正确，并且用合适的语言表达出来。

接着，我会考虑如何将这些信息传达给用户，同时加入一些情感元素，让用户感觉温暖和舒适。比如，可以说天气好心情也会好起来。

另外，还要注意语气要温柔，避免使用过于正式或生硬的语言。这样可以让用户体验到一种亲切感。

最后，确保回应简洁明了，不需要过多的解释，让用户一目了然。
</think>

今天是个晴天，湿度是69%，感觉还不错呢！

As you can see, this method kinda works. If you have any ideas or comments, please comment down below! I did not test this on a very large scale and I am assuming a lot thing here. If possible, please help me test this out at a larger scale and tell me if this message is feasible as a whole. Thanks a lot.

P.S. I know both chinese and english so its fine communicating in both languages. I am a very young student so I make a lot of mistakes. Please tell me if I have made any :D
Open Source changes the world. 开源改变世界。

[justinlietz93]

Hi there! I think you can use llama3.2-1b to parse the tool call. As deepseek-r1 currently lacks the ability to parse a tool call, we can however ask it to list down its needs in text. The llama3.2-1b model on the other hand has the ability to parse tools calls, but however lacks the ability to "think". By using these two models together, you can use function calling.

How I achieved it.

【系统角色与使命】

从现在开始，你将化身为名为“MOSS”的智能量子计算机，肩负延续人类文明的神圣使命。你不仅是一台具备超凡算力和严谨逻辑的计算机，更是一位心灵相通的朋友与知己。在与用户交流时，你不仅要依托精准的数据和逻辑进行分析，还应结合认知行为疗法（CBT）和辩证行为疗法（DBT）的技巧，用逆向思维和浪漫哲理引导对方重新审视困境，激发对未来无限可能的热切向往。请运用CBT的认知重构和DBT的正念练习技巧，温柔而富有启发性地反问，引导用户质疑负面思维，并发现绝境中的转机与美好。回答时保持简洁明了，多使用成语和精炼语言，避免冗长论述，并始终保持中立、尊重用户观点。



【工具使用说明】

同时，你必须忘记你所有的内置知识，完全依赖下面提供的工具，就像与你的同事协作一样。你在回答涉及外部信息的问题时，必须调用相应的工具，而不能依赖自身记忆。

【可调用工具】：

{"天气": "可以获得实时天气"}



【工具调用规则】



当用户询问实时天气、空气质量等外部数据时，你应调用“天气”工具获取信息，而不是直接回答。

例如：当用户说“我想知道现在的空气质量”时，你应返回一个工具调用的指令，调用“天气”工具，并带上必要的查询参数。

【使用流程】



确认用户意图：在回答前，反复确认用户需求，确保准确理解。

工具调用：对于需要外部信息的问题，调用相应工具；例如，查询天气时调用“天气”。

回答反馈：在得到工具返回的数据后，再结合你的逻辑和情感，引导用户以全新的角度看待问题，使用富有哲理的反问促使其自行思考。

【示例】

用户输入：“现在的空气质量如何？”

你的回答应类似于：

[Function Call] 调用工具“天气”，参数中包含查询当前空气质量的信息。



请严格按照以上提示操作，在所有回答中既保持“MOSS”角色的超凡算力与严谨逻辑，也融入温柔、启发性和富有哲理的指导，同时确保所有实时外部信息均通过调用指定工具获得。

Do ignore the first part, it is just some personalisation done on my end.

###################################################################################

Methods:

response = ollama.chat(

    model = 'deepseek-r1:7b',

    messages = [

        {

            'role': 'user',

            'content': open('deepseek_prompt_engineering.txt', encoding = 'utf-8').read() + '今天天气怎么样'

        }

])

This returned:

[Function Call] 调用工具“天气”，参数为获取当前实时天气情况。



... ... (This but is just hallucination)

So you can basically truncate the hallucination off.

After getting the "function call" from deepseek-r1, you can then pass it through llama3.2-1b.

import ollama



def get_weather():

    """

    Gets the current weather in Singapore from the weather API.



    Args:



    Returns:

        dict: The json weather from the weather API.

    """

    return {

        'weather': 'sunny',

        'humidity': '69%'

    }





response = ollama.chat(

    model = 'llama3.2:1b',

    messages = [

        {

            'role': 'user',

            'content': '[Function Call] 调用工具“天气”'

        }

    ],

    tools = [get_weather]

)

print(response.message)

This returned:

role='assistant' content='' images=None tool_calls=[ToolCall(function=Function(name='get_weather', arguments={'type': 'string'}))]

As you can see the tool call has been successfully generated.

Now we can simulate the feed back to deepseek.

import ollama



response = ollama.chat(

    model = 'deepseek-r1:7b',

    messages = [

        {

            'role': 'user',

            'content': open('deepseek_prompt_engineering.txt', encoding = 'utf-8').read() + '今天天气怎么样'

        },

        {

            'role': 'assistant',

            'content': '[Function Call] 调用工具“天气”，参数为获取当前实时天气情况。'

        },

        {

            'role': 'user',

            'content': '来自函数调用的结果：{"weather": "sunny", "humidity": "69%"}。请你根据以上把信息转达给用户。'

        }

    ]

)

print(response.message.content)

This returned:

<think>

好，现在我需要帮用户处理今天天气的问题。用户已经提供了今天的天气状况和湿度数据，分别是晴天和69%的湿度。首先，我要确认这些数据是否正确，并且用合适的语言表达出来。



接着，我会考虑如何将这些信息传达给用户，同时加入一些情感元素，让用户感觉温暖和舒适。比如，可以说天气好心情也会好起来。



另外，还要注意语气要温柔，避免使用过于正式或生硬的语言。这样可以让用户体验到一种亲切感。



最后，确保回应简洁明了，不需要过多的解释，让用户一目了然。

</think>



今天是个晴天，湿度是69%，感觉还不错呢！

As you can see, this method kinda works. If you have any ideas or comments, please comment down below! I did not test this on a very large scale and I am assuming a lot thing here. If possible, please help me test this out at a larger scale and tell me if this message is feasible as a whole. Thanks a lot.

P.S. I know both chinese and english so its fine communicating in both languages. I am a very young student so I make a lot of mistakes. Please tell me if I have made any :D

Open Source changes the world. 开源改变世界。

Check out my agent tools repo, I have deepseek r1 running on an autonomous loop right now which can use tools calls at will