Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[.Net][Document]: Tool call using ollama and OpenAIChatAgent #3248

Closed
LittleLittleCloud opened this issue Jul 29, 2024 · 1 comment
Closed
Assignees

Comments

@LittleLittleCloud
Copy link
Collaborator

Is your feature request related to a problem? Please describe.

Starting form 0.3.0, ollama supports tool call in both its api and openai server mode

Adding a document on how to use tool with ollama

  • model: llama3.1 (needs to be a model that support tool call)

code

// Copyright (c) Microsoft Corporation. All rights reserved.
// Tool_Call_With_Ollama_And_LiteLLM.cs

using AutoGen.Core;
using AutoGen.OpenAI.Extension;
using Azure.AI.OpenAI;
using Azure.Core.Pipeline;

namespace AutoGen.OpenAI.Sample;

#region Function
public partial class Function
{
    [Function]
    public async Task<string> GetWeatherAsync(string city)
    {
        return await Task.FromResult("The weather in " + city + " is 72 degrees and sunny.");
    }
}
#endregion Function

public class Tool_Call_With_Ollama_And_LiteLLM
{
    public static async Task RunAsync()
    {
        // Before running this code, make sure you have
        // - Ollama:
        //  - Version > 0.3.0
        //  - Ollama running on http://localhost:11434

        # region Create_tools
        var functions = new Function();
        var functionMiddleware = new FunctionCallMiddleware(
            functions: [functions.GetWeatherAsyncFunctionContract],
            functionMap: new Dictionary<string, Func<string, Task<string>>>
            {
                { functions.GetWeatherAsyncFunctionContract.Name!, functions.GetWeatherAsyncWrapper },
            });
        #endregion Create_tools
        #region Create_Agent
        var liteLLMUrl = "https://56lbqqkh-11434.asse.devtunnels.ms";
        using var httpClient = new HttpClient(new CustomHttpClientHandler(liteLLMUrl));
        var option = new OpenAIClientOptions(OpenAIClientOptions.ServiceVersion.V2024_04_01_Preview)
        {
            Transport = new HttpClientTransport(httpClient),
        };

        // api-key is not required for local server
        // so you can use any string here
        var openAIClient = new OpenAIClient("api-key", option);

        var agent = new OpenAIChatAgent(
            openAIClient: openAIClient,
            name: "assistant",
            modelName: "llama3.1",
            systemMessage: "You are a helpful AI assistant")
            .RegisterMessageConnector()
            .RegisterMiddleware(functionMiddleware)
            .RegisterPrintMessage();

        var reply = await agent.SendAsync("what's the weather in new york");
        #endregion Create_Agent
    }
}

Describe the solution you'd like

No response

Additional context

No response

@LittleLittleCloud
Copy link
Collaborator Author

fix by #3402

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant