Skip to content

The model argument is ignored in RequestModelUnloadAsync #386

Description

@yamanaiyuki

The method signature suggests it unloads a specific model, but it actually unloads client.SelectedModel, which can lead to unexpected behavior.

/// <summary>
/// Sends a request to /api/generate with <c>keep_alive</c> set to 0 to immediately unload a model from memory.
/// </summary>
/// <param name="client">The client used to execute the command.</param>
/// <param name="model">The name of the model to unload.</param>
/// <param name="cancellationToken">The token to cancel the operation with.</param>
/// <returns>A task that completes when the unload request has been sent.</returns>
public static async Task RequestModelUnloadAsync(this IOllamaApiClient client, string model, CancellationToken cancellationToken = default)
{
var request = new GenerateRequest
{
Model = client.SelectedModel,
Stream = false,
KeepAlive = "0s"
};
await foreach (var _ in client.GenerateAsync(request, cancellationToken))
{
break;
}
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions