The method signature suggests it unloads a specific model, but it actually unloads client.SelectedModel, which can lead to unexpected behavior.
|
/// <summary> |
|
/// Sends a request to /api/generate with <c>keep_alive</c> set to 0 to immediately unload a model from memory. |
|
/// </summary> |
|
/// <param name="client">The client used to execute the command.</param> |
|
/// <param name="model">The name of the model to unload.</param> |
|
/// <param name="cancellationToken">The token to cancel the operation with.</param> |
|
/// <returns>A task that completes when the unload request has been sent.</returns> |
|
public static async Task RequestModelUnloadAsync(this IOllamaApiClient client, string model, CancellationToken cancellationToken = default) |
|
{ |
|
var request = new GenerateRequest |
|
{ |
|
Model = client.SelectedModel, |
|
Stream = false, |
|
KeepAlive = "0s" |
|
}; |
|
await foreach (var _ in client.GenerateAsync(request, cancellationToken)) |
|
{ |
|
break; |
|
} |
|
} |
The method signature suggests it unloads a specific model, but it actually unloads client.SelectedModel, which can lead to unexpected behavior.
OllamaSharp/src/OllamaSharp/OllamaApiClientExtensions.cs
Lines 94 to 113 in 6ef00f4