Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions docs/website/content/docs/reference/api/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1294,9 +1294,10 @@ console.log(await response.text);

Unloads a previously loaded model from the server.

When the last model is unloaded (no more models remain), this function
automatically closes the RPC connection, allowing the process to exit
naturally without requiring manual cleanup.
When no models or providers remain active, the SDK can auto-close the RPC
connection so the host process exits naturally. The default flips by
runtime: **on** for Node/Electron, **off** for Bare. Pass `autoClose` to
override.

**Signature**:

Expand All @@ -1308,6 +1309,18 @@ function unloadModel(params: UnloadModelParams): Promise<void>;

- `QvacErrorBase` — When the response type is invalid or when the unload operation fails

**Example**:

```ts
await unloadModel({ modelId });

// Force close on Bare
await unloadModel({ modelId, autoClose: true });

// Keep the connection open on Node/Electron
await unloadModel({ modelId, autoClose: false });
```

---

## Objects
Expand Down
11 changes: 8 additions & 3 deletions packages/sdk/client/api/unload-model.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { isBare } from "which-runtime";
import { type UnloadModelRequest, type UnloadModelParams } from "@/schemas";
import { send, close } from "@/client/rpc/rpc-client";
import { stopLoggingStreamForModel } from "@/client/logging-stream-registry";
Expand All @@ -13,12 +14,15 @@ const logger = getClientLogger();
* Unloads a previously loaded model from the server.
*
* When the last model is unloaded (no more models remain), this function
* automatically closes the RPC connection, allowing the process to exit
* naturally without requiring manual cleanup.
* automatically closes the RPC connection on Node/Electron, allowing the
* process to exit naturally without requiring manual cleanup. On Bare the
* connection is left open by default so long-lived workers survive a routine
* unload; pass `autoClose: true` to opt in to closing.
*
* @param params - The parameters for unloading the model
* @param params.modelId - The unique identifier of the model to unload
* @param params.clearStorage - Whether to clear the storage for the model
* @param params.autoClose - Override the runtime-default auto-close behavior
* @throws {QvacErrorBase} When the response type is invalid or when the unload operation fails
*/
export async function unloadModel(params: UnloadModelParams) {
Expand All @@ -39,8 +43,9 @@ export async function unloadModel(params: UnloadModelParams) {

stopLoggingStreamForModel(params.modelId);

// Auto-close when no models remain AND no providers are active
const shouldAutoClose = params.autoClose ?? !isBare;
if (
shouldAutoClose &&
response.hasActiveModels === false &&
response.hasActiveProviders === false
) {
Expand Down
5 changes: 4 additions & 1 deletion packages/sdk/schemas/unload-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,13 @@ import { z } from "zod";
export const unloadModelParamsSchema = z.object({
modelId: z.string(),
clearStorage: z.boolean().default(false),
autoClose: z.boolean().optional(),
});

export const unloadModelRequestSchema = unloadModelParamsSchema.extend({
export const unloadModelRequestSchema = z.object({
type: z.literal("unloadModel"),
modelId: z.string(),
clearStorage: z.boolean().default(false),
});

export const unloadModelResponseSchema = z.object({
Expand Down
Loading