QVAC-24251 feat[bc|api]: drop n_discarded from the SDK config schema

.cursor/rules/sdk/docs/kv-cache-system.mdc

@@ -107,15 +107,15 @@ Named string-key caches are not inferred from their directory names and remain c
 
 | File | Purpose |
 |------|---------|
-| `server/bare/plugins/llamacpp-completion/ops/kv-cache-session.ts` | **`KvCacheSession` — single owner of the three KV-cache bookkeeping layers** (on-disk `.bin`, `initializedCaches`, `cachedMessageCounts`). Exposes `beginTurn` / `commitTurn` / `rollback` / `dropStaleSavedCount` plus the module-level `deleteKvCacheState(...)` administrative API. (QVAC-18182). |
-| `server/bare/plugins/llamacpp-completion/ops/completion-stream.ts` | Completion handler. Calls `session.beginTurn(...)`, registers `scope.defer(() => session.rollback(turn))` once, and calls `session.commitTurn(...)` on the happy path (which suppresses the deferred rollback). No direct references to the three layers. |
-| `server/bare/plugins/llamacpp-completion/ops/kv-cache-state.ts` | Pure `decideCachedHistorySlice(...)` helper used by the session — slice decision for the next addon call. No state. |
-| `server/bare/ops/kv-cache-retention.ts` | Plans idle and quota-based auto-cache evictions while excluding active turn paths. |
-| `server/bare/ops/kv-cache-utils.ts` | Path / hash / fs utilities: `getCacheFilePath`, `generateConfigHash`, `findMatchingCache`, `getCurrentCacheInfo`, `renameCacheFile`, `deleteCache`. No in-memory state. |
-| `server/bare/plugins/llamacpp-completion/ops/cache-logger.ts` | Debug logging for cache operations |
-| `server/rpc/handlers/delete-cache.ts` | `handleDeleteCache` RPC entry point. Delegates to `deleteKvCacheState(...)` — zero direct references to the three layers. |
-| `server/utils/cache.ts` | `getKVCacheDir()` base directory |
-| `client/api/delete-cache.ts` | Client-side delete cache API |
+| `packages/inference/src/plugins/builtin/llamacpp-completion/ops/kv-cache-session.ts` | **`KvCacheSession` — single owner of the KV-cache bookkeeping layers** (on-disk `.bin`, `initializedCaches`, `cachedPrefixes`, per-path write locks/active refs, auto-cache markers). Exposes `beginTurn` / `commitTurn` / `rollback` / `releaseTurn` (non-destructive: a thrown overflow, an enumerated pre-mutation refusal, or a pre-addon missing-attachment rejection never persists the turn, so the committed cache stays valid; a cache the failing turn itself primed rolls back instead) / `dropStaleSavedCount` plus the module-level `deleteKvCacheState(...)` administrative API. (QVAC-18182). |
+| `packages/inference/src/plugins/builtin/llamacpp-completion/ops/completion-stream.ts` | Completion handler. Calls `session.beginTurn(...)`, registers one deferred cleanup (rollback, or the non-destructive `releaseTurn` on a thrown addon overflow, an enumerated pre-mutation refusal, or a pre-addon missing-attachment rejection), and calls `session.commitTurn(...)` on the happy path (which suppresses it). No direct references to the session's layers. |
+| `packages/inference/src/plugins/builtin/llamacpp-completion/ops/kv-cache-state.ts` | Pure `decideCachedHistorySlice(...)` helper used by the session — slice decision for the next addon call. No state. |
+| `packages/inference/src/plugins/ops/kv-cache-retention.ts` | Plans idle and quota-based auto-cache evictions while excluding active turn paths. |
+| `packages/inference/src/plugins/ops/kv-cache-utils.ts` | Path / hash / fs utilities: `getCacheFilePath`, `generateConfigHash`, `findMatchingCache`, `getCurrentCacheInfo`, `renameCacheFile`, `deleteCache`. No in-memory state. |
+| `packages/inference/src/plugins/builtin/llamacpp-completion/ops/cache-logger.ts` | Debug logging for cache operations |
+| `packages/inference/src/handlers/delete-cache.ts` | `handleDeleteCache` RPC entry point. Delegates to `deleteKvCacheState(...)` — zero direct references to the session's layers. |
+| `packages/inference/src/utils/cache/paths.ts` | `getKVCacheDir()` base directory |
+| `packages/sdk/src/client/api/delete-cache.ts` | Client-side delete cache API |
 
 ## Key Behaviors
 
@@ -129,21 +129,23 @@ Before 0.11.0 the completion handler coordinated three independent bookkeeping l
 
 Three near-identical cleanup blocks in `completion-stream.ts` had to touch all three on every cancel / zero-token / rename-failed / tool-call exit. Any one of those blocks forgetting a layer produced the drift bugs the pitch documents (QVAC-17780 family).
 
-**0.11.0 collapses this into `KvCacheSession`**, the **single mutation point** for the three layers. The handler's loop is now:
+**0.11.0 collapses this into `KvCacheSession`**, the **single mutation point** for the bookkeeping layers (five today — see the table above). The handler's loop is now:
 
 ```typescript
 const session = createKvCacheSession(modelId);
 const turn = await session.beginTurn({ ... });          // primes cache if missing, returns handle
-scope.defer(() => session.rollback(turn));              // ONE cleanup hook for every exit path
+let preserveCacheOnUnwind = false;                      // set on a recognised pre-addon or pre-mutation failure
+scope.defer(() =>                                       // ONE cleanup hook for every exit path
+  preserveCacheOnUnwind ? session.releaseTurn(turn) : session.rollback(turn));
 // ... run model ...
-if (shouldCommit) await session.commitTurn(turn, ...);  // suppresses the deferred rollback
+if (shouldCommit) await session.commitTurn(turn, ...);  // suppresses the deferred cleanup
 ```
 
-`commitTurn` flips an internal flag on the turn handle; `rollback` reads the flag and short-circuits. The happy path commits and the deferred rollback becomes a no-op. Every other exit path (cancel, zero-token, addon error, rename failure, tool-call turn) lets `scope[Symbol.asyncDispose]` run the deferred rollback, which atomically unlinks the `.bin` file, deletes the `initializedCaches` entry, and forgets the `cachedMessageCounts` entry.
+`commitTurn` flips an internal flag on the turn handle; `rollback` reads the flag and short-circuits. The happy path commits and the deferred rollback becomes a no-op. Every other exit path (cancel, zero-token, addon error, rename failure, tool-call turn) lets `scope[Symbol.asyncDispose]` run the deferred cleanup: `rollback` unlinks the `.bin` file and clears the `initializedCaches`/`cachedPrefixes` entries, while a thrown addon overflow, an enumerated pre-mutation refusal (scheduler submit guards, generationParams validation), or a pre-addon missing-attachment rejection takes `releaseTurn`, which frees locks and refs but keeps the committed file and prefix valid for a retry — unless the failing turn itself primed the cache, in which case there is nothing committed to keep and it rolls back.
 
 **Late-commit vs cancel is intentional, not a race.** A commit only proceeds past its abort re-check for a turn that fully generated and whose file passed `verifySaveAndRecord`. If a cancel lands during the async commit (after that check, during rename/verify), the commit finishes: a **complete, verified** turn is persisted while the operation reports cancelled. This is by design — the persisted cache matches the tokens the client already received, and committing fully-generated verified work beats discarding it. A strict "cancel wins" rollback here would need target-aware cleanup and risks deleting valid completed state, so it is not done.
 
-The maps that backed `clearCacheRegistry` and `cachedMessageCounts` are **private to `kv-cache-session.ts`**. No other module reads or writes them.
+The maps behind the session (`initializedCaches`, `cachedPrefixes`, locks and refs) are **private to `kv-cache-session.ts`**. No other module reads or writes them.
 
 ### Cache Initialization (primeIfMissing) and Addon Non-Transactional Save
 
@@ -197,9 +199,9 @@ await deleteCache({ kvCacheKey: "session-a" });                // remove session
 await deleteCache({ kvCacheKey: "session-a", modelId: "..." }); // remove only one model's session-a cache
 ```
 
-`handleDeleteCache` delegates to `deleteKvCacheState(...)` — a module-level export from `kv-cache-session.ts` that owns the same three layers. It removes the matching on-disk directory tree, prefix-cleans `cachedMessageCounts` against the removed path, and scope-clears `initializedCaches` by `(kvCacheKey[, modelId])`. The RPC handler itself has **zero direct references** to `fsPromises.unlink`, the in-memory `initializedCaches` set, or the `cachedMessageCounts` map.
+`handleDeleteCache` delegates to `deleteKvCacheState(...)` — a module-level export from `kv-cache-session.ts` that owns the same layers. It removes the matching on-disk directory tree and scope-clears the `initializedCaches`/`cachedPrefixes` entries by `(kvCacheKey[, modelId])`. The RPC handler itself has **zero direct references** to `fsPromises.unlink` or the session's in-memory maps.
 
-See `schemas/delete-cache.ts` for the request schema.
+See `schemas/model-ops.ts` for the request schema.
 
 ## Cache Persistence
 
@@ -210,23 +212,41 @@ See `schemas/delete-cache.ts` for the request schema.
 
 ## Context Overflow with KV Cache
 
-When using KV Cache with sliding window (`n_discarded`), the context doesn't overflow because:
-1. Cache stores the compressed KV state, not raw tokens
-2. Sliding window discards oldest tokens when context fills
-3. **First message (system prompt + tools) is NEVER removed** — protected from `n_discarded`
-4. Only the last message is appended to the cached context
-
-**Required config for long conversations:**
+Nothing is evicted from the cache to make room. A conversation that grows past
+`ctx_size` runs out of context, and there are two ways to see it:
+
+1. A prompt that does not fit is refused before any decoding, with a typed `ContextOverflowError` carrying the parsed sizes.
+2. A generation that fills the window stops and reports `stopReason: "length"`, keeping the tokens it already produced.
+
+The second one is a normal completion, not an error. `stopReason` alone does not
+distinguish a full context from a positive `predict` cutoff — the SDK maps both
+to `"length"` (with `predict: -1` there is no cutoff, so `"length"` can only
+mean the boundary). With `parallel >= 2` the scheduler treats a positive
+`predict` as a reservation: a budget that cannot fit the slot is rejected at
+admission instead of stopping at the boundary. Those admission refusals also
+surface as `ContextOverflowError` — `requiredTokens` carries the cache-plus-
+prompt total (plus the reservation on the `n_predict` form) and `ctxSize` the
+per-slot cap — recognised by wording, since the transport strips the addon's
+status; giving them a real `ContextOverflow` status addon-side is a recorded
+follow-up. Use `predict: -1`/`-2` or a budget within the slot to avoid the
+reservation refusal.
+
+**Config for long conversations:** size `ctx_size` for the whole conversation you
+intend to hold. With `parallel > 1` the window is split across slots, so each
+request's effective ceiling — and the `ctxSize` an overflow error reports — is
+`ctx_size / parallel`, not the configured total.
 
 ```typescript
 const modelId = await loadModel({
   modelSrc: MODEL,
   modelType: "llm",
-  modelConfig: { ctx_size: 2048, n_discarded: 512 },
+  modelConfig: { ctx_size: 8192 },
 });
 ```
 
-**Note:** `n_discarded` only works during token generation, not during prefill. Ensure your initial prompt fits in `ctx_size`.
+**Note:** the sliding-window option `n_discarded` was removed. `loadModel()`
+rejects it at schema validation; only a direct addon caller sees it fail model
+load as an unknown option.
 
 ## Debug Logging
 
@@ -244,7 +264,7 @@ KV Cache works with MCP tools. Complete canonical tool definitions are hashed: o
 | Issue | Cause | Fix |
 |-------|-------|-----|
 | Cache not reused (full history sent, slow TTFT) | Different system prompt or tools between requests | Ensure consistent system prompt and tools for same cache key |
-| Context overflow at prefill | `n_discarded` not configured | Add `n_discarded` to `modelConfig` |
+| Context overflow at prefill | Conversation outgrew `ctx_size`; nothing is evicted to make room | Raise `ctx_size`, or start a new cache key for a fresh conversation |
 | Tools changed mid-session | Expected behavior | New cache created automatically; old cache preserved |
 
 ## See Also

packages/inference/package.json

@@ -198,7 +198,7 @@
     "@qvac/diffusion-cpp": "^0.21.0",
     "@qvac/embed-llamacpp": "^0.36.0",
     "@qvac/langdetect-text": "^0.1.2",
-    "@qvac/llm-llamacpp": "^0.47.0",
+    "@qvac/llm-llamacpp": "^0.48.0",
     "@qvac/ocr-ggml": "^0.20.0",
     "@qvac/translation-nmtcpp": "^0.12.0",
     "@qvac/tts-ggml": "^0.8.0",
@@ -254,7 +254,7 @@
     "@qvac/diffusion-cpp": "^0.21.0",
     "@qvac/embed-llamacpp": "^0.36.0",
     "@qvac/langdetect-text": "^0.1.2",
-    "@qvac/llm-llamacpp": "^0.47.0",
+    "@qvac/llm-llamacpp": "^0.48.0",
     "@qvac/ocr-ggml": "^0.20.0",
     "@qvac/translation-nmtcpp": "^0.12.0",
     "@qvac/tts-ggml": "^0.8.0",

packages/inference/src/errors/index.ts

@@ -270,49 +270,117 @@ export class CompletionFailedError extends QvacErrorBase {
   }
 }
 
+/** The overflow sizes as one record; fields admit explicit `undefined` so a parser result assigns directly. */
+export type ContextOverflowErrorSizes = {
+  promptTokens?: number | undefined
+  cachedTokens?: number | undefined
+  requiredTokens?: number | undefined
+  ctxSize?: number | undefined
+}
+
+function normalizeContextOverflowArgs(
+  sizesOrPromptTokens?: ContextOverflowErrorSizes | number,
+  modelIdOrCtxSize?: string | number,
+  causeOrModelId?: unknown,
+  cause?: unknown,
+  extraSizes?: { cachedTokens?: number | undefined; requiredTokens?: number | undefined }
+): ContextOverflowErrorSizes & { modelId?: string | undefined; cause?: unknown } {
+  if (typeof sizesOrPromptTokens === 'object' && sizesOrPromptTokens !== null) {
+    return {
+      ...sizesOrPromptTokens,
+      modelId: modelIdOrCtxSize as string | undefined,
+      cause: causeOrModelId
+    }
+  }
+  // Only the two supported extra fields — a spread would let a wider,
+  // structurally assignable extras object override the positional args.
+  return {
+    promptTokens: sizesOrPromptTokens,
+    ctxSize: modelIdOrCtxSize as number | undefined,
+    modelId: causeOrModelId as string | undefined,
+    cause,
+    cachedTokens: extraSizes?.cachedTokens,
+    requiredTokens: extraSizes?.requiredTokens
+  }
+}
+
 /**
- * Thrown when the prompt exceeds the loaded model's configured context
- * window — distinct from a generic `CompletionFailedError` so consumers
+ * Thrown when a request no longer fits the model's effective context
+ * capacity — distinct from a generic `CompletionFailedError` so consumers
  * can drive UX (truncate, summarize, or surface a "increase ctx_size /
  * start a new thread" CTA) instead of treating it as an opaque failure.
  *
- * Carries the addon-reported prompt size and the model's context window
- * when the addon's error message includes them (the C++ overflow paths
- * in `TextLlmContext.cpp` and `MtmdLlmContext.cpp` format both numbers
- * into the message; the bare `processPromptImpl: context overflow`
- * fallback in `LlamaModel.cpp` carries neither — both fields are
- * therefore optional). `modelId` is supplied by the handler that wraps
- * the addon error.
+ * All size fields are optional: they carry whatever the addon guard
+ * reported (a cold prompt, a warm cached-plus-prompt split, or a bare
+ * overflow with no numbers). `modelId` is supplied by the handler that
+ * wraps the addon error.
  *
  * Serializes its typed fields (`toErrorResponseFields`) so a receiver can
  * rebuild it after the error crosses a serialization boundary (an RPC
  * response).
  */
 export class ContextOverflowError extends QvacErrorBase {
+  /** The prompt alone, in tokens; unset when the source reported KV cells. */
   readonly promptTokens?: number
+  /** Cached conversation on a warm-cache overflow, in `ctxSize` units. */
+  readonly cachedTokens?: number
+  /** Total the failing guard reported, in `ctxSize` units; can equal the window (`>=` triggers). */
+  readonly requiredTokens?: number
+  /** Effective per-request ceiling: `ctx_size` split across slots at `parallel > 1`. */
   readonly ctxSize?: number
   readonly modelId?: string
 
-  constructor(promptTokens?: number, ctxSize?: number, modelId?: string, cause?: unknown) {
+  /** Canonical form: the parsed sizes as one record. */
+  // lunte-disable-next-line constructor-super -- overload declaration, no body; the implementation calls super
+  constructor(contextSizes: ContextOverflowErrorSizes, modelId?: string, cause?: unknown)
+  /** @deprecated Kept for external consumers of the published package; in-repo callers use the record form. */
+  // lunte-disable-next-line constructor-super -- overload declaration, no body; the implementation calls super
+  constructor(
+    promptTokens?: number,
+    ctxSize?: number,
+    modelId?: string,
+    cause?: unknown,
+    extraSizes?: { cachedTokens?: number | undefined; requiredTokens?: number | undefined }
+  )
+  constructor(
+    sizesOrPromptTokens?: ContextOverflowErrorSizes | number,
+    modelIdOrCtxSize?: string | number,
+    causeOrModelId?: unknown,
+    cause?: unknown,
+    extraSizes?: { cachedTokens?: number | undefined; requiredTokens?: number | undefined }
+  ) {
+    const n = normalizeContextOverflowArgs(
+      sizesOrPromptTokens,
+      modelIdOrCtxSize,
+      causeOrModelId,
+      cause,
+      extraSizes
+    )
     super(
       createErrorOptions(
         ERROR_CODES.CONTEXT_OVERFLOW,
         [
-          promptTokens !== undefined ? String(promptTokens) : '',
-          ctxSize !== undefined ? String(ctxSize) : '',
-          modelId ?? ''
+          n.promptTokens !== undefined ? String(n.promptTokens) : '',
+          n.ctxSize !== undefined ? String(n.ctxSize) : '',
+          n.modelId ?? '',
+          n.cachedTokens !== undefined ? String(n.cachedTokens) : '',
+          n.requiredTokens !== undefined ? String(n.requiredTokens) : ''
         ],
-        cause
+        n.cause
       )
     )
-    if (promptTokens !== undefined) this.promptTokens = promptTokens
-    if (ctxSize !== undefined) this.ctxSize = ctxSize
-    if (modelId !== undefined) this.modelId = modelId
+    if (n.promptTokens !== undefined) this.promptTokens = n.promptTokens
+    if (n.ctxSize !== undefined) this.ctxSize = n.ctxSize
+    if (n.modelId !== undefined) this.modelId = n.modelId
+    if (n.cachedTokens !== undefined) this.cachedTokens = n.cachedTokens
+    if (n.requiredTokens !== undefined) this.requiredTokens = n.requiredTokens
   }
 
   toErrorResponseFields(): Record<string, unknown> {
     return {
       ...(this.promptTokens !== undefined && { promptTokens: this.promptTokens }),
+      ...(this.cachedTokens !== undefined && { cachedTokens: this.cachedTokens }),
+      ...(this.requiredTokens !== undefined && { requiredTokens: this.requiredTokens }),
       ...(this.ctxSize !== undefined && { ctxSize: this.ctxSize }),
       ...(this.modelId !== undefined && { modelId: this.modelId })
     }