QVAC-18182 feat[api]: typed cancel outcomes on the wire + atomic KV-cache via KvCacheSession

[simon-iribarren]

🎯 What problem does this PR solve?

• Cancelled completions had no typed outcome. With M1's signal-based cancel the request did stop, but the wire just emitted a normal-looking completionDone and the client's final / text / toolCalls / stats promises resolved with empty/partial values — indistinguishable from a real EOS. Callers couldn't tell "you cancelled this" from "the model returned nothing".
• Three KV-cache layers drifted under cancel/error paths. cachedMessageCounts (in-memory map), initializedCaches (in-memory set), and the on-disk .bin file each had their own cleanup site. completion-stream.ts repeated the same fsPromises.unlink(...) + clearCacheRegistry(...) + cachedMessageCounts.delete(...) triplet in three places (cancelled-custom-key turn, cancelled-auto-cache turn, auto-cache rename failure). A miss in any one of them left observable cross-turn corruption.
• The same-tick stop-button race lost cancels. M1's RequestRegistry only matched cancel({ requestId }) against in-flight requests, so a cancel that landed before the worker called registry.begin(...) silently disappeared. The eventual begin ran without an aborted signal and the cancelled request streamed to completion.

📝 How does it solve it?

• Typed cancel outcomes (D1–D3). stopReason: "cancelled" joins "eos" | "length" | "stopSequence" on the completionDone wire event so the events stream still terminates cleanly. The client-side aggregator in client/api/completion-stream.ts detects the cancelled terminator and rejects the promise-aggregates (final, text, toolCalls, stats) with a new InferenceCancelledError(requestId, partial). partial.text / partial.toolCalls / partial.stats carry whatever the aggregator accumulated up to the cancel point — callers instanceof-check and reach for whichever partial field they need. The error class is defined server-side (utils/errors-server.ts, code 52419), constructed client-side (so the partial payload survives the RPC envelope), and re-exported from @qvac/sdk for instanceof checks.
• KvCacheSession as the atomic three-layer owner (D4). New server/bare/plugins/llamacpp-completion/ops/kv-cache-session.ts owns cachedMessageCounts, initializedCaches, and .bin file lifecycle behind a beginTurn / commitTurn / rollback API. completion-stream.ts registers scope.defer(() => session.rollback(turn)) at the top of a turn and calls session.commitTurn(...) only on the success path — so cancel / addon failure / auto-cache rename failure / scope unwind all funnel through the same rollback. The three duplicated cleanup blocks in the pre-PR completion-stream.ts are gone; the file dropped from ~756 lines to ~661 with the cancellation logic centralised.
• Cross-model administrative delete (D5). A new module-level deleteKvCacheState({ kvCacheKey?, modelId?, all? }) in kv-cache-session.ts handles wipe-by-key (optionally scoped to a model) and wipe-all, touching all three layers atomically. The RPC handler server/rpc/handlers/delete-cache.ts now delegates straight to it — no more direct fsPromises / module-set juggling in the handler.
• Bounded cancelledBeforeBegin map closes the race (D6, Option A). RequestRegistry keeps a 128-entry / 30 s TTL map of requestId → { at, reason }. cancel(...) records into the map when no in-flight context is found; begin(...) consults it and aborts the freshly-minted controller before returning the ManagedRequestContext. The wire surface for cancel is unchanged — internal-only fix, no [bc]. Eviction is size-cap + TTL combined.
• server/bare/runtime/request-registry.ts begin() now derives terminal state ("cancelled" | "completed") from ctx.signal.aborted so a same-tick cancel-then-begin pair lands as state: "cancelled", observable via __requestRegistryTestHooks in unit tests.
• Cursor rules updated in this PR (the brief's requirement): .cursor/rules/sdk/request-lifecycle-primitives.mdc, .cursor/rules/sdk/docs/request-lifecycle-system.mdc, .cursor/rules/sdk/docs/kv-cache-system.mdc, .cursor/rules/sdk/error-handling.mdc — error codes table, cancelled completion semantics, KvCacheSession ownership, FAQ entry for the race close, anti-patterns.

🧪 How was it tested?

Functional-equivalence vs. pre-PR completion-stream.ts (the D5/D7 mitigation listed in the brief):

Scenario	Pre-PR cleanup site	M2 coverage
Custom kvCache key, cancelled mid-stream	completion-stream.ts lines ~595–607 (unlink + clearCacheRegistry + cachedMessageCounts.delete)	KvCacheSession.rollback(turn) via scope.defer. Verified by cancel-then-resume-kv-cache e2e + kv-cache-session.test.ts "rollback wipes all three layers atomically".
Custom kvCache key, zero-token turn	Same block, branch on shouldRecordSavedCount	Same rollback path — commitTurn is never called, so the deferred rollback runs unconditionally.
Auto kvCache (kvCache: true), cancelled mid-stream	lines ~688–704 (unlink + cachedMessageCounts.delete)	Same KvCacheSession.rollback(turn) via scope.defer.
Auto kvCache, zero-token EOS turn (no content to promote)	Same block, alt branch	Same rollback.
Auto kvCache, file-rename failure during commitTurn	lines ~725–740 (unlink + cachedMessageCounts.delete)	commitTurn returns false / throws → deferred rollback wipes everything (kv-cache-session.test.ts "commitTurn rolls back if the addon did not persist the file").
Successful commit	trailing cachedMessageCounts.delete(cachePathToUse) + recordCacheSaveCount(...)	commitTurn(turn, newPath, savedCount) records the new path/count and drops the rollback hook in one transaction.

Unit tests (packages/sdk/test/unit/):

• runtime/request-registry.test.ts — same-tick cancel-before-begin now retroactively aborts (was: 0 matched, now: signal aborted on the new context); UUID-retry path tested; bounded-set eviction tested (200 cancel-before-begin calls don't grow the internal map past 128 entries; eviction preserves the freshest entries).
• runtime/kv-cache-session.test.ts — beginTurn priming + reuse, commitTurn records new count + suppresses rollback, rollback wipes all three layers, rollback tolerates missing on-disk file, double-rollback is idempotent, dropStaleSavedCount forgets count without touching file/init flag, deleteKvCacheState({ kvCacheKey }) + deleteKvCacheState({ all }) wipe correctly, commitTurn triggers rollback when the addon doesn't persist. Tests are bareTest-gated (dynamic import, runs in the Bare consumer; skipped under Bun because kv-cache-session.ts transitively imports bare-fs / bare-path).
• completion-event-schemas.test.ts — successDoneSchema.parse({ type: "completionDone", seq, stopReason: "cancelled" }) accepted; cancelled cases added to the event-type routing test.
• kv-cache-cancel.test.ts — refactored away from the deprecated module-scoped cachedMessageCounts map; decideCachedHistorySlice is now exercised purely with the explicit savedCount parameter (matching the M2 API).

E2E tests (packages/sdk/tests-qvac/tests/cancellation-tests.ts + shared/executors/cancellation-executor.ts):

• cancel-mid-stream-completion — fire cancel({ requestId }) after the third contentDelta, assert (a) events end with completionDone.stopReason === "cancelled", (b) run.final rejects with InferenceCancelledError, (c) error.partial.text equals the concatenated contentDelta.text the test observed on run.events.
• cancel-before-begin-completion — fire cancel({ requestId }) synchronously after completion(...) returns (before the completionStream RPC settles), assert the eventual outcome is a typed cancelled regardless of whether the cancel landed before or after begin(...) ran on the worker.
• cancel-then-resume-kv-cache — same as the existing kv-cache-cancel-then-new-prompt but additionally asserts the first turn's InferenceCancelledError rejection and the stopReason: "cancelled" event, then verifies the second turn on the same kvCache key re-primes cleanly and produces the expected answer (proving KvCacheSession.rollback wiped the three layers).

Pre-merge gates (run from packages/sdk/):

bun lint        # eslint + tsc, clean
bun run build   # clean
bun run test:unit  # 10/10 files, all asserts pass

🔌 API Changes

stopReason: "cancelled" is a new value on the successDoneSchema enum; InferenceCancelledError is a new export. Both are additive — existing consumers ignoring the new enum value or not catching the new error class still work.

import { completion, cancel, InferenceCancelledError } from "@qvac/sdk";

const run = completion({ modelId, history, stream: true });

// Iterating events stays unchanged — events end naturally with the
// cancelled terminator on cancel, no thrown error:
for await (const event of run.events) {
  if (event.type === "completionDone" && event.stopReason === "cancelled") {
    // request was cancelled — handle here if you want
  }
}

// Awaiting promise-aggregates throws InferenceCancelledError on cancel:
try {
  await cancel({ requestId: run.requestId });
  const text = await run.text;
  // ...
} catch (err) {
  if (err instanceof InferenceCancelledError) {
    console.log(`cancelled: requestId=${err.requestId}, partial=${err.partial.text}`);
    // err.partial.toolCalls and err.partial.stats are also available
  }
}

The internal RequestRegistry cancel-before-begin race close is wire-invisible — cancel({ requestId }) and cancel({ operation: "inference", modelId }) behave identically on the wire as before. Same-tick stop clicks just stop being silently dropped.

[simon-iribarren]

E2E test analysis (run 25748140796 on commit 1c4ffdb)

Investigated the failing e2e tests. None of the failures are caused by the M2 changes in this PR. Summary across all 5 platforms:

Platform	Total	Passed	Failed	Skipped	Verify suite (M2 cancellation tests)	kv-cache category
Desktop Windows	408	405	1	2	3/3 (100%)	17/17 (100%)
Desktop Ubuntu	408	404	2	2	3/3 (100%)	17/17 (100%)
Desktop macOS (M4)	408	405	1	2	3/3 (100%)	17/17 (100%)
iOS (iPhone 16 Pro)	408	280	1	127	3/3 (100%)	17/17 (100%)
Android (Samsung S25 Ultra)	408	77	329	2	0/3 (never ran)	0/17 (never ran)

What actually failed

1. completion-response-format-json-schema — fails on iOS, win, ubuntu, mac.

Model returned prose instead of valid JSON:

Error: json_schema output is not valid JSON: JSON Parse error: Unexpected character: H. Output: Here is the person info as JSON:

Pre-existing model-quality flake on the 1B Llama (the test forces response_format: json_schema and the small model occasionally prefixes prose). Touches none of the M2 surface (cancellation, KvCacheSession, request registry).

2. vision-text-extraction — ubuntu desktop only.

Vision OCR-style test, unrelated to M2. Pre-existing.

3. 329 android failures — root cause is a device hang, not test failures.

Producer log on android shows the consumer device went unresponsive at 21:07 while running transcribe-stream-events-happy:

21:06:39  ⏳ transcribe-stream-events-happy → consumer-mobile-SM-... (running, 63s / 180s)
21:07:09  ⏳ transcribe-stream-events-happy → consumer-mobile-SM-... (running, 93s / 180s)
21:07:39  ⏳ transcribe-stream-events-happy → consumer-mobile-SM-... (running, 123s / 180s)
21:07:39  💀 Consumer mobile-SM unresponsive for 123s — marking as dead
21:07:39  ❌ All consumers are dead. Terminating batch.

After the device died, the producer counted all 327 unrun queued tests as failures (which includes our 3 cancellation tests, the entire 17-test kv-cache suite, embedding, rag, translation, tools, ocr, tts, etc — all 0/N in the per-category report). This is a known device-farm/transcription flakiness pattern (cf. QVAC-18460 for the iOS transcription crash already tracked under skips).

M2 surface verification

Where M2-touched paths actually executed (4 of 5 platforms):

• Verify suite (the new cancellation e2e: cancel-mid-stream-completion, cancel-before-begin-completion, cancel-then-resume-kv-cache): 3/3 pass on every platform (iOS 317ms / 109ms / 637ms; desktop ~210ms / ~100ms / similar).
• kv-cache category: 17/17 pass on every platform — the existing kv-cache suite (including kv-cache-cancel-then-new-prompt which exercises broad-cancel backward-compat) is functionally equivalent under the KvCacheSession refactor.
• completion category: 45/48 pass on every platform (45 + 2 skipped + 1 failed = 48; the one failure is the json-schema flake above, not ours).

Verdict: the e2e failures on this run are entirely pre-existing (model-quality flake, vision flake) or device-hang artefacts. The M2 surface (typed cancel outcomes, InferenceCancelledError, KvCacheSession, stop-button race close) shows 100% pass on every platform that ran it.

[github-actions]

Tier-based Approval Status

**PR Tier:** TIER1

**Current Status:** ✅ APPROVED

**Requirements:**
- 1 Team Member approval ✅ (1/1)
- 1 Team Lead OR Management approval ✅ (2/1)



---
*This comment is automatically updated when reviews change.*

[github-actions]

QVAC E2E — ios — ⚠️ no results

Config: suite=smoke · filter=(none) · exclude=(none)
View run

The test job did not produce a results artifact. Check the run for job-level failures.

[github-actions]

QVAC E2E — windows — ✅ all tests passed (91/91, 445s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts: reports

[github-actions]

QVAC E2E — linux — ✅ all tests passed (91/91, 349s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts: reports

[github-actions]

QVAC E2E — macos — ✅ all tests passed (91/91, 341s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts: reports

[github-actions]

QVAC E2E — android — ✅ all tests passed (83/91, 1996s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts: reports · Device Farm logs

[simon-iribarren]

/review