Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions packages/sdk/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,34 @@
# Changelog

## [0.8.3]

📦 **NPM:** https://www.npmjs.com/package/@qvac/sdk/v/0.8.3

This is a patch release that fixes a race condition in the KV cache save path during tool-calling completions, improving stability for multi-turn conversations that use tool integration.

---

## 🐞 Bug Fixes

### KV Cache Save Race Condition in Tool-Calling Completions

The KV cache save during tool-calling completions could race with ongoing inference because the session path was not passed to the save command and the save response was not awaited. This could result in corrupted or missing session state between tool-call rounds.

The fix ensures the save command receives the correct session path and the SDK awaits the save response before proceeding. If the save fails, the error is now logged as a warning instead of propagating as an unhandled exception, so inference can continue gracefully.

**What changed:**

- The cache save now explicitly passes the session path alongside the save instruction, preventing the addon from writing to a stale or missing path
- The save response is awaited so subsequent inference steps don't race against an in-flight disk write
- A new `logCacheSaveError` helper captures save failures as warnings, keeping the completion stream alive even if the cache write fails

---

## 📘 Documentation

- Added npm keywords to `package.json` for better discoverability on the npm registry, covering AI/ML, inference engines, supported platforms, and P2P capabilities.
- Added a link to the consolidated plaintext documentation export (`llms-full.txt`) in the SDK README for AI/LLM tool consumption.

## [0.8.2]

📦 **NPM:** https://www.npmjs.com/package/@qvac/sdk/v/0.8.2
Expand Down
2 changes: 2 additions & 0 deletions packages/sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

See [https://docs.qvac.tether.io/sdk/getting-started](https://docs.qvac.tether.io/sdk/getting-started) for the comprehensive QVAC documentation.

For AI/LLM tools, use [https://docs.qvac.tether.io/llms-full.txt](https://docs.qvac.tether.io/llms-full.txt) as the consolidated plaintext documentation export.

## Supported environments and installation

See https://docs.qvac.tether.io/sdk/getting-started/installation
Expand Down
12 changes: 12 additions & 0 deletions packages/sdk/changelog/0.8.3/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Changelog v0.8.3

Release Date: 2026-04-13

## 🐞 Bug Fixes

- Fix KV cache save race condition in tool-calling completions. The cache save now correctly passes the session path and awaits the save response, preventing a race where the session file could be written concurrently with ongoing tool-call inference. Errors during cache save are now logged as warnings instead of crashing. (see PR [#1298](https://github.com/tetherto/qvac/pull/1298))

## 📘 Docs

- Add npm keywords for better discoverability on the npm registry.
- Add llms.txt link to SDK README for AI/LLM tool consumption.
28 changes: 28 additions & 0 deletions packages/sdk/changelog/0.8.3/CHANGELOG_LLM.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# QVAC SDK v0.8.3 Release Notes

📦 **NPM:** https://www.npmjs.com/package/@qvac/sdk/v/0.8.3

This is a patch release that fixes a race condition in the KV cache save path during tool-calling completions, improving stability for multi-turn conversations that use tool integration.

---

## 🐞 Bug Fixes

### KV Cache Save Race Condition in Tool-Calling Completions

The KV cache save during tool-calling completions could race with ongoing inference because the session path was not passed to the save command and the save response was not awaited. This could result in corrupted or missing session state between tool-call rounds.

The fix ensures the save command receives the correct session path and the SDK awaits the save response before proceeding. If the save fails, the error is now logged as a warning instead of propagating as an unhandled exception, so inference can continue gracefully.

**What changed:**

- The cache save now explicitly passes the session path alongside the save instruction, preventing the addon from writing to a stale or missing path
- The save response is awaited so subsequent inference steps don't race against an in-flight disk write
- A new `logCacheSaveError` helper captures save failures as warnings, keeping the completion stream alive even if the cache write fails

---

## 📘 Documentation

- Added npm keywords to `package.json` for better discoverability on the npm registry, covering AI/ML, inference engines, supported platforms, and P2P capabilities.
- Added a link to the consolidated plaintext documentation export (`llms-full.txt`) in the SDK README for AI/LLM tool consumption.
47 changes: 46 additions & 1 deletion packages/sdk/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@qvac/sdk",
"version": "0.8.2",
"version": "0.8.3",
"license": "Apache-2.0",
"repository": {
"type": "git",
Expand All @@ -9,6 +9,51 @@
},
"bugs": "https://github.com/tetherto/qvac/issues",
"homepage": "https://github.com/tetherto/qvac/tree/main/packages/sdk#readme",
"keywords": [
"ai",
"bare",
"cross-platform",
"deep-learning",
"diffusion",
"edge-ai",
"embeddings",
"expo",
"gguf",
"gpu",
"holepunch",
"hyperswarm",
"image-generation",
"inference",
"large-language-model",
"llama-cpp",
"llm",
"local",
"local-ai",
"machine-learning",
"ml",
"model-registry",
"node",
"ocr",
"offline",
"on-device",
"p2p",
"peer-to-peer",
"private-ai",
"qvac",
"rag",
"react-native",
"sdk",
"self-hosted",
"speech-to-text",
"stt",
"tether",
"text-to-speech",
"transcription",
"translation",
"tts",
"typescript",
"whisper"
],
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,9 @@ export function logCacheDisabled(): void {
export function logCacheSave(sessionPath: string): void {
logger.debug(`[kv-cache] Saving session: ...${sessionPath.slice(-20)}`);
}

export function logCacheSaveError(sessionPath: string, err: unknown): void {
logger.warn(
`[kv-cache] Failed to save session: ...${sessionPath.slice(-20)} — ${err instanceof Error ? err.message : String(err)}`,
);
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {
logCacheDisabled,
logCacheInit,
logCacheSave,
logCacheSaveError,
logCacheStatus,
logMessagesToAddon,
} from "@/server/bare/plugins/llamacpp-completion/ops/cache-logger";
Expand Down Expand Up @@ -232,8 +233,17 @@ async function* processModelResponse(
const sessionMsg = messagesToSend.find((m) => m.role === "session");
if (sessionMsg?.content) {
logCacheSave(sessionMsg.content);
const cachePath = sessionMsg.content;
const saveResp = await model.run([
{ role: "session", content: cachePath },
{ role: "session", content: "save" },
]);
try {
await saveResp.await();
} catch (err: unknown) {
logCacheSaveError(cachePath, err);
}
}
await model.run([{ role: "session", content: "save" }]);
}

const responseWithStats = response as unknown as ResponseWithStats;
Expand Down
Loading