Skip to content

QVAC-16480 fix: fix kv-cache save race condition in tool-calling completions#1298

Merged
NamelsKing merged 3 commits into
tetherto:mainfrom
gusttav-lang:fix/tool-cache
Apr 3, 2026
Merged

QVAC-16480 fix: fix kv-cache save race condition in tool-calling completions#1298
NamelsKing merged 3 commits into
tetherto:mainfrom
gusttav-lang:fix/tool-cache

Conversation

@gusttav-lang

@gusttav-lang gusttav-lang commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

🎯 What problem does this PR solve?

  • Multi-turn tool-calling completions crash with "Cannot set new job: a job is already set or being processed" on Linux AppImage/Snap builds
  • The crash is a race condition: the KV-cache save from iteration N is still in-flight when the next completion call begins, because _hasActiveResponse was never cleared
  • Only reproduces in packaged Linux environments (FUSE-mounted squashfs) where I/O latency widens the race window; dev mode timing masks the bug

📝 How does it solve it?

  • Include the cache file path as the first session message in the save command array — without it the native addon silently rejects the save
  • Await saveResp.await() so _hasActiveResponse is cleared (via its .finally()) before the generator yields stats and the next completion() call begins
  • Move the model.run() save call inside the if (sessionMsg?.content) guard since saving without a valid cache path always fails

🐛 Bug screenshot

image

🧪 How was it tested?

  • Reproduced the crash on Linux AppImage with Skill tool calls (e.g. weather Skill), which always triggered the race condition; confirmed the error no longer occurs after the fix
  • Verified MCP tool calls (which were not affected) still work correctly
  • Verified single-turn and multi-turn completions with KV-cache enabled still work correctly on dev mode (bun run dev)

gianni-cor
gianni-cor previously approved these changes Apr 1, 2026
NamelsKing
NamelsKing previously approved these changes Apr 2, 2026
@NamelsKing

Copy link
Copy Markdown
Contributor

/review

@github-actions

github-actions Bot commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Tier-based Approval Status

**PR Tier:** TIER1

**Current Status:** ✅ APPROVED

**Requirements:**
- 1 Team Member approval ✅ (1/1)
- 1 Team Lead OR Management approval ✅ (2/1)

**Bypass rule:** Triggered (2+ Team Lead approvals (Tier 1 exception)). This PR is approved regardless of tier.

---
*This comment is automatically updated when reviews change.*

@NamelsKing NamelsKing merged commit 5999875 into tetherto:main Apr 3, 2026
13 checks passed
opaninakuffo pushed a commit to opaninakuffo/qvac that referenced this pull request Apr 13, 2026
opaninakuffo added a commit that referenced this pull request Apr 13, 2026
* QVAC-16480 fix: fix kv-cache save race condition in tool-calling completions (#1298)

* doc: add npm keywords and llms.txt link to README

* chore: qvac-sdk v0.8.3 release

---------

Co-authored-by: gbl <59838276+gusttav-lang@users.noreply.github.com>
opaninakuffo added a commit that referenced this pull request Apr 13, 2026
* chore: qvac-sdk v0.8.1 release (#1285)

* fix: freeze sdk deps installs in publish and sdk-pod checks (#1288)

* chore: update changelog (#1295)

* doc[notask]: rewrite SDK README with streamlined quickstart and updated docs links (#1456)

* chore[skiplog|notask]: sdk v0.8.3 release, changelog (#1537)

* QVAC-16480 fix: fix kv-cache save race condition in tool-calling completions (#1298)

* doc: add npm keywords and llms.txt link to README

* chore: qvac-sdk v0.8.3 release

---------

Co-authored-by: gbl <59838276+gusttav-lang@users.noreply.github.com>

* chore[notask|skiplog]: cherry-pick workflow migration from qvac-devops to oss-actions (#1545)

* chore: migrate npm publish workflows to use npm trust oidc action of qvac-devops (#1362)

* chore: migrate npm publish workflows to use npm trust oidc action

* fix: remove the secret token from the workflows

* fix: permissions on the workflows for npm trust

* refactor: changed the qvac-devops actions to v1.1.8

* refactor: added NPM_TOKEN

* feat: reflect package.json changes on the PR

* chore: migrate qvac-devops to oss-action (#1478)

---------

Co-authored-by: sidj-thr <siddharth.jain@tether.io>

* chore: trigger sdk publish workflow (#1548)

---------

Co-authored-by: Dmytro Medvinskyi <functionsilence@gmail.com>
Co-authored-by: gbl <59838276+gusttav-lang@users.noreply.github.com>
Co-authored-by: sidj-thr <siddharth.jain@tether.io>
Proletter pushed a commit that referenced this pull request May 24, 2026
…letions (#1298)

* fix: fix kv-cache save race condition in tool-calling completions

* fix: log save errors

---------

Co-authored-by: Dmytro Medvinskyi <functionsilence@gmail.com>
Proletter pushed a commit that referenced this pull request May 24, 2026
* chore: qvac-sdk v0.8.1 release (#1285)

* fix: freeze sdk deps installs in publish and sdk-pod checks (#1288)

* chore: update changelog (#1295)

* doc[notask]: rewrite SDK README with streamlined quickstart and updated docs links (#1456)

* chore[skiplog|notask]: sdk v0.8.3 release, changelog (#1537)

* QVAC-16480 fix: fix kv-cache save race condition in tool-calling completions (#1298)

* doc: add npm keywords and llms.txt link to README

* chore: qvac-sdk v0.8.3 release

---------

Co-authored-by: gbl <59838276+gusttav-lang@users.noreply.github.com>

* chore[notask|skiplog]: cherry-pick workflow migration from qvac-devops to oss-actions (#1545)

* chore: migrate npm publish workflows to use npm trust oidc action of qvac-devops (#1362)

* chore: migrate npm publish workflows to use npm trust oidc action

* fix: remove the secret token from the workflows

* fix: permissions on the workflows for npm trust

* refactor: changed the qvac-devops actions to v1.1.8

* refactor: added NPM_TOKEN

* feat: reflect package.json changes on the PR

* chore: migrate qvac-devops to oss-action (#1478)

---------

Co-authored-by: sidj-thr <siddharth.jain@tether.io>

* chore: trigger sdk publish workflow (#1548)

---------

Co-authored-by: Dmytro Medvinskyi <functionsilence@gmail.com>
Co-authored-by: gbl <59838276+gusttav-lang@users.noreply.github.com>
Co-authored-by: sidj-thr <siddharth.jain@tether.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants