QVAC-12232 fix: handle unhandled worker process termination in Bare runtime

[simon-iribarren]

🎯 What problem does this PR solve?

• Worker process only handles SIGTERM/SIGINT for cleanup — uncaught exceptions, unhandled rejections, parent crashes, and SIGKILL all leave resources locked
• Corestore directories and RAG instances remain locked after ungraceful termination, causing "resource busy" / "File descriptor could not be locked" errors on restart
• unloadAllModels skips FilesystemDL.close() that the single-model unloadModel path performs, leaking file handles during shutdown
• Registry client's getRegistryClient() fails instantly on fd-lock contention with no retry — any transient lock (e.g., previous process still releasing) causes immediate ModelLoadFailedError
• QVACRegistryClient creates Corestore with wait: false (default), using tryLock instead of waitForLock — concurrent SDK instances on the same machine collide on ~/.qvac/registry-corestore/<key>

Related: Slack thread from Zbigniew documenting the fd-lock error chain in production

📝 How does it solve it?

A. Crash handlers (worker-core.ts)

• Register uncaughtException and unhandledRejection handlers on bare-process to trigger full cleanup on unexpected crashes
• Non-graceful shutdowns exit with code 1

B. IPC disconnect detection (create-server.ts, worker-core.ts)

• Listen for IPC socket close event in desktop mode — fires when parent process (Electron) crashes or is killed
• Triggers cleanup via callback (avoids circular imports)

C. Stale lock file (server/utils/worker-lock.ts)

• New PID-based lock file at ~/.qvac/.worker.lock written on startup, removed on graceful shutdown
• On startup, detects stale locks from dead PIDs (handles SIGKILL, OOM, power loss)
• Uses process.kill(pid, 0) for cross-platform liveness check

D. Fix unloadAllModels (model-registry.ts)

• Now calls loader.close() before model.unload(), matching unloadModel behavior
• Per-model try/catch so one failing unload doesn't block others

E. Registry client fd-lock retry (registry-client.ts)

• Retry getRegistryClient() up to 3 times with exponential backoff (500ms, 1000ms, 2000ms) on "File descriptor could not be locked" errors
• Only fd-lock errors trigger retry; other errors propagate immediately

F. Corestore wait option plumbing (qvac-lib-registry-server/client)

• QVACRegistryClient now accepts corestoreOpts and passes them to the Corestore constructor
• Type definition updated with corestoreOpts?: { wait?: boolean }
• SDK will pass corestoreOpts: { wait: true } once @qvac/registry-client dependency is bumped (TODO in code)

🧪 How was it tested?

Manual verification — no existing test infrastructure covers worker lifecycle. Suggested e2e tests as follow-up:

1. Stale lock recovery: create fake lock with dead PID → start worker → verify stale detection log and clean startup
2. Uncaught exception cleanup: inject setTimeout(() => { throw ... }, 3000) in worker → verify cleanup runs and lock file removed
3. SIGKILL recovery: start worker → kill -9 → verify lock file persists → restart → verify stale detection and no "resource busy"
4. IPC disconnect: start desktop worker → kill parent → verify worker exits cleanly
5. fd-lock retry: simulate contention → verify retry logs and eventual success

Files changed

File	Change
packages/sdk/server/utils/worker-lock.ts	New — PID lock file: acquire, release, stale detection
packages/sdk/server/worker-core.ts	Add crash/rejection handlers, lock lifecycle, IPC disconnect callback, expanded shutdown reasons
packages/sdk/server/rpc/create-server.ts	Add onDisconnect callback option to createIPCClient, socket close listener
packages/sdk/server/bare/registry/model-registry.ts	Fix unloadAllModels to close loaders, add per-model error handling
packages/sdk/server/bare/registry/registry-client.ts	Add retry with exponential backoff on fd-lock errors
packages/qvac-lib-registry-server/client/lib/client.js	Plumb corestoreOpts through to new Corestore()
packages/qvac-lib-registry-server/client/index.d.ts	Add corestoreOpts to QVACRegistryClientOptions type

Follow-up needed

• Bump @qvac/registry-client → release with corestoreOpts support
• Bump SDK's @qvac/registry-client dep → enable corestoreOpts: { wait: true }

[simon-iribarren]

/review

[github-actions]

Tier-based Approval Status

**PR Tier:** TIER1

**Current Status:** ✅ APPROVED

**Requirements:**
- 1 Team Member approval ✅ (2/1)
- 1 Team Lead OR Management approval ✅ (1/1)



---
*This comment is automatically updated when reviews change.*