QVAC-19900 feat[api]: add managed mode to @qvac/ai-sdk-provider

[simon-iribarren]

🎯 What problem does this PR solve?

@qvac/ai-sdk-provider v1 (0.1.0) is external-mode only: the user must hand-author a qvac.config.json and keep qvac serve openai running in a separate terminal before the provider works. This PR adds managed mode (QVAC-19900) so the provider runs qvac serve itself.

The hard part isn't spawning a serve — it's the lifecycle once you do it for real:

• A naive "spawn on start, kill on close" design orphans serves (a hard crash leaves qvac serve running forever — we hit exactly this in testing) and duplicates them (every session/tool reloads the model into memory).
• Coding-agent harnesses (OpenCode, Cline, Aider) connect straight to the baseURL, so any "is it still in use?" heuristic based on the provider's own request traffic is blind to them.
• The serve default ctx_size of 1024 is far too small for an agent's tool-laden system prompt, and reasoning models spam <think> blocks.

So managed mode is built as a shared, self-cleaning serve daemon that is robust standalone and usable by any tool, not just one in-process session. External mode is left byte-for-byte unchanged.

📝 How does it solve it?

createQvac({ mode: 'managed', models: [...] }) synthesizes an ephemeral qvac.config.json from SDK model-constant names and brings up a serve — but the serve is owned by a detached runner, not by your process:

• Reuse via a fleet key (src/managed/fleet-key.ts): a hash of { model set + per-model config + host } keys a cross-process registry under ~/.qvac/managed-serves/. createQvac attaches to a matching healthy serve instead of cold-starting a duplicate. Different models/config ⇒ different key ⇒ its own serve.
• Detached runner owns the serve (src/managed/runner.ts): it spawns qvac serve, publishes the registry record, and reaps the serve once no consumer process has been alive for serveIdleTimeout (default 5 min). Liveness — not request traffic — is the signal, so it works for tools that hit baseURL directly. This survives your script exiting, so a serve can be shared and still cleaned up.
• close() detaches, it doesn't kill: it deregisters this session as a consumer. A serve still in use by another session keeps running; an unused one is reaped after the idle timeout. Abrupt exits (Ctrl-C/crash) are handled by dead-PID pruning.
• Non-destructive sweep (src/managed/registry.ts): every start reaps only dead/orphaned serves — never a healthy serve a live runner owns. This fixes the original bug where a second session would SIGKILL a serve another session was still downloading into.
• Respawn-on-failure: the provider's fetch wrapper transparently re-resolves (reattach or spawn) and retries once on ECONNREFUSED, so a serve crash mid-session self-heals.
• Per-model config (QvacManagedModel): models entries may be { name, config, preload, default }, so callers set ctx_size / reasoning_budget per model — the agent-friendly setup. Duplicate model names are rejected up front (DuplicateManagedModelError) instead of silently overwriting.
• Private serves: reuse: false (or pinning servePort) gives a dedicated, non-shared serve reaped as soon as its owner exits.

src/managed/serve-process.ts holds the low-level spawn / health-poll / SIGTERM→grace→SIGKILL primitives used by the runner. @qvac/cli is an optional peer dependency, resolved via process.execPath + its entry (or an explicit serveBinPath). The whole managed subsystem is lazily dynamic-imported only when mode: 'managed' is set, so external mode stays synchronous and pays nothing. Portable node: APIs only — identical on Node 20+ and Bun.

🧪 How was it tested?

• bun run lint (tsc), bun run build, bun run test — green: 51 pass / 1 skip (the skip is the real-model integration test, gated by QVAC_INTEGRATION_TEST=1).
• New/updated unit + integration suites:
  • managed-registry — record round-trip, consumer markers + dead-PID pruning, findReusableServe health gating, and a non-destructive sweep that drops dead records, kills runner-orphaned serves, and leaves a healthy owned serve untouched.
  • managed-fleet-key — stable, order-insensitive key; changes with per-model config and host.
  • managed-runner — pure idle-reap decision (consumer liveness + timeout, incl. zero-timeout private serves).
  • managed-serve-process — fake-serve driven: healthy, start-timeout, early-exit/crash, SIGKILL-escalation.
  • managed-provider — end-to-end through the real detached runner: auto-spawn, reuse (second createQvac → same serve pid), and idle-reap (serve gone + registry cleared after the timeout).
  • managed-config — per-model config emission + duplicate-name rejection.
• Manually verified end-to-end against a real cached Qwen3-8B serve (tasks harness): healthy spawn → real /v1/chat/completions → second session reused the same pid → close() left it running while a consumer remained → idle-reaped (process gone + registry empty) once the last consumer left.

🔌 API Changes

New mode: 'managed' overload of createQvac (returns a Promise<ManagedQvacProvider>), plus the QvacManagedOptions / QvacManagedModel / ManagedQvacProvider types. External createQvac is unchanged.

import { createQvac } from '@qvac/ai-sdk-provider'
import { generateText } from 'ai'

// Auto-spawns (or REUSES) a shared `qvac serve`; resolves once it's healthy.
const qvac = await createQvac({
  mode: 'managed',
  models: [
    // Per-model serve config — agents need a real ctx window + no <think> blocks.
    { name: 'QWEN3_8B_INST_Q4_K_M', config: { ctx_size: 16384, reasoning_budget: 0 }, default: true }
  ],
  serveIdleTimeout: 300_000 // keep the shared serve 5 min after the last user exits
})

try {
  const { text } = await generateText({
    model: qvac('QWEN3_8B_INST_Q4_K_M'),
    prompt: 'Write a haiku about local-first AI.'
  })
  console.log(text)
} finally {
  await qvac.close() // detaches this session; a shared serve keeps running for others
}

// AsyncDisposable — `await using` handles detach automatically:
// await using qvac = await createQvac({ mode: 'managed', models: ['QWEN3_600M_INST_Q4'] })

QvacManagedOptions (all optional besides mode/models): servePort, serveHost, serveStartTimeout, serveBinPath, reuse (default true; false ⇒ private serve), serveIdleTimeout (default 300000), plus the shared apiKey / headers / fetch. QvacManagedModel: { name, config?, preload?, default? }. ManagedQvacProvider exposes close(), [Symbol.asyncDispose], port, pid, baseURL.

---

Follow-up (separate task): version bump to 0.2.0 + publish via the fork → release branch → main flow.

[lauripiisang]

few notes that probably should be addressed.
Also naming isn't super obvious re: const re= await reresolve()

[simon-iribarren]

/review

[github-actions]

Tier-based Approval Status

**PR Tier:** TIER1

**Current Status:** ✅ APPROVED

**Requirements:**
- 1 Team Member approval ✅ (2/1)
- 1 Team Lead OR Management approval ✅ (1/1)



---
*This comment is automatically updated when reviews change.*