fix(db): prevent Windows native-driver hang from stalling all requests (#10627)

[jonlwheat2-gif]

Summary

Fixes #10627 — a Windows-only total request hang where every request stalls indefinitely (0 bytes returned, no logs, ~0% CPU) while the port still accepts connections. Reported on v3.8.48/49; the reporter's omniroute doctor flagged better-sqlite3 as broken on their global install.

The fix makes a broken native driver fail fast and fail over instead of deadlocking the process, and moves the native driver load off the first-request path in the proxy runtime.

The problem

Symptom (from #10627)

• Windows (native), v3.8.48 + v3.8.49
• Every request hangs — /, /api/*, /v1/* — 0 bytes, no request logs
• Port listening, TCP connections accepted, ~0% CPU
• omniroute doctor reports better-sqlite3 broken/mismatched on the global install

Root cause

OmniRoute's SQLite driver cascade (src/lib/db/adapters/driverFactory.ts) is a try/catch chain: bun:sqlite → better-sqlite3 → node:sqlite → sql.js. Each branch assumes a broken native addon throws on load (e.g. ERR_DLOPEN_FAILED, Module did not self-register), so the fallback runs.

That assumption holds on POSIX but not on Windows: a mismatched-ABI addon can hang inside DllMain (the OS loader lock) instead of throwing. A hang never reaches the catch, so:

1. The cascade never falls through to node:sqlite / sql.js
2. The first request that touches the DB — the proxy runtime's cold getCachedSettings() → dynamic import("@/lib/db/settings") → native addon load — stalls forever
3. Because it's a loader-lock deadlock, the whole process is stuck at ~0% CPU with the event loop idle

Two compounding factors:

• The proxy runs in its own Next.js runtime that never executes instrumentation-node.ts's startup warm-ups, so its first DB touch happened on the request path, cold, with no timeout.
• runtimeRequire("better-sqlite3") (webpack external) resolves the addon in-process, so a hang is in-process and unstoppable — no timeout can interrupt a synchronous require once DllMain is entered.

The fix (2 changes, 3 files)

1. Windows driver-hang probe — src/lib/db/adapters/driverFactory.ts

New createBetterSqliteProbe() gates the better-sqlite3 branch:

• Loads better-sqlite3 in a child process with a bounded 5s timeout
• A DllMain hang → child killed by timeout (status === null) → verdict "bad"
• Verdict cached per process — the child spawn happens at most once
• Windows-only: on POSIX the probe returns true without spawning — broken addons throw there (already handled by the cascade), and we don't pay a subprocess on Linux/CI boot
• Injectable via the existing createSyncDriverFactory(load, probe?) seam — tests can simulate both worlds without a real child

Result: same machine, same broken binary — previously a deadlock, now an instant failover to node:sqlite (built into Node 22.5+, no addon) → sql.js.

2. Proxy boot warm — src/proxy.ts

Warm the settings cache at proxy-runtime boot (fire-and-forget, mirrors the void warmModelCatalogCache() pattern):

• The native driver load happens at startup, never on the first request
• A driver failure now surfaces as a logged startup error, and requests fall back to default limits — never an indefinite hang
• First real request starts with a hot cache (also removes the first-request latency spike on healthy installs)

Files changed

File	Change
src/lib/db/adapters/driverFactory.ts	+79/−3 — probe type, createBetterSqliteProbe(), cascade gating, production wiring
src/proxy.ts	+19 — boot-time settings warm
tests/unit/db-adapters/driverFactory.test.ts	+118 — 7 new tests

Tests

7 new unit tests:

• Probe: non-Windows skips the child and reports ok (no spawn)
• Probe: successful child probe is cached (spawned at most once)
• Probe: non-zero child exit rejects better-sqlite3
• Probe: child spawn throw rejects better-sqlite3
• Probe: timed-out child (status === null) rejects better-sqlite3 — the fix(startup): v3.8.49 (and v3.8.48) HTTP requests hang indefinitely on Windows — not the #6800/#8654 warmup window #10627 hang case
• Cascade: rejected probe skips better-sqlite3, falls through to node:sqlite, queries work
• Cascade: passed probe still prefers better-sqlite3 (healthy path unchanged)

Verification performed

Check	Result
New tests (7)	✅ all pass
db-adapters suite (54 tests)	✅ 54 pass / 1 fail — the fail is a pre-existing Windows libuv UV_HANDLE_CLOSING teardown crash, reproduced identically on the unmodified base
authz suite (227)	225 pass — 2 fail on pre-existing EPERM on %TEMP% (files don't import the changed modules)
Typecheck (tsconfig.typecheck-core)	✅ 0 errors in changed files (only the known pre-existing omniglyph node_modules artifact)
ESLint (3 changed files)	✅ clean
E2E on Windows (Node 24, production wiring, real spawnSync)	✅ see matrix below

E2E matrix (real Windows machine):

World	Driver picked	Time
Healthy (probe passes)	better-sqlite3 (unchanged)	80ms probe, once
Broken addon (probe "bad")	node:sqlite	1ms — no hang

Platform behavior

	Before	After
Windows + healthy addon	better-sqlite3	better-sqlite3 (unchanged; +80ms probe once at first DB open)
Windows + broken addon	indefinite hang, all requests dead	node:sqlite fallback, requests served
Linux/macOS + broken addon	throws → node:sqlite/sql.js fallback	identical (probe is a no-op)
Linux/macOS + healthy addon	better-sqlite3	better-sqlite3 (identical, no subprocess)

Affected users

• v3.8.48/49 on Windows with a broken better-sqlite3 binary: app now works out of the box — no reinstall needed — because the failover reaches node:sqlite (built into Node 22.5+)
• Users wanting the faster native driver back can repair it (npm rebuild better-sqlite3)

Related

• Issue: fix(startup): v3.8.49 (and v3.8.48) HTTP requests hang indefinitely on Windows — not the #6800/#8654 warmup window #10627
• Not related to base-red: production build fails — 28 Node-builtin resolution errors from server-only code reaching the client bundle (dashboard/providers page → db/core) #10692/fix(ops): judge the canary install by the SHA on disk, not npm's exit code #10699/test(guard): widen the client-bundle guard to every "use client" entry point (#10692) #10700/feat(docker): expose DASHBOARD_ALLOW_EMBED as a build argument #10701 (those landed independently on the same release line)

[jonlwheat2-gif]

Confirmed: this is still live on the v3.8.50 line (the base of this PR)

Verified against the current release/v3.8.50 tip (b754e44e2, package.json version 3.8.50):

Check	Result
Commits since v3.8.49 tag	#10691, #10692/#10695, #10698, #10697, #10699, #10700 — none touch the driver factory
createBetterSqliteProbe / probe gating in shipped driverFactory.ts	0 matches
Any hang-guard machinery (child_process / spawnSync / AbortSignal.timeout) in the shipped load path	0 matches — still the unguarded in-process _require("better-sqlite3") in a plain try/catch
Shipped proxy.ts	26 lines, no warm — first request still triggers the cold native load on the request path

The vulnerable shape on the 3.8.50 tip is exactly what #10627 describes:

• driverFactory.ts (~line 136): const BetterSqlite = _require("better-sqlite3") — in-process, synchronous, no timeout, no child-process boundary
• The catch fires only on throws; a Windows DllMain loader-lock hang never throws → no fallthrough → first DB touch stalls forever

Two precision notes:

1. Tagging: the release branch carries version 3.8.50 but there is no v3.8.50 tag yet — tags stop at v3.8.49. The reporter's v3.8.48/49 are the last tagged builds; the 3.8.50 line is the current untagged tip, and the bug code is byte-identical on it.
2. Not a regression: the bug is structural (the throw-only cascade assumption) and predates 3.8.48/49 — the unguarded load exists since at least the May v3.8.5 tag. It's Windows-only and machine-dependent (broken addon ABI), which is why Linux CI never caught it and it shipped.

This PR's two changes are the first to touch that path on this release line.