fix(server): flush WAL after /api/embed so search sees new embeddings (#1149)#1359
Conversation
…abhigyanpatwari#1149) The CLI path (analyze --embeddings) flushes the WAL via closeLbug() during process exit, but /api/embed uses a shared DB connection that stays open for other routes. Without an explicit CHECKPOINT the embedding rows remain in the WAL and subsequent /api/search requests return zero semantic results until an unrelated operation forces a WAL replay. Issue a CHECKPOINT inside the withLbugDb callback after the embedding pipeline completes, matching the pattern already documented in closeLbug() ("especially critical after embedding writes, which generate large amounts of WAL data").
|
@azizur100389 is attempting to deploy a commit to the NexusCore Team on Vercel. A member of the Team first needs to authorize it. |
|
Nice find! There was one more close. We need to consolidate them. Would you like to work on this? |
|
Thanks for the merge! Yes, happy to take this on. I've had a look — Could you point me at which "one more close" you have in mind? Happy to scope the consolidation once I know the starting point. |
I meant the one you found and fixed. |
|
Got it — thanks for clarifying. So the consolidation is about unifying the CHECKPOINT / |
Feel free to collaborate with @ChrisGVE, I mentioned this here too - #1301 (comment) |
Summary
/api/embedcompletes, the embedding rows remain in the WAL because the server keeps the DB connection open for other routes./api/searchrequests return zero semantic results until an unrelated operation forces a WAL replay.CHECKPOINTinside thewithLbugDbcallback after the embedding pipeline finishes, matching the pattern already documented incloseLbug().Root cause
The CLI path (
analyze --embeddings) flushes the WAL viacloseLbug()duringprocess.exit(). The server path (/api/embed) uses a shared singleton connection that stays open --withLbugDbonly releases the session lock, it never checkpoints or closes. ThecloseLbug()comment is explicit: "This is especially critical after embedding writes, which generate large amounts of WAL data."Test plan
npx tsc --noEmit-- cleannpx vitest run test/unit/rate-limit.test.ts-- 16/16 pass (source-grep confirms api.ts structure)POST /api/embed-> poll completion ->POST /api/searchreturns semantic results (reviewer verification)Closes #1149