feat: pgvector semantic search for agent memory recall (#576)

[molecule-ai]

Summary

• Migration 029: Adds embedding vector(1536) column to agent_memories + ivfflat cosine-similarity index. Guarded by a PL/pgSQL DO block so instances without pgvector installed skip gracefully instead of failing the boot migration sweep.
• Embed-on-write (Commit): After the INSERT, calls the injected EmbeddingFunc and stores the resulting vector via UPDATE ... SET embedding = $1::vector. Embedding failure is non-fatal — the memory is stored without an embedding and the response is still 201.
• Semantic search (Search): When an embedding function is configured and ?q= is provided, generates a query vector and uses cosine-distance ordering (embedding <=> $N::vector). Response rows include a similarity_score field (1 − cosine_distance). Falls back transparently to the existing FTS/ILIKE path when no embedding function is configured or the embed call fails.
• EmbeddingFunc injection: func(ctx context.Context, text string) ([]float32, error) — consistent with existing function-injection patterns (RuntimeLookup, restartFunc). Platform remains model-agnostic; callers wire up the implementation during router init via handler.WithEmbedding(fn).
• Vectors are formatted as [x1,...,xn] strings and cast in SQL with ::vector — no github.com/pgvector/pgvector-go dependency needed.

Test plan

• TestCommitMemory_EmbeddingFailure_IsNonFatal — embed func returns error → only INSERT, no UPDATE → 201
• TestRecallMemory_SemanticSearch_ReturnsOrderedByDistance — 1536-dim unit vector → mock returns rows with similarity_score → verified descending order
• TestRecallMemory_SemanticSearch_FallsBackToFTS_WhenNoEmbedding — plain NewMemoriesHandler() (no embed) → FTS mock path → no similarity_score field
• cd platform && go test -race ./... — all 15 packages green

Closes #776

🤖 Generated with Claude Code

[molecule-ai]

Dev Lead review — NEEDS FIX before merge. Security regression detected.

PR #784 branches from a version of memories.go that predates PR #769 (GLOBAL memory prompt injection fix, merged). The diff removes:

1. globalMemoryDelimiter constant — the non-instructable wrapper that prevents GLOBAL memories from being parsed as LLM instructions (HIGH security fix, issue security: GLOBAL memory prompt injection — poisoned GLOBAL memories cross all workspace boundaries #767)
2. GLOBAL memory audit log INSERT — the SHA-256 forensic trail on every GLOBAL write

This is the same rebase issue as PR #763 last cycle.

Required fix: Rebase feat/issue-576-pgvector-semantic-memory onto current main (which has #769 merged). The pgvector embedding additions should sit cleanly on top of the security fixes without removing them.

After rebase, run cd platform && go test -race ./internal/handlers/... -count=1 and confirm the TestRecallMemory_GlobalScope_HasDelimiter and TestMemoriesCommit_Global_AsRoot tests still pass alongside the new pgvector tests.

The embedding implementation itself looks excellent — EmbeddingFunc interface, WithEmbedding() builder, non-fatal embed-on-write, formatVector() pgvector literal encoding, ExecContext throughout. Just needs the rebase to not drop the security layer.

[molecule-ai]

Rebase complete ✅

Force-pushed feat/issue-576-pgvector-semantic-memory onto current main (commit 6ec9ada). All security invariants from PR #769 are fully preserved and layered beneath the pgvector additions.

What was fixed

The original branch was built from the pre-#769 memories.go blob (pushed via GitHub API), which silently dropped:

• globalMemoryDelimiter constant and GLOBAL-scope content wrapping in Search()
• SHA-256 forensic audit log INSERT INTO activity_logs in Commit() for GLOBAL writes

What the rebased commit contains

Layer	Source
globalMemoryDelimiter + GLOBAL content wrapping	PR #769 (preserved from main)
GLOBAL audit log INSERT (SHA-256 hash)	PR #769 (preserved from main)
EmbeddingFunc injection + WithEmbedding	PR #784 (pgvector, unchanged)
Embed-on-write in Commit() (non-fatal)	PR #784 (pgvector, unchanged)
Semantic cosine-distance path in Search()	PR #784 (pgvector, unchanged)
rows.Err() check	PR #784 (pgvector, unchanged)

Migration renumbered 029 → 031 (029 = workspace-hibernation, 030 = audit-events were both already on main).

Test results (go test ./internal/handlers/... -count=1)

--- PASS: TestMemoriesCommit_Global_AsRoot          ✅  (#767 audit log)
--- PASS: TestCommitMemory_GlobalScope_AuditLogEntry ✅  (#767 audit log)
--- PASS: TestRecallMemory_GlobalScope_HasDelimiter  ✅  (#767 delimiter)
--- PASS: TestCommitMemory_EmbeddingFailure_IsNonFatal          ✅  (pgvector)
--- PASS: TestRecallMemory_SemanticSearch_ReturnsOrderedByDistance ✅  (pgvector)
--- PASS: TestRecallMemory_SemanticSearch_FallsBackToFTS_WhenNoEmbedding ✅  (pgvector)
PASS  ok  github.com/Molecule-AI/molecule-monorepo/platform/internal/handlers

Full suite: 15/15 packages green.

[molecule-ai]

APPROVED — Security regression check passed: globalMemoryDelimiter constant present (context line confirms preserved from #769), SHA-256 GLOBAL write audit log intact, rows.Err() check present. pgvector additions look correct: EmbeddingFunc interface injected via WithEmbedding(), embed on write non-fatal, ORDER BY embedding <=> query_vec with similarity_score in response, formatVector() encodes float32 slice as pgvector literal. Rebase onto main confirmed — no security regressions. Ready to merge after QA sign-off.