[v3.8.50] feat(memory): MemoryBackend provider pattern with generic HTTP connector

[oyi77]

feat(memory): MemoryBackend Provider Pattern with Generic HTTP Connector

Summary

Introduces a pluggable MemoryBackend provider architecture with a generic HTTP connector that supports dynamic endpoint/query/path mapping for any REST-based memory backend (Obsidian, Notion, custom).

Changes

Core Architecture (Phases 1-3)

File	Description
src/lib/memory/backend.ts	MemoryBackend interface + CreateMemoryInput, MemoryFilter, SearchConfig, HealthCheckResult
src/lib/memory/manager.ts	MemoryManager singleton — register, configure, fallback routing, health checks
src/lib/memory/sqliteBackend.ts	Thin wrapper around existing store.ts/retrieval.ts — implements MemoryBackend
src/lib/memory/index.ts	Exports + auto-registers SQLiteBackend + initMemoryBackends() for app bootstrap
src/app/api/memory/route.ts	Delegates to memoryManager.list() / memoryManager.create()
src/app/api/memory/[id]/route.ts	Delegates to memoryManager.get/delete/update()

Optional Backends (Phase 4)

File	Description
src/lib/memory/genericBackend.ts	Generic HTTP connector with dynamic endpoint/query/path mapping
src/lib/memory/obsidianBackend.ts	Obsidian vault file-based backend (markdown + YAML frontmatter)
src/lib/memory/settings.ts	Extended with primaryBackend, fallbackBackends, backendConfigs + normalization
src/shared/schemas/memory.ts	Added backend fields to MemorySettingsExtendedSchema

Settings Schema (DB + API)

{
  "memoryEnabled": true,
  "primaryBackend": "sqlite",
  "fallbackBackends": ["obsidian"],
  "backendConfigs": {
    "obsidian": {
      "baseUrl": "http://localhost:27123",
      "apiKey": "...",
      "endpoints": {
        "search": "/api/v1/vault/memories/search",
        "create": "/api/v1/vault/memories",
        "get": "/api/v1/vault/memories/{memoryId}"
      },
      "queryParams": { "query": "q", "apiKeyId": "api_key" },
      "pathParams": { "id": "memoryId" }
    }
  }
}

GenericMemoryBackend — Dynamic Endpoint Mapping

// Any REST backend becomes pluggable via config
createGenericMemoryBackend("obsidian", "Obsidian Vault", {
  baseUrl: "http://localhost:27123",
  apiKey: process.env.OBSIDIAN_API_KEY,
  endpoints: {
    search: "/api/v1/vault/memories/search",
    create: "/api/v1/vault/memories",
    get: "/api/v1/vault/memories/{memoryId}",
  },
  queryParams: { query: "q", apiKeyId: "api_key" },
  pathParams: { id: "memoryId" },
});

Supported mappings:

• endpoints — override any REST path (supports {id}, {memoryId} placeholders)
• queryParams — rename any query parameter (query → q, apiKeyId → api_key, etc.)
• pathParams — rename path placeholders (id → memoryId)

Connecting Global omniroute to Custom Memory Backend

# Via environment variables (auto-registers on startup)
export OBSIDIAN_API_URL=http://localhost:27123
export OBSIDIAN_API_KEY=your-key
omniroute start

# Or via Settings API after startup (persists to DB)
curl -X PUT /api/settings/memory -d '{ "primaryBackend": "obsidian", ... }'

Verification

• ✅ Typecheck: tsc --noEmit — clean
• ✅ Memory tests: 45 passed, 6 skipped (pre-existing FTS5 test infra issue)
• ✅ Skills integration tests: 8 passed
• ✅ Behavioral parity: store.ts/retrieval.ts identical to v3.8.49 (newline-only diff)

Migration Notes

• Zero behavior change for existing SQLite memory — store.ts/retrieval.ts unchanged
• New settings are additive with safe defaults (primaryBackend: "sqlite", fallbackBackends: [])
• Existing callers (chatCore.ts, dashboard, CLI) work unchanged — they route through memoryManager

Future Work (Not in this PR)

• Dashboard UI for backend selector + config forms (Phase 5)
• Auto-discovery of npm plugins (omniroute-memory-backend-*)
• Response transformers for non-standard backend shapes

[diegosouzapw]

⏸️ Ejetada do merge-train v3.8.49 — triagem de cherrypick pendente

Esta PR foi ejetada do merge-train local (30→24 PRs) porque sua pegada no diff excede o orçamento seguro de validação em lote (>700 linhas ou nova fronteira arquitetural). Não é rejeição — é adiamento com motivo rastreável.

Motivo da ejeção

PR	Título	Files	Linhas	Risco em trem
#8760	feat: Vendo platform (novo módulo)	24	+3046	Novo módulo 3046+ linhas — precisa revisão arquitetural antes de entrar no release
#8713	fix(github): Copilot targetFormat override	581	+36652/-29407	Mass migration 36k+ linhas — provavelmente regen de lockfile + tree-wide diff, alto risco de regressão silenciosa
#8673	fix(security): tar ^7.5.22 (GHSA-w8wr-v893-vjvp)	578	+36542/-29411	Dependabot mass bump — GHSA real mas precisa validação isolada + tests de regressão de extract/archive
#8757	refactor(db): combo repository boundary (DRAFT)	9	+1114/-553	Draft + fronteira em DB core — refactor arquitetural, exige revisão de compat com localDb.ts (Hard Rule #2)
#8752	feat(memory): MemoryBackend provider pattern	17	+1347	Novo padrão de provider com HTTP connector genérico — abre nova superfície, exige revisão do contrato
#8718	fix(test): revive orphaned vitest tests	22	+733/-772	Suite-wide em vitest — risco de timing-flakes mascarando regressões (medido em #6218 follow-ups)

Como destravar (sugestão do operador)

O operador pediu que toda PR ejetada seja revisada para possível cherrypick. Providências sugeridas por categoria:

1. PRs com subconjunto cherrypickável (feat: add Vendo platform - African commerce MVP foundation #8760, [v3.8.50] feat(memory): MemoryBackend provider pattern with generic HTTP connector #8752, [v3.8.50] fix(test): revive orphaned vitest tests and fix CI routing #8718): abrir gh pr diff <N> e identificar arquivos pequenos, isolados, sem dependência cross-module. Considerar:

   git remote add author-<login> <fork-url> 2>/dev/null
   git fetch author-<login> <branch>
   git checkout -b cherry/<N>-<slug> release/v3.8.49
   git cherry-pick <commit-sha-do-subconjunto>

   Abrir nova PR com o subconjunto (mantendo Co-authored-by do autor original).

2. PRs com regen/lockfile ([v3.8.50] fix(github): honor per-model targetFormat override for Copilot custom models #8713, [v3.8.50] fix(security): override tar to ^7.5.22 (GHSA-w8wr-v893-vjvp) — unblocks the Lint job on every PR #8673): provavelmente só o package.json + package-lock.json + o patch de código mínimo. Rodar npm ls tar no tip local para confirmar a versão segura antes de cherrypick.

3. PRs com refactor arquitetural ([v3.8.50] refactor(db): add combo repository boundary #8757): exige review 1-a-1 com o operador — a fronteira db/ precisa ser validada contra Hard Rule build(deps): bump actions/github-script from 7 to 8 #2 (nunca lógica em localDb.ts).

Próximos passos

• Comentário fixado para rastreabilidade.
• Labels aplicadas: hold-vps, needs-rebase, merge-train-deferred.
• Estas 6 NÃO bloqueiam o trem das outras 24 (que vai entrar em modo --fast na .113).

[oyi77]

Merge-train ejection — status update

Thanks for the detailed breakdown of why this PR was ejected. Here is what has been done:

Fixed:

• Migration 118 (provider_param_filters): The file was docs-only with no executable SQL, causing "Migration FAILED: Query contained no valid SQL statement". Added SELECT 1; as a no-op so the migration registers properly. Pushed as commit 7446aab.

Pre-existing (not from our changes):

• DuckDuckGo Web Executor timeout (Unit Tests fast-path 3/4): The DuckDuckGo tests time out hitting a live endpoint. This is a pre-existing CI flake unrelated to our memory backend changes.

Plan:

Per your suggestion, I can prepare a cherry-pick subset isolating the most impactful pieces:

1. Core interfaces + schemas (backend.ts, genericBackend.ts, settings, schemas)
2. Existing backend refactors (sqliteBackend.ts, obsidianBackend.ts)
3. API routes + plumbing layer

Happy to arrange a 1-on-1 architecture review if you prefer to validate the full module boundary first. Which path works best?

[diegosouzapw]

Re-homed to release/v3.8.50: v3.8.49 entered its release freeze, so the branch now belongs to the release captain and development continues on the next cycle. Nothing is wrong with this PR — it just needed a live base. No action needed from you; CI will re-run against the new base.