feat(embedding): add OpenAI-compatible /v1/embeddings backend

[maximilize]

What does this PR do?

Adds an embedding_model: "openai-compat" option that computes embeddings via any OpenAI-compatible /v1/embeddings endpoint (LM Studio, llama.cpp, vLLM, Ollama's OpenAI shim, or a self-hosted server) instead of a local ONNX model — useful for larger / multilingual embedders (e.g. Qwen3-Embedding) or GPU offload.

• New OpenAICompatEmbeddingFunction (stdlib urllib only — no new dependency): batches requests, sends encoding_format: "float" + a mempalace/<ver> User-Agent (avoids the Cloudflare 403 from openai-compat provider probe fails with 403 on Cloudflare-fronted endpoints (Python-urllib User-Agent blocked) #1570), validates the response (contiguous 0..n-1 indices + well-formed vectors), and L2-normalizes for the cosine collection. Implements embed_query (ChromaDB 1.5 dispatches query embedding through it). name() encodes the model id so switching it surfaces as a clear rebuild-index signal. Failures raise a module-specific EmbeddingAPIError.
• Endpoint settings are resolved by MempalaceConfig as a single source of truth: embedding_api_url / embedding_api_model / embedding_api_key, each overridable via the matching MEMPALACE_EMBEDDING_API_* env var.
• Opt-in, local-first preserved: the default stays minilm; nothing changes unless you select openai-compat. When the endpoint is on your machine/LAN no content leaves your network — consistent with the zero-API-by-default principle (the endpoint is the user's explicit choice and may be fully local).
• Mirrors the existing openai-compat LLM provider naming. The selector reuses embedding_model (not embedding_device) because the device axis is for hardware accelerators.

Generalizes the Ollama-only approach in #982 (currently conflicting) to any OpenAI-compatible endpoint — Ollama works via its /v1 shim. Does not touch reranking (#1032).

How to test

Point at any OpenAI-compatible server via ~/.mempalace/config.json:

{
  "embedding_model": "openai-compat",
  "embedding_api_url": "http://localhost:1234",
  "embedding_api_model": "<server-side model id>"
}

(or the equivalent MEMPALACE_EMBEDDING_API_* env vars), then:

mempalace init <dir> && mempalace mine <dir> && mempalace search "<query>"

Unit tests need no server and no network (urlopen is mocked):

uv run pytest tests/test_embedding_api.py -v

Checklist

• Tests pass (uv run pytest tests/ -v — 2304 passed, 3 skipped; no network / API keys needed)
• No hardcoded paths
• Linter passes (ruff check .)

Related issues

Closes #1559. Related: #756, #1261, #1563, #903, #1561. Generalizes the Ollama-only #982.

[gemini-code-assist]

Code Review

This pull request introduces support for generating embeddings via any OpenAI-compatible /v1/embeddings endpoint by adding the openai-compat embedding model option. It implements the OpenAICompatEmbeddingFunction using standard library urllib to handle batching, response sorting, and L2-normalization, alongside configuration updates and new unit tests. The review feedback focuses on enhancing robustness by catching additional exceptions (http.client.HTTPException and ValueError) during API requests, validating that the JSON response is a dictionary to prevent AttributeError crashes, and adding corresponding unit tests for these error scenarios.

[gemini-code-assist]

The current except block misses two critical exception classes that can be raised by urlopen and cause unhandled crashes:

1. http.client.HTTPException: Raised on low-level HTTP protocol violations (e.g., BadStatusLine, IncompleteRead), which are common with local or overloaded LLM servers.
2. ValueError: Raised by urlopen if the configured URL has an invalid or missing scheme (e.g., localhost:1234 instead of http://localhost:1234). Note that json.JSONDecodeError is a subclass of ValueError, so catching ValueError covers both cases.

Adding these to the except tuple ensures all connection and configuration failures are cleanly wrapped in EmbeddingAPIError.

Suggested change

	except (HTTPError, URLError, OSError, json.JSONDecodeError) as e:
	raise EmbeddingAPIError(
	f"Embedding API request to {self._url} failed: {e}. Check that the "
	f"server is reachable and MEMPALACE_EMBEDDING_API_URL / embedding_api_url "
	f"is correct."
	) from e
	except (HTTPError, URLError, OSError, http.client.HTTPException, ValueError) as e:
	raise EmbeddingAPIError(
	f"Embedding API request to {self._url} failed: {e}. Check that the "
	f"server is reachable and MEMPALACE_EMBEDDING_API_URL / embedding_api_url "
	f"is correct."
	) from e

[gemini-code-assist]

If the API returns a non-dictionary JSON response (such as null, a JSON list, or a raw string), data will not be a dictionary. Calling data.get("data") on a non-dictionary object will raise an unhandled AttributeError and crash the application.

We should explicitly validate that data is a dictionary before attempting to call .get() on it.

Suggested change

	rows = data.get("data")
	if not isinstance(rows, list):
	raise EmbeddingAPIError(
	f"Embedding API at {self._url} returned no 'data' array: {data.get('error', data)}"
	)
	if not isinstance(data, dict):
	raise EmbeddingAPIError(
	f"Embedding API at {self._url} returned a non-object response: {data}"
	)
	rows = data.get("data")
	if not isinstance(rows, list):
	raise EmbeddingAPIError(
	f"Embedding API at {self._url} returned no 'data' array: {data.get('error', data)}"
	)

[gemini-code-assist]

To handle low-level HTTP protocol exceptions (like BadStatusLine or IncompleteRead) raised by urlopen, we need to import http.client so we can catch http.client.HTTPException.

Suggested change

	import json
	from urllib.error import HTTPError, URLError
	from urllib.request import Request, urlopen
	import http.client
	import json
	from urllib.error import HTTPError, URLError
	from urllib.request import Request, urlopen

[gemini-code-assist]

Let's add unit tests to verify that both http.client.HTTPException and ValueError (e.g., from an invalid URL scheme) are correctly caught and translated into EmbeddingAPIError.

def test_raises_on_transport_error(monkeypatch):
    from urllib.error import URLError

    def boom(req, timeout=None):
        raise URLError("connection refused")

    monkeypatch.setattr("urllib.request.urlopen", boom)
    ef = embedding.OpenAICompatEmbeddingFunction("http://h", "m")
    with pytest.raises(RuntimeError, match="failed"):
        ef(["a"])


def test_raises_on_http_exception(monkeypatch):
    from http.client import HTTPException

    def boom(req, timeout=None):
        raise HTTPException("incomplete read")

    monkeypatch.setattr("urllib.request.urlopen", boom)
    ef = embedding.OpenAICompatEmbeddingFunction("http://h", "m")
    with pytest.raises(embedding.EmbeddingAPIError, match="failed"):
        ef(["a"])


def test_raises_on_invalid_url_scheme(monkeypatch):
    # urlopen raises ValueError on invalid/missing schemes
    ef = embedding.OpenAICompatEmbeddingFunction("invalid_scheme://h", "m")
    with pytest.raises(embedding.EmbeddingAPIError, match="failed"):
        ef(["a"])

[gemini-code-assist]

Let's add a unit test to verify that non-dictionary JSON responses (like a JSON list) are handled gracefully and raise a clear EmbeddingAPIError instead of an unhandled AttributeError.

Suggested change

	def test_raises_and_surfaces_server_error_body(monkeypatch):
	# HTTP 200 with an OpenAI-style error envelope (no "data") — surface it.
	def err(req, timeout=None):
	return _FakeResp(json.dumps({"error": {"message": "model not found"}}).encode())
	monkeypatch.setattr("urllib.request.urlopen", err)
	ef = embedding.OpenAICompatEmbeddingFunction("http://h", "m")
	with pytest.raises(embedding.EmbeddingAPIError, match="model not found"):
	ef(["a"])
	def test_raises_and_surfaces_server_error_body(monkeypatch):
	# HTTP 200 with an OpenAI-style error envelope (no "data") — surface it.
	def err(req, timeout=None):
	return _FakeResp(json.dumps({"error": {"message": "model not found"}}).encode())
	monkeypatch.setattr("urllib.request.urlopen", err)
	ef = embedding.OpenAICompatEmbeddingFunction("http://h", "m")
	with pytest.raises(embedding.EmbeddingAPIError, match="model not found"):
	ef(["a"])
	def test_raises_on_non_dict_response(monkeypatch):
	def bad(req, timeout=None):
	return _FakeResp(json.dumps([1, 2, 3]).encode()) # JSON list instead of dict
	monkeypatch.setattr("urllib.request.urlopen", bad)
	ef = embedding.OpenAICompatEmbeddingFunction("http://h", "m")
	with pytest.raises(embedding.EmbeddingAPIError, match="non-object response"):
	ef(["a"])

[maximilize]

Thanks @gemini-code-assist — all three addressed in c3e6692:

• except now also wraps http.client.HTTPException (BadStatusLine / IncompleteRead) and ValueError (the latter subsumes json.JSONDecodeError and covers an invalid/missing URL scheme).
• _vectors_from_response rejects a non-dict top-level response before .get(), so a JSON list / null / string raises a clear EmbeddingAPIError instead of an unhandled AttributeError.
• Added tests for all three. The suggested transport-error and server-error-body tests already existed, so I didn't duplicate them.

---

For maintainers — design summary (re #1559):

• Opt-in. Default stays minilm; nothing changes unless embedding_model: "openai-compat" is selected. Zero regression risk.
• Single source of truth. Endpoint/model/key resolved by MempalaceConfig: embedding_api_url / embedding_api_model / embedding_api_key, each overridable via the matching MEMPALACE_EMBEDDING_API_* env var.
• No new dependency — stdlib urllib only. Stays local (no external-API warning) when the endpoint is on the machine/LAN, consistent with the zero-API-by-default principle.
• Selector reuses embedding_model (not embedding_device, which is for hardware accelerators) and mirrors the existing openai-compat LLM provider naming.
• Generalizes the Ollama-only feat(embeddings): opt-in Ollama embedding backend for GPU acceleration #982 (currently conflicting) to any OpenAI-compatible /v1/embeddings server — Ollama works via its /v1 shim. Does not touch reranking (feat: add Weibull decay and rerank pipeline for search results #1032). Also relates to Add MemPalace OpenAI 3072-dim embedding support or something similar opensource. #756, Feature request: configurable embedding_function (currently hardcoded to all-MiniLM-L6-v2) #1261, Feature Request: Configuration-driven embedding model switching (no source code edits) #1563, bug: embedding model mismatch — MCP server uses MiniLM (384-dim) while ingest can use mpnet (768-dim) #903, Feature Request: Persist and verify embedding model metadata in palace collections #1561.

Full suite is green and the unit tests need no server/network. Happy to adjust the config surface or naming before merge — wanted to give a concrete, tested proposal rather than just a design sketch.

[gemini-code-assist]

The refinements to _vectors_from_response and the expanded exception handling significantly improve the robustness of the OpenAICompatEmbeddingFunction. Specifically, the explicit validation of the response structure and the inclusion of http.client.HTTPException ensure that the implementation handles common network-level failures gracefully without leaking cryptic errors. The added unit tests in tests/test_embedding_api.py provide excellent coverage for these edge cases, ensuring that the integration remains stable under various server-side failure modes.

[jhsmith409]

+1 — this is exactly what a shared / multi-machine setup needs.

When several hosts share one vector store (e.g. a qdrant or pgvector collection), every host has to embed with the identical model or the vectors are mutually incompatible. Today that means each node runs its own copy of the same local ONNX model. An OpenAI-compatible /v1/embeddings option lets the whole fleet point at one GPU-hosted embedder (Qwen3-Embedding, etc.), which both guarantees consistency across nodes and offloads embedding from CPU-only hosts.

It would pair naturally with the shared-backend / network-backend efforts (#1963 and the qdrant/pgvector work). Would love to see this land.

[cuttie1979]

Hey maintainers — we'd really love to see this land. 👋

Use case: We run several hosts against one shared vector store (Qdrant), and every node has to embed with the identical model or the vectors are mutually incompatible. Today that means running a local ONNX model copy on every machine — wasteful on CPU-only hosts and hard to keep consistent. An OpenAI-compatible /v1/embeddings option lets the whole fleet point at one GPU-hosted embedder (Qwen3-Embedding), guaranteeing vector consistency across nodes while offloading embedding entirely.

Status: code review done (all bot findings addressed in c3e6692), branch mergeable, tests pass, strictly opt-in (embedding_model: "openai-compat") with zero regression risk to the default path. Open since June 1 with no maintainer feedback yet.

Would pair naturally with the shared-backend work (#1963, Qdrant/pgvector backends). @milla-jovovich @bensig @igorls — any chance of a review? Happy to rebase or adjust anything if needed.

[igorls]

Maintainer rebase onto current develop

Rebased this branch onto latest origin/develop (was ~238 commits behind) and force-pushed to feat/openai-compat-embeddings with --force-with-lease.

Rebase result

• Clean rebase — the original 2 commits applied with no conflicts.

• +1 post-rebase commit so the suite is green on current develop:

  Commit	What
  d471a9e	feat: OpenAI-compatible /v1/embeddings backend (original)
  f272c84	fix: harden API error handling (original review follow-up)
  d9a24c7	post-rebase: opt test_embedding_api out of conftest’s stable EF mock + move CHANGELOG entry to Unreleased

Why the extra commit was needed

Current develop has an autouse fixture in tests/conftest.py that stubs get_embedding_function for most modules (avoids loading ONNX on Windows). Without opting test_embedding_api into _REAL_EMBEDDING_TEST_MODULES, the selection-branch tests received _StableTestEmbeddingFunction and failed (3 failures). Same pattern as test_embedding / test_embeddinggemma.

CHANGELOG context had drifted: the feature bullet rebased under released 3.7.0 → Performance. Moved it to Unreleased → Features.

Verification (local)

uv run pytest tests/test_embedding_api.py tests/test_embedding.py tests/test_embeddinggemma.py -v
# 75 passed
uv run ruff check mempalace/embedding.py mempalace/config.py tests/test_embedding_api.py tests/conftest.py
# All checks passed

Still open (non-blocking for CI, useful before merge)

1. EF cache lock — ONNX path uses _EF_CACHE_LOCK; openai-compat still does lock-free get/set (double-create race under concurrent first init).
2. Privacy warning — mirror llm_client._endpoint_is_local when the embedding URL is not loopback/LAN.
3. CI — waiting on checks after this push.

@maximilize — sorry for the force-push on your branch; maintainerCanModify was on. Happy to adjust anything if you’d rather take the rebase yourself next time.

Closes the “open since June / needs rebase” blocker called out on #1559.