Skip to content

test(backends): cover the _embed_texts fallback and empty-batch guard - #2191

Merged
igorls merged 1 commit into
MemPalace:developfrom
mbeacom:mbeacom-fix-numpy2-embedding-floats
Aug 11, 2026
Merged

test(backends): cover the _embed_texts fallback and empty-batch guard#2191
igorls merged 1 commit into
MemPalace:developfrom
mbeacom:mbeacom-fix-numpy2-embedding-floats

Conversation

@mbeacom

@mbeacom mbeacom commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Refs #2190. Follow-up to #2187tests only, no production code changes.

Note on scope. I independently diagnosed and fixed #2190, then found #2187 had already landed the same one-line .tolist() fix while I was verifying. Rather than open a duplicate, I rebased onto develop and kept only the part that isn't already covered. mempalace/backends/embedding_wrapper.py is byte-identical to develop on this branch.

What does this PR do?

#2187 fixed the default-backend ingest failure and covered the numpy path well — an ndarray-returning EF plus a real EmbeddingCollection.upsert round-trip. Two branches of the fixed function are still unexercised:

1. The float(x) fallback arm. _embed_texts branches on hasattr(v, "tolist"):

return [
    v.tolist() if hasattr(v, "tolist") else [float(x) for x in v]  # numpy | plain sequence
    for v in vectors
]

Only the numpy side runs under test. The fallback is the arm serving embedders that return plain sequences — custom/BYO EFs, and rows that arrive as tuples. I confirmed the gap by mutation: replacing that arm with list(v) leaves the suite fully green. A regression there would surface only in the field, on a non-default embedder, as the same production ValueError #2190 describes.

2. The if not texts: return [] guard. Also untested. Callers pass empty batches (a drawer set fully filtered by dedup), and constructing the EF is the expensive part — on the ONNX default it spins up a native session. Deleting the early return is likewise green today.

Both tests are written so they assert the behaviour, not just the shape:

  • the plain-sequence EF yields Decimal, so == [[0.5, 0.25], ...] plus type(x) is float proves a real conversion happened rather than values passing through unchanged;
  • the empty-batch test asserts by making get_embedding_function raise, so it verifies the EF is never constructed, not merely that [] came back.

Neither test depends on the NumPy version. They live in tests/test_embedding.py for exactly the reason #2187 documented: conftest's autouse _stable_embedding_function_for_tests replaces _embed_texts outright for every module outside _REAL_EMBEDDING_TEST_MODULES, so tests placed elsewhere exercise the stub and pass against broken code.

How to test

Mutation-verified — each new test was proven to fail against a deliberately broken tree, then the tree was restored:

Mutation Result
fallback arm → list(v) test_embed_texts_handles_plain_sequence_embedders fails (type(x) is float → False)
delete if not texts: return [] test_embed_texts_short_circuits_on_empty_input fails (get_embedding_function must not be called for an empty batch)
unmodified develop ✅ both pass

The load-bearing point: deleting either branch of the shipped function leaves the suite entirely green today. That is the argument for these tests existing.

uv run pytest tests/ -q --ignore=tests/benchmarks   #  3854 passed, 31 skipped
uv run ruff check .                                 #  All checks passed!
uv run ruff format --check .                        #  212 files already formatted

git diff develop -- mempalace/ is empty — the change is confined to tests/test_embedding.py (+49).

Correcting the root-cause framing (this was mine, and it was wrong)

An earlier revision of this PR described #2190 as a "NumPy 2.x" failure, and I repeated that on the issue. That attribution is incorrect and I want it on the record rather than quietly edited away.

np.float32 has never been a float subclass in any NumPy major. Verified directly across the full supported range (pyproject.toml pins numpy>=1.24):

NumPy isinstance(np.float32(0.5), float) isinstance(np.float64(0.5), float)
1.24.4 False True
1.26.4 False True
2.4.4 False True

float32 mro on 1.26.4: ['float32', 'floating', 'inexact', 'number', 'generic', 'object'] — no float.

So normalize_embeddings would have rejected list(arr) output on NumPy 1.x just the same; there was no version boundary. The accurate framing is the one #2187's own docstring used, which pointedly never claimed version-specificity:

A latent defect exposed by a capability change0e79797 added requires_explicit_embeddings, routing the default Chroma backend through _embed_texts for the first time — and missed because conftest stubs _embed_texts out of every test module outside _REAL_EMBEDDING_TEST_MODULES.

A corollary worth stating explicitly: a numpy<2 upper bound would not have prevented this, and shouldn't be added on this basis.

None of this affects the change here — both tests target version-independent branches and remain mutation-proven.

Independent confirmation that #2190 is resolved on develop

  • Pre-fix: _embed_texts(['hello world'])<class 'numpy.float32'>, and normalize_embeddings raises.
  • On develop: → <class 'float'>, normalize_embeddings OK, dim 384.
  • End-to-end, real mine (not --dry-run) of 3 Claude transcripts into a throwaway palace: pre-fix 0 drawers filed, aborting in _validate_and_prepare_upsert_request; on develop 86 drawers filed, and mempalace search returns the drawer verbatim.

Checklist

  • Tests pass (python -m pytest tests/ -v) — 3854 passed, 31 skipped
  • No hardcoded paths
  • Linter passes (ruff check .) — plus ruff format --check . clean

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@mbeacom
mbeacom force-pushed the mbeacom-fix-numpy2-embedding-floats branch from 141c902 to 4826529 Compare August 8, 2026 13:41
@mbeacom mbeacom changed the title fix(backends): return Python floats from _embed_texts for NumPy 2.x test(backends): cover the _embed_texts fallback and empty-batch guard Aug 8, 2026
MemPalace#2187 fixed the default-backend ingest failure (MemPalace#2190) and covered the
numpy path: an ndarray-returning EF plus a real EmbeddingCollection.upsert.
Two branches of the fixed function are still unexercised.

(The defect was not NumPy-2.x-specific. `np.float32` has never been a
`float` subclass in any NumPy major -- verified on 1.24.4, 1.26.4 and
2.4.4 -- so `list(arr)` output was always rejected. It was latent until
0e79797 added `requires_explicit_embeddings` and routed the default
Chroma backend through `_embed_texts` for the first time.)

`_embed_texts` branches on `hasattr(v, "tolist")`. The `float(x)` arm is
the one that serves embedders returning plain sequences — custom/BYO EFs,
and rows arriving as tuples. Nothing ran it, so dropping the `float()`
call there stays green and reaches users as the same production
ValueError, just on a non-default embedder.

The `if not texts: return []` guard is likewise untested. Callers pass
empty batches (a drawer set fully filtered by dedup), and loading the EF
is the expensive part — on the ONNX default it spins up a native session.

Both tests are mutation-verified against this tree:

  - replacing the fallback with `list(v)` fails
    test_embed_texts_handles_plain_sequence_embedders
  - deleting the early return fails
    test_embed_texts_short_circuits_on_empty_input

The plain-sequence EF yields `Decimal`, so the assertion proves a real
conversion rather than values passing through unchanged, and the
empty-batch test asserts by making `get_embedding_function` raise, so it
verifies the EF is never constructed rather than only checking the
return value.

Tests only — no production code changes. They live in `test_embedding.py`
for the reason MemPalace#2187 documented: conftest's autouse
`_stable_embedding_function_for_tests` replaces `_embed_texts` outright
for every module outside `_REAL_EMBEDDING_TEST_MODULES`.

Refs MemPalace#2190

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@mbeacom
mbeacom force-pushed the mbeacom-fix-numpy2-embedding-floats branch from 1a06310 to 656e0a4 Compare August 8, 2026 14:00

@igorls igorls left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wave 3 minimal for 3.7.0: LGTM. Docs/tests/tiny fix only.

@igorls
igorls merged commit b6eae9b into MemPalace:develop Aug 11, 2026
@mbeacom
mbeacom deleted the mbeacom-fix-numpy2-embedding-floats branch August 15, 2026 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chroma ingest fails on NumPy 2.x: _embed_texts returns np.float32 scalars that normalize_embeddings rejects

2 participants