Skip to content

fix(embedding): download embeddinggemma external-data sibling (.onnx_data) - #1667

Open
JeanBaptisteRenard wants to merge 1 commit into
MemPalace:developfrom
JeanBaptisteRenard:fix/embeddinggemma-external-data
Open

fix(embedding): download embeddinggemma external-data sibling (.onnx_data)#1667
JeanBaptisteRenard wants to merge 1 commit into
MemPalace:developfrom
JeanBaptisteRenard:fix/embeddinggemma-external-data

Conversation

@JeanBaptisteRenard

Copy link
Copy Markdown

Problem

MEMPALACE_EMBEDDING_MODEL=embeddinggemma crashes on first use on a clean cache:

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL :
External data path validation failed for initializer: model.embed_tokens.weight_quantized.
External data path does not exist: .../onnx/model_quantized.onnx_data

The q8 export (onnx-community/embeddinggemma-300m-ONNX) keeps its weights in a sibling model_quantized.onnx_data file that the .onnx references by relative path. EmbeddinggemmaONNX._lazy_load() only hf_hub_downloads model_quantized.onnx, so the weights file is never fetched and onnxruntime can't resolve it.

Fix

Also fetch the .onnx_data sibling into the same snapshot dir (one extra hf_hub_download), wrapped in try/except so inlined-weight exports without a sibling stay non-fatal.

Repro / verification

  • Before: clean HF cache → embeddinggemma → crash above (only model_quantized.onnx present in the snapshot onnx/ dir).
  • After: the .onnx_data lands next to the .onnx; embeddinggemma loads and embeds. Verified the model works once both files are co-located — cross-lingual FR/EN cosine 0.89 (parallel) vs 0.40 (unrelated) on a 6-pair probe, matching the model card's multilingual claim.

No behavior change for minilm. Single-file additive fetch; honors the local-first principle (still HF-hosted weights, same as before — just the missing half).

The q8 ONNX export (onnx-community/embeddinggemma-300m-ONNX) stores its
weights in a sibling model_quantized.onnx_data file referenced by the .onnx
via a relative path. _lazy_load() only fetched model_quantized.onnx, so on a
clean cache the first embeddinggemma use crashes:

  onnxruntime ... FAIL: External data path validation failed ...
  External data path does not exist: .../model_quantized.onnx_data

Fetch the .onnx_data sibling into the same snapshot dir so onnxruntime can
resolve the relative path. Wrapped in try/except since inlined-weight exports
have no sibling (non-fatal).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates mempalace/embedding.py to download the external-data sibling file for the quantized ONNX model if it exists, preventing runtime failures when weights are stored externally. The review feedback recommends catching a specific EntryNotFoundError from huggingface_hub.utils instead of a broad Exception to avoid swallowing unrelated errors, and suggests pinning the repository revision to ensure consistency between downloads.

Comment thread mempalace/embedding.py
Comment on lines +200 to +208
try:
hf_hub_download(
_EMBEDDINGGEMMA_REPO,
subfolder="onnx",
filename=_EMBEDDINGGEMMA_ONNX + "_data",
)
except Exception:
# Some exports inline their weights — a missing sibling is non-fatal.
logger.debug("No external-data sibling for %s", _EMBEDDINGGEMMA_ONNX)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Specific Exception Handling & Revision Pinning

  1. Avoid Broad Exception Catching: Catching Exception can swallow unrelated errors (such as network timeouts, DNS failures, permission issues, or disk full errors) and lead to confusing ONNX Runtime errors later. Instead, import and catch EntryNotFoundError from huggingface_hub.utils to specifically handle the case where the sibling file does not exist on the hub.
  2. Revision Pinning Recommendation: Currently, the downloads do not specify a revision (defaulting to "main"). If a new commit is pushed to the repository between the download of the .onnx file and the .onnx_data file, they could be resolved to different commit hashes, leading to mismatched files or a missing sibling error. Pinning the revision to a specific commit hash (e.g., via a _EMBEDDINGGEMMA_REVISION constant) ensures consistency, prevents breaking changes from upstream updates, and allows huggingface_hub to load cached files instantly without making remote HEAD requests.
Suggested change
try:
hf_hub_download(
_EMBEDDINGGEMMA_REPO,
subfolder="onnx",
filename=_EMBEDDINGGEMMA_ONNX + "_data",
)
except Exception:
# Some exports inline their weights — a missing sibling is non-fatal.
logger.debug("No external-data sibling for %s", _EMBEDDINGGEMMA_ONNX)
try:
from huggingface_hub.utils import EntryNotFoundError
hf_hub_download(
_EMBEDDINGGEMMA_REPO,
subfolder="onnx",
filename=_EMBEDDINGGEMMA_ONNX + "_data",
)
except EntryNotFoundError:
# Some exports inline their weights — a missing sibling is non-fatal.
logger.debug("No external-data sibling for %s", _EMBEDDINGGEMMA_ONNX)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant