change model for embedding

[DevLenn]

Hey,

how can i change the "nomic-embed-text-v1.5.f16.gguf" model in "gpt4all/resources" to the Q5_K_M quantized one?

just removing the old one and pasting the new one doesn't work.

Thank you in advance
Lenn

[cosmic-snow]

You posted this in Bindings, so it's mildly confusing as well (language bindings are not quite yet up-to-date and you can choose a model; specifically, the Python SDK v2.7.0 has not yet incorporated recent changes made in the GUI and I don't know what, if anything, will change in the next version).

However, this looks like a question regarding the chat application in v3.0.0 or higher. At least for the time being the name of the embedding model is hardcoded, see logic in embllm.cpp and here are the constants:

gpt4all/gpt4all-chat/embllm.cpp

Lines 33 to 34 in 6b8e0f7

	static const QString EMBEDDING_MODEL_NAME = u"nomic-embed-text-v1.5"_s;
	static const QString LOCAL_EMBEDDING_MODEL = u"nomic-embed-text-v1.5.f16.gguf"_s;

As a hacky workaround, you could probably try renaming your quantised one to that -- it might work, or maybe it won't because there are other checks somewhere, I don't know.

[DevLenn]

you are amazing, ty

[DevLenn]

@cosmic-snow seems to actually work, i'll test if it's faster then the original one, update the discussion and close it. THANK YOU!

[DevLenn]

What i've experienced: using the Q5_K_M Version is somewhat quicker, with not too worse results.
didn't test on GPU
would NOT recommend doing on CPU

[blackccpie]

Hello,

On the same topic, is it in the roadmap to be able to configure its own embedding model (mpnet for example), like we do for the chat models?

thx