[ML] VoyageAI Integration - Clean Version#139812
Conversation
- text embedding models - contextual models - multimodal models
|
@fzowl please enable the option "Allow edits and access to secrets by maintainers" on your PR. For more information, see the documentation. |
- text embedding models - contextual models - multimodal models
|
@DonalEvans Can you please take a look? |
- text embedding models - contextual models - multimodal models
|
Pinging @elastic/search-inference-team (Team:Search - Inference) |
|
@fzowl Sorry for taking so long to get to this, I'm just coming back from vacation. I've had a quick look at the changes in this PR, and I feel that the approach used to implement multimodal embeddings for Voyage is at odds with the way we plan to implement them for other providers and will need to be reworked. This PR which adds support for the I had hoped to get that PR merged or at least ready for review before the Christmas break because I knew that this PR would need to be based on it, but I ran out of time, so I apologise for asking you to rework this PR as a result. Until the Jina AI embedding PR is merged, I don't think it will be possible to really complete this PR, because the Jina PR includes some changes to the Some other broad concerns around the approach in this PR after a quick look at it are:
Using the `embedding` task input format to specify a list of lists of String for a contextual embedding |
|
@DonalEvans I'm closing this one and will open a fresh PR. |
Summary
This PR contains the cleaned up VoyageAI integration with the following improvements:
Testing:
All tests compile successfully
Text embeddings functionality verified
Multimodal embeddings support added
Rerank functionality included