v1.10.2 🦩

Fixes 🦋

Activate the Swedish tokenization Pipeline

The Swedish tokenization pipeline were deactivated in the previous versions, now it is activated when specifying the index Language in the settings:

PATCH /indexes/:index-name/settings

{
  "localizedAttributes": [ { "locales": ["swe"], "attributePatterns": ["*"] } ]
}

related PR: #4949

v1.11.0-rc.0 🐿️

Warning

Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Meilisearch v1.11 introduces AI-powered search performance improvements thanks to binary quantization and various usage changes, all of which are steps towards a future stabilization of the feature. We have also improved federated search usage following user feedback.

New features and updates 🔥

Experimental - AI-powered search improvements

This release is Meilisearch's first step towards the stabilization of AI-powered search and introduces a few breaking changes to its API. Consult the PRD for full usage details.

Done by @dureuill in #4906, #4920, #4892, and #4938.

⚠️ Breaking changes

• When performing AI-powered searches, hybrid.embedder is now a mandatory parameter in GET and POST /indexes/{:indexUid}/search
• As a consequence, it is now mandatory to pass hybrid even for pure semantic searches
• embedder is now a mandatory parameter in GET and POST /indexes/{:indexUid}/similar
• Meilisearch now ignores semanticRatio and performs a pure semantic search for queries that include vector but not q

Addition & improvements

• The default model for OpenAI is now text-embedding-3-small instead of text-embedding-ada-002
• This release introduces a new embedder option: documentTemplateMaxBytes. Meilisearch will truncate a document's template text when it goes over the specified limit
• Fields in documentTemplate include a new field.is_searchable property. It is also possible to filter out document fields:

v1.11:

{% for field in fields %}
  {% if field.is_searchable and not field.value == nil %}
    {{ field.name }}: {{ field.value }}\n
  {% endif %}
{% endfor %}

v1.10:

{% for field in fields %}
  {{ field.name }}: {{ field.value }}\n
{% endfor %}

Changes to document templates are backwards compatible. Embedders using the v1.10 document template will continue working as before The new default document template will only work newly created embedders.

Vector database indexing performance improvements

v1.11 introduces a new embedder option, binaryQuantized:

curl \
  -X PATCH 'http://localhost:7700/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "embedders": {
      "image2text": {
        "binaryQuantized": true
      }
    }
  }'

Enable binary quantization to convert embeddings of floating point numbers into embeddings of boolean values. This will negatively impact the relevancy of AI-powered searches, but significantly improve performance in large collections with more than 100 dimensions.

In our benchmarks, this reduced the size of the database by a factor of 10, and divided the indexing time by a factor of 6 with little impact on search times.

Warning

Enabling this feature will update all of your vectors to contain only 1s or -1s, impacting the relevancy significantly.

You cannot revert this option once you enable it. Before setting binaryQuantized to true, Meilisearch recommends testing it in a smaller or duplicate index in a development environment.

Done by @irevoire in #4941.

Federated search improvements

Facet distribution and stats for federated searches

This release adds two new federated search options, facetsByIndex and mergeFacets. These allow you to request facet distributions and stats data for a federated search.

Facet information by index

To obtain facet distribution and stats for each separate index, use facetsByIndex when querying the POST /multi-search endpoint:

POST /multi-search
{
  "federation": {
    "limit": 20,
    "offset": 0,
	"facetsByIndex": {
	  "movies": ["title", "id"],
	  "comics": ["title"],
	}
  },
  "queries": [
    {
      "q": "Batman",
      "indexUid": "movies"
    },
    {
      "q": "Batman",
      "indexUid": "comics"
    }
  ]
}

The multi-search response will include a new field, facetsByIndex with facet data separated per index:

{
  "hits": […],
  …
  "facetsByIndex": {
      "movies": {
        "distribution": {
          "title": {
            "Batman returns": 1
          },
          "id": {
            "42": 1
          }
        },
        "stats": {
          "id": {
            "min": 42,
            "max": 42
          }
        }
      },
     …
  }
}

Merged facet information

To obtain facet distribution and stats for all index merged into a single ,use both facetsByIndex and mergeFacets when querying the POST /multi-search endpoint:

POST /multi-search
{

  "federation": {
    "limit": 20,
    "offset": 0,
	  "facetsByIndex": {
	    "movies": ["title", "id"],
	    "comics": ["title"],
	  },
	  "mergeFacets": {
	    "maxValuesPerFacet": 10,
	  }
  }
  "queries": [
    {
      "q": "Batman",
      "indexUid": "movies"
    },
    {
      "q": "Batman",
      "indexUid": "comics"
    }
  ]
}

The response includes two new fields, facetDistribution and facetStarts:

{
  "hits": […],
  …
  "facetDistribution": {
    "title": {
      "Batman returns": 1
      "Batman: the killing joke":
    },
    "id": {
      "42": 1
    }
  },
  "facetStats": {
    "id": {
      "min": 42,
      "max": 42
    }
  }
}

Done by @dureuill in #4929.

Experimental — New STARTS_WITH filter operator

Enable the experimental feature to use the STARTS_WITH filter operator:

curl \
  -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "containsFilter": true
  }'

Use the STARTS_WITH operator when filtering:

curl \
  -X POST http://localhost:7700/indexes/movies/search \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "filter": "hero STARTS_WITH spider"
  }'

🗣️ This is an experimental feature, and we need your help to improve it! Share your thoughts and feedback on this GitHub discussion.

Done by @Kerollmops in #4939.

Other improvements

• Language support and localizedAttributes settings by @ManyTheFish in #4937
  • Add ISO-639-1 variants
  • Convert ISO-639-1 into ISO-639-3
• Add a German language tokenizer by @luflow in meilisearch/charabia#303 and in #4945

Fixes 🐞

• ⚠️ When using federated search, query.facets was silently ignored at the query level, but should not have been. It now returns the appropriate error. Use federation.facetsByIndex instead if you want facets to be applied during federated search.
• Prometheus /metrics return the route pattern instead of the real route when returning the HTTP requests total by @irevoire in #4839
• Truncate values at the end of a list of facet values when the number of facet values is larger than maxValuesPerFacet. For example, setting maxValuesPerFacet to 2 could result in ["blue", "red", "yellow"], being truncated to ["blue", "yellow"] instead of ["blue", "red"]`. By @dureuill in #4929
• Swedish support: the characters å, ä, ö are no longer normalized to a and o. By @ManyTheFish in #4945

Misc

• Dependencies updates
  • Security dependency upgrade: bump quinn-proto from 0.11.3 to 0.11.8 by @dependabot in #4911
• CIs and tests
  • Make the tests run faster by @irevoire in #4808
• Documentation
  • Fix broken links in README by @iornstein in #4943
• Misc
  • Allow Meilitool to upgrade from v1.9 to v1.10 without a dump in some conditions, by @dureuill in #4912

❤️ Thanks again to our external contributors:

• Meilisearch: @iornstein.
• Charabia: @luflow

v1.10.1 🦩

Fixes 🦋

Better search handling under heavy loads

All of the next PR should make meilisearch behave better under heavy loads:

• Only spawn one search queue in actix-web by @irevoire in #4893
• Make sure the index scheduler never stops running by @irevoire in #4896
• Explicitly drop the search permits by @irevoire in #4898
• Stop trying to process searches after one minute by @irevoire in #4899

Speed improvement 🐎

We made the autobatching of the document deletion with the document deletion by filter possible which should uncklog the task queue of the people using these two operations heavily.
Meilisearch still cannot autobatch the document deletion by filter and the document addition, though.

• Autobatch document deletion by filter by @irevoire in #4901
• Do not fail the whole batch when a single document deletion by filter fails by @irevoire in #4905

Full Changelog: v1.10.0...v1.10.1

v1.9.1 🦎

Fixes 🪲

• Return an empty list of embeddings for embedders that have no document for an embedder. by @dureuill in #4889

This fixes an issue where dumps created for indexes with:

1. A user-provided embedder
2. At least one documents that opt-out of vectors for that user-provided embedder

would fail to import correctly.

Upgrade path to v1.10.0 🚀

If you are a Cloud user affected by the above issue, please contact customer support so we perform the upgrade for you.

If you are an OSS user affected by the above, perform the following operations:

1. Upgrade from v1.9.0 to v1.9.1 without using a dump
2. Upgrade to v1.10.0 using a dump created from v1.9.1

Full Changelog

v1.10.0 🦩

Meilisearch v1.10 introduces federated search. This innovative feature allows you to receive a single list of results for multi-search requests. v1.10 also includes a setting to manually define which language or languages are present in your documents, and two new new experimental features: the CONTAINS filter operator and the ability to update a subset of your dataset with a function.

🧰 All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment happens between 4 to 48 hours after a new version becomes available.

Some SDKs might not include all new features. Consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that ❤️).

New features and updates 🔥

Federated search

Use the new federation setting of the /multi-search route to return a single search result object:

curl \
  -X POST 'http://localhost:7700/multi-search' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "federation": {
      "offset": 5,
      "limit": 10
    }
    "queries": [
      {
        "q": "Batman",
        "indexUid": "movies"
      },
      {
        "q": "Batman",
        "indexUid": "comics"
      }
    ]
  }'

Response:

{
  "hits": [
    {
      "id": 42,
      "title": "Batman returns",
      "overview": "..",
      "_federation": {
        "indexUid": "movies",
        "queriesPosition": 0
      }
    },
    {
      "comicsId": "batman-killing-joke",
      "description": "..",
      "title": "Batman: the killing joke",
      "_federation": {
        "indexUid": "comics",
        "queriesPosition": 1
      }
    },
    …
 ],
  processingTimeMs: 0,
  limit: 20,
  offset: 0,
  estimatedTotalHits: 2,
  semanticHitCount: 0,
}

When performing a federated search, Meilisearch merges the results coming from different sources in descending ranking score order.

If federation is empty ({}), Meilisearch sets offset and limit to 0 and 20 respectively.

If federation is null or missing, multi-search returns one list of search result objects for each index.

Federated results relevancy

When performing federated searches, use federationOptions in the request's queries array to configure the relevancy and the weight of each index:

curl \
 -X POST 'http://localhost:7700/multi-search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
  "federation": {},
  "queries": [
    {
      "q": "apple red",
      "indexUid": "fruits",
      "filter": "BOOSTED = true",
      "_showRankingScore": true,
      "federationOptions": {
        "weight": 3.0
      }
    },
    {
      "q": "apple red",
      "indexUid": "fruits",
      "_showRankingScore": true,
    }
  ]
}'

federationOptions must be an object. It supports a single field, weight, which must be a positive floating-point number:

• if weight < 1.0, results from this index are less likely to appear in the results
• if weight > 1.0, results from this index are more likely to appear in the results
• if not specified, weight defaults to 1.0

📖 Consult the usage page for more information about the merge algorithm.

Done by @dureuill in #4769.

Experimental: CONTAINS filter operator

Enable the containsFilter experimental feature to use the CONTAINS filter operator:

curl \
  -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "containsFilter": true
  }'

CONTAINS filters results containing partial matches to the specified string, similar to a SQL LIKE:

curl \
  -X POST http://localhost:7700/indexes/movies/search \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "q": "super hero",
    "filter": "synopsis CONTAINS spider"
  }'

🗣️ This is an experimental feature, and we need your help to improve it! Share your thoughts and feedback on this GitHub discussion.

Done by @irevoire in #4804.

Language settings

Use the new localizedAttributes index setting and the locales search parameter to explicitly set the languages used in document fields and the search query itself. This is particularly useful for <=v1.9 users who have to occasionally resort to alternative Meilisearch images due to language auto-detect issues in Swedish and Japanese datasets.

Done by @ManyTheFish in #4819.

Set language during indexing with localizedAttributes

Use the newly introduced localizedAttributes setting to explicitly declare which languages correspond to which document fields:

curl \
  -X PATCH 'http://localhost:7700/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "localizedAttributes": [
      {"locales": ["jpn"], "attributePatterns": ["*_ja"]},
      {"locales": ["eng"], "attributePatterns": ["*_en"]},
      {"locales": ["cmn"], "attributePatterns": ["*_zh"]},
      {"locales": ["fra", "ita"], "attributePatterns": ["latin.*"]},
      {"locales": [], "attributePatterns": ["*"]}
    ]
  }'

locales is a list of ISO-639-3 language codes to assign to a pattern. The currently supported languages are: epo, eng, rus, cmn, spa, por, ita, ben, fra, deu, ukr, kat, ara, hin, jpn, heb, yid, pol, amh, jav, kor, nob, dan, swe, fin, tur, nld, hun, ces, ell, bul, bel, mar, kan, ron, slv, hrv, srp, mkd, lit, lav, est, tam, vie, urd, tha, guj, uzb, pan, aze, ind, tel, pes, mal, ori, mya, nep, sin, khm, tuk, aka, zul, sna, afr, lat, slk, cat, tgl, hye.

attributePattern is a pattern that can start or end with a * to match one or several attributes.

If an attribute matches several rules, only the first rule in the list will be applied. If the locales list is empty, then Meilisearch is allowed to auto-detect any language in the matching attributes.

These rules are applied to the searchableAttributes, the filterableAttributes, and the sortableAttributes.

Set language at search time with locales

The /search route accepts a new parameter, locales. Use it to define the language used in the current query:

curl \
  -X POST http://localhost:7700/indexes/movies/search \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "q": "進撃の巨人",
    "locales": ["jpn"]
  }'

The locales parameter overrides eventual locales in the index settings.

Experimental: Edit documents with a Rhai function

Use a Rhai function to edit documents in your database directly from Meilisearch:

First, activate the experimental feature:

curl \
  -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "editDocumentsByFunction": true
  }'

Then query the /documents/edit route with the editing function:

curl http://localhost:7700/indexes/movies/documents/edit \
  -H 'content-type: application/json' \
  -d '{
   "function": "doc.title = `✨ ${doc.title.to_upper()} ✨`",
   "filter": "id > 3000"
  }'

/documents/edit accepts three parameters in its payload: function, filter, and context.

function must be a string with a Rhai function. filter must be a filter expression.. context must be an object with data you want to make available for the editing function.

📖 More information here.

🗣️ This is an experimental feature and we need your help to improve it! Share your thoughts and feedback on this GitHub discussion.

Done by @Kerollmops in #4626.

Experimental AI-powered search: quality of life improvements

For the purpose of future stabilization of the feature, we are applying changes and quality-of-life improvements.

Done by @dureuill in #4801, #4815, #4818, #4822.

⚠️ Breaking changes: Changing the parameters of the REST API

The old parameters of the REST API are too numerous and confusing.

Removed parameters: query , inputField, inputType, pathToEmbeddings and embeddingObject.
Replaced by:

• request : A JSON value that represents the request made by Meilisearch to the remote embedder. The text to embed must be replaced by the placeholder value “{{text}}”.
• response: A JSON value that represents a fragment of the response made by the remote embedder to Meilisearch. The embedding must be replaced by the placeholder value "{{embedding}}".

Before:

// v1.10 version ✅
{
  "source": "rest",
  "url": "https://localhost:10006",
  "request": {
    "model": "minillm",
    "prompt": "{{text}}"
  },
  "response": {
    "embedding": "{{embedding}}"
  }
}

// v1.9 version ❌
{
  "source": "rest",
  "url": "https://localhost:10006",
  "query": {
    "model": "minillm",
  },
  "inputField": ["prompt"],
  "inputType": "text",
  "embeddingObject": ["embedding"]
}

Caution

This is a breaking change to the configuration of REST embedders.
Importing a dump containing a REST embedder configur...

v1.10.0-rc.3 🦩

What's Changed

• also intersect the universe for searchOnAttributes by @dureuill in #4858
• Don't remove facet value when multiple original values map to the same normalized value by @dureuill in #4864
• Fix perf regression facet strings by @dureuill in #4845
• Infer locales from index settings by @ManyTheFish in #4881

v1.10.0-rc.2

⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

What's Changed

Bugs fixes 🐞

Use a fixed date format regardless of features

If you were compiling Meilisearch yourself, we found out that using one version compiled from the root of the repository and then one from the meilisearch directory could cause database corruption.

Fixed by @dureuill in #4850

Fix edit by function deletion of documents

When deleting documents from the new edit by function route, if you had more than one thread, some documents would randomly not be deleted.

Fixed by @dureuill in #4853

Maintainance 🐢

• Add OpenAI tests by @dureuill in #4846

Full Changelog: v1.10.0-rc.1...v1.10.0-rc.2

v1.10.0-rc.1 🦩

⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Bug fixes 🪲

• Add missing PUT and GET /indexes/{indexUid}/settings/localized-attributes routes by @dureuill in #4836

Other improvements

• Errors emitted at the main level of the Meilisearch binary are now logged with level ERROR by @dureuill in #4835

Full Changelog

v1.10.0-rc.0 🦩

Warning

Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

With Meilisearch v1.10 we keep innovating by introducing the really demanded federated search! You can now apply multi-search requests and get one single list of results 🎉. This version also includes a setting to define your index languages (even if multiple languages are in your documents!), and new experimental features like the CONTAINS operator and the ability to update a subset of your dataset by using a simple function.

New features and updates 🔥

Federated search

By using the POST /multi-search endpoint, you can now return a single search result object, whose list of hits is built by merging the hits coming from all the queries in descending ranking score order.

curl \
 -X POST 'http://localhost:7700/multi-search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
  "federation": {
    "offset": 5,
    "limit": 10,
 }
  "queries": [
 {
      "q": "Batman",
      "indexUid": "movies"
 },
 {
      "q": "Batman",
      "indexUid": "comics"
 }
 ]
}'

Response:

{
  "hits": [
    {
      "id": 42,
      "title": "Batman returns",
      "overview": "..",
      "_federation": {
        "indexUid": "movies",
        "queriesPosition": 0
      }
    },
    {
      "comicsId": "batman-killing-joke",
      "description": "..",
      "title": "Batman: the killing joke",
      "_federation": {
        "indexUid": "comics",
        "queriesPosition": 1
      }
    },
    ...
 ],
  processingTimeMs: 0,
  limit: 20,
  offset: 0,
  estimatedTotalHits: 2,
  semanticHitCount: 0,
}

If federation is empty ({}) default values of offset and limit are used, so respectively 0 and 20.
If federation is null or missing, a classic multi-search will be applied, so a list of search result objects for each index will be returned.

To customize the relevancy and the weight applied to each index in the search result, use the federationOptions parameter in your request:

curl \
 -X POST 'http://localhost:7700/multi-search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
  "federation": {},
  "queries": [
    {
      "q": "apple red",
      "indexUid": "fruits",
      "filter": "BOOSTED = true",
      "_showRankingScore": true,
      "federationOptions": {
        "weight": 3.0
      }
    },
    {
      "q": "apple red",
      "indexUid": "fruits",
      "_showRankingScore": true,
    }
  ]
}'

weight must be positive (>=0)

• if < 1.0, the hits from this query are less likely to appear in the results.
• if > 1.0, the hits from this query are more likely to appear in the results.
• if missing, the default value is applied (1.0)

📖 More information about the merge algorithm on the here.

Done by @dureuill in #4769.

Experimental: CONTAINS filter operator

Enabling the experimental feature will make a new CONTAINS operator available while filtering on strings.
This is similar to the SQL LIKE operator used with %.

Activate the experimental feature:

curl \
 -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json' \
 --data-binary '{
    "containsFilter": true
 }'

Use the newly introduced CONTAINS operator:

curl \
 -X POST http://localhost:7700/indexes/movies/search \
  -H 'Content-Type: application/json' \
 --data-binary '{
      "q": "super hero",
      "filter": "synopsis NOT CONTAINS spider"
 }'

🗣️ This is an experimental feature, and we need your help to improve it! Share your thoughts and feedback on this GitHub discussion.

Done by @irevoire in #4804.

Languages settings

You can now set up the language of your index in your settings and during the search. This will prevent users from using alternative Meilisearch images we were separately created until now (like for Swedish and Japanese)

Done by @ManyTheFish in #4819.

Index settings

Use the newly introduced localizedAttributes setting (here is an example of handling multi-language documents):

curl \
  -X PATCH 'http://localhost:7700/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "localizedAttributes": [
      {"locales": ["jpn"], "attributePatterns": ["*_ja"]},
      {"locales": ["eng"], "attributePatterns": ["*_en"]},
      {"locales": ["cmn"], "attributePatterns": ["*_zh"]},
      {"locales": ["fra", "ita"], "attributePatterns": ["latin.*"]},
      {"locales": [], "attributePatterns": ["*"]}
    ]
}'

locales is a list of language codes to assign to a pattern, the supported codes are: epo, eng, rus, cmn, spa, por, ita, ben, fra, deu, ukr, kat, ara, hin, jpn, heb, yid, pol, amh, jav, kor, nob, dan, swe, fin, tur, nld, hun, ces, ell, bul, bel, mar, kan, ron, slv, hrv, srp, mkd, lit, lav, est, tam, vie, urd, tha, guj, uzb, pan, aze, ind, tel, pes, mal, ori, mya, nep, sin, khm, tuk, aka, zul, sna, afr, lat, slk, cat, tgl, hye.

attributePattern is a pattern that can start or end with a * to match one or several attributes.

"locales": [], "attributePatterns": ["*"] means the is the default rule.

Notes:

• if an attribute matches several rules, only the first rule in the list will be applied
• if the locales list is empty, then Meilisearch is allowed to auto-detect any language in the matching attributes
• These rules are applied to the searchableAttributes, the filterableAttributes, and the sortableAttributes.

At search time

The search route accepts a new parameter, locales allowing the end-user to define the language used in the current query:

curl \
 -X POST http://localhost:7700/indexes/movies/search \
  -H 'Content-Type: application/json' \
 --data-binary '{
  "q": "進撃の巨人",
  "locales": ["jpn"]
}'

The locales parameter overrides eventual locales in the index settings.

Experimental: edit documents by using a function

You can edit documents by executing a Rhai function on all the documents of your database or a subset of them that you can select by a Meilisearch filter.

Activate the experimental feature:

curl \
 -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json' \
 --data-binary '{
    "editDocumentsByFunction": true
 }'

By indexing this movies dataset, you can run the following Rhai function on all of them. This function will uppercase the titles of the movies with an id > 3000 and add sparkles around it. Rhai templating syntax is applied here:

curl http://localhost:7700/indexes/movies/documents/edit \
 -H 'content-type: application/json' \
  -d '{
 "filter": "id > 3000",
 "function": "doc.title = `✨ ${doc.title.to_upper()} ✨`"
 }'

📖 More information here.

🗣️ This is an experimental feature and we need your help to improve it! Share your thoughts and feedback on this GitHub discussion.

Done by @Kerollmops in #4626.

Experimental AI-powered search: quality of life improvements

For the purpose of future stabilization of the feature, we are applying changes and quality-of-life improvements.

Done by @dureuill in #4801, #4815, #4818, #4822.

⚠️ Breaking changes - changing the parameters of the REST API

The old parameters of the REST API are too numerous and confusing.

Removed parameters: query , inputField, inputType, pathToEmbeddings and embeddingObject.
Replaced by

• request : A JSON value that represents the request made by Meilisearch to the remote embedder. The text to embed must be replaced by the placeholder value “{{text}}”.
• response: A JSON value that represents a fragment of the response made by the remote embedder to Meilisearch. The embedding must be replaced by the placeholder value "{{embedding}}".

Before:

// v1.9 (old) version
{
  "source": "rest",
  "url": "https://localhost:10006",
  "query": {
    "model": "minillm",
 },
  "inputField": ["prompt"],
  "inputType": "text",
  "embeddingObject": ["embedding"]
}

// v1.10 (new) version
{
  "source": "rest",
  "url": "https://localhost:10006",
  "request": {
    "model": "minillm",
    "prompt": "{{text}}"
 },
  "response": {
    "embedding": "{{embedding}}"
 }
}

Caution

This is a breaking change to the configuration of REST embedders.
Importing a dump containing a REST embedder configuration will fail in v1.10 with an error: "Error: unknown field query, expected one of source, model, revision, apiKey, dimensions, documentTemplate, url, request, response, distribution at line 1 column 752".

Upgrade procedure (Cloud):

1. Remove any embedder with source "rest"
2. Follow the usual steps described [here in the documentation](https://www.meilisearch.com/docs/lear...

v1.8.4 🪼

Improvements

• Generate vectors in dumps by @dureuill in #4796
  • this version provides an upgrade path to v1.9 without regenerating embeddings for autogenerating embedders.
  • to upgrade from v1.8.x to v1.9.0 without regenerating embeddings, please follow the following procedure:
    1. Upgrade from v1.8.x to v1.8.4 by simply restarting Meilisearch v1.8.4 on you v1.8.x DB. No dump is necessary at this step
    2. Create a dump.
    3. Import the created dump in a Meilisearch v1.9

Full Changelog