Skip to content

v1.2: Delete and get documents by filter #2421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Parent: v1.2
merged 19 commits into from
Jun 1, 2023
Merged

v1.2: Delete and get documents by filter #2421

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
1db7ebe
update get and post routes
maryamsulemani97 May 11, 2023
8a61092
delete documents by filter
maryamsulemani97 May 11, 2023
077d3d4
more content
maryamsulemani97 May 15, 2023
2c3cc32
update stuff
maryamsulemani97 May 15, 2023
a955600
add warning
maryamsulemani97 May 15, 2023
bdd91a0
Update reference/api/documents.mdx
maryamsulemani97 May 15, 2023
c7576e5
update Get documents with POST
maryamsulemani97 May 16, 2023
d0fb247
more updates
maryamsulemani97 May 16, 2023
959cc9f
Update learn/core_concepts/documents.mdx
maryamsulemani97 May 16, 2023
72088f4
update based on review
maryamsulemani97 May 17, 2023
7461ca0
add warning boxes
maryamsulemani97 May 17, 2023
1627e4c
rename delete_documents_1
maryamsulemani97 May 18, 2023
fa2d34d
Apply suggestions from code review
maryamsulemani97 May 18, 2023
0104c2c
move note
maryamsulemani97 May 18, 2023
62bbfe1
update based on review+update errors
maryamsulemani97 May 29, 2023
f8c5538
update based on review
maryamsulemani97 May 29, 2023
cc66141
update based on review
maryamsulemani97 May 30, 2023
932d7cf
add version
maryamsulemani97 May 30, 2023
23b0f42
remove versioning
maryamsulemani97 May 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion .code-samples.meilisearch.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ get_one_document_1: |-
-X GET 'http://localhost:7700/indexes/movies/documents/25684?fields=id,title,poster,release_date'
get_documents_1: |-
curl \
-X GET 'http://localhost:7700/indexes/movies/documents?limit=2'
-X GET 'http://localhost:7700/indexes/movies/documents?limit=2&filter=genres=action'
add_or_replace_documents_1: |-
curl \
-X POST 'http://localhost:7700/indexes/movies/documents' \
Expand Down Expand Up @@ -1065,3 +1065,21 @@ filtering_update_settings_1: |-
"genres",
"director"
]'
fetch_documents_1: |-
curl \
-X POST http://localhost:7700/indexes/books/documents/fetch \
-H 'Content-Type: application/json' \
--data-binary '{
"filter": "(rating > 3 AND (genres = Adventure OR genres = Fiction)) AND language = English",
"fields": ["title", "genres", "rating", "language"],
"limit": 3
}'
delete_documents_by_filter_1: |-
curl \
-X POST http://localhost:7700/indexes/movies/documents/delete \
-H 'Content-Type: application/json' \
--data-binary '{
"filter": "genres = action AND genres = adventure"
}'


4 changes: 4 additions & 0 deletions learn/core_concepts/documents.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ sidebarDepth: 3

A document is an object composed of one or more fields. Each field consists of an **attribute** and its associated **value**. Documents function as containers for organizing data, and are the basic building blocks of a Meilisearch database. To search for a document, you must first add it to an [index](/learn/core_concepts/indexes).

<Capsule intent="note">
Meilisearch orders documents depending on the order they were inserted into the database.
</Capsule>

## Structure

![Diagram illustration Meilisearch's document structure](https://raw.githubusercontent.com/meilisearch/documentation/main/assets/images/document_structure.svg)
Expand Down
2 changes: 1 addition & 1 deletion learn/inner_workings/known_limitations.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ user = 1 OR user = 2 […] OR user = 1500 OR user = 1501 […] OR user = 2000 OR

**Limitation:** By default, Meilisearch returns up to 1000 documents per search.

**Explanation:** Meilisearch limits the maximum amount of returned search results to protect your database from malicious scraping. You may change this by using the `maxTotalHits` property of the [pagination index settings](/reference/api/settings#pagination-object). `maxTotalHits` only applies to the [search route](/reference/api/search) and has no effect on the [get documents endpoint](/reference/api/documents#get-documents).
**Explanation:** Meilisearch limits the maximum amount of returned search results to protect your database from malicious scraping. You may change this by using the `maxTotalHits` property of the [pagination index settings](/reference/api/settings#pagination-object). `maxTotalHits` only applies to the [search route](/reference/api/search) and has no effect on the [get documents with POST](/reference/api/documents#get-documents-with-post) and [get documents with GET](/reference/api/documents#get-documents-with-get) endpoints.

## Large datasets and internal errors

Expand Down
211 changes: 170 additions & 41 deletions reference/api/documents.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,105 @@ The `/documents` route allows you to create, manage, and delete documents.

[Learn more about documents.](/learn/core_concepts/documents)

## Get documents
## Get documents with POST

<RouteHighlighter method="GET" route="/indexes/{index_uid}/documents"/>
<RouteHighlighter method="POST" route="/indexes/{index_uid}/documents/fetch"/>

Get a set of documents.

Using the query parameters `offset` and `limit`, you can browse through all your documents.
Using `offset` and `limit`, you can browse through all your documents. `filter` will not work without first explicitly adding attributes to the [`filterableAttributes` list](/reference/api/settingss#update-filterable-attributes). [Learn more about filters in our dedicated guide.](/learn/advanced/filtering#filter-basics)

### Path parameters

| Name | Type | Description |
| :---------------- | :----- | :--------------------------------------------------------------------- |
| **`index_uid`** * | String | [`uid`](/learn/core_concepts/indexes#index-uid) of the requested index |

### Body

| Name | Type | Default Value | Description |
| :----------- | :-------------------------------------- | :------------ | :-------------------------------------------------------------------- |
| **`offset`** | Integer | `0` | Number of documents to skip |
| **`limit`** | Integer | `20` | Number of documents to return |
| **`fields`** | Array of strings/`null` | `*` | Document attributes to show (case-sensitive, comma-separated) |
| **`filter`** | String/Array of array of strings/`null` | N/A | Refine results based on attributes in the `filterableAttributes` list |

<Capsule intent="note">
Documents are ordered by Meilisearch depending on the hash of their id.
Sending an empty payload (`--data-binary '{}'`) will return all documents in the index.
</Capsule>

### Response

| Name | Type | Description |
| :------------ | :------ | :------------------------------------- |
| **`results`** | Array | An array of documents |
| **`offset`** | Integer | Number of documents skipped |
| **`limit`** | Integer | Number of documents returned |
| **`total`** | Integer | Total number of documents in the index |

### Example

<CodeSamples id="fetch_documents_1" />

#### Response: `200 Ok`

```json
{
"results":[
{
"title":"The Travels of Ibn Battuta",
"genres":[
"Travel",
"Adventure"
],
"language":"English",
"rating":4.5
},
{
"title":"Pride and Prejudice",
"genres":[
"Classics",
"Fiction",
"Romance",
"Literature"
],
"language":"English",
"rating":4
},
…
],
"offset":0,
"limit":3,
"total":5
}
```

## Get documents with GET

<Capsule intent="danger">
We recommend using POST `/indexes/{index_uid}/documents/fetch` as this endpoint will be deprecated in the near future.
</Capsule>

<RouteHighlighter method="GET" route="/indexes/{index_uid}/documents"/>

Get a set of documents.

Using the query parameters `offset` and `limit`, you can browse through all your documents. `filter` will not work without first explicitly adding attributes to the [`filterableAttributes` list](/reference/api/settingss#update-filterable-attributes). [Learn more about filters in our dedicated guide.](/learn/advanced/filtering#filter-basics)

### Path parameters

| Name | Type | Description |
| :---------------- | :----- | :------------------------------------------------------------------------ |
| Name | Type | Description |
| :---------------- | :----- | :--------------------------------------------------------------------- |
| **`index_uid`** * | String | [`uid`](/learn/core_concepts/indexes#index-uid) of the requested index |

### Query parameters

| Query Parameter | Default Value | Description |
| :-------------- | :------------ | :------------------------------------------------------------ |
| **`offset`** | `0` | Number of documents to skip |
| **`limit`** | `20` | Number of documents to return |
| **`fields`** | `*` | Document attributes to show (case-sensitive, comma-separated) |
| Query Parameter | Default Value | Description |
| :-------------- | :------------ | :-------------------------------------------------------------------- |
| **`offset`** | `0` | Number of documents to skip |
| **`limit`** | `20` | Number of documents to return |
| **`fields`** | `*` | Document attributes to show (case-sensitive, comma-separated) |
| **`filter`** | N/A | Refine results based on attributes in the `filterableAttributes` list |

### Response

Expand All @@ -47,25 +121,34 @@ Documents are ordered by Meilisearch depending on the hash of their id.

```json
{
"results": [
"results":[
{
"id": 25684,
"release_date": "1993-01-01",
"poster": "https://image.tmdb.org/t/p/w1280/iuAQVI4mvjI83wnirpD8GVNRVuY.jpg",
"title": "American Ninja 5",
"overview": "When a scientists daughter is kidnapped, American Ninja, attempts to find her, but this time he teams up with a youngster he has trained in the ways of the ninja."
"id":364,
"title":"Batman Returns",
"overview":"While Batman deals with a deformed man calling himself the Penguin, an employee of a corrupt businessman transforms into the Catwoman.",
"genres":[
"Action",
"Fantasy"
],
"poster":"https://image.tmdb.org/t/p/w500/jKBjeXM7iBBV9UkUcOXx3m7FSHY.jpg",
"release_date":708912000
},
{
"id": 468219,
"title": "Dead in a Week (Or Your Money Back)",
"release_date": "2018-09-12",
"poster": "https://image.tmdb.org/t/p/w1280/f4ANVEuEaGy2oP5M0Y2P1dwxUNn.jpg",
"overview": "William has failed to kill himself so many times that he outsources his suicide to aging assassin Leslie. But with the contract signed and death assured within a week (or his money back), William suddenly discovers reasons to live... However Leslie is under pressure from his boss to make sure the contract is completed."
"id":13851,
"title":"Batman: Gotham Knight",
"overview":"A collection of key events mark Bruce Wayne's life as he journeys from beginner to Dark Knight.",
"genres":[
"Animation",
"Action",
"Adventure"
],
"poster":"https://image.tmdb.org/t/p/w500/f3xUrqo7yEiU0guk2Ua3Znqiw6S.jpg",
"release_date":1215475200
}
],
"offset": 0,
"limit": 2,
"total": 500134
"offset":0,
"limit":2,
"total":5403
}
```

Expand All @@ -77,8 +160,8 @@ Get one document using its unique id.

### Path parameters

| Name | Type | Description |
| :------------------ | :------------- | :--------------------------------------------------------------------------------------- |
| Name | Type | Description |
| :------------------ | :------------- | :------------------------------------------------------------------------------------ |
| **`index_uid`** * | String | [`uid`](/learn/core_concepts/indexes#index-uid) of the requested index |
| **`document_id`** * | String/Integer | [Document id](/learn/core_concepts/primary_key#document-id) of the requested document |

Expand Down Expand Up @@ -119,15 +202,15 @@ This endpoint accepts the following content types:

### Path parameters

| Name | Type | Description |
| :---------------- | :----- | :------------------------------------------------------------------------ |
| Name | Type | Description |
| :---------------- | :----- | :--------------------------------------------------------------------- |
| **`index_uid`** * | String | [`uid`](/learn/core_concepts/indexes#index-uid) of the requested index |

### Query parameters

| Query Parameter | Default Value | Description |
| :----------------- | :------------ | :-------------------------------------------------------------------------------------------------------------------------------------- |
| **`primaryKey`** | `null` | [Primary key](/learn/core_concepts/primary_key#primary-key-2) of the index |
| **`primaryKey`** | `null` | [Primary key](/learn/core_concepts/primary_key#primary-key-2) of the index |
| **`csvDelimiter`** | `","` | Configure the character separating CSV fields. Must be a string containing [one ASCII character](https://www.rfc-editor.org/rfc/rfc20). |

<Capsule intent="warning">
Expand Down Expand Up @@ -190,15 +273,15 @@ This endpoint accepts the following content types:

### Path parameters

| Name | Type | Description |
| :---------------- | :----- | :------------------------------------------------------------------------ |
| Name | Type | Description |
| :---------------- | :----- | :--------------------------------------------------------------------- |
| **`index_uid`** * | String | [`uid`](/learn/core_concepts/indexes#index-uid) of the requested index |

### Query parameters

| Query Parameter | Default Value | Description |
| :----------------- | :------------ | :-------------------------------------------------------------------------------------------------------------------------------------- |
| **`primaryKey`** | `null` | [Primary key](/learn/core_concepts/primary_key#primary-key-2) of the documents |
| **`primaryKey`** | `null` | [Primary key](/learn/core_concepts/primary_key#primary-key-2) of the documents |
| **`csvDelimiter`** | `","` | Configure the character separating CSV fields. Must be a string containing [one ASCII character](https://www.rfc-editor.org/rfc/rfc20). |

<Capsule intent="warning">
Expand Down Expand Up @@ -248,8 +331,8 @@ Delete all documents in the specified index.

### Path parameters

| Name | Type | Description |
| :---------------- | :----- | :------------------------------------------------------------------------ |
| Name | Type | Description |
| :---------------- | :----- | :--------------------------------------------------------------------- |
| **`index_uid`** * | String | [`uid`](/learn/core_concepts/indexes#index-uid) of the requested index |

### Example
Expand Down Expand Up @@ -278,8 +361,8 @@ Delete one document based on its unique id.

### Path parameters

| Name | Type | Description |
| :------------------ | :------------- | :--------------------------------------------------------------------------------------- |
| Name | Type | Description |
| :------------------ | :------------- | :------------------------------------------------------------------------------------ |
| **`index_uid`** * | String | [`uid`](/learn/core_concepts/indexes#index-uid) of the requested index |
| **`document_id`** * | String/Integer | [Document id](/learn/core_concepts/primary_key#document-id) of the requested document |

Expand All @@ -301,21 +384,67 @@ Delete one document based on its unique id.

You can use this `taskUid` to get more details on [the status of the task](/reference/api/tasks#get-one-task).

## Delete documents by filter

<RouteHighlighter method="POST" route="/indexes/{index_uid}/documents/delete"/>

Delete a selection of documents based on a filter.

### Path parameters

| Name | Type | Description |
| :---------------- | :----- | :--------------------------------------------------------------------- |
| **`index_uid`** * | String | [`uid`](/learn/core_concepts/indexes#index-uid) of the requested index |

### Body

A filter expression written as a string or array of array of strings for the documents to be deleted. `filter` will not work without first explicitly adding attributes to the [`filterableAttributes` list](/reference/api/settings#update-filterable-attributes). [Learn more about filters in our dedicated guide.](/learn/advanced/filtering#filter-basics)

```
"filter": "rating > 3.5"
```

<Capsule intent="warning">
Sending an empty payload (`--data-binary '{}'`) will return a `bad_request` error.
</Capsule>

### Example

<CodeSamples id="delete_documents_by_filter_1" />

#### Response: `202 Accepted`

```json
{
"taskUid": 1,
"indexUid": "movies",
"status": "enqueued",
"type": "documentDeletion",
"enqueuedAt": "2023-05-15T08:38:48.024551Z"
}
```

You can use this `taskUid` to get more details on [the status of the task](/reference/api/tasks#get-one-task).

## Delete documents by batch

<Capsule intent="danger">
We recommend using POST `/indexes/{index_uid}/documents/delete` as this endpoint will be deprecated in the near future.
</Capsule>

<RouteHighlighter method="POST" route="/indexes/{index_uid}/documents/delete-batch"/>

Delete a selection of documents based on an array of document id's.
Delete a selection of documents based on an array of document ids.

### Path parameters

| Name | Type | Description |
| :---------------- | :----- | :------------------------------------------------------------------------ |
| Name | Type | Description |
| :---------------- | :----- | :--------------------------------------------------------------------- |
| **`index_uid`** * | String | [`uid`](/learn/core_concepts/indexes#index-uid) of the requested index |

### Body

An array of numbers containing the unique id's of the documents to be deleted.
An array of numbers containing the unique ids of the documents to be deleted.

```json
[23488, 153738, 437035, 363869]
Expand Down
Loading