Skip to content

Commit 35d2ac0

Browse files
simon-dewsarahlwelton
simon-dew
and
sarahlwelton
committed
[DOC-12432] Document Usage of {"ctl": { "validate" : true }, } for Search Queries (#302)
* [DOC-12432] Clarifying search_after and search_before behaviour, along with sorting and general pagination tips. Adding documentation on the new validate object - in search-request-params and as examples in the simple-search topics. Adding query results to the example in the REST API topic, and expanding on what those results mean. Fixing error in field path in the example full-request. * Add 7.6.4 version labels for validate property --------- Co-authored-by: Sarah Welton <[email protected]>
1 parent 6616ff5 commit 35d2ac0

10 files changed

+1025
-20
lines changed

‎modules/search/examples/query-sample-results-unindexed.jsonc‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
{
2+
"status": {
3+
"total": 1,
4+
"failed": 0,
5+
"successful": 1
6+
},
7+
"request": {
8+
"query": {
9+
"location": [
10+
-2.235143,
11+
53.482358
12+
],
13+
"distance": "100mi",
14+
"field": "geo"
15+
},
16+
"size": 10,
17+
"from": 0,
18+
"highlight": {
19+
"style": null,
20+
"fields": null
21+
},
22+
"fields": [
23+
"content"
24+
],
25+
"facets": null,
26+
"explain": true,
27+
"sort": [
28+
"-_score"
29+
],
30+
"includeLocations": false,
31+
"search_after": null,
32+
"search_before": null,
33+
"knn": null,
34+
"knn_operator": ""
35+
},
36+
"hits": [],
37+
"total_hits": 0,
38+
"cost": 0,
39+
"max_score": 0,
40+
"took": 4150150,
41+
"facets": null
42+
}

‎modules/search/examples/query-sample-results-validate.jsonc‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
"error": "rest_index: Query, indexName: travel-sample.inventory.landmark-content-index, err: bleve: QueryBleve query validation failed against index, err: query_validate: field not indexed, name: geo, type: geopoint",
3+
"request": {
4+
"ctl": {
5+
"validate": true
6+
},
7+
"explain": true,
8+
"fields": [
9+
"content"
10+
],
11+
"from": 0,
12+
"highlight": {},
13+
"query": {
14+
"distance": "100mi",
15+
"field": "geo",
16+
"location": {
17+
"lat": 53.482358,
18+
"lon": -2.235143
19+
}
20+
},
21+
"size": 10
22+
},
23+
"status": "fail"
24+
}

‎modules/search/examples/query-sample-results.jsonc‎

Lines changed: 740 additions & 0 deletions
Large diffs are not rendered by default.

‎modules/search/examples/run-search-full-request.jsonc‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
"bool": false
1010
},
1111
{
12-
"field": "ratings.Cleanliness",
12+
"field": "reviews.ratings.Cleanliness",
1313
"min": 1,
1414
"max": 3,
1515
"inclusive_min": true,
@@ -71,8 +71,9 @@
7171
// end::vectors[]
7272
"level": "at_plus",
7373
"results": "complete"
74-
}
74+
},
7575
// end::consistency[]
76+
"validate": true
7677
},
7778
// end::ctl[]
7879
"size": 10,
@@ -147,9 +148,9 @@
147148
"by": "field",
148149
"field": "field2",
149150
"desc": false,
150-
"mode": "max",
151+
"mode": "default",
151152
"missing": "last",
152-
"type": "number"
153+
"type": "string"
153154
},
154155
"-_score",
155156
"-_id"
@@ -158,8 +159,8 @@
158159
// end::sort[]
159160
"includeLocations": false,
160161
"score": "none",
161-
"search_after": ["field1Value", "5", "10.033205341869529", "1234"],
162-
"search_before": ["field1Value", "5", "10.033205341869529", "1234"],
162+
"search_after": ["field1String", "field2String", "10.033205341869529", "hotel_1234"],
163+
"search_before": ["field1String", "field2String", "10.033205341869529", "hotel_1234"],
163164
"limit": 10,
164165
"offset": 0,
165166
"collections": ["collection1", "collection2"]

‎modules/search/examples/run-search-unindexed-field.sh‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
curl -XPOST -H "Content-Type: application/json" \
2+
-u ${CB_USERNAME}:${CB_PASSWORD} http://${CB_HOSTNAME}:8094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index/query \
3+
-d '{
4+
"explain": true,
5+
"fields": [
6+
"content"
7+
],
8+
"highlight": {},
9+
"query": {
10+
"location": {
11+
"lon": -2.235143,
12+
"lat": 53.482358
13+
},
14+
"distance": "100mi",
15+
"field": "geo"
16+
},
17+
"size": 10,
18+
"from": 0
19+
}'

‎modules/search/examples/run-search-validate-ui.jsonc‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"explain": true,
3+
"fields": [
4+
"content"
5+
],
6+
"highlight": {},
7+
"query": {
8+
"location": {
9+
"lon": -2.235143,
10+
"lat": 53.482358
11+
},
12+
"distance": "100mi",
13+
"field": "geo"
14+
},
15+
"ctl": {
16+
"validate": true
17+
},
18+
"size": 10,
19+
"from": 0
20+
}

‎modules/search/examples/run-search-validate.sh‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
curl -XPOST -H "Content-Type: application/json" \
2+
-u ${CB_USERNAME}:${CB_PASSWORD} http://${CB_HOSTNAME}:8094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index/query \
3+
-d '{
4+
"explain": true,
5+
"fields": [
6+
"content"
7+
],
8+
"highlight": {},
9+
"query": {
10+
"location": {
11+
"lon": -2.235143,
12+
"lat": 53.482358
13+
},
14+
"distance": "100mi",
15+
"field": "geo"
16+
},
17+
"ctl": {
18+
"validate": true
19+
},
20+
"size": 10,
21+
"from": 0
22+
}'

‎modules/search/pages/search-request-params.adoc‎

Lines changed: 53 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,14 @@ Set the total number of results to return for a single page of search results.
4949

5050
If you provide both the `size` and `limit` properties, the Search Service uses the `size` value.
5151

52+
The Search Service returns the `size` number of results:
53+
54+
* Starting at the offset in the `from` or `offset` property.
55+
* Starting from the key specified in the <<search_after,search_after property>>.
56+
* Starting backward from the key specified in the <<search_before,search_before property>>.
57+
58+
The `size` property is added by default to all Search requests, if not otherwise specified, with a value of `10`.
59+
5260
|from/offset |Integer |No a|
5361

5462
Set an offset value to change where pagination starts for search results, based on the Search query's <<sort,>>.
@@ -57,6 +65,8 @@ For example, if you set a `size` value of `5` and a `from` value of `10`, the Se
5765

5866
If you provide both the `from` and `offset` properties, the Search Service uses the `from` value.
5967

68+
The `from` property is added by default to all Search requests, if not otherwise specified, with a value of `0`.
69+
6070
|highlight |Object |No a|
6171

6272
Contains properties to control search result highlighting.
@@ -87,7 +97,7 @@ To create an explanation for a search result's score in search results, set `exp
8797

8898
To turn off explanations for search result scoring, set `explain` to `false`.
8999

90-
|sort |Array |No a|
100+
|[[sort_arr]]sort |Array |No a|
91101

92102
Contains an array of strings or JSON objects to set how to sort search results.
93103

@@ -114,31 +124,45 @@ To turn off document relevancy scoring in search results, set `score` to `none`.
114124

115125
To turn on document relevancy scoring in search results, remove the `score` property.
116126

117-
|search_after |Array |No a|
127+
|[[search_after]]search_after |Array |No a|
118128

119-
NOTE: If you use `search_after` in a search request, you can't use `search_before`. Both properties are included in the example code to show the correct syntax.
129+
NOTE: If you use `search_after` in a search request, you can't use `search_before`.
130+
Both properties are included in the example code to show the correct syntax.
120131

121132
Use `search_after` with `from/offset` and `sort` to control pagination in search results.
122133

123-
Give a value for each string or JSON object in the `sort` array to the `search_after` array.
124-
The Search Service starts search result pagination after the document with those values.
134+
Give a value for each string or JSON object in the <<sort_arr,sort array>> to the `search_after` array.
135+
You must provide the values in the same order that they appear in the <<sort_arr,sort array>>.
136+
Values in the `search_after` array must be strings.
137+
You cannot use `search_after` with numbers or other field data types - if your `sort` array includes fields with a `date` or `number` type, you cannot use `search_after`.
138+
Only result relevancy score values can be entered as strings in the array.
125139

126-
You must provide the values in the same order that they appear in the `sort` array.
140+
The Search Service starts search result pagination after the document with the values you provide in the array.
127141

128142
For example, if you had a set of 10 documents to sort based on `_id` values of 1-10, with `from` set to `2` and `search_after` set to `8`, documents 9-10 appear on the same page.
129143

130-
|search_before |Array |No a|
144+
To reduce the resource costs of deeper pagination on your Search queries, try to always include your document ID values as the final sort criteria in your <<sort_arr,sort array>>.
145+
Set the `search_after` property to include the values from the last result on your previous page of search results to effectively paginate.
131146

132-
NOTE: If you use `search_before` in a search request, you can't use `search_after`. Both properties are included in the example code to show the correct syntax.
147+
|[[search_before]]search_before |Array |No a|
148+
149+
NOTE: If you use `search_before` in a search request, you can't use `search_after`.
150+
Both properties are included in the example code to show the correct syntax.
133151

134152
Use `search_before` with `from/offset` and `sort` to control pagination in search results.
135153

136154
Give a value for each string or JSON object in the `sort` array to the `search_before` array.
137-
The Search Service starts search result pagination before the document with those values.
155+
You must provide the values in the same order that they appear in the `sort` array.
156+
Values in the `search_before` array must be strings.
157+
You cannot use `search_before` with numbers or other field data types - if your `sort` array includes fields with a `date` or `number` type, you cannot use `search_before`.
158+
Only result relevancy score values can be entered as strings in the array.
159+
160+
The Search Service starts search result pagination before the document with the values you provide in the array.
138161

139-
You must provide the values in the same order that they appear in the `sort` array.
162+
For example, if you had a set of 10 documents to sort based on `_id` values of 1-10, with `from` set to `2` and `search_before` set to `8`, documents 2-6 appear on the same page.
140163

141-
For example, if you had a set of 10 documents to sort based on `_id` values of 1-10, with `from` set to `2` and `search_before` set to `8`, documents 2-6 appear on the same page.
164+
To reduce the resource costs of deeper pagination on your Search queries, try to always include your document ID values as the final sort criteria in your <<sort_arr,sort array>>.
165+
Set the `search_before` property to include the values from the last result on your previous page of search results to effectively paginate.
142166

143167
|[[collections]]collections |Array |No |Contains an array of strings that specify the collections where you want to run the query.
144168

@@ -1889,7 +1913,9 @@ The Search Service uses a consistency vector to synchronize the last document wr
18891913
include::example$run-search-full-request.jsonc[tag=ctl]
18901914
----
18911915

1892-
The `ctl` object contains the following properties:
1916+
In Couchbase Server 7.6.4 and later, you can also use the `ctl` object with the `validate` property to add an extra check to your queries and get help troubleshooting when a query does not return results.
1917+
1918+
The `ctl` object can contain the following properties:
18931919

18941920
[cols="1,1,1,4"]
18951921
|====
@@ -1908,6 +1934,18 @@ An object that contains a `vectors` object and the `level` and `results` propert
19081934

19091935
For more information, see <<consistency,>>.
19101936

1937+
|[[validate]]validate |Boolean |No a|
1938+
1939+
[.status]#Couchbase Server 7.6.4#
1940+
1941+
Add the `validate` property with a value of `true` to add extra validation checks to your Search query.
1942+
1943+
For example, the Search Service can tell you through the Web Console or the REST API that a field in your Search query is not in your Search index:
1944+
1945+
----
1946+
err: query_validate: field not indexed, field: ratings.Cleanliness, type: number
1947+
----
1948+
19111949
|====
19121950

19131951
[#consistency]
@@ -2166,6 +2204,9 @@ The following `sort` object orders search results by the values in `field1`, the
21662204
include::example$run-search-full-request.jsonc[tag=sort]
21672205
----
21682206

2207+
This means that if 2 documents have the same value in `field1`, then the Search Service will sort them again based on their `field2` values.
2208+
If they have the same value in `field2`, then sorting will happen again based on each document's score, and then finally the documents' ID values.
2209+
21692210
The `sort` object can contain the following string values:
21702211

21712212
[cols="1,2"]

0 commit comments

Comments
 (0)