[DOC-12432] Document Usage of {"ctl": { "validate" : true }, } for Search Queries (#302)

* [DOC-12432] Clarifying search_after and search_before behaviour, along with sorting and general pagination tips.

Adding documentation on the new validate object - in search-request-params and as examples in the simple-search topics.

Adding query results to the example in the REST API topic, and expanding on what those results mean.

Fixing error in field path in the example full-request.

* Add 7.6.5 version labels for validate property

---------

Co-authored-by: Sarah Welton <[email protected]>

Author: simon-dew

modules/search/examples/run-search-full-request.jsonc

@@ -9,7 +9,7 @@
                     "bool": false
                 },
                 {
-                    "field": "ratings.Cleanliness",
+                    "field": "reviews.ratings.Cleanliness",
                     "min": 1,
                     "max": 3,
                     "inclusive_min": true,
@@ -71,8 +71,9 @@
             // end::vectors[]
             "level": "at_plus",
             "results": "complete"
-        }
+        },
         // end::consistency[]
+        "validate": true
     },
     // end::ctl[]
     "size": 10,
@@ -147,9 +148,9 @@
                 "by": "field",
                 "field": "field2",
                 "desc": false,
-                "mode": "max",
+                "mode": "default",
                 "missing": "last",
-                "type": "number"
+                "type": "string"
             },
             "-_score",
             "-_id"
@@ -158,8 +159,8 @@
     // end::sort[]
     "includeLocations": false,
     "score": "none",
-    "search_after": ["field1Value", "5", "10.033205341869529", "1234"],
-    "search_before": ["field1Value", "5", "10.033205341869529", "1234"],
+    "search_after": ["field1String", "field2String", "10.033205341869529", "hotel_1234"],
+    "search_before": ["field1String", "field2String", "10.033205341869529", "hotel_1234"],
     "limit": 10,
     "offset": 0,
     "collections": ["collection1", "collection2"]

modules/search/examples/run-search-validate-ui.jsonc

@@ -0,0 +1,20 @@
+{
+    "explain": true,
+    "fields": [
+      "content"
+    ],
+    "highlight": {},
+    "query": {
+        "location": {
+            "lon": -2.235143,
+            "lat": 53.482358
+        },
+            "distance": "100mi",
+            "field": "geo"
+    },
+    "ctl": {
+        "validate": true
+    },
+    "size": 10,
+    "from": 0
+}

modules/search/pages/search-request-params.adoc

@@ -48,8 +48,13 @@ Set the total number of results to return for a single page of search results.
 
 If you provide both the `size` and `limit` properties, the Search Service uses the `size` value.
 
-If you do not provide a `from`, `size`, or other pagination settings in your query, the Search Service defaults to a `size` value of `10` and a `from` value of `0`.
-This means the Search Service does not offset results, and returns the first `10` matches to your query.
+The Search Service returns the `size` number of results: 
+
+* Starting at the offset in the `from` or `offset` property.
+* Starting from the key specified in the <<search_after,search_after property>>.
+* Starting backward from the key specified in the <<search_before,search_before property>>.
+
+The `size` property is added by default to all Search requests, if not otherwise specified, with a value of `10`.
 
 |from/offset |Integer |No a|
 
@@ -59,8 +64,7 @@ For example, if you set a `size` value of `5` and a `from` value of `10`, the Se
 
 If you provide both the `from` and `offset` properties, the Search Service uses the `from` value.
 
-If you do not provide a `from`, `size`, or other pagination settings in your query, the Search Service defaults to a `size` value of `10` and a `from` value of `0`.
-This means the Search Service does not offset results, and returns the first `10` matches to your query.
+The `from` property is added by default to all Search requests, if not otherwise specified, with a value of `0`.
 
 |highlight |Object |No a|
 
@@ -92,7 +96,7 @@ To create an explanation for a search result's score in search results, set `exp
 
 To turn off explanations for search result scoring, set `explain` to `false`.
 
-|[[sort-array]]sort |Array |No a|
+|[[sort_arr]]sort |Array |No a|
 
 Contains an array of strings or JSON objects to set how to sort search results.
 By default, the Search Service sorts results based on score values, from highest to lowest.
@@ -122,39 +126,45 @@ To turn off document relevancy scoring in search results, set `score` to `none`.
 
 To turn on document relevancy scoring in search results, remove the `score` property.
 
-|search_after |Array |No a|
+|[[search_after]]search_after |Array |No a|
 
-NOTE: If you use `search_after` in a search request, you cannot use `search_before`. Both properties are included in the example code to show the correct syntax.
+NOTE: If you use `search_after` in a search request, you can't use `search_before`.
+Both properties are included in the example code to show the correct syntax.
 
 Use `search_after` with `from/offset` and `sort` to control pagination in search results. 
 
-For example, if you had a set of 10 documents to sort based on `_id` values of 1-10, with `from` set to `2` and `search_after` set to `8`, documents 9-10 appear on the same page. 
+Give a value for each string or JSON object in the <<sort_arr,sort array>> to the `search_after` array.
+You must provide the values in the same order that they appear in the <<sort_arr,sort array>>.
+Values in the `search_after` array must be strings.
+You cannot use `search_after` with numbers or other field data types - if your `sort` array includes fields with a `date` or `number` type, you cannot use `search_after`. 
+Only result relevancy score values can be entered as strings in the array. 
 
-You must give a value for each string or JSON object in the `sort` array to the `search_after` array.
-The Search Service starts search result pagination after the document with those values. 
+The Search Service starts search result pagination after the document with the values you provide in the array.
 
-You must provide the values in the same order that they appear in the `sort` array.
-Your `sort` array must force a total order on your search results. 
+For example, if you had a set of 10 documents to sort based on `_id` values of 1-10, with `from` set to `2` and `search_after` set to `8`, documents 9-10 appear on the same page. 
 
-Use `search_after` to make the memory requirements of deeper page searches more manageable, when compared to using only `from/offset`.
-`search_after` lets you start your search results from a specific result, rather than needing to process a number of search results to skip.
+To reduce the resource costs of deeper pagination on your Search queries, try to always include your document ID values as the final sort criteria in your <<sort_arr,sort array>>. 
+Set the `search_after` property to include the values from the last result on your previous page of search results to effectively paginate. 
 
-|search_before |Array |No a|
+|[[search_before]]search_before |Array |No a|
 
-NOTE: If you use `search_before` in a search request, you cannot use `search_after`. Both properties are included in the example code to show the correct syntax.
+NOTE: If you use `search_before` in a search request, you can't use `search_after`.
+Both properties are included in the example code to show the correct syntax.
 
 Use `search_before` with `from/offset` and `sort` to control pagination in search results.
 
-For example, if you had a set of 10 documents to sort based on `_id` values of 1-10, with `from` set to `2` and `search_before` set to `8`, documents 2-6 appear on the same page. 
+Give a value for each string or JSON object in the `sort` array to the `search_before` array.
+You must provide the values in the same order that they appear in the `sort` array.
+Values in the `search_before` array must be strings.
+You cannot use `search_before` with numbers or other field data types - if your `sort` array includes fields with a `date` or `number` type, you cannot use `search_before`. 
+Only result relevancy score values can be entered as strings in the array.
 
-You must give a value for each string or JSON object in the `sort` array to the `search_before` array.
-The Search Service starts search result pagination before the document with those values. 
+The Search Service starts search result pagination before the document with the values you provide in the array.   
 
-You must provide the values in the same order that they appear in the `sort` array.
-Your `sort` array must force a total order on your search results.
+For example, if you had a set of 10 documents to sort based on `_id` values of 1-10, with `from` set to `2` and `search_before` set to `8`, documents 2-6 appear on the same page.
 
-Use `search_before` to make the memory requirements of deeper page searches more manageable, when compared to using only `from/offset`.
-`search_before` lets you start your search results back from a specific result, rather than needing to process a number of search results to skip.
+To reduce the resource costs of deeper pagination on your Search queries, try to always include your document ID values as the final sort criteria in your <<sort_arr,sort array>>. 
+Set the `search_before` property to include the values from the last result on your previous page of search results to effectively paginate. 
 
 |[[collections]]collections |Array |No |Contains an array of strings that specify the collections where you want to run the query. 
 
@@ -1904,7 +1914,9 @@ The Search Service uses a consistency vector to synchronize the last document wr
 include::example$run-search-full-request.jsonc[tag=ctl]
 ----
 
-The `ctl` object contains the following properties:
+In Couchbase Server 7.6.5 and later, you can also use the `ctl` object with the `validate` property to add an extra check to your queries and get help troubleshooting when a query does not return results. 
+
+The `ctl` object can contain the following properties:
 
 [cols="1,1,1,4"]
 |====
@@ -1923,6 +1935,18 @@ An object that contains a `vectors` object and the `level` and `results` propert
 
 For more information, see <<consistency,>>.
 
+|[[validate]]validate |Boolean |No a|
+
+[.status]#Couchbase Server 7.6.5#
+
+Add the `validate` property with a value of `true` to add extra validation checks to your Search query.
+
+For example, the Search Service can tell you through the Web Console or the REST API that a field in your Search query is not in your Search index: 
+
+----
+err: query_validate: field not indexed, field: ratings.Cleanliness, type: number
+----
+
 |====
 
 [#consistency]
@@ -2181,7 +2205,8 @@ The following `sort` object orders search results by the values in `field1`, the
 include::example$run-search-full-request.jsonc[tag=sort]
 ----
 
-TIP: For the best results with sorting and page navigation in search results, always include your document ID values (`_id` or `-_id`) as the final sort criteria in your `sort` object.
+This means that if 2 documents have the same value in `field1`, then the Search Service will sort them again based on their `field2` values.
+If they have the same value in `field2`, then sorting will happen again based on each document's score, and then finally the documents' ID values. 
 
 The `sort` object can contain the following string values: 
 

modules/search/pages/simple-search-ui.adoc

@@ -31,7 +31,7 @@ To run a simple search with the {page-ui-name}:
 . Press kbd:[Enter].
 . (Optional) To view a document and its source collection, click a document name in the search results list. 
 
-=== Example
+=== Example: Simple Text Search
 
 For example, the following query searches for the strings `view`, `food`, and `beach`:
 
@@ -45,6 +45,24 @@ It also returns all available fields in the index, and returns 10 results per pa
 
 TIP: Use the xref:search-request-params.adoc#collections[`collections` parameter] in your request to specify an array of collections to search from the Search index. 
 
+=== Example: Validate a Search Query
+
+[.status]#Couchbase Server 7.6.5#
+
+For example, the following query searches a Search index, `landmark-content-index`, using a xref:search-request-params.adoc#geopoint-queries-distance[Distance/Radius-Based Geopoint Query] on the `geo` field.
+The query includes the xref:search-request-params.adoc#ctl[ctl object] with the `validate` property to validate the query:
+
+[source,json]
+----
+include::example$run-search-validate-ui.jsonc[]
+----
+
+Since the `landmark-content-index` does not include a mapping for the `geo` field and the `validate` property is included in the query, the Web Console returns the following error: 
+
+----
+query_validate: field not indexed, name: geo, type: geopoint
+----
+
 == Next Steps 
 
 If you do not get the search results you were expecting, you can change the xref:search-request-params.adoc[JSON payload for your Search query].