[DOCS] EQL/SQL: Document runtime_fields parameter (#71487)

jrodewig · web-flow · commit 07fade1d27bd · 2021-04-19T09:15:12.000-04:00
diff --git a/docs/reference/eql/eql-search-api.asciidoc b/docs/reference/eql/eql-search-api.asciidoc
@@ -294,6 +294,8 @@ command].
 NOTE: This parameter may change the set of returned hits. However, it does not
 change the sort order of hits in the response.
 
+include::{es-repo-dir}/search/search.asciidoc[tag=runtime-mappings-def]
+
 `size`::
 (Optional, integer or float)
 For <<eql-basic-syntax,basic queries>>, the maximum number of matching events to
diff --git a/docs/reference/eql/eql.asciidoc b/docs/reference/eql/eql.asciidoc
@@ -442,7 +442,7 @@ GET /my-data-stream/_eql/search?filter_path=-hits.events._source
     "process.*",                <1>
     {
       "field": "@timestamp",    <2>
-      "format": "epoch_millis"  
+      "format": "epoch_millis"
     }
   ]
 }
@@ -532,6 +532,75 @@ The values are returned as a flat list in the `fields` section of each hit:
 // TESTRESPONSE[s/"_id": "OQmfCaduce8zoHT93o4H"/"_id": $body.hits.events.0._id/]
 // TESTRESPONSE[s/"_id": "xLkCaj4EujzdNSxfYLbO"/"_id": $body.hits.events.1._id/]
 
+[discrete]
+[[eql-use-runtime-fields]]
+=== Use runtime fields
+
+Use the `runtime_mappings` parameter to extract and create <<runtime,runtime
+fields>> during a search. Use the `fields` parameter to include runtime fields
+in the response.
+
+The following search creates a `day_of_week` runtime field from the `@timestamp`
+and returns it in the response.
+
+[source,console]
+----
+GET /my-data-stream/_eql/search?filter_path=-hits.events._source
+{
+  "runtime_mappings": {
+    "day_of_week": {
+      "type": "keyword",
+      "script": "emit(doc['@timestamp'].value.dayOfWeekEnum.toString())"
+    }
+  },
+  "query": """
+    process where process.name == "regsvr32.exe"
+  """,
+  "fields": [
+    "@timestamp",
+    "day_of_week"
+  ]
+}
+----
+// TEST[setup:sec_logs]
+
+The API returns:
+
+[source,console-result]
+----
+{
+  "is_partial": false,
+  "is_running": false,
+  "took": 60,
+  "timed_out": false,
+  "hits": {
+    "total": {
+      "value": 2,
+      "relation": "eq"
+    },
+    "events": [
+      {
+        "_index": ".ds-my-data-stream-2099.12.07-000001",
+        "_id": "OQmfCaduce8zoHT93o4H",
+        "fields": {
+          "@timestamp": [
+            "2099-12-07T11:07:09.000Z"
+          ],
+          "day_of_week": [
+            "MONDAY"
+          ]
+        }
+      },
+      ...
+    ]
+  }
+}
+----
+// TESTRESPONSE[s/"took": 60/"took": $body.took/]
+// TESTRESPONSE[s/"_index": ".ds-my-data-stream-2099.12.07-000001"/"_index": $body.hits.events.0._index/]
+// TESTRESPONSE[s/"_id": "OQmfCaduce8zoHT93o4H"/"_id": $body.hits.events.0._id/]
+// TESTRESPONSE[s/\.\.\./$body.hits.events.1/]
+
 [discrete]
 [[specify-a-timestamp-or-event-category-field]]
 === Specify a timestamp or event category field
diff --git a/docs/reference/search/search.asciidoc b/docs/reference/search/search.asciidoc
@@ -428,36 +428,47 @@ Period of time used to extend the life of the PIT.
 <<query-dsl,Query DSL>>.
 
 [[search-api-body-runtime]]
+// tag::runtime-mappings-def[]
 `runtime_mappings`::
-(Optional, object)
-Defines a <<runtime-search-request,runtime field>> in the search request that
-exist only as part of the query. Fields defined in the search request take
-precedence over fields defined with the same name in the index mappings.
+(Optional, object of objects)
+Defines one or more <<runtime-search-request,runtime fields>> in the search
+request. These fields take precedence over mapped fields with the same name.
 +
 .Properties of `runtime_mappings` objects
 [%collapsible%open]
 ====
+
+`<field-name>`::
+(Required, object)
+Configuration for the runtime field. The key is the field name.
++
+.Properties of `<field-name>`
+[%collapsible%open]
+=====
+
 `type`::
 (Required, string)
-<<mapping-types,Data type>> of the object, which can be any of the following:
+<<mapping-types,Field type>>, which can be any of the following:
 +
 include::{es-ref-dir}/mapping/runtime.asciidoc[tag=runtime-data-types]
 
 `script`::
 (Optional, string)
-<<modules-scripting-using,Painless script>> that is executed at query time. The
+<<modules-scripting-using,Painless script>> executed at query time. The
 script has access to the entire context of a document, including the original
 `_source` and any mapped fields plus their values.
 +
-Your script must include `emit` to return calculated values. For
-example:
+This script must include `emit` to return calculated values. For example:
 +
 [source,js,indent=0]
 ----
 include::search.asciidoc[tag=runtime-script]
 ----
 // NOTCONSOLE
+
+=====
 ====
+// end::runtime-mappings-def[]
 
 ////
 [source,console]
diff --git a/docs/reference/sql/endpoints/rest.asciidoc b/docs/reference/sql/endpoints/rest.asciidoc
@@ -9,6 +9,7 @@
 * <<sql-rest-filtering>>
 * <<sql-rest-columnar>>
 * <<sql-rest-params>>
+* <<sql-runtime-fields>>
 * <<sql-rest-fields>>
 
 [[sql-rest-overview]]
@@ -522,6 +523,44 @@ POST /_sql?format=txt
 [IMPORTANT]
 The recommended way of passing values to a query is with question mark placeholders, to avoid any attempts of hacking or SQL injection.
 
+[[sql-runtime-fields]]
+=== Use runtime fields
+
+Use the `runtime_mappings` parameter to extract and create <<runtime,runtime
+fields>>, or columns, from existing ones during a search.
+
+The following search creates a `release_day_of_week` runtime field from
+`release_date` and returns it in the response.
+
+[source,console]
+----
+POST _sql?format=txt
+{
+  "runtime_mappings": {
+    "release_day_of_week": {
+      "type": "keyword",
+      "script": """
+        emit(doc['release_date'].value.dayOfWeekEnum.toString())
+      """
+    }
+  },
+  "query": """
+    SELECT * FROM library WHERE page_count > 300 AND author = 'Frank Herbert'
+  """
+}
+----
+// TEST[setup:library]
+
+The API returns:
+
+[source,txt]
+----
+    author     |     name      |  page_count   |      release_date      |release_day_of_week
+---------------+---------------+---------------+------------------------+-------------------
+Frank Herbert  |Dune           |604            |1965-06-01T00:00:00.000Z|TUESDAY
+----
+// TESTRESPONSE[non_json]
+
 [[sql-rest-fields]]
 === Supported REST parameters
 
@@ -530,7 +569,7 @@ the request time-outs or localization information (such as timezone).
 
 The table below lists the supported parameters:
 
-[cols="^m,^m,^5"]
+[cols="<m,<m,<5"]
 
 |===
 
@@ -579,6 +618,11 @@ More information available https://docs.oracle.com/javase/8/docs/api/java/time/Z
 |none
 |Optional list of parameters to replace question mark (`?`) placeholders inside the query.
 
+|runtime_mappings
+|none
+|Defines one or more <<runtime-search-request,runtime fields>> in the search
+request. These fields take precedence over mapped fields with the same name.
+
 |===
 
 Do note that most parameters (outside the timeout and `columnar` ones) make sense only during the initial query - any follow-up pagination request only requires the `cursor` parameter as explained in the <<sql-pagination, pagination>> chapter.
