Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 151 additions & 7 deletions docs/reference/cat/transforms.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,167 @@

Returns configuration and usage information about {transforms}.


[[cat-transforms-api-request]]
==== {api-request-title}

`GET /_cat/transforms/<transform_id>` +

`GET /_cat/transforms/_all` +

`GET /_cat/transforms/*` +

`GET /_cat/transforms`

[[cat-transforms-api-prereqs]]
==== {api-prereq-title}

* If the {es} {security-features} are enabled, you must have `monitor_transform`
cluster privileges to use this API. The built-in `transform_user` role has these
privileges. For more information, see <<security-privileges>> and
<<built-in-roles>>.

//[[cat-transforms-api-desc]]
//==== {api-description-title}

[[cat-transforms-api-path-params]]
==== {api-path-parms-title}

`<transform_id>`::
(Optional, string)
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id-wildcard]

[[cat-transforms-api-query-params]]
==== {api-query-parms-title}

`allow_no_match`::
(Optional, boolean)
include::{docdir}/rest-api/common-parms.asciidoc[tag=allow-no-match-transforms1]

include::{docdir}/rest-api/common-parms.asciidoc[tag=http-format]

`from`::
(Optional, integer)
include::{docdir}/rest-api/common-parms.asciidoc[tag=from-transforms]

include::{docdir}/rest-api/common-parms.asciidoc[tag=cat-h]
+
If you do not specify which columns to include, the API returns the default
columns. If you explicitly specify one or more columns, it returns only the
specified columns.
+
Valid columns are:

`changes_last_detection_time`, `cldt`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=checkpointing-changes-last-detected-at]

`checkpoint_duration_time_exp_avg`, `cdtea`, `checkpointTimeExpAvg`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-checkpoint-duration-ms]

`create_time`, `ct`, `createTime`:::
(Default)
The time the {transform} was created.

`description`, `d`:::
(Default)
The description of the {transform}.

`dest_index`, `di`, `destIndex`:::
(Default)
include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-index]

`document_total`, `dt`:::
The total number of documents.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

document_total

I don't see an equivalent stat in the "get transform stats" API. Is it based on documents_processed or documents_indexed?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to the PR it is documents_processed


`frequency`, `f`:::
(Default)
include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency]

`id`:::
(Default)
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id]

`index_failure`, `if`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-failures]

`index_time`, `itime`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-time-ms]

`index_total `, `it`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-total]

`indexed_documents_exp_avg`, `idea`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-indexed]

`invocation_total`, `itotal`:::
The total number of invocations.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

invocation_total

Is this related to continuous transforms? I couldn't see a matching item in the "get transform stats" API.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is called trigger_count in stats


`max_page_search_size`, `mpsz`:::
(Default)
The maximum page search size.

`page_total`, `pt`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=pages-processed]

`pipeline`, `p`:::
(Default)
The {transform} pipeline.

`processed_documents_exp_avg`, `pdea`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-processed]

`reason`, `r`:::
The reason.

`search_failure`, `sf`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=search-failures]

`search_time`, `stime`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=search-time-ms]

`search_total`, `st`:::
include::{docdir}/rest-api/common-parms.asciidoc[tag=search-total]

`source_index`, `si`, `sourceIndex`:::
(Default)
include::{docdir}/rest-api/common-parms.asciidoc[tag=source-index-transforms]

`state`, `s`:::
(Default)
include::{docdir}/rest-api/common-parms.asciidoc[tag=state-transform]

`transform_type`, `tt`:::
(Default)
Indicates whether it's a batch or continuous {transform}.

`version`, `v`:::
(Default)
The version of {es} that existed on the node when the {transform} was
created.

include::{docdir}/rest-api/common-parms.asciidoc[tag=help]

include::{docdir}/rest-api/common-parms.asciidoc[tag=cat-s]

`size`::
(Optional, integer)
include::{docdir}/rest-api/common-parms.asciidoc[tag=size-transforms]

include::{docdir}/rest-api/common-parms.asciidoc[tag=time]

//[[cat-transforms-api-query-params]]
//==== {api-query-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=cat-v]

[[cat-transforms-api-examples]]
==== {api-examples-title}

//[[cat-transforms-api-response-codes]]
//==== {api-response-codes-title}
[source,console]
--------------------------------------------------
GET /_cat/transforms?v
--------------------------------------------------
// TEST[skip:kibana sample data]

//[[cat-transforms-api-examples]]
//==== {api-examples-title}
[source,console-result]
----
id create_time version source_index dest_index pipeline description transform_type frequency max_page_search_size state
ecommerce-customer-transform 2020-03-18T23:39:29.525Z 8.0.0 kibana_sample_data_ecommerce ecommerce-customers batch 1m 500 STOPPED
----
// TESTRESPONSE[skip:kibana sample data]
5 changes: 0 additions & 5 deletions docs/reference/redirects.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -385,8 +385,3 @@ coming::[7.x]
=== Component template APIs

coming::[7.x]

[role="exclude",id="cat-transform"]
=== cat transform API

See <<cat-transforms>>.
62 changes: 62 additions & 0 deletions docs/reference/rest-api/common-parms.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,10 @@ tag::bytes[]
(Optional, <<byte-units,byte size units>>) Unit used to display byte values.
end::bytes[]

tag::checkpointing-changes-last-detected-at[]
The timestamp when changes were last detected in the source indices.
end::checkpointing-changes-last-detected-at[]

tag::cluster-health-status[]
(string)
Health status of the cluster, based on the state of its primary and replica
Expand Down Expand Up @@ -207,6 +211,20 @@ Wildcard expressions are not accepted.
--
end::expand-wildcards[]

tag::exponential-avg-checkpoint-duration-ms[]
Exponential moving average of the duration of the checkpoint, in milliseconds.
end::exponential-avg-checkpoint-duration-ms[]

tag::exponential-avg-documents-indexed[]
Exponential moving average of the number of new documents that have been
indexed.
end::exponential-avg-documents-indexed[]

tag::exponential-avg-documents-processed[]
Exponential moving average of the number of documents that have been
processed.
end::exponential-avg-documents-processed[]

tag::field_statistics[]
`field_statistics`::
(Optional, boolean) If `true`, the response includes the document count, sum of document frequencies,
Expand Down Expand Up @@ -380,6 +398,18 @@ tag::index[]
used to limit the request.
end::index[]

tag::index-failures[]
The number of indexing failures.
end::index-failures[]

tag::index-time-ms[]
The amount of time spent indexing, in milliseconds.
end::index-time-ms[]

tag::index-total[]
The number of indices created.
end::index-total[]

tag::bulk-index[]
`_index`::
(Optional, string)
Expand Down Expand Up @@ -574,6 +604,11 @@ tag::pipeline[]
(Optional, string) ID of the pipeline to use to preprocess incoming documents.
end::pipeline[]

tag::pages-processed[]
The number of search or bulk index operations processed. Documents are
processed in batches instead of individually.
end::pages-processed[]

tag::path-pipeline[]
`<pipeline>`::
(Optional, string) Comma-separated list or wildcard expression of pipeline IDs
Expand Down Expand Up @@ -706,13 +741,25 @@ tag::scroll_size[]
Defaults to 100.
end::scroll_size[]

tag::search-failures[]
The number of search failures.
end::search-failures[]

tag::search-time-ms[]
The amount of time spent searching, in milliseconds.
end::search-time-ms[]

tag::search_timeout[]
`timeout`::
(Optional, <<time-units, time units>>)
Explicit timeout for each search request.
Defaults to no timeout.
end::search_timeout[]

tag::search-total[]
The number of search operations on the source index for the {transform}.
end::search-total[]

tag::search_type[]
`search_type`::
(Optional, string) The type of the search operation. Available options:
Expand Down Expand Up @@ -802,6 +849,21 @@ A query clause that retrieves a subset of data from the source index. See
<<query-dsl>>.
end::source-query-transforms[]

tag::state-transform[]
The status of the {transform}, which can be one of the following values:
+
--
* `aborting`: The {transform} is aborting.
* `failed`: The {transform} failed. For more information about the failure,
check the reason field.
* `indexing`: The {transform} is actively processing data and creating new
documents.
* `started`: The {transform} is running but not actively indexing data.
* `stopped`: The {transform} is stopped.
* `stopping`: The {transform} is stopping.
--
end::state-transform[]

tag::stats[]
`stats`::
(Optional, string) Specific `tag` of the request for logging and statistical
Expand Down
50 changes: 24 additions & 26 deletions docs/reference/transform/apis/get-transform-stats.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,8 @@ informational; you cannot update their values.
`checkpointing`::
(object) Contains statistics about <<transform-checkpoints,checkpoints>>.
`checkpointing`.`changes_last_detected_at`:::
(date) The timestamp when changes were last detected in the source indices.
(date)
include::{docdir}/rest-api/common-parms.asciidoc[tag=checkpointing-changes-last-detected-at]
`checkpointing`.`last`:::
(object) Contains statistics about the last completed checkpoint.
`checkpointing`.`last`.`checkpoint`::::
Expand Down Expand Up @@ -128,18 +129,8 @@ started.
(string) The host and port where transport HTTP connections are accepted. For
example, `127.0.0.1:9300`.
`state`::
(string) The status of the {transform}, which can be one of the following values:
+
--
* `aborting`: The {transform} is aborting.
* `failed`: The {transform} failed. For more information about the failure,
check the reason field.
* `indexing`: The {transform} is actively processing data and creating new
documents.
* `started`: The {transform} is running but not actively indexing data.
* `stopped`: The {transform} is stopped.
* `stopping`: The {transform} is stopping.
--
(string)
include::{docdir}/rest-api/common-parms.asciidoc[tag=state-transform]

`stats`::
(object) An object that provides statistical information about the {transform}.
Expand All @@ -150,28 +141,35 @@ for the {transform}.
(long) The number of documents that have been processed from the source index of
the {transform}.
`stats`.`exponential_avg_checkpoint_duration_ms`:::
(double) Exponential moving average of the duration of the checkpoint, in milliseconds.
(double)
include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-checkpoint-duration-ms]
`stats`.`exponential_avg_documents_indexed`:::
(double) Exponential moving average of the number of new documents that have been
indexed.
(double)
include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-indexed]
`stats`.`exponential_avg_documents_processed`:::
(double) Exponential moving average of the number of documents that have been
processed.
(double)
include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-processed]
`stats`.`index_failures`:::
(long) The number of indexing failures.
(long)
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-failures]
`stats`.`index_time_in_ms`:::
(long) The amount of time spent indexing, in milliseconds.
(long)
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-time-ms]
`stats`.`index_total`:::
(long) The number of indices created.
(long)
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-total]
`stats`.`pages_processed`:::
(long) The number of search or bulk index operations processed. Documents are
processed in batches instead of individually.
(long)
include::{docdir}/rest-api/common-parms.asciidoc[tag=pages-processed]
`stats`.`search_failures`:::
(long) The number of search failures.
(long)
include::{docdir}/rest-api/common-parms.asciidoc[tag=search-failures]
`stats`.`search_time_in_ms`:::
(long) The amount of time spent searching, in milliseconds.
(long)
include::{docdir}/rest-api/common-parms.asciidoc[tag=search-time-ms]
`stats`.`search_total`:::
(long) The number of search operations on the source index for the {transform}.
(long)
include::{docdir}/rest-api/common-parms.asciidoc[tag=search-total]
`stats`.`trigger_count`:::
(long) The number of times the {transform} has been triggered by the scheduler.
For example, the scheduler triggers the {transform} indexer to check for updates
Expand Down