diff --git a/_api-reference/alias.md b/_api-reference/alias.md index 8c0ff8bcaf6..349f4ac6359 100644 --- a/_api-reference/alias.md +++ b/_api-reference/alias.md @@ -29,7 +29,6 @@ POST _aliases "alias": "old-index-alias" } } - ] } ``` diff --git a/_api-reference/analyze-apis/perform-text-analysis.md b/_api-reference/analyze-apis/perform-text-analysis.md index 6ce733bfdaa..25b81a878e4 100644 --- a/_api-reference/analyze-apis/perform-text-analysis.md +++ b/_api-reference/analyze-apis/perform-text-analysis.md @@ -85,6 +85,7 @@ GET /_analyze "text" : ["first array element", "second array element"] } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -150,6 +151,7 @@ GET /_analyze "text" : "OpenSearch text analysis" } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -196,6 +198,7 @@ GET /books2/_analyze "text" : "Le garçon m'a SUIVI." } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -248,6 +251,7 @@ GET /_analyze "text" : "OpenSearch filter" } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -277,6 +281,7 @@ GET /_analyze "text" : "Leave right now!" } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -308,6 +313,7 @@ GET /_analyze "text" : "how to train your dog in five steps" } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -372,6 +378,7 @@ GET /books/_analyze "text" : "OpenSearch analyze test" } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -414,6 +421,7 @@ GET /books/_analyze "text" : "OpenSearch analyze test" } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -446,6 +454,7 @@ GET /books2/_analyze "text" : "OpenSearch analyze test" } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -492,6 +501,7 @@ GET /books2/_analyze "text" : "C'est le garçon qui m'a suivi." } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -522,6 +532,7 @@ GET /_analyze "text" : "That is the boy who followed me." } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -555,6 +566,7 @@ GET /_analyze "attributes" : ["keyword"] } ```` +{% include copy-curl.html %} The previous request returns the following fields: @@ -635,6 +647,8 @@ PUT /books2 } } ```` +{% include copy-curl.html %} + The preceding request is an index API rather than an analyze API. See [DYNAMIC INDEX SETTINGS]({{site.url}}{{site.baseurl}}/api-reference/index-apis/create-index/#dynamic-index-settings) for additional details. {: .note} diff --git a/_api-reference/cat/cat-aliases.md b/_api-reference/cat/cat-aliases.md index fb4f0e2222d..7118465dfb1 100644 --- a/_api-reference/cat/cat-aliases.md +++ b/_api-reference/cat/cat-aliases.md @@ -18,18 +18,21 @@ The CAT aliases operation lists the mapping of aliases to indices, plus routing ```json GET _cat/aliases?v ``` +{% include copy-curl.html %} To limit the information to a specific alias, add the alias name after your query: ```json GET _cat/aliases/?v ``` +{% include copy-curl.html %} If you want to get information for more than one alias, separate the alias names with commas: ```json GET _cat/aliases/alias1,alias2,alias3 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-allocation.md b/_api-reference/cat/cat-allocation.md index 021b99e0586..96a93950538 100644 --- a/_api-reference/cat/cat-allocation.md +++ b/_api-reference/cat/cat-allocation.md @@ -18,18 +18,21 @@ The CAT allocation operation lists the allocation of disk space for indices and ```json GET _cat/allocation?v ``` +{% include copy-curl.html %} To limit the information to a specific node, add the node name after your query: ```json GET _cat/allocation/ ``` +{% include copy-curl.html %} If you want to get information for more than one node, separate the node names with commas: ```json GET _cat/allocation/node_name_1,node_name_2,node_name_3 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-cluster_manager.md b/_api-reference/cat/cat-cluster_manager.md index 1538c703507..2508fce675b 100644 --- a/_api-reference/cat/cat-cluster_manager.md +++ b/_api-reference/cat/cat-cluster_manager.md @@ -18,6 +18,7 @@ The CAT cluster manager operation lists information that helps identify the elec ``` GET _cat/cluster_manager?v ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-count.md b/_api-reference/cat/cat-count.md index ab40540685d..e0c41c078b7 100644 --- a/_api-reference/cat/cat-count.md +++ b/_api-reference/cat/cat-count.md @@ -18,18 +18,21 @@ The CAT count operation lists the number of documents in your cluster. ```json GET _cat/count?v ``` +{% include copy-curl.html %} To see the number of documents in a specific index or alias, add the index or alias name after your query: ```json GET _cat/count/?v ``` +{% include copy-curl.html %} If you want to get information for more than one index or alias, separate the index or alias names with commas: ```json GET _cat/count/index_or_alias_1,index_or_alias_2,index_or_alias_3 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-field-data.md b/_api-reference/cat/cat-field-data.md index a2c4346225d..10c32939c1a 100644 --- a/_api-reference/cat/cat-field-data.md +++ b/_api-reference/cat/cat-field-data.md @@ -18,18 +18,21 @@ The CAT fielddata operation lists the memory size used by each field per node. ```json GET _cat/fielddata?v ``` +{% include copy-curl.html %} To limit the information to a specific field, add the field name after your query: ```json GET _cat/fielddata/?v ``` +{% include copy-curl.html %} If you want to get information for more than one field, separate the field names with commas: ```json GET _cat/fielddata/field_name_1,field_name_2,field_name_3 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-health.md b/_api-reference/cat/cat-health.md index 12613a48a5b..5c5fab0e5b7 100644 --- a/_api-reference/cat/cat-health.md +++ b/_api-reference/cat/cat-health.md @@ -18,12 +18,14 @@ The CAT health operation lists the status of the cluster, how long the cluster h ```json GET _cat/health?v ``` +{% include copy-curl.html %} ## Path and HTTP methods ``` GET _cat/health?v ``` +{% include copy-curl.html %} ## URL parameters diff --git a/_api-reference/cat/cat-indices.md b/_api-reference/cat/cat-indices.md index 75bc5f4d312..32c983bf07d 100644 --- a/_api-reference/cat/cat-indices.md +++ b/_api-reference/cat/cat-indices.md @@ -18,18 +18,21 @@ The CAT indices operation lists information related to indices⁠—how much dis ``` GET _cat/indices?v ``` +{% include copy-curl.html %} To limit the information to a specific index, add the index name after your query. ``` GET _cat/indices/?v ``` +{% include copy-curl.html %} If you want to get information for more than one index, separate the indices with commas: ```json GET _cat/indices/index1,index2,index3 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-nodeattrs.md b/_api-reference/cat/cat-nodeattrs.md index b7cfe0873cf..ebfb58cb42e 100644 --- a/_api-reference/cat/cat-nodeattrs.md +++ b/_api-reference/cat/cat-nodeattrs.md @@ -18,6 +18,7 @@ The CAT nodeattrs operation lists the attributes of custom nodes. ``` GET _cat/nodeattrs?v ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-nodes.md b/_api-reference/cat/cat-nodes.md index d05e6eda99c..ace8a287e2c 100644 --- a/_api-reference/cat/cat-nodes.md +++ b/_api-reference/cat/cat-nodes.md @@ -20,6 +20,7 @@ A few important node metrics are `pid`, `name`, `cluster_manager`, `ip`, `port`, ``` GET _cat/nodes?v ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-pending-tasks.md b/_api-reference/cat/cat-pending-tasks.md index 2657fab3916..fa9a5810174 100644 --- a/_api-reference/cat/cat-pending-tasks.md +++ b/_api-reference/cat/cat-pending-tasks.md @@ -18,6 +18,7 @@ The CAT pending tasks operation lists the progress of all pending tasks, includi ``` GET _cat/pending_tasks?v ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-plugins.md b/_api-reference/cat/cat-plugins.md index f6c29867497..2f53512775e 100644 --- a/_api-reference/cat/cat-plugins.md +++ b/_api-reference/cat/cat-plugins.md @@ -18,6 +18,7 @@ The CAT plugins operation lists the names, components, and versions of the insta ``` GET _cat/plugins?v ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-recovery.md b/_api-reference/cat/cat-recovery.md index 6dd20885e1f..c1df5e3a6d8 100644 --- a/_api-reference/cat/cat-recovery.md +++ b/_api-reference/cat/cat-recovery.md @@ -18,18 +18,21 @@ The CAT recovery operation lists all completed and ongoing index and shard recov ``` GET _cat/recovery?v ``` +{% include copy-curl.html %} To see only the recoveries of a specific index, add the index name after your query. ``` GET _cat/recovery/?v ``` +{% include copy-curl.html %} If you want to get information for more than one index, separate the indices with commas: ```json GET _cat/recovery/index1,index2,index3 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-repositories.md b/_api-reference/cat/cat-repositories.md index 34811b04244..997b993cdd8 100644 --- a/_api-reference/cat/cat-repositories.md +++ b/_api-reference/cat/cat-repositories.md @@ -18,6 +18,7 @@ The CAT repositories operation lists all completed and ongoing index and shard r ``` GET _cat/repositories?v ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-segments.md b/_api-reference/cat/cat-segments.md index 2014e466826..131a7a74a57 100644 --- a/_api-reference/cat/cat-segments.md +++ b/_api-reference/cat/cat-segments.md @@ -18,18 +18,21 @@ The cat segments operation lists Lucene segment-level information for each index ``` GET _cat/segments?v ``` +{% include copy-curl.html %} To see only the information about segments of a specific index, add the index name after your query. ``` GET _cat/segments/?v ``` +{% include copy-curl.html %} If you want to get information for more than one index, separate the indices with commas: ``` GET _cat/segments/index1,index2,index3 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-shards.md b/_api-reference/cat/cat-shards.md index c614a3fd894..d07127c46ea 100644 --- a/_api-reference/cat/cat-shards.md +++ b/_api-reference/cat/cat-shards.md @@ -18,18 +18,21 @@ The CAT shards operation lists the state of all primary and replica shards and h ``` GET _cat/shards?v ``` +{% include copy-curl.html %} To see only the information about shards of a specific index, add the index name after your query. ``` GET _cat/shards/?v ``` +{% include copy-curl.html %} If you want to get information for more than one index, separate the indices with commas: ``` GET _cat/shards/index1,index2,index3 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-snapshots.md b/_api-reference/cat/cat-snapshots.md index 6f59401be72..fcdd3c61ba4 100644 --- a/_api-reference/cat/cat-snapshots.md +++ b/_api-reference/cat/cat-snapshots.md @@ -18,6 +18,7 @@ The CAT snapshots operation lists all snapshots for a repository. ``` GET _cat/snapshots?v ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-tasks.md b/_api-reference/cat/cat-tasks.md index 4e51973da6e..d80fb11cc04 100644 --- a/_api-reference/cat/cat-tasks.md +++ b/_api-reference/cat/cat-tasks.md @@ -18,6 +18,7 @@ The CAT tasks operation lists the progress of all tasks currently running on you ``` GET _cat/tasks?v ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/cat-templates.md b/_api-reference/cat/cat-templates.md index acdb9a114d8..f361ae26f48 100644 --- a/_api-reference/cat/cat-templates.md +++ b/_api-reference/cat/cat-templates.md @@ -18,18 +18,21 @@ The CAT Templates operation lists the names, patterns, order numbers, and versio ``` GET _cat/templates?v ``` +{% include copy-curl.html %} If you want to get information for a specific template or pattern: ``` GET _cat/templates/ ``` +{% include copy-curl.html %} ## Path and HTTP methods ``` GET _cat/templates ``` +{% include copy-curl.html %} ## URL parameters diff --git a/_api-reference/cat/cat-thread-pool.md b/_api-reference/cat/cat-thread-pool.md index 5355098c03b..9a48b0d017b 100644 --- a/_api-reference/cat/cat-thread-pool.md +++ b/_api-reference/cat/cat-thread-pool.md @@ -17,18 +17,21 @@ The CAT thread pool operation lists the active, queued, and rejected threads of ``` GET _cat/thread_pool?v ``` +{% include copy-curl.html %} If you want to get information for more than one thread pool, separate the thread pool names with commas: ``` GET _cat/thread_pool/thread_pool_name_1,thread_pool_name_2,thread_pool_name_3 ``` +{% include copy-curl.html %} If you want to limit the information to a specific thread pool, add the thread pool name after your query: ``` GET _cat/thread_pool/?v ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cat/index.md b/_api-reference/cat/index.md index 6a52863a697..6064dead8e7 100644 --- a/_api-reference/cat/index.md +++ b/_api-reference/cat/index.md @@ -21,6 +21,7 @@ To see the available operations in the CAT API, use the following command: ``` GET _cat ``` +{% include copy-curl.html %} ## Optional query parameters @@ -45,6 +46,7 @@ To query aliases and get verbose output that includes all column headings in the ```json GET _cat/aliases?v ``` +{% include copy-curl.html %} The response provides more details, such as names of each column in the response. diff --git a/_api-reference/cluster-allocation.md b/_api-reference/cluster-api/cluster-allocation.md similarity index 98% rename from _api-reference/cluster-allocation.md rename to _api-reference/cluster-api/cluster-allocation.md index b5052f628b1..31b55537564 100644 --- a/_api-reference/cluster-allocation.md +++ b/_api-reference/cluster-api/cluster-allocation.md @@ -1,7 +1,9 @@ --- layout: default title: Cluster allocation explain -nav_order: 15 +nav_order: 10 +parent: Cluster APIs +has_children: false --- # Cluster allocation explain @@ -23,7 +25,7 @@ GET _cluster/allocation/explain?include_yes_decisions=true "primary": true } ``` - +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cluster-awareness.md b/_api-reference/cluster-api/cluster-awareness.md similarity index 94% rename from _api-reference/cluster-awareness.md rename to _api-reference/cluster-api/cluster-awareness.md index 2197bfd2023..8e7073d965a 100644 --- a/_api-reference/cluster-awareness.md +++ b/_api-reference/cluster-api/cluster-awareness.md @@ -1,14 +1,16 @@ --- layout: default title: Cluster routing and awareness -nav_order: 16 +nav_order: 20 +parent: Cluster APIs +has_children: false --- # Cluster routing and awareness To control the distribution of search or HTTP traffic, you can use the weights per awareness attribute to control the distribution of search or HTTP traffic across zones. This is commonly used for zonal deployments, heterogeneous instances, and routing traffic away from zones during zonal failure. -## HTTP and path methods +## Path and HTTP methods ``` PUT /_cluster/routing/awareness//weights @@ -50,6 +52,7 @@ The following example request creates a round robin shard allocation for search ### Request +```json PUT /_cluster/routing/awareness/zone/weights { "weights": @@ -60,6 +63,8 @@ PUT /_cluster/routing/awareness/zone/weights } "_version" : 1 } +``` +{% include copy-curl.html %} ### Response @@ -76,9 +81,10 @@ The following example request gets weights for all zones. ### Request -``` +```json GET /_cluster/routing/awareness/zone/weights ``` +{% include copy-curl.html %} ### Response @@ -103,9 +109,10 @@ You can remove your weight ratio for each zone using the `DELETE` method. ### Request -``` +```json DELETE /_cluster/routing/awareness/zone/weights ``` +{% include copy-curl.html %} ### Response diff --git a/_api-reference/cluster-decommission.md b/_api-reference/cluster-api/cluster-decommission.md similarity index 93% rename from _api-reference/cluster-decommission.md rename to _api-reference/cluster-api/cluster-decommission.md index 6a9a4880834..6d0fcec3d80 100644 --- a/_api-reference/cluster-decommission.md +++ b/_api-reference/cluster-api/cluster-decommission.md @@ -1,7 +1,9 @@ --- layout: default title: Cluster decommission -nav_order: 20 +nav_order: 30 +parent: Cluster APIs +has_children: false --- # Cluster decommission @@ -35,15 +37,17 @@ You can use the following example requests to decommission and recommission a zo The following example request decommissions `zone-a`: -``` +```json PUT /_cluster/decommission/awareness// ``` +{% include copy-curl.html %} If you want to recommission a decommissioned zone, you can use the `DELETE` method: -``` +```json DELETE /_cluster/decommission/awareness ``` +{% include copy-curl.html %} ### Response @@ -60,10 +64,10 @@ The following example requests returns the decommission status of all zones. ### Request -``` +```json GET /_cluster/decommission/awareness/zone/_status ``` - +{% include copy-curl.html %} ### Response diff --git a/_api-reference/cluster-health.md b/_api-reference/cluster-api/cluster-health.md similarity index 98% rename from _api-reference/cluster-health.md rename to _api-reference/cluster-api/cluster-health.md index 37fcd59f9a4..a87b796dc63 100644 --- a/_api-reference/cluster-health.md +++ b/_api-reference/cluster-api/cluster-health.md @@ -1,7 +1,9 @@ --- layout: default title: Cluster health -nav_order: 17 +nav_order: 40 +parent: Cluster APIs +has_children: false --- # Cluster health @@ -19,6 +21,7 @@ This request waits 50 seconds for the cluster to reach the yellow status or bett ``` GET _cluster/health?wait_for_status=yellow&timeout=50s ``` +{% include copy-curl.html %} If the cluster health becomes yellow or green before 50 seconds elapse, it returns a response immediately. Otherwise it returns a response as soon as it exceeds the timeout. @@ -56,6 +59,7 @@ The following sample request retrieves cluster health for all indexes in the clu ```json GET _cluster/health ``` +{% include copy-curl.html %} #### Sample response @@ -112,6 +116,7 @@ To check cluster health by awareness attribute (for example, zone or rack), spec ```json GET _cluster/health?level=awareness_attributes ``` +{% include copy-curl.html %} The response contains cluster health metrics partitioned by awareness attribute: @@ -196,6 +201,7 @@ If you're interested in a particular awareness attribute, you can include the na ```json GET _cluster/health?level=awareness_attributes&awareness_attribute=zone ``` +{% include copy-curl.html %} In response to the preceding request, OpenSearch returns cluster health information only for the `zone` awareness attribute. diff --git a/_api-reference/cluster-settings.md b/_api-reference/cluster-api/cluster-settings.md similarity index 99% rename from _api-reference/cluster-settings.md rename to _api-reference/cluster-api/cluster-settings.md index 366c7422c3e..96375ab094b 100644 --- a/_api-reference/cluster-settings.md +++ b/_api-reference/cluster-api/cluster-settings.md @@ -1,7 +1,9 @@ --- layout: default title: Cluster settings -nav_order: 18 +nav_order: 50 +parent: Cluster APIs +has_children: false --- # Cluster settings @@ -33,6 +35,7 @@ timeout (PUT only) | Time unit | The amount of time to wait for a response from ```json GET _cluster/settings?include_defaults=true ``` +{% include copy-curl.html %} #### Sample response @@ -114,6 +117,7 @@ PUT _cluster/settings } } ``` +{% include copy-curl.html %} For more information about transient settings, persistent settings, and precedence, see [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/). diff --git a/_api-reference/cluster-stats.md b/_api-reference/cluster-api/cluster-stats.md similarity index 99% rename from _api-reference/cluster-stats.md rename to _api-reference/cluster-api/cluster-stats.md index b384b8f8927..8097c27cd6a 100644 --- a/_api-reference/cluster-stats.md +++ b/_api-reference/cluster-api/cluster-stats.md @@ -1,7 +1,9 @@ --- layout: default title: Cluster stats -nav_order: 18 +nav_order: 60 +parent: Cluster APIs +has_children: false --- # Cluster stats @@ -15,6 +17,7 @@ The cluster stats API operation returns statistics about your cluster. ```json GET _cluster/stats/nodes/_cluster_manager ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/cluster-api/index.md b/_api-reference/cluster-api/index.md new file mode 100644 index 00000000000..9ce04695f62 --- /dev/null +++ b/_api-reference/cluster-api/index.md @@ -0,0 +1,12 @@ +--- +layout: default +title: Cluster APIs +has_children: true +nav_order: 15 +redirect_from: + - /opensearch/api-reference/cluster-api/ +--- + +# Cluster APIs + +The cluster APIs allow you to manage your cluster. You can use them to check cluster health, modify settings, retrieve statistics, and more. diff --git a/_api-reference/count.md b/_api-reference/count.md index 34ecf978836..fd9f3113ca6 100644 --- a/_api-reference/count.md +++ b/_api-reference/count.md @@ -26,6 +26,7 @@ GET opensearch_dashboards_sample_data_logs/_count } } ``` +{% include copy-curl.html %} The following call to the search API produces equivalent results: @@ -41,12 +42,14 @@ GET opensearch_dashboards_sample_data_logs/_search "track_total_hits": true } ``` +{% include copy-curl.html %} To see the number of documents in an index: ```json GET opensearch_dashboards_sample_data_logs/_count ``` +{% include copy-curl.html %} To check for the number of documents in a [data stream]({{site.url}}{{site.baseurl}}/opensearch/data-streams/), replace the index name with the data stream name. @@ -55,6 +58,7 @@ To see the number of documents in your cluster: ```json GET _count ``` +{% include copy-curl.html %} Alternatively, you could use the [cat indices]({{site.url}}{{site.baseurl}}/api-reference/cat/cat-indices/) and [cat count]({{site.url}}{{site.baseurl}}/api-reference/cat/cat-count/) APIs to see the number of documents per index or data stream. {: .note } diff --git a/_api-reference/document-apis/bulk.md b/_api-reference/document-apis/bulk.md index d479461684c..e7bd87324f9 100644 --- a/_api-reference/document-apis/bulk.md +++ b/_api-reference/document-apis/bulk.md @@ -25,6 +25,7 @@ POST _bulk { "doc" : { "title": "World War Z" } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/document-apis/delete-by-query.md b/_api-reference/document-apis/delete-by-query.md index a33459b0580..7622beef943 100644 --- a/_api-reference/document-apis/delete-by-query.md +++ b/_api-reference/document-apis/delete-by-query.md @@ -23,6 +23,7 @@ POST sample-index1/_delete_by_query } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/document-apis/delete-document.md b/_api-reference/document-apis/delete-document.md index 2c20e7248bc..e98a651193f 100644 --- a/_api-reference/document-apis/delete-document.md +++ b/_api-reference/document-apis/delete-document.md @@ -16,6 +16,7 @@ If you no longer need a document in your index, you can use the delete document ``` DELETE /sample-index1/_doc/1 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/document-apis/get-documents.md b/_api-reference/document-apis/get-documents.md index 92a75692b91..6e541606dd4 100644 --- a/_api-reference/document-apis/get-documents.md +++ b/_api-reference/document-apis/get-documents.md @@ -16,6 +16,7 @@ After adding a JSON document to your index, you can use the get document API ope ```json GET sample-index1/_doc/1 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/document-apis/index-document.md b/_api-reference/document-apis/index-document.md index e681b74b376..b9eb53dbcc5 100644 --- a/_api-reference/document-apis/index-document.md +++ b/_api-reference/document-apis/index-document.md @@ -19,6 +19,7 @@ PUT sample-index/_doc/1 "Description": "To be or not to be, that is the question." } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/document-apis/multi-get.md b/_api-reference/document-apis/multi-get.md index 5ff3b9ff453..e2dd6e451a0 100644 --- a/_api-reference/document-apis/multi-get.md +++ b/_api-reference/document-apis/multi-get.md @@ -31,6 +31,7 @@ GET _mget ] } ``` +{% include copy-curl.html %} ## Example of specifying index in URL @@ -50,6 +51,7 @@ GET sample-index1/_mget ] } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/document-apis/reindex.md b/_api-reference/document-apis/reindex.md index a24c739ef02..93aef5dac59 100644 --- a/_api-reference/document-apis/reindex.md +++ b/_api-reference/document-apis/reindex.md @@ -24,6 +24,7 @@ POST /_reindex } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/document-apis/update-by-query.md b/_api-reference/document-apis/update-by-query.md index 51bb27e8a62..4af274b1523 100644 --- a/_api-reference/document-apis/update-by-query.md +++ b/_api-reference/document-apis/update-by-query.md @@ -30,6 +30,7 @@ POST test-index1/_update_by_query } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/document-apis/update-document.md b/_api-reference/document-apis/update-document.md index 7dcc47b7e06..e2694f7fe30 100644 --- a/_api-reference/document-apis/update-document.md +++ b/_api-reference/document-apis/update-document.md @@ -22,6 +22,7 @@ POST /sample-index1/_update/1 } } ``` +{% include copy-curl.html %} ## Script example @@ -33,6 +34,7 @@ POST /test-index1/_update/1 } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/explain.md b/_api-reference/explain.md index 4398970f948..2d79baefbe3 100644 --- a/_api-reference/explain.md +++ b/_api-reference/explain.md @@ -30,6 +30,7 @@ POST opensearch_dashboards_sample_data_ecommerce/_search?explain=true } } ``` +{% include copy-curl.html %} More often, you want the output for a single document. In that case, specify the document ID in the URL: @@ -43,6 +44,7 @@ POST opensearch_dashboards_sample_data_ecommerce/_explain/EVz1Q3sBgg5eWQP6RSte } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/index-apis/clear-index-cache.md b/_api-reference/index-apis/clear-index-cache.md index 42885831fa1..f44504a944a 100644 --- a/_api-reference/index-apis/clear-index-cache.md +++ b/_api-reference/index-apis/clear-index-cache.md @@ -46,6 +46,7 @@ The following request clears the fields cache only: ````json POST /my-index/_cache/clear?fielddata=true ```` +{% include copy-curl.html %}
@@ -54,6 +55,7 @@ The following request clears the query cache only: ````json POST /my-index/_cache/clear?query=true ```` +{% include copy-curl.html %}
@@ -62,6 +64,7 @@ The following request clears the request cache only: ````json POST /my-index/_cache/clear?request=true ```` +{% include copy-curl.html %} ##### Clear the cache for specific fields @@ -70,6 +73,7 @@ The following request clears the fields caches of `fielda` and `fieldb`: ````json POST /my-index/_cache/clear?fields=fielda,fieldb ```` +{% include copy-curl.html %} ##### Clear caches for specific data streams and indexes @@ -78,6 +82,7 @@ The following request clears the cache for two specific indexes: ````json POST /my-index,my-index2/_cache/clear ```` +{% include copy-curl.html %} ##### Clear caches for all data streams and indexes @@ -86,6 +91,7 @@ The following request clears the cache for all data streams and indexes: ````json POST /_cache/clear ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/index-apis/clone.md b/_api-reference/index-apis/clone.md index a9fa7cf8545..07391ce8f1e 100644 --- a/_api-reference/index-apis/clone.md +++ b/_api-reference/index-apis/clone.md @@ -25,6 +25,7 @@ PUT /sample-index1/_clone/cloned-index1 } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/index-apis/close-index.md b/_api-reference/index-apis/close-index.md index 7ed65e9e618..a914a7ea677 100644 --- a/_api-reference/index-apis/close-index.md +++ b/_api-reference/index-apis/close-index.md @@ -16,6 +16,7 @@ The close index API operation closes an index. Once an index is closed, you cann ```json POST /sample-index/_close ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/index-apis/dangling-index.md b/_api-reference/index-apis/dangling-index.md index a34c5bf0238..f654b108141 100644 --- a/_api-reference/index-apis/dangling-index.md +++ b/_api-reference/index-apis/dangling-index.md @@ -56,12 +56,14 @@ The following are sample requests and a sample response. ````bash GET /_dangling ```` +{% include copy-curl.html %} #### Sample import ````bash POST /_dangling/msdjernajxAT23RT-BupMB?accept_data_loss=true ```` +{% include copy-curl.html %} #### Sample delete diff --git a/_api-reference/index-apis/delete-index.md b/_api-reference/index-apis/delete-index.md index f66b415ae9b..e938b5ea7c2 100644 --- a/_api-reference/index-apis/delete-index.md +++ b/_api-reference/index-apis/delete-index.md @@ -16,6 +16,7 @@ If you no longer need an index, you can use the delete index API operation to de ```json DELETE /sample-index ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/index-apis/exists.md b/_api-reference/index-apis/exists.md index 2b2407b87be..45e68793bbb 100644 --- a/_api-reference/index-apis/exists.md +++ b/_api-reference/index-apis/exists.md @@ -16,6 +16,7 @@ The index exists API operation returns whether or not an index already exists. ```json HEAD /sample-index ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/index-apis/get-index.md b/_api-reference/index-apis/get-index.md index 3a45fd0a234..60b5976d18e 100644 --- a/_api-reference/index-apis/get-index.md +++ b/_api-reference/index-apis/get-index.md @@ -16,6 +16,7 @@ You can use the get index API operation to return information about an index. ```json GET /sample-index ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/index-apis/get-settings.md b/_api-reference/index-apis/get-settings.md index 4974ea50901..6fa558a7609 100644 --- a/_api-reference/index-apis/get-settings.md +++ b/_api-reference/index-apis/get-settings.md @@ -16,6 +16,7 @@ The get settings API operation returns all the settings in your index. ```json GET /sample-index1/_settings ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/index-apis/open-index.md b/_api-reference/index-apis/open-index.md index fa2c8e73bc1..9df21b597cf 100644 --- a/_api-reference/index-apis/open-index.md +++ b/_api-reference/index-apis/open-index.md @@ -16,6 +16,7 @@ The open index API operation opens a closed index, letting you add or search for ```json POST /sample-index/_open ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/index-apis/put-mapping.md b/_api-reference/index-apis/put-mapping.md index db06d32c758..7bf9b243e74 100644 --- a/_api-reference/index-apis/put-mapping.md +++ b/_api-reference/index-apis/put-mapping.md @@ -78,6 +78,7 @@ PUT /sample-index/_mapping } } ``` +{% include copy-curl.html %} #### Sample Response diff --git a/_api-reference/index-apis/shrink-index.md b/_api-reference/index-apis/shrink-index.md index 3a723914873..1df92699499 100644 --- a/_api-reference/index-apis/shrink-index.md +++ b/_api-reference/index-apis/shrink-index.md @@ -23,6 +23,7 @@ POST /my-old-index/_shrink/my-new-index } } ``` +{% include copy-curl.html %} ## Path and HTTP methods @@ -56,10 +57,21 @@ timeout | Time | How long to wait for the request to return a response. Default You can use the request body to configure some index settings for the target index. All fields are optional. Field | Type | Description -alias | Object | Sets an alias for the target index. Can have the fields `filter`, `index_routing`, `is_hidden`, `is_write_index`, `routing`, and `search_routing`. See [Index Aliases]({{site.url}}{{site.baseurl}}/api-reference/alias/#request-body). +:--- | :--- | :--- +alias | Object | Sets an alias for the target index. Can have the fields `filter`, `index_routing`, `is_hidden`, `is_write_index`, `routing`, or `search_routing`. See [Index Aliases]({{site.url}}{{site.baseurl}}/api-reference/alias/#request-body). settings | Object | Index settings you can apply to your target index. See [Index Settings]({{site.url}}{{site.baseurl}}/api-reference/index-apis/create-index/#index-settings). +[max_shard_size](#the-max_shard_size-parameter) | Bytes | Specifies the maximum size of a primary shard in the target index. Because `max_shard_size` conflicts with the `index.number_of_shards` setting, you cannot set both of them at the same time. + +### The `max_shard_size` parameter + +The `max_shard_size` parameter specifies the maximum size of a primary shard in the target index. OpenSearch uses `max_shard_size` and the total storage for all primary shards in the source index to calculate the number of primary shards and their size for the target index. + +The primary shard count of the target index is the smallest factor of the source index's primary shard count for which the shard size does not exceed `max_shard_size`. For example, if the source index has 8 primary shards, they occupy a total of 400 GB of storage, and the `max_shard_size` is equal to 150 GB, OpenSearch calculates the number of primary shards in the target index using the following algorithm: + +1. Calculate the minimum number of primary shards as 400/150, rounded to the nearest whole integer. The minimum number of primary shards is 3. +1. Calculate the number of primary shards as the smallest factor of 8 that is greater than 3. The number of primary shards is 4. + +The maximum number of primary shards for the target index is equal to the number of primary shards in the source index because the shrink operation is used to reduce the primary shard count. As an example, consider a source index with 5 primary shards that occupy a total of 600 GB of storage. If `max_shard_size` is 100 GB, the minimum number of primary shards is 600/100, which is 6. However, because the number of primary shards in the source index is smaller than 6, the number of primary shards in the target index is set to 5. -{% comment %} -The new parameter to be called max_shard_size. -max_primary_shard_size | Bytes | Sets the maximum size of a primary shard in the target index. For example, if this field is set to 100 GB, and the source index's primary shards total to 300 GB, then the target index has 3 primary shards of 100 GB each. -{% endcomment %} +The minimum number of primary shards for the target index is 1. +{: .note} \ No newline at end of file diff --git a/_api-reference/index-apis/split.md b/_api-reference/index-apis/split.md index 03c9ec41831..96032d0915e 100644 --- a/_api-reference/index-apis/split.md +++ b/_api-reference/index-apis/split.md @@ -25,6 +25,7 @@ PUT /sample-index1/_split/split-index1 } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/index-apis/update-settings.md b/_api-reference/index-apis/update-settings.md index 410e6bd92b8..fd507fd19d7 100644 --- a/_api-reference/index-apis/update-settings.md +++ b/_api-reference/index-apis/update-settings.md @@ -24,6 +24,7 @@ PUT /sample-index1/_settings } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/ingest-apis/create-update-ingest.md b/_api-reference/ingest-apis/create-update-ingest.md index 767c747411c..03e5d1f3fd7 100644 --- a/_api-reference/ingest-apis/create-update-ingest.md +++ b/_api-reference/ingest-apis/create-update-ingest.md @@ -25,6 +25,7 @@ PUT _ingest/pipeline/12345 ] } ``` +{% include copy-curl.html %} ## Path and HTTP methods ``` diff --git a/_api-reference/ingest-apis/delete-ingest.md b/_api-reference/ingest-apis/delete-ingest.md index 1dcae3c87d2..0f2b6e950c6 100644 --- a/_api-reference/ingest-apis/delete-ingest.md +++ b/_api-reference/ingest-apis/delete-ingest.md @@ -14,6 +14,7 @@ If you no longer want to use an ingest pipeline, use the delete ingest pipeline ``` DELETE _ingest/pipeline/12345 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/ingest-apis/get-ingest.md b/_api-reference/ingest-apis/get-ingest.md index b91838df4a6..c92d8e5101e 100644 --- a/_api-reference/ingest-apis/get-ingest.md +++ b/_api-reference/ingest-apis/get-ingest.md @@ -14,6 +14,7 @@ After you create a pipeline, use the get ingest pipeline API operation to return ``` GET _ingest/pipeline/12345 ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/ingest-apis/simulate-ingest.md b/_api-reference/ingest-apis/simulate-ingest.md index e37304aa48b..db22336aa5b 100644 --- a/_api-reference/ingest-apis/simulate-ingest.md +++ b/_api-reference/ingest-apis/simulate-ingest.md @@ -32,6 +32,7 @@ POST /_ingest/pipeline/35678/_simulate ] } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/multi-search.md b/_api-reference/multi-search.md index 1680fe73b0a..b895f0c5f0b 100644 --- a/_api-reference/multi-search.md +++ b/_api-reference/multi-search.md @@ -21,6 +21,7 @@ GET _msearch { "query": { "match_all": {} } } ``` +{% include copy-curl.html %} ## Path and HTTP methods diff --git a/_api-reference/nodes-apis/index.md b/_api-reference/nodes-apis/index.md index 2a93233c4de..c6574f96615 100644 --- a/_api-reference/nodes-apis/index.md +++ b/_api-reference/nodes-apis/index.md @@ -50,12 +50,14 @@ To get statistics from the elected cluster manager node only, use the following ```json GET /_nodes/_cluster_manager/stats ``` +{% include copy-curl.html %} To get statistics from nodes that are data-only nodes, use the following query: ```json GET /_nodes/data:true/stats ``` +{% include copy-curl.html %} ### Order of resolution mechanisms @@ -66,9 +68,11 @@ To get statistics from all the nodes except the cluster manager node, use the fo ```json GET /_nodes/_all,cluster_manager:false/stats ``` +{% include copy-curl.html %} However, if you switch the resolution mechanisms, the result will include all the cluster nodes, including the cluster manager node: ```json GET /_nodes/cluster_manager:false,_all/stats -``` \ No newline at end of file +``` +{% include copy-curl.html %} \ No newline at end of file diff --git a/_api-reference/nodes-apis/nodes-hot-threads.md b/_api-reference/nodes-apis/nodes-hot-threads.md index 98a12e3bee8..23019d800fd 100644 --- a/_api-reference/nodes-apis/nodes-hot-threads.md +++ b/_api-reference/nodes-apis/nodes-hot-threads.md @@ -14,6 +14,7 @@ The nodes hot threads endpoint provides information about busy JVM threads for s ```json GET /_nodes/hot_threads ``` +{% include copy-curl.html %} ## Path and HTTP methods @@ -48,6 +49,7 @@ timeout | Time | Sets the time limit for node response. Default value is `30s`. ```json GET /_nodes/hot_threads ``` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/nodes-apis/nodes-info.md b/_api-reference/nodes-apis/nodes-info.md index e2469f1336f..b04f06ec46d 100644 --- a/_api-reference/nodes-apis/nodes-info.md +++ b/_api-reference/nodes-apis/nodes-info.md @@ -23,12 +23,14 @@ To get information about all nodes in a cluster, use the following query: ```json GET /_nodes ``` +{% include copy-curl.html %} To get thread pool information about the cluster manager node only, use the following query: ```json GET /_nodes/master:true/thread_pool ``` +{% include copy-curl.html %} ## Path and HTTP methods @@ -82,6 +84,7 @@ The following query requests the `process` and `transport` metrics from the clus ```json GET /_nodes/cluster_manager:true/process,transport ``` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/nodes-apis/nodes-reload-secure.md b/_api-reference/nodes-apis/nodes-reload-secure.md index 24415ce3386..467df77060e 100644 --- a/_api-reference/nodes-apis/nodes-reload-secure.md +++ b/_api-reference/nodes-apis/nodes-reload-secure.md @@ -41,6 +41,7 @@ The following is an example API request: ``` POST _nodes/reload_secure_settings ``` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/nodes-apis/nodes-stats.md b/_api-reference/nodes-apis/nodes-stats.md index f582a67be2c..fbd0511bfb7 100644 --- a/_api-reference/nodes-apis/nodes-stats.md +++ b/_api-reference/nodes-apis/nodes-stats.md @@ -57,6 +57,7 @@ GET _nodes/stats/ GET _nodes/stats/_all GET _nodes/stats/indices ``` + The following index metrics are supported: - docs @@ -80,6 +81,7 @@ For example, the following query requests statistics for `docs` and `search`: ```json GET _nodes/stats/indices/docs,search ``` +{% include copy-curl.html %} ## Query parameters @@ -100,6 +102,7 @@ include_segment_file_sizes | Boolean | If segment statistics are requested, this ```json GET _nodes/stats/ ``` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/nodes-apis/nodes-usage.md b/_api-reference/nodes-apis/nodes-usage.md index 5cfa25b77b0..d9a1af24495 100644 --- a/_api-reference/nodes-apis/nodes-usage.md +++ b/_api-reference/nodes-apis/nodes-usage.md @@ -43,6 +43,7 @@ The following request returns usage details for all nodes: ``` GET _nodes/usage ``` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/rank-eval.md b/_api-reference/rank-eval.md index 5ed5ca6261c..f2c5d931b8b 100644 --- a/_api-reference/rank-eval.md +++ b/_api-reference/rank-eval.md @@ -72,6 +72,7 @@ GET shakespeare/_rank_eval ] } ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/reload-search-analyzer.md b/_api-reference/reload-search-analyzer.md index 291efe6712e..d8e217396f8 100644 --- a/_api-reference/reload-search-analyzer.md +++ b/_api-reference/reload-search-analyzer.md @@ -34,6 +34,7 @@ The following are an example request and response. ````json POST /shakespeare/_reload_search_analyzers ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/remote-info.md b/_api-reference/remote-info.md index 9275459d08e..9f2e702923a 100644 --- a/_api-reference/remote-info.md +++ b/_api-reference/remote-info.md @@ -18,7 +18,7 @@ The response is more comprehensive and useful than a call to `_cluster/settings` ``` GET _remote/info ``` - +{% include copy-curl.html %} ## Response diff --git a/_api-reference/script-apis/create-stored-script.md b/_api-reference/script-apis/create-stored-script.md index 1f73b09ed86..f50e6d504ce 100644 --- a/_api-reference/script-apis/create-stored-script.md +++ b/_api-reference/script-apis/create-stored-script.md @@ -75,6 +75,8 @@ PUT _scripts/my-first-script } } ```` +{% include copy.html %} + The example above uses the syntax of the Dev Tools console in OpenSearch Dashboards. You can also use a curl request. {: .note } @@ -89,6 +91,7 @@ curl -XPUT "http://opensearch:9200/_scripts/my-first-script" -H 'Content-Type: a } }' ```` +{% include copy.html %} See [Execute Painless stored script]({{site.url}}{{site.baseurl}}/api-reference/script-apis/exec-stored-script/) for information about running the script. diff --git a/_api-reference/script-apis/delete-script.md b/_api-reference/script-apis/delete-script.md index 2c075ac2706..be53a08182a 100644 --- a/_api-reference/script-apis/delete-script.md +++ b/_api-reference/script-apis/delete-script.md @@ -31,6 +31,7 @@ The following request deletes the `my-first-script` script: ````json DELETE _scripts/my-script ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/script-apis/exec-script.md b/_api-reference/script-apis/exec-script.md index 30aa8e47a08..073eae4bcca 100644 --- a/_api-reference/script-apis/exec-script.md +++ b/_api-reference/script-apis/exec-script.md @@ -40,6 +40,7 @@ GET /_scripts/painless/_execute } } ``` +{% include copy-curl.html %} #### Sample response @@ -94,6 +95,7 @@ PUT /testindex1 } } ``` +{% include copy-curl.html %} Run a script to determine if a student is eligible to graduate with honors: @@ -116,6 +118,7 @@ POST /_scripts/painless/_execute } } ``` +{% include copy-curl.html %} The response contains the result: @@ -151,6 +154,7 @@ PUT /testindex1 } } ``` +{% include copy-curl.html %} Run a script that converts a GPA on a 4.0 scale into a different scale that is provided as a parameter: @@ -172,6 +176,7 @@ POST /_scripts/painless/_execute } } ``` +{% include copy-curl.html %} The response contains the result: diff --git a/_api-reference/script-apis/exec-stored-script.md b/_api-reference/script-apis/exec-stored-script.md index cbef9baa993..9a640365a54 100644 --- a/_api-reference/script-apis/exec-stored-script.md +++ b/_api-reference/script-apis/exec-stored-script.md @@ -44,6 +44,7 @@ GET books/_search } } ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/script-apis/get-script-contexts.md b/_api-reference/script-apis/get-script-contexts.md index 679893a84a2..e39dadc0dff 100644 --- a/_api-reference/script-apis/get-script-contexts.md +++ b/_api-reference/script-apis/get-script-contexts.md @@ -14,6 +14,7 @@ Retrieves all contexts for stored scripts. ````json GET _script_context ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/script-apis/get-script-language.md b/_api-reference/script-apis/get-script-language.md index 2a94b225a8b..1405191ea28 100644 --- a/_api-reference/script-apis/get-script-language.md +++ b/_api-reference/script-apis/get-script-language.md @@ -14,6 +14,7 @@ The get script language API operation retrieves all supported script languages a ```json GET _script_language ``` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/script-apis/get-stored-script.md b/_api-reference/script-apis/get-stored-script.md index b781f79484f..35eba7b2e6a 100644 --- a/_api-reference/script-apis/get-stored-script.md +++ b/_api-reference/script-apis/get-stored-script.md @@ -28,6 +28,7 @@ The following retrieves the `my-first-script` stored script. ````json GET _scripts/my-first-script ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/scroll.md b/_api-reference/scroll.md index 96b4cc2ed7f..4f373627c52 100644 --- a/_api-reference/scroll.md +++ b/_api-reference/scroll.md @@ -25,6 +25,7 @@ GET shakespeare/_search?scroll=10m "size": 10000 } ``` +{% include copy-curl.html %} OpenSearch caches the results and returns a scroll ID to access them in batches: @@ -41,6 +42,7 @@ GET _search/scroll "scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAAUWdmpUZDhnRFBUcWFtV21nMmFwUGJEQQ==" } ``` +{% include copy-curl.html %} Using this scroll ID, you get results in batches of 10,000 as long as the search context is still open. Typically, the scroll ID does not change between requests, but it *can* change, so make sure to always use the latest scroll ID. If you don't send the next scroll request within the set search context, the `scroll` operation does not return any results. @@ -59,6 +61,7 @@ GET shakespeare/_search?scroll=10m } } ``` +{% include copy-curl.html %} With a single scroll ID, you get back 10 results. You can have up to 10 IDs. @@ -68,12 +71,14 @@ Close the search context when you’re done scrolling, because the `scroll` oper ```json DELETE _search/scroll/DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAAcWdmpUZDhnRFBUcWFtV21nMmFwUGJEQQ== ``` +{% include copy-curl.html %} To close all open scroll contexts: ```json DELETE _search/scroll/_all ``` +{% include copy-curl.html %} The `scroll` operation corresponds to a specific timestamp. It doesn't consider documents added after that timestamp as potential results. diff --git a/_api-reference/search.md b/_api-reference/search.md index aaff20b6675..4422afa50f0 100644 --- a/_api-reference/search.md +++ b/_api-reference/search.md @@ -22,6 +22,7 @@ GET /movies/_search } } ``` +{% include copy-curl.html %} ## Path and HTTP Methods diff --git a/_api-reference/snapshots/create-repository.md b/_api-reference/snapshots/create-repository.md index 912bdfb1714..22c96152e16 100644 --- a/_api-reference/snapshots/create-repository.md +++ b/_api-reference/snapshots/create-repository.md @@ -73,6 +73,7 @@ PUT _snapshot/my-opensearch-repo } } ``` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/snapshots/create-snapshot.md b/_api-reference/snapshots/create-snapshot.md index d4c720bb195..f0cbed4ac3b 100644 --- a/_api-reference/snapshots/create-snapshot.md +++ b/_api-reference/snapshots/create-snapshot.md @@ -53,6 +53,7 @@ The following request creates a snapshot called `my-first-snapshot` in an S3 rep ```json POST _snapshot/my-s3-repository/my-first-snapshot ``` +{% include copy-curl.html %} ##### Request with a body @@ -67,6 +68,7 @@ PUT _snapshot/my-s3-repository/2 "partial": false } ``` +{% include copy-curl.html %} #### Sample responses @@ -120,6 +122,7 @@ The snapshot definition is returned. } } ``` + #### Response fields | Field | Data type | Description | diff --git a/_api-reference/snapshots/delete-snapshot-repository.md b/_api-reference/snapshots/delete-snapshot-repository.md index 440e0311aae..49173bc6255 100644 --- a/_api-reference/snapshots/delete-snapshot-repository.md +++ b/_api-reference/snapshots/delete-snapshot-repository.md @@ -26,6 +26,7 @@ The following request deletes the `my-opensearch-repo` repository: ````json DELETE _snapshot/my-opensearch-repo ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/snapshots/delete-snapshot.md b/_api-reference/snapshots/delete-snapshot.md index 67b3627197d..a3f896e7c13 100644 --- a/_api-reference/snapshots/delete-snapshot.md +++ b/_api-reference/snapshots/delete-snapshot.md @@ -24,9 +24,12 @@ snapshot | String | Snapshot to delete. | #### Sample request -The following request deletes a snapshot called `my-first-snapshot` from the `my-opensearch-repo` repository. +The following request deletes a snapshot called `my-first-snapshot` from the `my-opensearch-repo` repository: -`DELETE _snapshot/my-opensearch-repo/my-first-snapshot` +```json +DELETE _snapshot/my-opensearch-repo/my-first-snapshot +``` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/snapshots/get-snapshot-repository.md b/_api-reference/snapshots/get-snapshot-repository.md index da30dbc4766..d365b93dfe1 100644 --- a/_api-reference/snapshots/get-snapshot-repository.md +++ b/_api-reference/snapshots/get-snapshot-repository.md @@ -34,6 +34,7 @@ The following request retrieves information for the `my-opensearch-repo` reposit ````json GET /_snapshot/my-opensearch-repo ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/snapshots/get-snapshot-status.md b/_api-reference/snapshots/get-snapshot-status.md index ba9e51345a9..a79c3c14d28 100644 --- a/_api-reference/snapshots/get-snapshot-status.md +++ b/_api-reference/snapshots/get-snapshot-status.md @@ -50,6 +50,7 @@ GET _snapshot/my-opensearch-repo/my-first-snapshot/_status "ignore_unavailable": true } ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/snapshots/get-snapshot.md b/_api-reference/snapshots/get-snapshot.md index 89bda2d2515..f65a43eb78a 100644 --- a/_api-reference/snapshots/get-snapshot.md +++ b/_api-reference/snapshots/get-snapshot.md @@ -30,6 +30,7 @@ The following request retrieves information for the `my-first-snapshot` located ````json GET _snapshot/my-opensearch-repo/my-first-snapshot ```` +{% include copy-curl.html %} #### Sample response diff --git a/_api-reference/tasks.md b/_api-reference/tasks.md index 029c2415b79..54dbe62a221 100644 --- a/_api-reference/tasks.md +++ b/_api-reference/tasks.md @@ -15,12 +15,14 @@ The following request returns information about all of your tasks: ``` GET _tasks ``` +{% include copy-curl.html %} By including a task ID, you can get information specific to a particular task. Note that a task ID consists of a node's identifying string and the task's numerical ID. For example, if your node's identifying string is `nodestring` and the task's numerical ID is `1234`, then your task ID is `nodestring:1234`. You can find this information by running the `tasks` operation: ``` GET _tasks/ ``` +{% include copy-curl.html %} Note that if a task finishes running, it won't be returned as part of your request. For an example of a task that takes a little longer to finish, you can run the [`_reindex`]({{site.url}}{{site.baseurl}}/opensearch/reindex-data) API operation on a larger document, and then run `tasks`. @@ -77,6 +79,7 @@ Note that if a task finishes running, it won't be returned as part of your reque } } ``` + You can also use the following parameters with your query. Parameter | Data type | Description | @@ -97,6 +100,7 @@ For example, this request returns tasks currently running on a node named `opens ``` GET /_tasks?nodes=opensearch-node1 ``` +{% include copy-curl.html %} **Sample Response** @@ -149,6 +153,7 @@ The following request will return detailed information about active search tasks ```bash curl -XGET "localhost:9200/_tasks?actions=*search&detailed ``` +{% include copy.html %} **Sample Response** @@ -203,6 +208,7 @@ After getting a list of tasks, you can cancel all cancelable tasks with the foll ``` POST _tasks/_cancel ``` +{% include copy-curl.html %} Note that not all tasks are cancelable. To see if a task is cancelable, refer to the `cancellable` field in the response to your `tasks` API request. @@ -211,12 +217,14 @@ You can also cancel a task by including a specific task ID. ``` POST _tasks//_cancel ``` +{% include copy-curl.html %} The `cancel` operation supports the same parameters as the `tasks` operation. The following example shows how to cancel all cancelable tasks on multiple nodes. ``` POST _tasks/_cancel?nodes=opensearch-node1,opensearch-node2 ``` +{% include copy-curl.html %} ## Attaching headers to tasks @@ -227,6 +235,7 @@ Usage: ```bash curl -i -H "X-Opaque-Id: 111111" "https://localhost:9200/_tasks" -u 'admin:admin' --insecure ``` +{% include copy.html %} The `_tasks` operation returns the following result. @@ -284,4 +293,5 @@ This operation supports the same parameters as the `tasks` operation. The follow ```bash curl -i -H "X-Opaque-Id: 123456" "https://localhost:9200/_tasks?nodes=opensearch-node1" -u 'admin:admin' --insecure -``` \ No newline at end of file +``` +{% include copy.html %} \ No newline at end of file diff --git a/_config.yml b/_config.yml index 10ba0964373..846b9efac77 100644 --- a/_config.yml +++ b/_config.yml @@ -40,6 +40,9 @@ collections: dashboards: permalink: /:collection/:path/ output: true + tuning-your-cluster: + permalink: /:collection/:path/ + output: true security: permalink: /:collection/:path/ output: true @@ -52,10 +55,7 @@ collections: im-plugin: permalink: /:collection/:path/ output: true - replication-plugin: - permalink: /:collection/:path/ - output: true - observability-plugin: + observing-your-data: permalink: /:collection/:path/ output: true ml-commons-plugin: @@ -64,13 +64,7 @@ collections: neural-search-plugin: permalink: /:collection/:path/ output: true - monitoring-plugins: - permalink: /:collection/:path/ - output: true - notifications-plugin: - permalink: /:collection/:path/ - output: true - job-scheduler-plugin: + monitoring-your-cluster: permalink: /:collection/:path/ output: true clients: @@ -109,6 +103,9 @@ just_the_docs: dashboards: name: OpenSearch Dashboards nav_fold: true + tuning-your-cluster: + name: Tuning your cluster + nav_fold: true security: name: Security in OpenSearch nav_fold: true @@ -121,11 +118,8 @@ just_the_docs: im-plugin: name: Index management plugin nav_fold: true - replication-plugin: - name: Replication plugin - nav_fold: true - observability-plugin: - name: Observability plugin + observing-your-data: + name: Observing your data nav_fold: true ml-commons-plugin: name: ML Commons plugin @@ -133,14 +127,8 @@ just_the_docs: neural-search-plugin: name: Neural Search plugin nav_fold: true - monitoring-plugins: - name: Monitoring plugins - nav_fold: true - notifications-plugin: - name: Notifications plugin - nav_fold: true - job-scheduler-plugin: - name: Job Scheduler plugin + monitoring-your-cluster: + name: Monitoring your cluster nav_fold: true clients: name: Clients diff --git a/_dashboards/discover/dql.md b/_dashboards/discover/dql.md new file mode 100644 index 00000000000..7185de16e79 --- /dev/null +++ b/_dashboards/discover/dql.md @@ -0,0 +1,164 @@ +--- +layout: default +title: Using Dashboards Query Language +parent: Exploring data with Discover +nav_order: 40 +redirect_from: + - /dashboards/dql/ +--- + +# Using Dashboards Query Language + +Dashboards Query Language (DQL) is a simple text-based query language for filtering data in OpenSearch Dashboards. Similar to [Query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/index), DQL uses an HTTP request body. For example, to display your site visitor data for a host in the United States, you would enter `geo.dest:US` in the search field, as shown in the following image. + +Search term using DQL toolbar in Dashboard + +Before you can search data in Dashboards, you must index it. In OpenSearch, the basic unit of data is a JSON document. Within an index, OpenSearch identifies each document using a unique ID. To learn more about indexing in OpenSearch, see [Index data]({{site.url}}{{site.baseurl}}/opensearch/index-data). +{: .note purple} + +## Searching with terms queries + +The most basic query specifies the search term, for example: + +``` +host:www.example.com +``` + +To access an object's nested field, list the complete path to the field separated by periods. For example, use the following path to retrieve the `lat` field in the `coordinates` object: + +``` +coordinates.lat:43.7102 +``` + +DQL supports leading and trailing wildcards, so you can search for any terms that match your pattern, for example: + +``` +host.keyword:*.example.com/* +``` + +To check whether a field exists or has any data, use a wildcard to see whether Dashboards returns any results,for example: + +``` +host.keyword:* +``` + +## Searching with Boolean queries + +To mix and match or combine multiple queries for more refined results, you can use the Boolean operators `and`, `or`, and `not`. DQL is not case sensitive, so `AND` and `and` are the same, for example: + +``` +host.keyword:www.example.com and response.keyword:200 +``` + +You also can use multiple Boolean operators in one query, for example: + +``` +geo.dest:US or response.keyword:200 and host.keyword:www.example.com +``` + +Remember that Boolean operators follow the logical precedence order of `not`, `and`, and `or`, so if you have an expression like the one in the preceding example, `response.keyword:200 and host.keyword:www.example.com` is evaluated first. + +To avoid confusion, use parentheses to dictate the order in which you want to evaluate operands. If you want to evaluate `geo.dest:US or response.keyword:200` first, you can use an expression like the following: + +``` +(geo.dest:US or response.keyword:200) and host.keyword:www.example.com +``` + +## Querying dates and ranges + +DQL supports numeric inequalities, for example, `bytes >= 15 and memory < 15`. + +You can use the same method to find a date before or after the date specified in the query. `>` indicates a search for a date after the specified date, and `<` returns dates before the specified date, for example, `@timestamp > "2020-12-14T09:35:33`. + +## Querying nested fields + +Searching a document with [nested fields]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/nested/) requires you to specify the full path of the field to be retrieved. In the following example document, the `superheroes` field has nested objects: + +```json +{ + "superheroes":[ + { + "hero-name": "Superman", + "real-identity": "Clark Kent", + "age": 28 + }, + { + "hero-name": "Batman", + "real-identity": "Bruce Wayne", + "age": 26 + }, + { + "hero-name": "Flash", + "real-identity": "Barry Allen", + "age": 28 + }, + { + "hero-name": "Robin", + "real-identity": "Dick Grayson", + "age": 15 + } + ] +} +``` +{% include copy.html %} + +To retrieve documents that match a specific field using DQL, specify the field, for example: + +``` +superheroes: {hero-name: Superman} +``` +{% include copy.html %} + +To retrieve documents that match multiple fields, specify all the fields, for example: + +``` +superheroes: {hero-name: Superman} and superheroes: {hero-name: Batman} +``` +{% include copy.html %} + +You can combine multiple Boolean and range queries to create a more refined query, for example: + +``` +superheroes: {hero-name: Superman and age < 50} +``` +{% include copy.html %} + +## Querying doubly nested objects + +If a document has doubly nested objects (objects nested inside other objects), retrieve a field value by specifying the full path to the field. In the following example document, the `superheroes` object is nested inside the `justice-league` object: + +```json +{ +"justice-league": [ +{ +"superheroes":[ +{ +"hero-name": "Superman", +"real-identity": "Clark Kent", +"age": 28 +}, +{ +"hero-name": "Batman", +"real-identity": "Bruce Wayne", +"age": 26 +}, +{ +"hero-name": "Flash", +"real-identity": "Barry Allen", +"age": 28 +}, +{ +"hero-name": "Robin", +"real-identity": "Dick Grayson", +"age": 15 +} +] +} +] +} +``` +{% include copy.html %} + +The following image shows the query result using the example notation `justice-league.superheroes: {hero-name:Superman}`. + +DQL query result diff --git a/_dashboards/discover/index-discover.md b/_dashboards/discover/index-discover.md new file mode 100644 index 00000000000..ebf46b8c234 --- /dev/null +++ b/_dashboards/discover/index-discover.md @@ -0,0 +1,104 @@ +--- +layout: default +title: Exploring data with Discover +nav_order: 20 +has_children: true +--- + +# Exploring data with Discover + +**Discover** in OpenSearch Dashboards helps you extract insights and get value out of data assets across your organization. Discover enables you to: + +1. **Explore data**. You can explore, customize, and filter data as well as search data using [Dashboards Query Language (DQL)]({{site.url}}{{site.baseurl}}/dashboards/dql/). +2. **Analyze data**. You can analyze data, view individual documents, and create tables summarizing data contents. +3. **Visualize data**. You can display findings from your saved searches in a single dashboard that combines different data visualization types. + +## Try it: Exploring sample data with Discover + +This tutorial shows you how to use Discover to analyze and understand a sample dataset. At the end of this tutorial, you should be ready to use Discover with your own data. + +Before starting this tutorial, make sure you've added the **Sample flight data**. See [Quickstart guide for OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/quickstart/) for information about how to get started. +{: .warning} + +### Setting up data + +Watch the following short video or start with the tutorial steps to learn how to set up a sample dataset in Discover. + +![Setting up the sample data in Discover]({{site.url}}{{site.baseurl}}/images/discover-setting-up-data.gif) + +1. Verify access to OpenSearch Dashboards by connecting to [http://localhost:5601](http://localhost:5601) from a browser. The default username and password are `admin`. +1. On the **Home** page, choose **Discover** in the navigation pane. +1. On the index pattern toolbar, select the **opensearch_dashboards_sample_data_flights** dataset. +1. On the time filter toolbar, choose the calendar icon and then change the time range to **Last 7 days**. + +### Exploring the data fields + +In the **Discover** panel, you'll see a table that shows all the documents that match your search. The table includes a list of data fields that are available in the document table, as shown in the following image. + +![Exploring data fields interface]({{site.url}}{{site.baseurl}}/images/discover-data-fields.png) + +Follow these steps to explore the data fields: + +1. View the list of **Available fields**. +1. Choose **Cancelled** to view the values (`true` and `false`). +1. Choose the plus (+) sign to add the field to the document table. The field will be automatically added to **Selected fields** and the document table. +1. Select **FlightDelay** from the **Available fields** list, and then choose the plus (+) sign to add the field to the document table. +1. Optional: Rearrange the table columns by selecting the table header and then choosing **Move left** or **Move right**. + +## Searching data + +You can use the search toolbar or enter a DQL query in the **DevTools** console to search data in Dashboards, as shown in the following image. The search toolbar is best for basic queries, such as searching by a field name. DQL is best for complex queries, such as searching data using a term, string, Boolean, date, range, or nested query. + +![Searching data interface]({{site.url}}{{site.baseurl}}/images/discover-search.png) + +Follow these steps to search data: + +1. In the search toolbar, enter the Boolean query. For example, enter `FlightDelay:true AND FlightDelayMin >= 60` to search the data for flights delayed by 60 minutes or more. +1. Choose **Update**. +1. Optional: Choose the arrow (`>`) in a table row to expand the row and view the document table details. + +## Filtering data + +Filters allow you to refine sets of documents to subsets of those documents. For example, you can filter data to include or exclude certain fields, as shown in the following image. + +![Filtering data interface]({{site.url}}{{site.baseurl}}/images/discover-filter.png) + +Follow these steps to filter data: + +1. In the filter bar, choose **Add filter**. +1. Select options from the **Field**, **Operator**, and **Value** dropdown lists. For example, `Cancelled`, `is`, and `true`. +1. Choose **Save**. +1. To remove the filter, choose the close icon (x) next to the filter name. +1. Optional: Add more filters to further explore the data. + +## Analyzing data in the document table + +You can view the document table fields to better understand the data and gather insights for more informed decision-making: + +1. Choose the arrow icon (>) to expand a table row. +1. View the fields and details. +1. Switch between the **Table** and **JSON** tabs to view the different formats, as shown in the following image. + +![Analyzing data in the document table]({{site.url}}{{site.baseurl}}/images/discover-analyze.png) + +## Saving the search + +Saving a search saves the query text, filters, and current data view. To save your search to use it later, generate a report, or build visualizations and dashboards: + +1. Choose the save icon in the toolbar. +1. Give the search a title, and then choose **Save**. +1. Choose the save icon to access the saved search, as shown in the following image. + +Save search interface + +## Visualizing the search + +You can quickly visualize an aggregated field from **Discover**: + +1. From the **Available fields** list, select `FlightDelayType` and then choose **Visualize**, as shown in the following image. + +Visualizing search queries from Discover + +Dashboards creates a visualization for this field, which in this case is a basic bar chart, as shown in the following image. + +Bar chart created from Discover diff --git a/_dashboards/discover/index.md b/_dashboards/discover/index.md deleted file mode 100644 index e8ffcef7442..00000000000 --- a/_dashboards/discover/index.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -layout: default -title: Discover -nav_order: 10 -has_children: true ---- - -# Discover - -**Discover** in OpenSearch Dashboards helps you extract insights and get value out of data assets across your organization. You can quickly ingest and query your data, display that data in visualizations and interactive dashboards, and deliver insights to your organization. - -Discover user interface - -## What's next? - -- [Running queries in the console]({{site.url}}{{site.baseurl}}/dashboards/run-queries/) -- [Creating a dashboard with multiple data sources]({{site.url}}{{site.baseurl}}/) diff --git a/_dashboards/discover/multi-data-sources.md b/_dashboards/discover/multi-data-sources.md index 629d9305165..0b19007d01a 100644 --- a/_dashboards/discover/multi-data-sources.md +++ b/_dashboards/discover/multi-data-sources.md @@ -1,11 +1,11 @@ --- layout: default -title: Multiple data sources -parent: Discover +title: Adding multiple data sources +parent: Exploring data with Discover nav_order: 5 --- -# Multiple data sources +# Adding multiple data sources The multiple data sources feature is an experimental feature released in OpenSearch 2.4. It can't be used in a production environment. For updates on the feature’s progress or to leave feedback on improving the feature, see the [OpenSearch Forum discussion](https://forum.opensearch.org/t/feedback-experimental-feature-connect-to-external-data-sources/11144). {: .warning } @@ -14,14 +14,14 @@ You can add multiple data sources to a single dashboard. OpenSearch Dashboards a In this tutorial we provide the steps for enabling the `data_source` setting in Dashboards; adding credentials, data source connections, and index patterns; and combining visualizations in a single dashboard. -## Try out the multiple data sources feature in your local environment +## Try it: Exploring the multiple data sources feature in your local environment This tutorial uses a preconfigured data source and index pattern, and you aren’t required to configure settings. However, you’ll need to enable the `data_source` setting in the configuration file before before getting started with exploring this feature. {: .note } The multiple data sources feature is experimental and can't be deployed into production. You can try it out with a sample data source and a sample index pattern. Before getting started, you must first edit the YAML configuration. The following section provides the steps for enabling the feature. -## Edit the YAML configuration to enable the multiple data sources feature +## Modifying the multiple data sources settings Dashboards is configured in the cluster settings, and the multiple data sources feature is disabled by default. To enable it, you need to edit the configuration in `opensearch_dashboards.yml` and then restart the cluster. @@ -33,7 +33,7 @@ To enable the feature: 4. Restart the Dashboards container. 5. Verify the feature configuration settings were created and configured properly by connecting to Dashboards through [http://localhost:5601](http://localhost:5601/) and viewing the **Stack Management** console. **Data Sources `Experimental`** will appear in the sidebar. Alternatively, you can open on [http://localhost:5601/app/management/opensearch-dashboards/dataSources](http://localhost:5601/app/management/opensearch-dashboards/dataSource). -## Create a data source connection +## Creating a data source connection A data source connection specifies the parameters needed to connect to a data source. These parameters form a connection string for the data source. In Dashboards, you can add new data source connections or edit existing connections. @@ -41,90 +41,103 @@ To create a new data source connection: 1. Open Dashboards. If you’re not running the security plugin, go to [`http://localhost:5601`](http://localhost:5601/). If you’re running the security plugin, go to [`https://localhost:5601`](https://localhost:5601/) and log in with the username `admin` and password `admin`. -2. In the Dashboards console, select **Stack Management > Data Sources `Experimental` > Data Sources > Create data source connection**. -Data sources user interface +1. In the Dashboards console, choose **Stack Management** > **Data Sources `Experimental`** > **Data Sources** > **Create data source connection**, as shown in the following image. -3. Add information to each field to configure **Connection Details**, **Endpoint**, and **Authentication** to connect to a data source. For this tutorial, the **Endpoint URL** is `http://localhost:5601/app/management/opensearch-dashboards/dataSources`. -Create a data source connection user interface + Data sources user interface -4. Select **Create data source connection** to save your settings. +1. Add information to each field to configure **Connection Details**, **Endpoint**, and **Authentication** to connect to a data source, as shown in the following image. For this tutorial, the **Endpoint URL** is `http://localhost:5601/app/management/opensearch-dashboards/dataSources`. -5. Return to the **Data Sources** main page to confirm that the newly created data source is listed under **Data Sources**. -Data sources list user interface + Create a data source connection user interface -6. (Optional): Select the data source to verify that the settings are configured properly. -Data sources settings verification user interface +1. Choose **Create data source connection** to save your settings. +1. Return to the **Data Sources** main page to confirm that the newly created data source is listed under **Data Sources**, as shown in the following image. -## Create an index pattern + Data sources list user interface + +1. Optional: Select the data source to verify that the settings are configured properly, as shown in the following image. + + Data sources settings verification user interface + +## Creating an index pattern Index patterns allow you to access the OpenSearch data that you want to explore. An index pattern selects the data to use and allows you to define the field properties. Learn how to load your own data and create an index pattern following these steps. This tutorial uses the preconfigured index pattern `opensearch_dashboards_sample_data_ecommerce Default`. -1. In the Dashboards console, select **Index Patterns > Create index pattern**. -Index pattern user interface +1. In the Dashboards console, choose **Index Patterns** > **Create index pattern**, as shown in the following image. + + Index pattern user interface + +1. Choose **Use external data source connection**. +1. Start typing in the **Search data sources** field to search for the data source you created earlier and then select the data source and **Next step**, as shown in the following image. + + Index pattern search user interface -2. Select **Use external data source connection**. -3. Start typing in the Search data sources field to search for the data source you created earlier and then select the data source and **Next step**. -Index pattern search user interface +1. Add an **Index pattern name** to define the index pattern and then choose **Next step**, as shown in the following image. -4. Add an **Index pattern name** to define the index pattern and then select **Next step**. -Index pattern define user interface + Index pattern define user interface -5. Select an option for the **Time field** and then choose **Create index pattern**. -Index pattern time field user interface +1. Select an option for the **Time field** and then choose **Create index pattern**, as shown in the following image. -## Search data + Index pattern time field user interface -Before you start searching for data, set up the time filter. The sample index pattern used for this tutorial contains time-based data. You can set a time filter that displays only the data within a specified time range, and you can select the time filter to change the time range or select a specific time range in the histogram. +## Searching data -### Use the time filter +Before you start searching for data, set up the time filter. The sample index pattern used for this tutorial contains time-based data. You can set a time filter that displays only the data within a specified time range, and you can choose the time filter to change the time range or select a specific time range in the histogram. -1. In the Dashboards console, select **Discover** and confirm the index pattern being used is `opensearch_dashboards_sample_data_ecommerce`. -2. Select the calendar icon to change the time field. The default is **Last 15 minutes**. -3. Change the time field to **Last 7 days** and select **Refresh**. -Time filter user interface +### Adjusting the time filter -4. To set the start and end times, select the bar next to the time filter. In the popup, select **Absolute**, **Relative**, or **Now** and then specify the required options. -Start and end times user interface +To adjust the time filter: -### Select a time range from the histogram +1. In the Dashboards console, choose **Discover** and confirm that the index pattern being used is `opensearch_dashboards_sample_data_ecommerce`. +2. Choose the calendar icon to change the time field. The default is **Last 15 minutes**. +3. Change the time field to **Last 7 days** and choose **Refresh**, as shown in the following image. + + Time filter user interface + +4. To set the start and end times, choose the bar next to the time filter. In the popup, select **Absolute**, **Relative**, or **Now** and then specify the required options, as shown in the following image. + + Start and end times user interface + +### Selecting a time range from the histogram To select a time range for the histogram, you can do one of the following: * Select the bar that represents the time range you want to zoom in on. -* Select the bar and drag to view a specific time range. You must start the selection with the cursor over the background of the chart—the cursor changes to a plus sign when you hover over a valid start point. +* Select the bar and drag to view a specific time range. You must start the selection with the cursor over the background of the chart (the cursor changes to a plus sign when you hover over a valid start point). * Select the dropdown and then select an interval. +The following image shows a date histogram with an interval dropdown list. + Histogram user interface -## Create visualizations +## Creating data visualizations for a dashboard -Follow these steps to learn how to connect your visualizations in a single dashboard: +Follow these steps to learn how to create data visualizations for a dashboard: -1. In the Dashboards console, select **Visualize** > **Create visualization**. -2. Select the visualization type. For this tutorial, select **Line**. -3. Choose a source. For this tutorial, select the index pattern `opensearch_dashboards_sample_data_ecommerce`. -4. Under **Buckets**, select **Add > X-axis**. -5. In the **Aggregation** field, select **Date Histogram** and then **Update**. -6. Select **Save** and add the file name. This tutorial uses preconfigured visualizations, so you won’t be able to save your visualization. +1. In the Dashboards console, choose **Visualize** > **Create visualization**. +2. Select the visualization type. For this tutorial, choose **Line**. +3. Select a source. For this tutorial, choose the index pattern `opensearch_dashboards_sample_data_ecommerce`. +4. Under **Buckets**, choose **Add** > **X-axis**. +5. In the **Aggregation** field, choose **Date Histogram** and then choose **Update**. +6. Optional: Choose **Save** and add the file name. This tutorial uses preconfigured data visualizations, so you can't save the file for this tutorial. -## Connect visualizations in a single dashboard +## Connecting visualizations in a single dashboard Follow these steps to connect your visualizations in a single dashboard: -1. In the Dashboards console, select **Dashboard > Create dashboard**. -2. Select **Add an existing** and then select the data you want to add. -3. Select **Save** and add the dashboard name in the **Title field**. This tutorial uses preconfigured dashboards, so you won’t be able to save your dashboard. +1. In the Dashboards console, choose **Dashboard** > **Create dashboard**. +2. Choose **Add an existing** and then select the data you want to add. +3. Choose **Save** and add the dashboard name in the **Title field**. This tutorial uses preconfigured dashboards, so you won’t be able to save your dashboard. 4. Click on the white space left of **Add panels** to view the visualizations in a single dashboard. -Your dashboard might look like this: +Your dashboard might look like the one in the following image. -Example dashboard using data visualizations from many data sources +Example dashboard using data visualizations from many data sources -You have now explored the data sources experimental feature. We look forward to your feedback on how we can improve this feature ahead of its release for production use. +You have now explored the data sources experimental feature. To provide feedback on how this feature can be improved ahead of its release for production use, comment in the [OpenSearch forum](https://forum.opensearch.org/). -## Limitations +## Understanding feature limitations -The following limitations apply to the OpenSearch 2.4 release of this experimental feature: +The following limitations apply to this experimental feature: * The multiple data sources feature is supported for index-pattern-based visualizations only. * The visualization types Time Series Visual Builder (TSVB), Vega and Vega-Lite, and timeline are not supported. @@ -132,5 +145,4 @@ The following limitations apply to the OpenSearch 2.4 release of this experiment ## Related topics -* [OpenSearch 2.4.0 is ready for download](https://opensearch.org/blog/) -* [OpenSearch Forum](https://forum.opensearch.org/) \ No newline at end of file +* [OpenSearch Forum](https://forum.opensearch.org/) diff --git a/_dashboards/discover/time-filter.md b/_dashboards/discover/time-filter.md new file mode 100644 index 00000000000..fe910aba132 --- /dev/null +++ b/_dashboards/discover/time-filter.md @@ -0,0 +1,33 @@ +--- +layout: default +title: Setting the time filter +parent: Exploring data with Discover +nav_order: 10 +--- + +# Setting the time filter + +You can change the time range to display dashboard data over minutes, hours, days, weeks, months, or years. + +The default time range is **Last 15 minutes**. You can change the time range at the dashboard level or under **Stack Management > Advanced Settings > Time filter defaults**. +{: .note} + +To change the time range at the dashboard level, perform the following steps: + +1. Select the calendar icon. +2. Select one of the time filter options, as shown in the following image: + - **Quick select:** Choose a time based on the last or next number of seconds, minutes, hours, days, or another time unit. + - **Commonly used:** Choose a common time range like **Today**, **Last 7 days**, or **Last 30 days**. + - **Recently used date ranges:** Select a previously used time range. + - **Refresh every:** Set an automatic refresh period. + + Time range interface + +3. Choose **Show dates** to set start and end times, and then select anywhere inside the toolbar to access the time filter pop-up window, as shown in the following image. + + Time filter pop-up window + +4. Select **Absolute**, **Relative**, or **Now** and specify ranges. +5. Choose **Update** to apply changes, as shown in the following image. + + Start and end times interface diff --git a/_dashboards/dql.md b/_dashboards/dql.md deleted file mode 100644 index c0b4fcf523c..00000000000 --- a/_dashboards/dql.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -layout: default -title: Dashboards query language -nav_order: 90 ---- - -# Dashboards Query Language - -Similar to the [Query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/index) that lets you use the HTTP request body to search for data, you can use the Dashboards Query Language (DQL) in OpenSearch Dashboards to search for data and visualizations. - -For example, if you want to see all visualizations of visits to a host based in the US, enter `geo.dest:US` into the search field, and Dashboards refreshes to display all related data. - -Just like the query DSL, DQL has a handful of query types, so use whichever best fits your use case. - -This section uses the OpenSearch Dashboards sample web log data. To add sample data in Dashboards, log in to OpenSearch Dashboards, choose **Home**, **Add sample data**, and then **Add data**. - ---- - -#### Table of contents -1. TOC -{:toc} - ---- - -## Terms query - -The most basic query is to just specify the term you're searching for. - -``` -host:www.example.com -``` - -To access an object's nested field, list the complete path to the field separated by periods. For example, to retrieve the `lat` field in the `coordinates` object: - -``` -coordinates.lat:43.7102 -``` - -DQL also supports leading and trailing wildcards, so you can search for any terms that match your pattern. - -``` -host.keyword:*.example.com/* -``` - -To check if a field exists or has any data, use a wildcard to see if Dashboards returns any results. - -``` -host.keyword:* -``` - -## Boolean query - -To mix and match, or even combine, multiple queries for more refined results, you can use the boolean operators `and`, `or`, and `not`. DQL is not case sensitive, so `AND` and `and` are the same. - -``` -host.keyword:www.example.com and response.keyword:200 -``` - -The following example demonstrates how to use multiple operators in one query. - -``` -geo.dest:US or response.keyword:200 and host.keyword:www.example.com -``` - -Remember that boolean operators follow the logical precedence order of `not`, `and`, and `or`, so if you have an expression like the previous example, `response.keyword:200 and host.keyword:www.example.com` gets evaluated first, and then Dashboards uses that result to compare with `geo.dest:US`. - -To avoid confusion, we recommend using parentheses to dictate the order you want to evaluate in. If you want to evaluate `geo.dest:US or response.keyword:200` first, your expression becomes: - -``` -(geo.dest:US or response.keyword:200) and host.keyword:www.example.com -``` - -## Date and range queries - -DQL also supports inequalities if you're using numeric inequalities. - -``` -bytes >= 15 and memory < 15 -``` - -Similarly, you can use the same method to find a date before or after your query. `>` indicates a search for a date after your specified date, and `<` returns dates before. - -``` -@timestamp > "2020-12-14T09:35:33" -``` - -## Nested field query - -If you have a document with nested fields, you have to specify which parts of the document you want to retrieve. - -Suppose that you have the following document: - -```json -{ - "superheroes":[ - { - "hero-name": "Superman", - "real-identity": "Clark Kent", - "age": 28 - }, - { - "hero-name": "Batman", - "real-identity": "Bruce Wayne", - "age": 26 - }, - { - "hero-name": "Flash", - "real-identity": "Barry Allen", - "age": 28 - }, - { - "hero-name": "Robin", - "real-identity": "Dick Grayson", - "age": 15 - } - ] -} -``` - -The following example demonstrates how to use DQL to retrieve a specific field. - -``` -superheroes: {hero-name: Superman} -``` - -If you want to retrieve multiple objects from your document, just specify all of the fields you want to retrieve. - -``` -superheroes: {hero-name: Superman} and superheroes: {hero-name: Batman} -``` - -The previous boolean and range queries still work, so you can submit a more refined query. - -``` -superheroes: {hero-name: Superman and age < 50} -``` - -If your document has an object nested within another object, you can still retrieve data by specifying all of the levels. - -``` -justice-league.superheroes: {hero-name:Superman} -``` diff --git a/_dashboards/get-started/quickstart-dashboards.md b/_dashboards/get-started/quickstart-dashboards.md new file mode 100644 index 00000000000..5cae8b05fa0 --- /dev/null +++ b/_dashboards/get-started/quickstart-dashboards.md @@ -0,0 +1,117 @@ +--- +layout: default +title: Quickstart guide for OpenSearch Dashboards +nav_order: 20 +has_children: false +--- + +# Quickstart guide for OpenSearch Dashboards + +This quickstart guide covers the core concepts that you need to understand to get started with OpenSearch Dashboards. You'll learn how to: + +- Add sample data. +- Explore and inspect data with **Discover**. +- Visualize data with **Dashboard**. +- Add sample data. +- Explore and inspect data with **Discover**. +- Visualize data with **Dashboard**. + +Before you get started, make sure you've installed OpenSearch and OpenSearch Dashboards. For information on installation and configuration, see [Install and configure OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/index/) and [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/). +{: .note} + +# Adding sample data + +Sample datasets come with visualizations, dashboards, and other tools to help you explore Dashboards before you add your own data. To add sample data, perform the following steps: + +1. Verify access to OpenSearch Dashboards by connecting to [http://localhost:5601](http://localhost:5601) from a browser. The default username and password are `admin`. +1. On the OpenSearch Dashboards **Home** page, choose **Add sample data**. +2. Choose **Add data** to add the datasets, as shown in the following image. + + Sample datasets + +# Exploring and inspecting data + +In [**Discover**](discover/index.md), you can: + +- Choose data to explore, set a time range for that data, search it using [Dashboards Query Language (DQL)]({{site.url}}{{site.baseurl}}/dashboards/dql/), and filter the results. +- Explore the data, view individual documents, and create tables summarizing the data's contents. +- Visualize your findings. + +## Try it: Getting familiar with Discover + +1. On the OpenSearch Dashboards **Home** page, choose **Discover**. +1. Change the [time filter](time-filter.md) to **Last 7 days**, as shown in the following image. + + Time filter interface + +2. Search using the DQL query `FlightDelay:true AND DestCountry: US AND FlightDelayMin >= 60` and then choose **Update**. You should see results for US-bound flights delayed by 60 minutes or more, as shown in the following image. + + DQL search field example + +3. To filter data, choose **Add filter** and then select an **Available field**. For example, select `FlightDelayType`, **is**, and **Weather delay** from the **Field**, **Operator**, and **Value** dropdown lists, as shown in the following image. + + Filter data by FlightDelayType field + +# Visualizing data + +Raw data can be difficult to comprehend and use. Data visualizations help you prepare and present data in a visual form. In **Dashboard** you can: + +- Display data in a single view. +- Build dynamic dashboards. +- Create and share reports. +- Embed analytics to differentiate your applications. + +## Try it: Getting familiar with Dashboard + +1. On the OpenSearch Dashboards **Home** page, choose **Dashboard**. +1. Choose **[Flights] Global Flight Data** in the **Dashboards** window, as shown in the following image. + + Data visualization dashboard + +1. To add panels to the dashboard, choose **Edit** and then **Add** from the toolbar. +1. In the **Add panels** window, choose the existing panel **[Flights] Delay Buckets**. You'll see a pop-up window on the lower right confirming that you've added the panel. +1. Select `x` to close the **Add panels** window. +1. View the added panel **[Flights] Delay Buckets**, which is added as the last panel on the dashboard, as shown in the following image. + + Add panel to dashboard + +## Try it: Creating a visualization panel + +Continuing with the preceding dashboard, you'll create a bar chart comparing the number of canceled flights and delayed flights to delay type and then add the panel to the dashboard: + +1. Change the default [time range]({{site.url}}{{site.baseurl}}/dashboards/get-started/time-filter/) from **24 hours** to **Last 7 days**. +1. In the toolbar, choose **Edit**, then **Create new**. +1. Select **VisBuilder** in the **New Visualizations** window. +1. In the **Data Source** dropdown list, choose `opensearch_dashboards_sample_data_flights`. +1. Drag the fields **Cancelled** and **FlightDelay** to the y-axis column. +1. Drag the field **FlightDelayType** to the x-axis column. +1. Choose **Save** and name the visualization in the **Title** field. +2. Choose **Save and return**. The following bar chart is added as the last panel on the dashboard, as shown in the following image. + +Creating a visualization panel + +# Interacting with data + +Interactive dashboards allow you analyze data in more depth and filter it in several ways. In Dashboards, you can interact directly with data on a dashboard by using dashboard-level filters. For example, continuing with the preceding dashboard, you can filter to show delays and cancellations for a specific airline. + +## Try it: Interacting with the sample flight data + +1. On the **[Flights] Airline Carrier** panel, choose **OpenSearch-Air**. The dashboard updates automatically. +1. Choose **Save** to save the customized dashboard. + +Alternatively, you can apply filters using the dashboard toolbar: + +1. In the dashboard toolbar, choose **Add filter**. +1. From the **Field**, **Operator**, and **Value** dropdown lists, choose **Carrier**, **is**, and **OpenSearch-Air**, respectively, as shown in the following image. + + Edit field interface + +1. Choose **Save**. The dashboard updates automatically, and the result is the dashboard shown in the following image. + + Dashboard view after applying Carrier filter + +# Next steps + +- **Visualize data**. To learn more about data visualizations in OpenSearch Dashboards, see [**Building data visualizations with Visualize**]({{site.url}}{{site.baseurl}}/dashboards/visualize/viz-index/). +- **Create dashboards**. To learn more about creating dashboards in OpenSearch Dashboards, see [**Creating dashboards with Dashboard**]({{site.url}}{{site.baseurl}}/dashboards/dashboard/). +- **Explore data**. To learn more about exploring data in OpenSearch Dashboards, see [**Exploring data with Discover**]({{site.url}}{{site.baseurl}}/dashboards/discover/). \ No newline at end of file diff --git a/_dashboards/index.md b/_dashboards/index.md deleted file mode 100644 index 1c0d5a433b6..00000000000 --- a/_dashboards/index.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -layout: default -title: Getting started with OpenSearch Dashboards -nav_order: 1 -has_children: false -has_toc: false -redirect_from: - - /docs/opensearch-dashboards/ - - /dashboards/ ---- - -{%- comment -%}The `/docs/opensearch-dashboards/` redirect is specifically to support the UI links in OpenSearch Dashboards 1.0.0.{%- endcomment -%} - -# Getting started with OpenSearch Dashboards - -OpenSearch Dashboards is an open-source, integrated visualization tool that makes it easy for users to explore their data in OpenSearch. From real-time application monitoring, threat detection, and incident management to personalized search, OpenSearch Dashboards gives you the data visualizations needed to graphically represent trends, outliers, and patterns in your data. The image below shows a sample of data visualizations in OpenSearch Dashboards. - -User interface showing several data visualizations - -## Use cases for OpenSearch Dashboards - -In OpenSearch Dashboards, a data dashboard is a collection of charts, graphs, gauges, and other visualizations that gives you a snapshot of the data that you're interested in and that you interact with. You can track, analyze, and display real-time search, monitoring, and analysis of business and operational data for use cases like application monitoring, log analytics, observability, and website search. The image below shows data visualization use cases for OpenSearch Dashboards. - - - - - - - - - - - - -
Visualize log and trace data with interactive log analytics.Log analyticsDetect and mitigate issues faster with anomaly detection.Anomaly detection
Diagnose performance issues and reduce application downtime.Observability interface and log monitoring featuresUnlock real-time search, monitoring, and analysis of business and operational data.Real-time search, monitoring, and analysis
- -## Explore the OpenSearch Dashboards playground - -You can interact with the demonstration datasets in the OpenSearch Dashboards playground by following the steps below: - -1. Go to the [OpenSearch Dashboards playground](https://playground.opensearch.org/app/home). -2. Choose **OpenSearch Dashboards > Dashboard**. -3. Explore the available datasets and select one: **[Flights] Global Flight Dashboard**, **[Logs] Web Traffic**, or **[eCommerce] Revenue Dashboard**. -4. Interact with the data on a dashboard. Choose **Add filter**, and specify the data you'd like to see. -5. Select one of the options from the **Field** menu, and specify an **Operator** to filter the results. - -The image below shows a [Logs] Web Traffic dashboard with filters applied. - -Logs web traffic dashboard with filters applied - -## Next steps - -You can run OpenSearch Dashboards on a local host after installing OpenSearch. See [Install and configure OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/index/) and [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/) for installation instructions. - -## Related links -- [Getting Started with OpenSearch]({{site.url}}{{site.baseurl}}). -- [Launch Highlight: OpenSearch Playground](https://www.opensearch.org/blog/community/2022/10/opensearch-playground/) -- [Upgrade from Kibana OSS to OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/upgrade-to/dashboards-upgrade-to/) -- [OpenSearch Frequently Asked Questions]({{site.url}}/faq/) -- [OpenSearch Dashboards Developer Guide](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/DEVELOPER_GUIDE.md) \ No newline at end of file diff --git a/_dashboards/visualize/maptiles.md b/_dashboards/visualize/maptiles.md new file mode 100644 index 00000000000..077b23466b4 --- /dev/null +++ b/_dashboards/visualize/maptiles.md @@ -0,0 +1,34 @@ +--- +layout: default +title: WMS map server +nav_order: 20 +redirect_from: + - /docs/opensearch-dashboards/maptiles/ + - /dashboards/maptiles/ +--- + +{%- comment -%}The `/docs/opensearch-dashboards/maptiles/` redirect is specifically to support the UI links in OpenSearch Dashboards 1.0.0.{%- endcomment -%} + +# Configure WMS map server + +OpenSearch Dashboards includes default map tiles, but if you need more specialized maps, you can configure OpenSearch Dashboards to use a WMS map server: + +1. Open OpenSearch Dashboards at `https://:`. For example, [https://localhost:5601](https://localhost:5601). +1. If necessary, log in. +1. Choose **Management** and **Advanced Settings**. +1. Locate `visualization:tileMap:WMSdefaults`. +1. Change `enabled` to true and add the URL of a valid WMS map server: + + ```json + { + "enabled": true, + "url": "", + "options": { + "format": "image/png", + "transparent": true + } + } + ``` + +Map services often have licensing fees or restrictions. You're responsible for all such considerations on any map server that you specify. +{: .note } diff --git a/_data-prepper/index.md b/_data-prepper/index.md index 20662a11499..73086d2b625 100644 --- a/_data-prepper/index.md +++ b/_data-prepper/index.md @@ -14,7 +14,7 @@ redirect_from: Data Prepper is a server-side data collector capable of filtering, enriching, transforming, normalizing, and aggregating data for downstream analytics and visualization. -Data Prepper lets users build custom pipelines to improve the operational view of applications. Two common uses for Data Prepper are trace and log analytics. [Trace analytics]({{site.url}}{{site.baseurl}}/observability-plugin/trace/index/) can help you visualize the flow of events and identify performance problems, and [log analytics]({{site.url}}{{site.baseurl}}/observability-plugin/log-analytics/) can improve searching, analyzing and provide insights into your application. +Data Prepper lets users build custom pipelines to improve the operational view of applications. Two common uses for Data Prepper are trace and log analytics. [Trace analytics]({{site.url}}{{site.baseurl}}/observing-your-data/trace/index/) can help you visualize the flow of events and identify performance problems, and [log analytics]({{site.url}}{{site.baseurl}}/observing-your-data/log-analytics/) can improve searching, analyzing and provide insights into your application. ## Concepts diff --git a/_data-prepper/pipelines.md b/_data-prepper/pipelines.md index ddbfbb9b1b4..7879467424b 100644 --- a/_data-prepper/pipelines.md +++ b/_data-prepper/pipelines.md @@ -125,7 +125,7 @@ This example uses weak security. We strongly recommend securing all plugins whic ### Trace analytics pipeline -The following example demonstrates how to build a pipeline that supports the [Trace Analytics OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observability-plugin/trace/ta-dashboards/). This pipeline takes data from the OpenTelemetry Collector and uses two other pipelines as sinks. These two separate pipelines index trace and the service map documents for the dashboard plugin. +The following example demonstrates how to build a pipeline that supports the [Trace Analytics OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observing-your-data/trace/ta-dashboards/). This pipeline takes data from the OpenTelemetry Collector and uses two other pipelines as sinks. These two separate pipelines index trace and the service map documents for the dashboard plugin. Starting from Data Prepper 2.0, Data Prepper no longer supports `otel_trace_raw_prepper` processor due to the Data Prepper internal data model evolution. Instead, users should use `otel_trace_raw`. diff --git a/_install-and-configure/install-opensearch/debian.md b/_install-and-configure/install-opensearch/debian.md index 6180b67deec..9f655ed528d 100644 --- a/_install-and-configure/install-opensearch/debian.md +++ b/_install-and-configure/install-opensearch/debian.md @@ -300,7 +300,7 @@ TLS certificates provide additional security for your cluster by allowing client ```bash sudo chown opensearch:opensearch admin-key.pem admin.pem node1-key.pem node1.pem root-ca-key.pem root-ca.pem root-ca.srl ``` -1. Add these certificates to `opensearch.yml` as described in [Generate Certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/generate-certificates/#add-distinguished-names-to-opensearchyml). Advanced users might also choose to append the settings using a script: +1. Add these certificates to `opensearch.yml` as described in [Generate Certificates]({{site.url}}{{site.baseurl}}/security/configuration/generate-certificates/#add-distinguished-names-to-opensearchyml). Advanced users might also choose to append the settings using a script: ```bash #! /bin/bash diff --git a/_install-and-configure/install-opensearch/docker.md b/_install-and-configure/install-opensearch/docker.md index 37965a8f2f6..4f7e7be974e 100644 --- a/_install-and-configure/install-opensearch/docker.md +++ b/_install-and-configure/install-opensearch/docker.md @@ -460,8 +460,4 @@ COPY --chown=opensearch:opensearch my-root-cas.pem /usr/share/opensearch/config/ - [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/) - [Performance analyzer]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/index/) - [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/) -<<<<<<< HEAD -- [About the security plugin]({{site.url}}{{site.baseurl}}/security/index/) -======= -- [About the security plugin]({{site.url}}{{site.baseurl}}/security-plugin/index/) ->>>>>>> main +- [About Security in OpenSearch]({{site.url}}{{site.baseurl}}/security/index/) diff --git a/_install-and-configure/plugins.md b/_install-and-configure/plugins.md index dabac1a0d62..4831e3959b6 100644 --- a/_install-and-configure/plugins.md +++ b/_install-and-configure/plugins.md @@ -291,7 +291,7 @@ Members of the OpenSearch community have built countless plugins for the service ## Related links -- [About Observability]({{site.url}}{{site.baseurl}}/observability-plugin/index/) +- [About Observability]({{site.url}}{{site.baseurl}}/observing-your-data/index/) - [About security analytics]({{site.url}}{{site.baseurl}}/security-analytics/index/) - [About the security plugin]({{site.url}}{{site.baseurl}}/security/index/) - [Alerting]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/index/) diff --git a/_ml-commons-plugin/cluster-settings.md b/_ml-commons-plugin/cluster-settings.md index eb4c71be515..f0ee867f4ac 100644 --- a/_ml-commons-plugin/cluster-settings.md +++ b/_ml-commons-plugin/cluster-settings.md @@ -143,13 +143,17 @@ The default value allows you to upload a model file from any http/https/ftp/loca ### Setting +The default URL value for this trusted URL setting is not secure. To ensure the security, please use you own regex string to the trusted repository that contains your models, for example `https://github.com/opensearch-project/ml-commons/blob/2.x/ml-algorithms/src/test/resources/org/opensearch/ml/engine/algorithms/text_embedding/*`. +{: .warning } + + ``` -plugins.ml_commons.trusted_url_regex: ^(https?\|ftp\|file)://[-a-zA-Z0-9+&@#/%?=~_\|!:,.;]*[-a-zA-Z0-9+&@#/%=~_\|] +plugins.ml_commons.trusted_url_regex: ``` ### Values -- Default value: `^(https?\|ftp\|file)://[-a-zA-Z0-9+&@#/%?=~_\|!:,.;]*[-a-zA-Z0-9+&@#/%=~_\|]` +- Default value: `"^(https?|ftp|file)://[-a-zA-Z0-9+&@#/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#/%=~_|]"` - Value range: Java regular expression (regex) string ## Assign task timeout diff --git a/_job-scheduler-plugin/index.md b/_monitoring-your-cluster/job-scheduler/index.md similarity index 99% rename from _job-scheduler-plugin/index.md rename to _monitoring-your-cluster/job-scheduler/index.md index 028a02e0986..938d79803ce 100644 --- a/_job-scheduler-plugin/index.md +++ b/_monitoring-your-cluster/job-scheduler/index.md @@ -4,6 +4,8 @@ title: Job Scheduler nav_order: 1 has_children: false has_toc: false +redirect_from: + - /job-scheduler-plugin/index/ --- # Job Scheduler diff --git a/_monitoring-plugins/pa/api.md b/_monitoring-your-cluster/pa/api.md similarity index 99% rename from _monitoring-plugins/pa/api.md rename to _monitoring-your-cluster/pa/api.md index f70481da244..8ead2b47a4a 100644 --- a/_monitoring-plugins/pa/api.md +++ b/_monitoring-your-cluster/pa/api.md @@ -3,6 +3,8 @@ layout: default title: API parent: Performance Analyzer nav_order: 1 +redirect_from: + - /monitoring-plugins/pa/api/ --- # Performance Analyzer API diff --git a/_monitoring-plugins/pa/dashboards.md b/_monitoring-your-cluster/pa/dashboards.md similarity index 99% rename from _monitoring-plugins/pa/dashboards.md rename to _monitoring-your-cluster/pa/dashboards.md index 561f6fa3ce0..300095d3af6 100644 --- a/_monitoring-plugins/pa/dashboards.md +++ b/_monitoring-your-cluster/pa/dashboards.md @@ -3,6 +3,8 @@ layout: default title: Create PerfTop Dashboards parent: Performance Analyzer nav_order: 2 +redirect_from: + - /monitoring-plugins/pa/dashboards/ --- # PerfTop dashboards diff --git a/_monitoring-plugins/pa/index.md b/_monitoring-your-cluster/pa/index.md similarity index 99% rename from _monitoring-plugins/pa/index.md rename to _monitoring-your-cluster/pa/index.md index 45acb55c966..d63b1795f8a 100644 --- a/_monitoring-plugins/pa/index.md +++ b/_monitoring-your-cluster/pa/index.md @@ -5,6 +5,7 @@ nav_order: 58 has_children: true redirect_from: - /monitoring-plugins/pa/ + - /monitoring-plugins/pa/index/ --- # Performance analyzer diff --git a/_monitoring-plugins/pa/rca/api.md b/_monitoring-your-cluster/pa/rca/api.md similarity index 96% rename from _monitoring-plugins/pa/rca/api.md rename to _monitoring-your-cluster/pa/rca/api.md index 2d3aeb3e80e..cb8762cd5fd 100644 --- a/_monitoring-plugins/pa/rca/api.md +++ b/_monitoring-your-cluster/pa/rca/api.md @@ -4,6 +4,8 @@ title: API parent: Root Cause Analysis grand_parent: Performance Analyzer nav_order: 1 +redirect_from: + - /monitoring-plugins/pa/rca/api/ --- # RCA API diff --git a/_monitoring-plugins/pa/rca/index.md b/_monitoring-your-cluster/pa/rca/index.md similarity index 95% rename from _monitoring-plugins/pa/rca/index.md rename to _monitoring-your-cluster/pa/rca/index.md index 765b9e23e00..cd63659529f 100644 --- a/_monitoring-plugins/pa/rca/index.md +++ b/_monitoring-your-cluster/pa/rca/index.md @@ -4,6 +4,8 @@ title: Root Cause Analysis nav_order: 50 parent: Performance Analyzer has_children: true +redirect_from: + - /monitoring-plugins/pa/rca/index/ --- # Root Cause Analysis diff --git a/_monitoring-plugins/pa/rca/reference.md b/_monitoring-your-cluster/pa/rca/reference.md similarity index 83% rename from _monitoring-plugins/pa/rca/reference.md rename to _monitoring-your-cluster/pa/rca/reference.md index 765942d0708..2805f894a69 100644 --- a/_monitoring-plugins/pa/rca/reference.md +++ b/_monitoring-your-cluster/pa/rca/reference.md @@ -4,6 +4,8 @@ title: RCA Reference parent: Root Cause Analysis grand_parent: Performance Analyzer nav_order: 3 +redirect_from: + - /monitoring-plugins/pa/rca/reference/ --- # RCA reference diff --git a/_monitoring-plugins/pa/reference.md b/_monitoring-your-cluster/pa/reference.md similarity index 99% rename from _monitoring-plugins/pa/reference.md rename to _monitoring-your-cluster/pa/reference.md index caa87a8f797..9fed7646b1e 100644 --- a/_monitoring-plugins/pa/reference.md +++ b/_monitoring-your-cluster/pa/reference.md @@ -3,6 +3,8 @@ layout: default title: Metrics Reference parent: Performance Analyzer nav_order: 3 +redirect_from: + - /monitoring-plugins/pa/reference/ --- # Metrics reference diff --git a/_monitoring-plugins/ad/api.md b/_observing-your-data/ad/api.md similarity index 99% rename from _monitoring-plugins/ad/api.md rename to _observing-your-data/ad/api.md index c8149424f7b..635914b99ae 100644 --- a/_monitoring-plugins/ad/api.md +++ b/_observing-your-data/ad/api.md @@ -3,6 +3,8 @@ layout: default title: Anomaly detection API parent: Anomaly detection nav_order: 1 +redirect_from: + - /monitoring-plugins/ad/api/ --- # Anomaly detection API diff --git a/_monitoring-plugins/ad/index.md b/_observing-your-data/ad/index.md similarity index 99% rename from _monitoring-plugins/ad/index.md rename to _observing-your-data/ad/index.md index 0c476878986..be4fabc5ae7 100644 --- a/_monitoring-plugins/ad/index.md +++ b/_observing-your-data/ad/index.md @@ -1,10 +1,11 @@ --- layout: default title: Anomaly detection -nav_order: 46 +nav_order: 90 has_children: true redirect_from: - /monitoring-plugins/ad/ + - /monitoring-plugins/ad/index/ --- # Anomaly detection diff --git a/_monitoring-plugins/ad/result-mapping.md b/_observing-your-data/ad/result-mapping.md similarity index 99% rename from _monitoring-plugins/ad/result-mapping.md rename to _observing-your-data/ad/result-mapping.md index 54d8670540f..7e1482a0134 100644 --- a/_monitoring-plugins/ad/result-mapping.md +++ b/_observing-your-data/ad/result-mapping.md @@ -3,6 +3,8 @@ layout: default title: Anomaly result mapping parent: Anomaly detection nav_order: 6 +redirect_from: + - /monitoring-plugins/ad/result-mapping/ --- # Anomaly result mapping diff --git a/_monitoring-plugins/ad/security.md b/_observing-your-data/ad/security.md similarity index 98% rename from _monitoring-plugins/ad/security.md rename to _observing-your-data/ad/security.md index 777bb416791..e309a410299 100644 --- a/_monitoring-plugins/ad/security.md +++ b/_observing-your-data/ad/security.md @@ -4,6 +4,8 @@ title: Anomaly detection security nav_order: 10 parent: Anomaly detection has_children: false +redirect_from: + - /monitoring-plugins/ad/security/ --- # Anomaly detection security diff --git a/_monitoring-plugins/ad/settings.md b/_observing-your-data/ad/settings.md similarity index 99% rename from _monitoring-plugins/ad/settings.md rename to _observing-your-data/ad/settings.md index ca0ddf4e226..66cfe6fe4ef 100644 --- a/_monitoring-plugins/ad/settings.md +++ b/_observing-your-data/ad/settings.md @@ -3,6 +3,8 @@ layout: default title: Settings parent: Anomaly detection nav_order: 4 +redirect_from: + - /monitoring-plugins/ad/settings/ --- # Settings diff --git a/_monitoring-plugins/alerting/api.md b/_observing-your-data/alerting/api.md similarity index 99% rename from _monitoring-plugins/alerting/api.md rename to _observing-your-data/alerting/api.md index 074c64d8940..811743d8f5a 100644 --- a/_monitoring-plugins/alerting/api.md +++ b/_observing-your-data/alerting/api.md @@ -3,6 +3,8 @@ layout: default title: API parent: Alerting nav_order: 15 +redirect_from: + - /monitoring-plugins/alerting/api/ --- # Alerting API diff --git a/_monitoring-plugins/alerting/cron.md b/_observing-your-data/alerting/cron.md similarity index 97% rename from _monitoring-plugins/alerting/cron.md rename to _observing-your-data/alerting/cron.md index bba64d067bb..b37d13e576d 100644 --- a/_monitoring-plugins/alerting/cron.md +++ b/_observing-your-data/alerting/cron.md @@ -4,6 +4,8 @@ title: Cron nav_order: 20 parent: Alerting has_children: false +redirect_from: + - /monitoring-plugins/alerting/cron/ --- # Cron expression reference diff --git a/_monitoring-plugins/alerting/index.md b/_observing-your-data/alerting/index.md similarity index 95% rename from _monitoring-plugins/alerting/index.md rename to _observing-your-data/alerting/index.md index 3495e4251b4..fa033ce1110 100644 --- a/_monitoring-plugins/alerting/index.md +++ b/_observing-your-data/alerting/index.md @@ -1,10 +1,11 @@ --- layout: default title: Alerting -nav_order: 34 +nav_order: 70 has_children: true redirect_from: - /monitoring-plugins/alerting/ + - /monitoring-plugins/alerting/index/ --- # Alerting diff --git a/_monitoring-plugins/alerting/monitors.md b/_observing-your-data/alerting/monitors.md similarity index 99% rename from _monitoring-plugins/alerting/monitors.md rename to _observing-your-data/alerting/monitors.md index 1cb92aa94c2..9872d3e2c97 100644 --- a/_monitoring-plugins/alerting/monitors.md +++ b/_observing-your-data/alerting/monitors.md @@ -4,6 +4,8 @@ title: Monitors nav_order: 1 parent: Alerting has_children: false +redirect_from: + - /monitoring-plugins/alerting/monitors/ --- # Monitors diff --git a/_monitoring-plugins/alerting/security.md b/_observing-your-data/alerting/security.md similarity index 99% rename from _monitoring-plugins/alerting/security.md rename to _observing-your-data/alerting/security.md index 0b384338d79..21685a64881 100644 --- a/_monitoring-plugins/alerting/security.md +++ b/_observing-your-data/alerting/security.md @@ -4,6 +4,8 @@ title: Alerting security nav_order: 10 parent: Alerting has_children: false +redirect_from: + - /monitoring-plugins/alerting/security/ --- # Alerting security diff --git a/_monitoring-plugins/alerting/settings.md b/_observing-your-data/alerting/settings.md similarity index 98% rename from _monitoring-plugins/alerting/settings.md rename to _observing-your-data/alerting/settings.md index 6e44d7ae5b0..20ef388b63a 100644 --- a/_monitoring-plugins/alerting/settings.md +++ b/_observing-your-data/alerting/settings.md @@ -3,6 +3,8 @@ layout: default title: Management parent: Alerting nav_order: 5 +redirect_from: + - /monitoring-plugins/alerting/settings/ --- # Management diff --git a/_observability-plugin/app-analytics.md b/_observing-your-data/app-analytics.md similarity index 95% rename from _observability-plugin/app-analytics.md rename to _observing-your-data/app-analytics.md index 03a3dfa07a8..44e1939aaa4 100644 --- a/_observability-plugin/app-analytics.md +++ b/_observing-your-data/app-analytics.md @@ -1,7 +1,9 @@ --- layout: default title: Application analytics -nav_order: 80 +nav_order: 10 +redirect_from: + - /observing-your-data/app-analytics/ --- # Application analytics @@ -39,7 +41,7 @@ To see your visualizations, choose the **Panel** tab. ### Configure availability -Availability is the status of your application determined by availability levels set on a [time series metric]({{site.url}}{{site.baseurl}}/observability-plugin/app-analytics/#time-series-metric). +Availability is the status of your application determined by availability levels set on a [time series metric]({{site.url}}{{site.baseurl}}/observing-your-data/app-analytics/#time-series-metric). To create an availability level, you must configure the following: - color: The color of the availability badge on the home page. diff --git a/_observability-plugin/event-analytics.md b/_observing-your-data/event-analytics.md similarity index 95% rename from _observability-plugin/event-analytics.md rename to _observing-your-data/event-analytics.md index 030315eb28b..780a1f0039e 100644 --- a/_observability-plugin/event-analytics.md +++ b/_observing-your-data/event-analytics.md @@ -1,7 +1,9 @@ --- layout: default title: Event analytics -nav_order: 10 +nav_order: 20 +redirect_from: + - /observing-your-data/event-analytics/ --- # Event analytics @@ -28,7 +30,7 @@ For more information about building PPL queries, see [Piped Processing Language] ## Save a visualization -After Dashboards generates a visualization, you must save it if you want to return to it at a later time or if you want to add it to an [operational panel]({{site.url}}{{site.baseurl}}/observability-plugin/operational-panels). +After Dashboards generates a visualization, you must save it if you want to return to it at a later time or if you want to add it to an [operational panel]({{site.url}}{{site.baseurl}}/observing-your-data/operational-panels). To save a visualization, expand the save dropdown menu next to **Refresh**, enter a name for your visualization, then choose **Save**. You can reopen any saved visualizations on the event analytics page. diff --git a/_observability-plugin/index.md b/_observing-your-data/index.md similarity index 57% rename from _observability-plugin/index.md rename to _observing-your-data/index.md index 304cf2dbbf9..41f6ea19d27 100644 --- a/_observability-plugin/index.md +++ b/_observing-your-data/index.md @@ -1,13 +1,14 @@ --- layout: default -title: About Observability +title: Observing your data nav_order: 1 has_children: false redirect_from: - - /observability-plugin/ + - /observing-your-data/ + - /observing-your-data/index/ --- -# About Observability +# Observing your data OpenSearch Dashboards {: .label .label-yellow :} @@ -16,12 +17,12 @@ Observability is collection of plugins and applications that let you visualize d Your experience of exploring data might differ, but if you're new to exploring data to create visualizations, we recommend trying a workflow like the following: 1. Explore data within a certain timeframe using [Piped Processing Language]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/index). -2. Use [event analytics]({{site.url}}{{site.baseurl}}/observability-plugin/event-analytics) to turn data-driven events into visualizations. +2. Use [event analytics]({{site.url}}{{site.baseurl}}/observing-your-data/event-analytics) to turn data-driven events into visualizations. ![Sample Event Analytics View]({{site.url}}{{site.baseurl}}/images/event-analytics.png) -3. Create [operational panels]({{site.url}}{{site.baseurl}}/observability-plugin/operational-panels) and add visualizations to compare data the way you like. +3. Create [operational panels]({{site.url}}{{site.baseurl}}/observing-your-data/operational-panels) and add visualizations to compare data the way you like. ![Sample Operational Panel View]({{site.url}}{{site.baseurl}}/images/operational-panel.png) -4. Use [log analytics]({{site.url}}{{site.baseurl}}/observability-plugin/log-analytics) to transform unstructured log data. -5. Use [trace analytics]({{site.url}}{{site.baseurl}}/observability-plugin/trace/index) to create traces and dive deep into your data. +4. Use [log analytics]({{site.url}}{{site.baseurl}}/observing-your-data/log-analytics) to transform unstructured log data. +5. Use [trace analytics]({{site.url}}{{site.baseurl}}/observing-your-data/trace/index) to create traces and dive deep into your data. ![Sample Trace Analytics View]({{site.url}}{{site.baseurl}}/images/observability-trace.png) -6. Leverage [notebooks]({{site.url}}{{site.baseurl}}/observability-plugin/notebooks) to combine different visualizations and code blocks that you can share with team members. +6. Leverage [notebooks]({{site.url}}{{site.baseurl}}/observing-your-data/notebooks) to combine different visualizations and code blocks that you can share with team members. ![Sample Notebooks View]({{site.url}}{{site.baseurl}}/images/notebooks.png) diff --git a/_observability-plugin/log-analytics.md b/_observing-your-data/log-analytics.md similarity index 98% rename from _observability-plugin/log-analytics.md rename to _observing-your-data/log-analytics.md index 8128d6d5610..7be152d17d8 100644 --- a/_observability-plugin/log-analytics.md +++ b/_observing-your-data/log-analytics.md @@ -1,7 +1,9 @@ --- layout: default title: Log analytics -nav_order: 70 +nav_order: 30 +redirect_from: + - /observing-your-data/log-analytics/ --- # Log Ingestion diff --git a/_observability-plugin/notebooks.md b/_observing-your-data/notebooks.md similarity index 98% rename from _observability-plugin/notebooks.md rename to _observing-your-data/notebooks.md index c4392696cbc..65d503f9ac7 100644 --- a/_observability-plugin/notebooks.md +++ b/_observing-your-data/notebooks.md @@ -2,7 +2,9 @@ layout: default title: Notebooks nav_order: 50 -redirect_from: /notebooks/ +redirect_from: + - /notebooks/ + - /observing-your-data/notebooks/ has_children: false --- diff --git a/_notifications-plugin/api.md b/_observing-your-data/notifications/api.md similarity index 99% rename from _notifications-plugin/api.md rename to _observing-your-data/notifications/api.md index 79a2180cf16..bc55099899a 100644 --- a/_notifications-plugin/api.md +++ b/_observing-your-data/notifications/api.md @@ -2,8 +2,10 @@ layout: default title: API nav_order: 50 -has_children: false +has_children: true +parent: Notifications redirect_from: + - /notifications-plugin/api/ --- # Notifications API diff --git a/_notifications-plugin/index.md b/_observing-your-data/notifications/index.md similarity index 99% rename from _notifications-plugin/index.md rename to _observing-your-data/notifications/index.md index 521d4384419..c04d9787639 100644 --- a/_notifications-plugin/index.md +++ b/_observing-your-data/notifications/index.md @@ -1,10 +1,11 @@ --- layout: default title: Notifications -nav_order: 1 +nav_order: 80 has_children: false redirect_from: - /notifications-plugin/ + - /notifications-plugin/index/ --- # Notifications diff --git a/_observability-plugin/observability-security.md b/_observing-your-data/observability-security.md similarity index 97% rename from _observability-plugin/observability-security.md rename to _observing-your-data/observability-security.md index aacf40c30ba..da3ad613085 100644 --- a/_observability-plugin/observability-security.md +++ b/_observing-your-data/observability-security.md @@ -3,6 +3,8 @@ layout: default title: Observability security nav_order: 5 has_children: false +redirect_from: + - /observing-your-data/security/ --- # Observability security diff --git a/_observability-plugin/operational-panels.md b/_observing-your-data/operational-panels.md similarity index 87% rename from _observability-plugin/operational-panels.md rename to _observing-your-data/operational-panels.md index 8b8db539a49..1b05b9ded8b 100644 --- a/_observability-plugin/operational-panels.md +++ b/_observing-your-data/operational-panels.md @@ -1,7 +1,9 @@ --- layout: default title: Operational panels -nav_order: 30 +nav_order: 60 +redirect_from: + - /observing-your-data/operational-panels/ --- # Operational panels @@ -16,7 +18,7 @@ If you want to start using operational panels without adding any data, expand th To create an operational panel and add visualizations: -1. From the **Add Visualization** dropdown menu, choose **Select Existing Visualization** or **Create New Visualization**, which takes you to the [event analytics]({{site.url}}{{site.baseurl}}/observability-plugin/event-analytics) explorer, where you can use PPL to create visualizations. +1. From the **Add Visualization** dropdown menu, choose **Select Existing Visualization** or **Create New Visualization**, which takes you to the [event analytics]({{site.url}}{{site.baseurl}}/observing-your-data/event-analytics) explorer, where you can use PPL to create visualizations. 1. If you're adding already existing visualizations, choose a visualization from the dropdown menu. 1. Choose **Add**. diff --git a/_observability-plugin/trace/get-started.md b/_observing-your-data/trace/getting-started.md similarity index 93% rename from _observability-plugin/trace/get-started.md rename to _observing-your-data/trace/getting-started.md index 852626a9e83..1e48b739a18 100644 --- a/_observability-plugin/trace/get-started.md +++ b/_observing-your-data/trace/getting-started.md @@ -1,11 +1,13 @@ --- layout: default -title: Get Started +title: Getting Started parent: Trace analytics nav_order: 1 +redirect_from: + - /observability-plugin/trace/get-started/ --- -# Get started with Trace Analytics +# Getting started with Trace Analytics OpenSearch Trace Analytics consists of two components---Data Prepper and the Trace Analytics OpenSearch Dashboards plugin---that fit into the OpenTelemetry and OpenSearch ecosystems. The Data Prepper repository has several [sample applications](https://github.com/opensearch-project/data-prepper/tree/main/examples) to help you get started. @@ -21,7 +23,7 @@ OpenSearch Trace Analytics consists of two components---Data Prepper and the Tra 1. [Data Prepper]({{site.url}}{{site.baseurl}}/clients/data-prepper/index/) processes the OpenTelemetry data, transforms it for use in OpenSearch, and indexes it on an OpenSearch cluster. -1. The [Trace Analytics OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observability-plugin/trace/ta-dashboards/) displays the data in near real-time as a series of charts and tables, with an emphasis on service architecture, latency, error rate, and throughput. +1. The [Trace Analytics OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observing-your-data/trace/ta-dashboards/) displays the data in near real-time as a series of charts and tables, with an emphasis on service architecture, latency, error rate, and throughput. ## Jaeger HotROD @@ -78,4 +80,4 @@ curl -X GET -u 'admin:admin' -k 'https://localhost:9200/otel-v1-apm-span-000001/ Navigate to `http://localhost:5601` in a web browser and choose **Trace Analytics**. You can see the results of your single click in the Jaeger HotROD web interface: the number of traces per API and HTTP method, latency trends, a color-coded map of the service architecture, and a list of trace IDs that you can use to drill down on individual operations. -If you don't see your trace, adjust the timeframe in OpenSearch Dashboards. For more information on using the plugin, see [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observability-plugin/trace/ta-dashboards/). +If you don't see your trace, adjust the timeframe in OpenSearch Dashboards. For more information on using the plugin, see [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observing-your-data/trace/ta-dashboards/). diff --git a/_observability-plugin/trace/index.md b/_observing-your-data/trace/index.md similarity index 94% rename from _observability-plugin/trace/index.md rename to _observing-your-data/trace/index.md index b2466d56c7e..55a83fffcbf 100644 --- a/_observability-plugin/trace/index.md +++ b/_observing-your-data/trace/index.md @@ -1,9 +1,11 @@ --- layout: default title: Trace analytics -nav_order: 60 +nav_order: 40 has_children: true has_toc: false +redirect_from: + - /observablity-plugin/trace/index/ --- # Trace analytics diff --git a/_observability-plugin/trace/ta-dashboards.md b/_observing-your-data/trace/ta-dashboards.md similarity index 96% rename from _observability-plugin/trace/ta-dashboards.md rename to _observing-your-data/trace/ta-dashboards.md index 31bb57ad575..0db398fe028 100644 --- a/_observability-plugin/trace/ta-dashboards.md +++ b/_observing-your-data/trace/ta-dashboards.md @@ -3,6 +3,8 @@ layout: default title: OpenSearch Dashboards plugin parent: Trace analytics nav_order: 50 +redirect_from: + - /observing-your-data/trace/ta-dashboards --- # Trace Analytics OpenSearch Dashboards plugin diff --git a/_observability-plugin/trace/trace-analytics-jaeger.md b/_observing-your-data/trace/trace-analytics-jaeger.md similarity index 99% rename from _observability-plugin/trace/trace-analytics-jaeger.md rename to _observing-your-data/trace/trace-analytics-jaeger.md index df5f63f37cc..f5af3c0503b 100644 --- a/_observability-plugin/trace/trace-analytics-jaeger.md +++ b/_observing-your-data/trace/trace-analytics-jaeger.md @@ -3,6 +3,8 @@ layout: default title: Analyzing Jaeger trace data parent: Trace analytics nav_order: 55 +redirect_from: + - /observability-plugin/trace/trace-analytics-jaeger/ --- # Analyzing Jaeger trace data diff --git a/_opensearch/supported-field-types/alias.md b/_opensearch/supported-field-types/alias.md index 61650763e13..f128ef65704 100644 --- a/_opensearch/supported-field-types/alias.md +++ b/_opensearch/supported-field-types/alias.md @@ -28,6 +28,7 @@ PUT movies } } ``` +{% include copy-curl.html %} ## Parameters @@ -70,6 +71,7 @@ To use an alias in the field capabilities API, specify it in the fields paramete ```json GET movies/_field_caps?fields=release_date ``` +{% include copy-curl.html %} ## Exceptions @@ -87,3 +89,4 @@ In search and field capabilities wildcard queries, both the original field and t ```json GET movies/_field_caps?fields=release* ``` +{% include copy-curl.html %} \ No newline at end of file diff --git a/_opensearch/supported-field-types/binary.md b/_opensearch/supported-field-types/binary.md index 4e2d64ae864..2bc95c31ab1 100644 --- a/_opensearch/supported-field-types/binary.md +++ b/_opensearch/supported-field-types/binary.md @@ -26,6 +26,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index a document with a binary value: @@ -35,6 +36,7 @@ PUT testindex/_doc/1 "binary_value" : "bGlkaHQtd29rfx4=" } ``` +{% include copy-curl.html %} Use `=` as a padding character. Embedded newline characters are not allowed. {: .note } diff --git a/_opensearch/supported-field-types/boolean.md b/_opensearch/supported-field-types/boolean.md index bccae2138f4..40affbde382 100644 --- a/_opensearch/supported-field-types/boolean.md +++ b/_opensearch/supported-field-types/boolean.md @@ -32,6 +32,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index a document with Boolean values: @@ -43,6 +44,7 @@ PUT testindex/_doc/1 "c" : "" } ``` +{% include copy-curl.html %} As a result, `a` and `b` will be set to `true`, and `c` will be set to `false`. @@ -58,6 +60,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} ## Parameters @@ -100,6 +103,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} The script returns the value of `a` as `true`, `key` returns the value of `a` as `1`, and `key_as_string` returns the value of `a` as `"true"`: diff --git a/_opensearch/supported-field-types/completion.md b/_opensearch/supported-field-types/completion.md index 29d239ef307..2f3123af853 100644 --- a/_opensearch/supported-field-types/completion.md +++ b/_opensearch/supported-field-types/completion.md @@ -30,6 +30,7 @@ PUT chess_store } } ``` +{% include copy-curl.html %} Index suggestions into OpenSearch: @@ -42,6 +43,7 @@ PUT chess_store/_doc/1 } } ``` +{% include copy-curl.html %} ## Parameters @@ -73,6 +75,7 @@ PUT chess_store/_doc/2 ] } ``` +{% include copy-curl.html %} As an alternative, you can use the following shorthand notation (note that you cannot provide the `weight` parameter in this notation): @@ -82,6 +85,7 @@ PUT chess_store/_doc/3 "suggestions" : [ "Chess clock", "Chess timer" ] } ``` +{% include copy-curl.html %} ## Querying completion field types @@ -102,6 +106,7 @@ GET chess_store/_search } } ``` +{% include copy-curl.html %} The response contains autocomplete suggestions: @@ -192,6 +197,7 @@ GET chess_store/_search } } ``` +{% include copy-curl.html %} The response contains the suggestions: @@ -297,6 +303,7 @@ GET chess_store/_search } } ``` +{% include copy-curl.html %} To use all default fuzziness options, specify `"fuzzy": {}` or `"fuzzy": true`. {: .tip} @@ -330,6 +337,7 @@ GET chess_store/_search } } ``` +{% include copy-curl.html %} The response matches the string "abcde": diff --git a/_opensearch/supported-field-types/date.md b/_opensearch/supported-field-types/date.md index b093a56a02f..9195fbbb3a9 100644 --- a/_opensearch/supported-field-types/date.md +++ b/_opensearch/supported-field-types/date.md @@ -34,6 +34,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} ## Parameters @@ -168,6 +169,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index a document with a date: @@ -177,6 +179,7 @@ PUT testindex/_doc/21 "release_date" : "03/21/2019" } ``` +{% include copy-curl.html %} When searching for an exact date, provide that date in the same format: @@ -192,6 +195,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} Range queries by default use the field's mapped format. You can also specify the range of dates in a different format by providing the `format` parameter: @@ -209,6 +213,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} ## Date math @@ -263,6 +268,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index two documents into the index: @@ -271,12 +277,16 @@ PUT testindex/_doc/1 { "release_date": "2022-09-14" } +``` +{% include copy-curl.html %} +```json PUT testindex/_doc/2 { "release_date": "2022-11-15" } ``` +{% include copy-curl.html %} The following query searches for documents with `release_date` within 2 months and 1 day of 09/14/2022. The lower boundary of the range is rounded to the beginning of the day on 09/14/2022: @@ -293,6 +303,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} The response contains both documents: diff --git a/_opensearch/supported-field-types/geo-point.md b/_opensearch/supported-field-types/geo-point.md index 99c91c5445d..c686907f92b 100644 --- a/_opensearch/supported-field-types/geo-point.md +++ b/_opensearch/supported-field-types/geo-point.md @@ -27,6 +27,7 @@ PUT testindex1 } } ``` +{% include copy-curl.html %} ## Formats @@ -43,6 +44,7 @@ PUT testindex1/_doc/1 } } ``` +{% include copy-curl.html %} - A string in the "`latitude`,`longitude`" format @@ -52,6 +54,7 @@ PUT testindex1/_doc/2 "point": "40.71,74.00" } ``` +{% include copy-curl.html %} - A geohash @@ -61,6 +64,7 @@ PUT testindex1/_doc/3 "point": "txhxegj0uyp3" } ``` +{% include copy-curl.html %} - An array in the [`longitude`, `latitude`] format @@ -70,6 +74,7 @@ PUT testindex1/_doc/4 "point": [74.00, 40.71] } ``` +{% include copy-curl.html %} - A [Well-Known Text](https://docs.opengeospatial.org/is/12-063r5/12-063r5.html) POINT in the "POINT(`longitude` `latitude`)" format @@ -79,6 +84,7 @@ PUT testindex1/_doc/5 "point": "POINT (74.00 40.71)" } ``` +{% include copy-curl.html %} - GeoJSON format, where the `coordinates` are in the [`longitude`, `latitude`] format @@ -91,6 +97,7 @@ PUT testindex1/_doc/6 } } ``` +{% include copy-curl.html %} ## Parameters diff --git a/_opensearch/supported-field-types/geo-shape.md b/_opensearch/supported-field-types/geo-shape.md index 2da10ff3a5e..e66dfed8f8c 100644 --- a/_opensearch/supported-field-types/geo-shape.md +++ b/_opensearch/supported-field-types/geo-shape.md @@ -27,6 +27,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} ## Formats @@ -68,6 +69,7 @@ PUT testindex/_doc/1 } } ``` +{% include copy-curl.html %} Index a point in WKT format: @@ -77,6 +79,7 @@ PUT testindex/_doc/1 "location" : "POINT (74.0060 40.7128)" } ``` +{% include copy-curl.html %} ## Linestring @@ -93,6 +96,7 @@ PUT testindex/_doc/2 } } ``` +{% include copy-curl.html %} Index a linestring in WKT format: @@ -102,6 +106,7 @@ PUT testindex/_doc/2 "location" : "LINESTRING (74.0060 40.7128, 71.0589 42.3601)" } ``` +{% include copy-curl.html %} ## Polygon @@ -126,6 +131,7 @@ PUT testindex/_doc/3 } } ``` +{% include copy-curl.html %} Index a polygon (triangle) in WKT format: @@ -135,6 +141,7 @@ PUT testindex/_doc/3 "location" : "POLYGON ((74.0060 40.7128, 71.0589 42.3601, 73.7562 42.6526, 74.0060 40.7128))" } ``` +{% include copy-curl.html %} The polygon may have holes inside. In this case, the `coordinates` field will contain multiple arrays. The first array represents the outer polygon, and each subsequent array represents a hole. Holes are represented as polygons and specified as arrays of coordinates. @@ -162,6 +169,7 @@ PUT testindex/_doc/4 } } ``` +{% include copy-curl.html %} Index a polygon (triangle) with a triangular hole in WKT format: @@ -171,6 +179,7 @@ PUT testindex/_doc/4 "location" : "POLYGON ((40.7128 74.0060, 42.3601 71.0589, 42.6526 73.7562, 40.7128 74.0060), (41.7658 72.6734, 41.5623 72.6506, 41.5582 73.0515, 41.7658 72.6734))" } ``` +{% include copy-curl.html %} In OpenSearch, you can specify a polygon by listing its vertices clockwise or counterclockwise. This works well for polygons that do not cross the date line (are narrower than 180°). However, a polygon that crosses the date line (is wider than 180°) might be ambiguous because WKT does not impose a specific order on vertices. Thus, you must specify polygons that cross the date line by listing their vertices counterclockwise. @@ -189,6 +198,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Subsequently indexed documents can override the `orientation` setting: @@ -207,6 +217,7 @@ PUT testindex/_doc/3 } } ``` +{% include copy-curl.html %} ## Multipoint @@ -226,6 +237,7 @@ PUT testindex/_doc/6 } } ``` +{% include copy-curl.html %} Index a multipoint in WKT format: @@ -235,6 +247,7 @@ PUT testindex/_doc/6 "location" : "MULTIPOINT (74.0060 40.7128, 71.0589 42.3601)" } ``` +{% include copy-curl.html %} ## Multilinestring @@ -254,6 +267,7 @@ PUT testindex/_doc/2 } } ``` +{% include copy-curl.html %} Index a linestring in WKT format: @@ -263,6 +277,7 @@ PUT testindex/_doc/2 "location" : "MULTILINESTRING ((74.0060 40.7128, 71.0589 42.3601), (73.7562 42.6526, 72.6734 41.7658))" } ``` +{% include copy-curl.html %} ## Multipolygon @@ -297,6 +312,7 @@ PUT testindex/_doc/4 } } ``` +{% include copy-curl.html %} Index a multipolygon in WKT format: @@ -306,8 +322,7 @@ PUT testindex/_doc/4 "location" : "MULTIPOLYGON (((40.7128 74.0060, 42.3601 71.0589, 42.6526 73.7562, 40.7128 74.0060), (41.7658 72.6734, 41.5623 72.6506, 41.5582 73.0515, 41.7658 72.6734)), ((73.9776 40.7614, 73.9554 40.7827, 73.9631 40.7812, 73.9776 40.7614)))" } ``` - - +{% include copy-curl.html %} ## Geometry collection @@ -333,6 +348,7 @@ PUT testindex/_doc/7 } } ``` +{% include copy-curl.html %} Index a geometry collection in WKT format: @@ -342,6 +358,7 @@ PUT testindex/_doc/7 "location" : "GEOMETRYCOLLECTION (POINT (74.0060 40.7128), LINESTRING(73.7562 42.6526, 72.6734 41.7658))" } ``` +{% include copy-curl.html %} ## Envelope @@ -358,6 +375,7 @@ PUT testindex/_doc/2 } } ``` +{% include copy-curl.html %} In WKT format, use `BBOX (minLon, maxLon, maxLat, minLat)`. @@ -369,6 +387,7 @@ PUT testindex/_doc/8 "location" : "BBOX (71.0589, 74.0060, 42.3601, 40.7128)" } ``` +{% include copy-curl.html %} ## Parameters diff --git a/_opensearch/supported-field-types/ip.md b/_opensearch/supported-field-types/ip.md index 00ef322e843..250e2da6a0c 100644 --- a/_opensearch/supported-field-types/ip.md +++ b/_opensearch/supported-field-types/ip.md @@ -29,6 +29,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index a document with an IP address: @@ -38,6 +39,7 @@ PUT testindex/_doc/1 "ip_address" : "10.24.34.0" } ``` +{% include copy-curl.html %} Query an index for a specific IP address: @@ -51,6 +53,7 @@ GET testindex/_doc/1 } } ``` +{% include copy-curl.html %} ## Searching for an IP address and its associated network mask @@ -68,6 +71,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} #### Example query in IPv6 format @@ -81,6 +85,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} If you use an IP address in IPv6 format in a `query_string` query, you need to escape `:` characters because they are parsed as special characters. You can accomplish this by wrapping the IP address in quotation marks and escaping those quotation marks with `\`. @@ -94,6 +99,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} ## Parameters diff --git a/_opensearch/supported-field-types/join.md b/_opensearch/supported-field-types/join.md index 173e29f0196..18a25637dbb 100644 --- a/_opensearch/supported-field-types/join.md +++ b/_opensearch/supported-field-types/join.md @@ -30,6 +30,7 @@ PUT testindex1 } } ``` +{% include copy-curl.html %} Then, index a parent document with a join field type: @@ -42,6 +43,7 @@ PUT testindex1/_doc/1 } } ``` +{% include copy-curl.html %} You can also use a shortcut without object notation to index a parent document: @@ -52,6 +54,7 @@ PUT testindex1/_doc/1 "product_to_brand" : "brand" } ``` +{% include copy-curl.html %} When indexing child documents, you have to specify the `routing` query parameter because parent and child documents in the same relation have to be indexed on the same shard. Each child document refers to its parent's ID in the `parent` field. @@ -66,7 +69,10 @@ PUT testindex1/_doc/3?routing=1 "parent": "1" } } +``` +{% include copy-curl.html %} +```json PUT testindex1/_doc/4?routing=1 { "name": "Product 2", @@ -76,6 +82,7 @@ PUT testindex1/_doc/4?routing=1 } } ``` +{% include copy-curl.html %} ## Querying a join field @@ -91,6 +98,7 @@ GET testindex1/_search } } ``` +{% include copy-curl.html %} The response indicates whether a document is a parent or a child: @@ -175,6 +183,7 @@ GET testindex1/_search } } ``` +{% include copy-curl.html %} The response contains Product 1 and Product 2, which are associated with Brand 1: @@ -247,6 +256,7 @@ GET testindex1/_search } } ``` +{% include copy-curl.html %} The response returns Brand 1 as Product 1's parent: @@ -303,6 +313,7 @@ PUT testindex1 } } ``` +{% include copy-curl.html %} ## Join field type notes diff --git a/_opensearch/supported-field-types/keyword.md b/_opensearch/supported-field-types/keyword.md index 8374b3501ca..cfa522f5993 100644 --- a/_opensearch/supported-field-types/keyword.md +++ b/_opensearch/supported-field-types/keyword.md @@ -30,6 +30,7 @@ PUT movies } } ``` +{% include copy-curl.html %} ## Parameters diff --git a/_opensearch/supported-field-types/nested.md b/_opensearch/supported-field-types/nested.md index 84f29e61c2d..bb27cd28ee8 100644 --- a/_opensearch/supported-field-types/nested.md +++ b/_opensearch/supported-field-types/nested.md @@ -26,6 +26,7 @@ PUT testindex1/_doc/100 ] } ``` +{% include copy-curl.html %} When these objects are stored, they are flattened, so their internal representation has an array of all values for each field: @@ -62,6 +63,7 @@ GET testindex1/_search } } ``` +{% include copy-curl.html %} The query correctly returns document 100: @@ -132,6 +134,7 @@ GET testindex1/_search } } ``` +{% include copy-curl.html %} However, this query still incorrectly returns document 100. This is because the relation between age and smoking was lost when arrays of values for individual fields were created. @@ -151,6 +154,7 @@ PUT testindex1 } } ``` +{% include copy-curl.html %} Then, index a document with a nested field type: @@ -163,6 +167,7 @@ PUT testindex1/_doc/100 ] } ``` +{% include copy-curl.html %} Now if you run the same query to search for patients older than 75 AND smokers, nothing is returned, which is correct. diff --git a/_opensearch/supported-field-types/numeric.md b/_opensearch/supported-field-types/numeric.md index 882d68dfabc..c12326e1723 100644 --- a/_opensearch/supported-field-types/numeric.md +++ b/_opensearch/supported-field-types/numeric.md @@ -45,6 +45,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index a document with an integer value: @@ -54,6 +55,7 @@ PUT testindex/_doc/1 "integer_value" : 123 } ``` +{% include copy-curl.html %} ## Scaled float field type @@ -79,6 +81,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index a document with a scaled_float value: @@ -88,6 +91,7 @@ PUT testindex/_doc/1 "scaled" : 2.3 } ``` +{% include copy-curl.html %} The `scaled` value will be stored as 23. diff --git a/_opensearch/supported-field-types/object.md b/_opensearch/supported-field-types/object.md index e4283d3e25b..68cb15cb83c 100644 --- a/_opensearch/supported-field-types/object.md +++ b/_opensearch/supported-field-types/object.md @@ -33,6 +33,7 @@ PUT testindex1/_mappings } } ``` +{% include copy-curl.html %} Index a document with an object field: @@ -45,6 +46,7 @@ PUT testindex1/_doc/1 } } ``` +{% include copy-curl.html %} Nested objects are stored as flat key/value pairs internally. To refer to a field in a nested object, use `parent field`.`child field` (for example, `patient.id`). @@ -60,6 +62,7 @@ GET testindex1/_search } } ``` +{% include copy-curl.html %} ## Parameters @@ -92,6 +95,7 @@ PUT testindex1/_mappings } } ``` +{% include copy-curl.html %} Then you index a document with a new `id` field in `patient`: @@ -104,6 +108,7 @@ PUT testindex1/_doc/1 } } ``` +{% include copy-curl.html %} As a result, the field `id` is added to the mappings: diff --git a/_opensearch/supported-field-types/percolator.md b/_opensearch/supported-field-types/percolator.md index 0dc498b0df1..393c9e0aaa9 100644 --- a/_opensearch/supported-field-types/percolator.md +++ b/_opensearch/supported-field-types/percolator.md @@ -38,6 +38,7 @@ PUT testindex1 } } ``` +{% include copy-curl.html %} Index a query: @@ -68,6 +69,7 @@ PUT testindex1/_doc/1 } } ``` +{% include copy-curl.html %} Fields referenced in the query must already exist in the mapping. {: .note } @@ -93,6 +95,7 @@ GET testindex1/_search } } ``` +{% include copy-curl.html %} The response contains the originally indexed query: diff --git a/_opensearch/supported-field-types/range.md b/_opensearch/supported-field-types/range.md index 83ba1f04adc..218b4f88002 100644 --- a/_opensearch/supported-field-types/range.md +++ b/_opensearch/supported-field-types/range.md @@ -39,6 +39,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index a document with a double range and a date range: @@ -55,6 +56,7 @@ PUT testindex/_doc/1 } } ``` +{% include copy-curl.html %} You can use a [Term query](#term-query) or a [Range query](#range-query) to search for values within range fields. @@ -76,6 +78,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} ### Range query @@ -108,6 +111,7 @@ GET testindex1/_search } } ``` +{% include copy-curl.html %} The above query will return document 1 for the `within` and `intersects` relations but will not return it for the `contains` relation. @@ -132,6 +136,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index a document with IP address ranges in both formats: @@ -145,6 +150,7 @@ PUT testindex/_doc/2 "ip_address_cidr" : "10.24.34.0/24" } ``` +{% include copy-curl.html %} ## Parameters diff --git a/_opensearch/supported-field-types/rank.md b/_opensearch/supported-field-types/rank.md index eaa8728e9d1..50aab1bc64b 100644 --- a/_opensearch/supported-field-types/rank.md +++ b/_opensearch/supported-field-types/rank.md @@ -45,6 +45,7 @@ PUT chessplayers } } ``` +{% include copy-curl.html %} Index three documents with a rank_feature field that boosts the score (`rating`) and a rank_feature field that decreases the score (`age`): @@ -55,14 +56,20 @@ PUT testindex1/_doc/1 "rating" : 2554, "age" : 75 } +``` +{% include copy-curl.html %} +```json PUT testindex1/_doc/2 { "name" : "Kwaku Mensah", "rating" : 2067, "age": 10 } +``` +{% include copy-curl.html %} +```json PUT testindex1/_doc/3 { "name" : "Nikki Wolf", @@ -70,6 +77,7 @@ PUT testindex1/_doc/3 "age" : 22 } ``` +{% include copy-curl.html %} ## Rank feature query @@ -98,6 +106,7 @@ GET chessplayers/_search } } ``` +{% include copy-curl.html %} When ranked by both age and rating, younger players and players who are more highly ranked score better: @@ -176,6 +185,7 @@ PUT testindex1 } } ``` +{% include copy-curl.html %} To index a document with a rank features field, use a hashmap with string keys and positive float values: @@ -188,7 +198,10 @@ PUT testindex1/_doc/1 "teens" : 25.9 } } +``` +{% include copy-curl.html %} +```json PUT testindex1/_doc/2 { "correlations": { @@ -197,6 +210,7 @@ PUT testindex1/_doc/2 } } ``` +{% include copy-curl.html %} Query the documents using a rank feature query: @@ -210,6 +224,7 @@ GET testindex1/_search } } ``` +{% include copy-curl.html %} The response is ranked by relevance score: diff --git a/_opensearch/supported-field-types/search-as-you-type.md b/_opensearch/supported-field-types/search-as-you-type.md index 17bd0915025..eb4d863ef02 100644 --- a/_opensearch/supported-field-types/search-as-you-type.md +++ b/_opensearch/supported-field-types/search-as-you-type.md @@ -29,6 +29,7 @@ PUT books } } ``` +{% include copy-curl.html %} In addition to the `suggestions` field, this creates `suggestions._2gram`, `suggestions._3gram`, and `suggestions._index_prefix` fields. @@ -40,6 +41,7 @@ PUT books/_doc/1 "suggestions": "one two three four" } ``` +{% include copy-curl.html %} To match terms in any order, use a bool_prefix or multi-match query. These queries rank the documents in which search terms are in the specified order higher than the documents in which terms are out of order. @@ -59,6 +61,7 @@ GET books/_search } } ``` +{% include copy-curl.html %} The response contains the matching document: @@ -105,6 +108,7 @@ GET books/_search } } ``` +{% include copy-curl.html %} The response contains the matching document: @@ -151,6 +155,7 @@ GET books/_search } } ``` +{% include copy-curl.html %} Response: diff --git a/_opensearch/supported-field-types/text.md b/_opensearch/supported-field-types/text.md index f2db314ac5e..37f3a6c07b0 100644 --- a/_opensearch/supported-field-types/text.md +++ b/_opensearch/supported-field-types/text.md @@ -31,6 +31,7 @@ PUT movies } } ``` +{% include copy-curl.html %} ## Parameters @@ -94,6 +95,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index a document with a text field: @@ -103,6 +105,7 @@ PUT testindex/_doc/1 "dob" : "The patient's date of birth." } ``` +{% include copy-curl.html %} Query for "date of birth" and highlight it in the original field: @@ -121,6 +124,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} The words "date of birth" are highlighted in the response: diff --git a/_opensearch/supported-field-types/token-count.md b/_opensearch/supported-field-types/token-count.md index 06e437cda09..c1795af3a46 100644 --- a/_opensearch/supported-field-types/token-count.md +++ b/_opensearch/supported-field-types/token-count.md @@ -33,19 +33,27 @@ PUT testindex } } ``` +{% include copy-curl.html %} Index three documents with text fields: ```json PUT testindex/_doc/1 { "sentence": "To be, or not to be: that is the question." } +``` +{% include copy-curl.html %} +```json PUT testindex/_doc/2 { "sentence": "All the world’s a stage, and all the men and women are merely players." } +``` +{% include copy-curl.html %} +```json PUT testindex/_doc/3 { "sentence": "Now is the winter of our discontent." } ``` +{% include copy-curl.html %} Search for sentences with fewer than 10 words: @@ -61,6 +69,7 @@ GET testindex/_search } } ``` +{% include copy-curl.html %} The response contains one matching sentence: diff --git a/_opensearch/supported-field-types/xy-point.md b/_opensearch/supported-field-types/xy-point.md index bcafa2c31f9..8909c43bc28 100644 --- a/_opensearch/supported-field-types/xy-point.md +++ b/_opensearch/supported-field-types/xy-point.md @@ -27,6 +27,7 @@ PUT testindex1 } } ``` +{% include copy-curl.html %} ## Formats @@ -43,6 +44,7 @@ PUT testindex1/_doc/1 } } ``` +{% include copy-curl.html %} - A string in the "`x`, `y`" format @@ -52,6 +54,7 @@ PUT testindex1/_doc/2 "point": "0.5, 4.5" } ``` +{% include copy-curl.html %} - An array in the [`x`, `y`] format @@ -61,6 +64,7 @@ PUT testindex1/_doc/3 "point": [0.5, 4.5] } ``` +{% include copy-curl.html %} - A [well-known text (WKT)](https://docs.opengeospatial.org/is/12-063r5/12-063r5.html) POINT in the "POINT(`x` `y`)" format @@ -70,6 +74,7 @@ PUT testindex1/_doc/4 "point": "POINT (0.5 4.5)" } ``` +{% include copy-curl.html %} - GeoJSON format @@ -82,6 +87,7 @@ PUT testindex1/_doc/5 } } ``` +{% include copy-curl.html %} In all xy point formats, the coordinates must be specified in the `x, y` order. {: .note} diff --git a/_opensearch/supported-field-types/xy-shape.md b/_opensearch/supported-field-types/xy-shape.md index dfb4ba09607..07cc85866b2 100644 --- a/_opensearch/supported-field-types/xy-shape.md +++ b/_opensearch/supported-field-types/xy-shape.md @@ -29,6 +29,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} ## Formats @@ -70,6 +71,7 @@ PUT testindex/_doc/1 } } ``` +{% include copy-curl.html %} Index a point in WKT format: @@ -79,6 +81,7 @@ PUT testindex/_doc/1 "location" : "POINT (0.5 4.5)" } ``` +{% include copy-curl.html %} ## Linestring @@ -95,6 +98,7 @@ PUT testindex/_doc/2 } } ``` +{% include copy-curl.html %} Index a linestring in WKT format: @@ -104,6 +108,7 @@ PUT testindex/_doc/2 "location" : "LINESTRING (0.5 4.5, -1.5 2.3)" } ``` +{% include copy-curl.html %} ## Polygon @@ -128,6 +133,7 @@ PUT testindex/_doc/3 } } ``` +{% include copy-curl.html %} Index a polygon (triangle) in WKT format: @@ -137,6 +143,7 @@ PUT testindex/_doc/3 "location" : "POLYGON ((0.5 4.5, 2.5 6.0, 1.5 2.0, 0.5 4.5))" } ``` +{% include copy-curl.html %} The polygon may have holes inside. In this case, the `coordinates` field will contain multiple arrays. The first array represents the outer polygon, and each subsequent array represents a hole. Holes are represented as polygons and specified as arrays of coordinates. @@ -164,6 +171,7 @@ PUT testindex/_doc/4 } } ``` +{% include copy-curl.html %} Index a polygon (triangle) with a triangular hole in WKT format: @@ -173,6 +181,8 @@ PUT testindex/_doc/4 "location" : "POLYGON ((0.5 4.5, 2.5 6.0, 1.5 2.0, 0.5 4.5), (1.0 4.5, 1.5 4.5, 1.5 4.0, 1.0 4.5))" } ``` +{% include copy-curl.html %} + By default, the vertices of the polygon are traversed in a counterclockwise order. You can define an [`orientation`](#parameters) parameter to specify the vertex traversal order at mapping time: ```json @@ -188,6 +198,7 @@ PUT testindex } } ``` +{% include copy-curl.html %} Subsequently indexed documents can override the `orientation` setting: @@ -206,6 +217,7 @@ PUT testindex/_doc/3 } } ``` +{% include copy-curl.html %} ## Multipoint @@ -225,6 +237,7 @@ PUT testindex/_doc/6 } } ``` +{% include copy-curl.html %} Index a multipoint in WKT format: @@ -234,6 +247,7 @@ PUT testindex/_doc/6 "location" : "MULTIPOINT (0.5 4.5, 2.5 6.0)" } ``` +{% include copy-curl.html %} ## Multilinestring @@ -253,6 +267,7 @@ PUT testindex/_doc/2 } } ``` +{% include copy-curl.html %} Index a linestring in WKT format: @@ -262,6 +277,7 @@ PUT testindex/_doc/2 "location" : "MULTILINESTRING ((0.5 4.5, 2.5 6.0), (1.5 2.0, 3.5 3.5))" } ``` +{% include copy-curl.html %} ## Multipolygon @@ -296,6 +312,7 @@ PUT testindex/_doc/4 } } ``` +{% include copy-curl.html %} Index a multipolygon in WKT format: @@ -305,6 +322,7 @@ PUT testindex/_doc/4 "location" : "MULTIPOLYGON (((0.5 4.5, 2.5 6.0, 1.5 2.0, 0.5 4.5), (1.0 4.5, 1.5 4.5, 1.5 4.0, 1.0 4.5)), ((2.0 0.0, 1.0 2.0, 3.0 1.0, 2.0 0.0)))" } ``` +{% include copy-curl.html %} ## Geometry collection @@ -330,6 +348,7 @@ PUT testindex/_doc/7 } } ``` +{% include copy-curl.html %} Index a geometry collection in WKT format: @@ -339,6 +358,7 @@ PUT testindex/_doc/7 "location" : "GEOMETRYCOLLECTION (POINT (0.5 4.5), LINESTRING(2.5 6.0, 1.5 2.0))" } ``` +{% include copy-curl.html %} ## Envelope @@ -355,6 +375,7 @@ PUT testindex/_doc/2 } } ``` +{% include copy-curl.html %} In WKT format, use `BBOX (minX, maxY, maxX, minY)`. @@ -366,6 +387,7 @@ PUT testindex/_doc/8 "location" : "BBOX (3.0, 2.0, 6.0, 0.0)" } ``` +{% include copy-curl.html %} ## Parameters diff --git a/_search-plugins/sql/ppl/index.md b/_search-plugins/sql/ppl/index.md index de6ee0f84ed..aca05b4b22d 100644 --- a/_search-plugins/sql/ppl/index.md +++ b/_search-plugins/sql/ppl/index.md @@ -8,7 +8,7 @@ has_toc: false redirect_from: - /search-plugins/sql/ppl - /search-plugins/ppl - - /observability-plugin/ppl + - /observing-your-data/ppl --- # PPL – Piped Processing Language diff --git a/_security-analytics/index.md b/_security-analytics/index.md index da10abce7fc..6fef848df4b 100644 --- a/_security-analytics/index.md +++ b/_security-analytics/index.md @@ -17,7 +17,7 @@ Security Analytics is a security information and event management (SIEM) solutio As part of the OpenSearch Project, Security Analytics exists in the open source community and benefits from the feedback and contributions of that community. To learn more about proposals for its development, options for making contributions, and general information on the platform, see the [Security Analytics repository](https://github.com/opensearch-project/security-analytics) at GitHub. -If you would like to leave feedback that could help improve Security Analytics, join the discussion on the [OpenSearch forum](https://forum.opensearch.org/t/feedback-experimental-feature-security-analytics/11418). +If you would like to leave feedback that could help improve Security Analytics, join the discussion on the [OpenSearch forum](https://forum.opensearch.org/c/plugins/security-analytics/73). ## Components and concepts @@ -45,7 +45,7 @@ Log types are specified during the creation of detectors, including steps for ma ### Rules -Rules, or threat detection rules, define the conditional logic applied to ingested log data that allows the system to identify an event of interest. Security Analytics uses pre-packaged, open source [Sigma rules](https://github.com/SigmaHQ/sigma) as a starting point for describing relevant log events. But with their inherently flexible format and easy portability, Sigma rules provide users of Security Analytics with options for importing and customizing the rules. You can take advantage of these options using either Dashboards or the API. +Rules, or threat detection rules, define the conditional logic applied to ingested log data that allows the system to identify an event of interest. Security Analytics uses prepackaged, open source [Sigma rules](https://github.com/SigmaHQ/sigma) as a starting point for describing relevant log events. But with their inherently flexible format and easy portability, Sigma rules provide users of Security Analytics with options for importing and customizing the rules. You can take advantage of these options using either Dashboards or the API. For information on configuring rules, see [Working with rules]({{site.url}}{{site.baseurl}}/security-analytics/usage/rules/). diff --git a/_security-analytics/sec-analytics-config/detectors-config.md b/_security-analytics/sec-analytics-config/detectors-config.md index 1d464a3cd55..743cb9c7f66 100644 --- a/_security-analytics/sec-analytics-config/detectors-config.md +++ b/_security-analytics/sec-analytics-config/detectors-config.md @@ -9,30 +9,30 @@ nav_order: 15 Security Analytics provides the options and functionality to monitor and respond to a wide range of security threats. Detectors are the essential components that determine what to look for and how to respond to those threats. This section covers their creation and configuration. -## Step 1. Define the detector +## Step 1. Define a detector -Defining a new detector involves naming the detector, selecting a data source and detector type, and specifying a detector schedule. You can also create alerts for the detector at this stage, although there are options to create alerts in other areas of the interface. Follow the steps in this section to define a new detector. +You can define a new detector by naming the detector, selecting a data source and detector type, and specifying a detector schedule. After defining a detector, you can also configure field mappings and set up alerts. Follow the steps in this section to accomplish all three of these setup tasks. 1. On the Detectors page, select the **Create detector** button. The Define detector page opens. 1. Give the detector a name and, as an option, add a description for the detector. 1. In the Data source section, select the dropdown arrow and select a source for the log data. 1. In the threat detection type section, select the data type. The Sigma security rules associated with the log data are automatically populated in the Detection rules section below it. -Selecting threat detector type to auto-populate rules +Selecting threat detector type to auto-populate rules You can skip the next step for mapping rules if you are satisfied with those automatically populated by the system. Otherwise, go to the next step to specify select rules. {: .note } 1. In the **Detection rules** section, specify only those rules you want mapped to the detector. -Select or deselect rules that detector will use for findings +Select or deselect rules that detector will use for findings * Use the toggle to the left of the rule name to select or deselect rules. -* Use the **Log type**, **Rule severity**, and **Source** dropdown menus to filter the rules you want to select from. +* Use the **Log type**, **Rule severity**, and **Source** dropdown lists to filter the rules you want to select from. * Use the **Search** bar to search for specific rules. To quickly select one or more known rules and dismiss others, first deselect all rules by moving the **Rule name** toggle to the left, then search for your target rule names and select each individually by moving its toggle to the right. {: .tip } 1. In the **Detector schedule** section, set how often the detector will run. Specify a unit of time and a corresponding number to set the interval. -1. Select the **Next** button in the lower-right corner of the screen to continue. The Configure field mapping page appears. +1. Select the **Next** button in the lower-right corner of the screen to continue. The **Configure field mapping** page appears. ## Step 2. Create field mappings @@ -40,6 +40,8 @@ The field mapping step matches field names from the rule with field names from t The data source (log index), log type, and detection rules specified in the first step determine which fields are available for mapping. For example, when "Windows logs" is selected as the log type, this parameter, along with the specific detection rules, determines the list of rule field names available for the mapping. Similarly, the selected data source (log index) determines the list of log field names that are available for the mapping. +Because the system uses prepackaged Sigma rules for detector creation, it can automatically map important fields for a specific log type with the corresponding fields in the Sigma rules. The field mapping step presents a view of automatically mapped fields while also providing the option to customize, change, or add new field mappings. When a detector includes custom rules, you can follow this step to manually map rule field names to log field names. + #### A note on field names The field mapping process requires that you are familiar with the field names in the log index and have an understanding of the data contained in those fields. If you have an understanding of the log fields in the index, the mapping is typically a straightforward process. @@ -48,9 +50,16 @@ Security Analytics takes advantage of prepackaged Sigma rules for security event Although the ECS rule field names are largely self-explanatory, you can find predefined mappings of the Sigma rule field names with ECS rule field names, for all supported log types, in the GitHub Security Analytics repository. Navigate to the [OSMappings](https://github.com/opensearch-project/security-analytics/tree/main/src/main/resources/OSMapping) folder, select the folder named for the log type, and open the `fieldmappings.yml` file. For example, to see the Sigma rule fields that correspond to ECS rule fields for the Windows log type, open the [fieldmappings.yml file](https://github.com/opensearch-project/security-analytics/blob/main/src/main/resources/OSMapping/windows/fieldmappings.yml) in the **windows** folder. +### Default field mappings + +Once you navigate to the **Configure field mapping** page, the system attempts to automatically map fields between the two sources. The **Default mapped fields** table contains mappings that the system created automatically after defining the detector. As shown in the image that follows, when the field names are similar to one another the system can successfully match the two. +
Field mapping example for pending mappings + +Although these automatic matches are normally dependable, it's still a good idea to review the mappings in the **Default mapped fields** table and verify that they are correct and matched as expected. If you find a mapping that doesn't appear to be accurate, you can use the dropdown list as described in the [Pending field mappings](#pending-field-mappings) section that follows to correct the field mapping. + ### Pending field mappings -Once you navigate to the **Configure field mapping** page, the system attempts to automatically map fields between the two sources. Those field names that are not automatically mapped appear in the **Pending field mapping** table. In this table you can manually map rule fields to log fields, as shown in the following image. +The field names that are not automatically mapped appear in the **Pending field mappings** table. In this table you can manually map rule fields to log fields, as shown in the following image.
Field mapping example for pending mappings While mapping fields, consider the following: @@ -59,35 +68,30 @@ While mapping fields, consider the following: * To map a rule field name to a log field name, use the dropdown arrow to open the list of log fields and select the log field name from the list. To search for names in the log field list, enter text in the **Select a mapping field** box.
Field mapping example for pending mappings * Once the log field name is selected and mapped to the rule field name, the icon in the Status column to the right changes to a green check mark. -* Make as many matches between field names as possible to complete an accurate mapping for rule and log fields. - -### Default field mappings - -The **Default mapped fields** table contains mappings that the system created automatically after defining the detector. As shown in the image that follows, when the field names are similar to one another the system can successfully match the two. -
Field mapping example for pending mappings - -Although these automatic matches are normally dependable, it's still a good idea to review the mappings and verify that they are correct and matched as expected. If you find a mapping that doesn't appear to be accurate, you can use the dropdown list as described in the [Pending field mappings](#pending-field-mappings) section above to correct the field mapping. +* Make as many matches between field names as possible to complete an accurate mapping for rule and log fields. After completing the mappings, select the **Next** button in the lower-right corner of the screen. The **Set up alerts** page appears and displays settings for an alert trigger. ## Step 3. Set up alerts -At this stage, setting up alerts is optional for creating a new detector. Alerts can be configured at any time, including from the Findings window. This section describes the process for defining the alert conditions during creation of a detector. To see how to initiate creation of alerts from the Findings window, see [The findings list]({{site.url}}{{site.baseurl}}/security-analytics/usage/findings/#the-findings-list). +The third step in creating a detector involves setting up alerts. Alerts are configured to create triggers that, when matched with a set of detection rule criteria, send a notification of a possible security event. You can select rule names, rule severity, and tags in any combination to define a trigger. Once a trigger is defined, the alert setup lets you choose the channel on which to be notified and provides options for customizing a message for the notification. + +At least one alert condition is required before a detector can begin generating findings. +{: .note } -To skip directly to generating findings from the detector, select the **Remove alert trigger** button and then the **Next** button in the lower-right corner of the screen. Review the detector's definition and then select the **Create** button in the lower-right corner of the screen. The detector is created. -{: .tip } +You can also configure alerts from the **Findings** window. To see how to set up alerts from the **Findings** window, see [The findings list]({{site.url}}{{site.baseurl}}/security-analytics/usage/findings/#the-findings-list). A final option for adding additional alerts is to edit a detector and navigate to the **Alert triggers** tab, where you can edit existing alerts as well as add new ones. For details, see [Editing a detector]({{site.url}}{{site.baseurl}}security-analytics/usage/detectors/#editing-a-detector). -To set up an alert for the detector at this stage of detector creation, continue with the following steps: +To set up an alert for a detector, continue with the following steps: 1. In the **Trigger name** box, enter a name for the trigger. 1. To define rule matches for the alert, select security rules, severity levels, and tags. -Rules used to define an alert -* Select one rule or multiple rules that will trigger the alert. Put the cursor in the **Rule names** box and type a name to search for it. To remove a rule name, select the **X** beside the name. To remove all rule names, select the **X** beside the dropdown menu's down arrow. -
Deletes all selected rules +
Rules used to define an alert +* Select one rule or multiple rules that will trigger the alert. Put the cursor in the **Rule names** box and type a name to search for it. To remove a rule name, select the **X** beside the name. To remove all rule names, select the **X** beside the dropdown list's down arrow. +
Deletes all selected rules * Select one or more rule severities as conditions for the alert. * Select from a list of tags to include as conditions for the alert. 1. To define a notification for the alert, assign an alert severity, select a channel for the notification, and customize a message generated for the alert. -Notification settings for the alert +
Notification settings for the alert * Assign a level of severity for the alert to give the recipient an indication of its urgency. * Select a channel for the notification. Examples include Slack, Chime, or email. Select the **Manage channels** link to the right of the field to link the notification to a preferred channel. * Select the **Show notify message** label to expand message preferences. You can add a subject for the message and a note to inform recipients of the nature of the message. diff --git a/_security-analytics/sec-analytics-config/index.md b/_security-analytics/sec-analytics-config/index.md index 7d63010d129..1925a97262b 100644 --- a/_security-analytics/sec-analytics-config/index.md +++ b/_security-analytics/sec-analytics-config/index.md @@ -21,4 +21,4 @@ Before Security Analytics can begin generating findings and sending alerts, admi 1. To get started, select the top menu on the Dashboards home page and then select **Security Analytics**. The Overview page for Security Analytics is displayed. 1. From the options on the left side of the page, select **Detectors** to begin creating a detector. -Navigating to create a detector page +Navigating to create a detector page diff --git a/_security-analytics/usage/alerts.md b/_security-analytics/usage/alerts.md index b21e823366f..1d2e24e34e0 100644 --- a/_security-analytics/usage/alerts.md +++ b/_security-analytics/usage/alerts.md @@ -15,21 +15,21 @@ You can select the **Refresh** button at any time to refresh information on the ## The Alerts graph -The Alerts graph can display alerts by their status or severity. Use the **Group by** dropdown menu to specify either Alert status or Alert severity. +The Alerts graph can display alerts by their status or severity. Use the **Group by** dropdown list to specify either Alert status or Alert severity. To specify the date range you would like the graph to display, first select the calendar dropdown arrow. The date selector window opens. -Date selector for findings graph +
Date selector for findings graph You can use the **Quick select** settings to specify an exact window of time. -* Select either **Last** or **Next** in the first dropdown menu to set the window of time behind the current setting or ahead of the current setting. -* Select a number in the second dropdown menu to define a value for the range. -* Select a unit of time in the third dropdown menu. Available options are seconds, minutes, hours, days, weeks, months, and years. +* Select either **Last** or **Next** in the first dropdown list to set the window of time behind the current setting or ahead of the current setting. +* Select a number in the second dropdown list to define a value for the range. +* Select a unit of time in the third dropdown list. Available options are seconds, minutes, hours, days, weeks, months, and years. Select the **Apply** button to apply the range of dates to the graph. Information on the graph changes accordingly. -Quick select settings example +
Quick select settings example
You can use the left and right arrows to move the window of time behind the current range of dates or ahead of the current range of dates. When you use these arrows, the start date and end date appear in the date range field. You can then select each one to set an absolute, relative, or current date and time. For absolute and relative changes, select the **Update** button to apply the changes. -Altering date range +
Altering date range -As an alternative, you can select an option in the **Commonly used** section (see the preceding image of the calendar dropdown menu) to conveniently set a window of time. Options include date ranges such as **Today**, **Yesterday**, **this week**, and **week to date**. +As an alternative, you can select an option in the **Commonly used** section (see the preceding image of the calendar dropdown list) to conveniently set a window of time. Options include date ranges such as **Today**, **Yesterday**, **this week**, and **week to date**. When one of the commonly used windows of time is selected, you can select the **Show dates** label in the date range field to populate the range of dates. Following that, you can select either the start date or end date to specify by an absolute, relative, or current date and time setting. For absolute and relative changes, select the **Update** button to apply the changes. @@ -38,5 +38,5 @@ As one more alternative, you can select an option from the **Recently used date ## The Alerts list The Alerts list displays all findings according to the time when the alert was triggered, the alert's trigger name, the detector that triggered the alert, the alert status, and alert severity. -Use the **Alert severity** dropdown menu to filter the list of alerts by severity. Use the **Status** dropdown menu to filter the list by alert status. +Use the **Alert severity** dropdown list to filter the list of alerts by severity. Use the **Status** dropdown list to filter the list by alert status. diff --git a/_security-analytics/usage/detectors.md b/_security-analytics/usage/detectors.md index 1570bd19944..1c692d66eea 100644 --- a/_security-analytics/usage/detectors.md +++ b/_security-analytics/usage/detectors.md @@ -8,24 +8,24 @@ nav_order: 30 # Working with detectors After creating a detector, it appears on the Threat detectors page along with others saved to the system. You can then perform a number of actions for each detector, from editing its details to changing its status. See the following sections for description of the available actions. -
Threat detector page +
Threat detector page ## Threat detector list -The list of threat detectors includes the search bar, the **Status** dropdown menu, and the **Log type** dropdown menu. +The list of threat detectors includes the search bar, the **Status** dropdown list, and the **Log type** dropdown list. * Use the search bar to filter by detector name. -* Select the **Status** dropdown menu to filter detectors in the list by Active and Inactive status. -* Select the **Log type** dropdown menu to filter detectors by any log type that appears in the list (the options depend on the detectors present in the list and their log types). +* Select the **Status** dropdown list to filter detectors in the list by Active and Inactive status. +* Select the **Log type** dropdown list to filter detectors by any log type that appears in the list (the options depend on the detectors present in the list and their log types). ### Editing a detector To edit a detector, begin by selecting the link to the detector in the Detector name column of the list. The detector's details window opens and shows details about the detector's configuration. -
Detector details window for editig the detector +
Detector details window for editig the detector * In the upper-left portion of the window, the details window shows the name of the detector and its status, either Active or Inactive. * In the upper-right corner of the window, you can select **View alerts** to go to the Alerts window or **View findings** to go to the Findings window. You can also select **Actions** to perform actions for the detector. See [Detector actions]({{site.url}}{{site.baseurl}}/security-analytics/usage/detectors/#detector-actions). * In the lower portion of the window, select the **Edit** button for either Detector details or Detection rules to make changes accordingly. * Finally, you can select the **Field mappings** tab to edit field mappings for the detector, or select the **Alert triggers** tab to make edits to alerts associated with the detector. -
Field mappings and Alert triggers tabs +
Field mappings and Alert triggers tabs After you select the **Alert triggers** tab, you also have the option to add additional alerts for the detector by selecting **Add another alert condition** at the bottom of the page. {: .tip } @@ -33,16 +33,16 @@ To edit a detector, begin by selecting the link to the detector in the Detector ## Detector actions Threat detector actions allow you to stop and start detectors or delete a detector. To enable actions, first select the checkbox beside one or more detectors in the list. -
Threat detector actions +
Threat detector actions ### Changing detector status -1. Select the detector or detectors in the list whose status you would like to change. The **Actions** dropdown menu becomes enabled. +1. Select the detector or detectors in the list whose status you would like to change. The **Actions** dropdown list becomes enabled. 1. Depending on whether the detector is currently active or inactive, select either **Stop detector** or **Start detector**. After a moment, the change in status of the detector appears in the detector list as either Inactive or Active. ### Deleting a detector -1. Select the detector or detectors in the list that you would like to delete. The **Actions** dropdown menu becomes enabled. -1. Select **Delete** in the dropdown menu. The Delete detector popup window opens and asks you to verify that you want to delete the detector or detectors. +1. Select the detector or detectors in the list that you would like to delete. The **Actions** dropdown list becomes enabled. +1. Select **Delete** in the dropdown list. The Delete detector popup window opens and asks you to verify that you want to delete the detector or detectors. 1. Select **Cancel** to decline the action. Select **Delete detector** to delete the detector or detectors permanently from the list. diff --git a/_security-analytics/usage/findings.md b/_security-analytics/usage/findings.md index 9eee29c6da1..432e87079bd 100644 --- a/_security-analytics/usage/findings.md +++ b/_security-analytics/usage/findings.md @@ -15,21 +15,21 @@ You can select the **Refresh** button at any time to refresh information on the ## The Findings graph -The findings graph can display findings by log type or rule severity. Use the **Group by** dropdown menu to specify either log type or rule severity. +The findings graph can display findings by log type or rule severity. Use the **Group by** dropdown list to specify either log type or rule severity. -To specify the date range you would like the graph to display, first select the calendar dropdown menu. The date selector window opens. -Date selector for findings graph +To specify the date range you would like the graph to display, first select the calendar dropdown list. The date selector window opens. +
Date selector for findings graph You can use the **Quick select** settings to specify an exact window of time. -* Select either **Last** or **Next** in the first dropdown menu to set the window of time behind the current setting or ahead of the current setting. -* Select a number in the second dropdown menu to define a value for the range. -* Select a unit of time in the third dropdown menu. Available options are seconds, minutes, hours, days, weeks, months, and years. +* Select either **Last** or **Next** in the first dropdown list to set the window of time behind the current setting or ahead of the current setting. +* Select a number in the second dropdown list to define a value for the range. +* Select a unit of time in the third dropdown list. Available options are seconds, minutes, hours, days, weeks, months, and years. Select the **Apply** button to apply the range of dates to the graph. Information on the graph changes accordingly. -Quick select settings example +
Quick select settings example
You can use the left and right arrows to move the window of time behind the current range of dates or ahead of the current range of dates. When you use these arrows, the start date and end date appear in the date range field. You can then select each one to set an absolute, relative, or current date and time. For absolute and relative changes, select the **Update** button to apply the changes. -Altering date range +
Altering date range -As an alternative, you can select an option in the **Commonly used** section (see the preceding image of the calendar dropdown menu) to conveniently set a window of time. Options include date ranges such as **Today**, **Yesterday**, **this week**, and **week to date**. +As an alternative, you can select an option in the **Commonly used** section (see the preceding image of the calendar dropdown list) to conveniently set a window of time. Options include date ranges such as **Today**, **Yesterday**, **this week**, and **week to date**. When one of the commonly used windows of time is selected, you can select the **Show dates** label in the date range field to populate the range of dates. Following that, you can select either the start date or end date to specify by an absolute, relative, or current date and time setting. For absolute and relative changes, select the **Update** button to apply the changes. @@ -38,8 +38,8 @@ As one more alternative, you can select an option from the **Recently used date ## The Findings list The Findings list displays all findings according to time of the finding, the finding ID, the rule name that generated the finding, the detector that captured the finding, and other details. -A list of all findings -Use the **Rule severity** dropdown menu to filter the list of findings by severity. Use the **log type** dropdown menu to filter the list by log type. +
A list of all findings +Use the **Rule severity** dropdown list to filter the list of findings by severity. Use the **log type** dropdown list to filter the list by log type. Each finding in the list includes a finding ID. You can select the ID to open the Finding details pane, which describes the finding by parameters defined when creating the detector. The Actions column includes two options for each finding: * The diagonal arrow provides another way to open the Findings detail pane. diff --git a/_security-analytics/usage/overview.md b/_security-analytics/usage/overview.md index d32fef9588f..ac7d8eb6ed7 100644 --- a/_security-analytics/usage/overview.md +++ b/_security-analytics/usage/overview.md @@ -21,7 +21,7 @@ Each section provides a summary description for each element of Security Analyti The upper portion of the Overview page contains two control buttons for refreshing information and getting started with Security Analytics. You can select the **Refresh** button to refresh all of the information on the page. You can also select the **Getting started** link to expand the Get started with Security Analytics window, which includes a summary of the setup steps as well as control buttons that allow you to jump to any of the steps. -
The overview page with getting started quick launch window +
The overview page with getting started quick launch window * In step 1 of setup, select **Create detector** to define a detector. * In step 2, select **View findings** to go to the Findings page. For details about this page, see [Working with findings]({{site.url}}{{site.baseurl}}/security-analytics/usage/findings/). * In step 3, select **View alerts** to go to the Security alerts page. For details about this page, see [Working with alerts]({{site.url}}{{site.baseurl}}/security-analytics/usage/alerts/). @@ -29,7 +29,7 @@ You can also select the **Getting started** link to expand the Get started with ## Findings and alert count -The Findings and alert count section provides a graph showing data on the latest findings. Use the **Group by** menu to select either **All findings** or **Log type**. +The Findings and alert count section provides a graph showing data on the latest findings. Use the **Group by** dropdown list to select either **All findings** or **Log type**. ## Recent alerts @@ -42,9 +42,9 @@ The Recent findings table displays recent findings by time, rule name, rule seve ## Most frequent detection rules This section provides a graphical representation of detection rules that trigger findings most often and how they compare to others as a percentage of the whole. The rule names represented by the graph are listed to the right. -
The detection rule graph on the Overview page +
The detection rule graph on the Overview page ## Detectors -Detectors displays a list of available detectors by detector name, status (active/inactive), and log type. Select **View all detectors** to go to the Detectors page. Select **Create detector** to go directly to the Define detector page. +The Detectors section displays a list of available detectors by detector name, status (active/inactive), and log type. Select **View all detectors** to go to the Detectors page. Select **Create detector** to go directly to the Define detector page. diff --git a/_security-analytics/usage/rules.md b/_security-analytics/usage/rules.md index 5c3666f1aaf..feed0f9f2d9 100644 --- a/_security-analytics/usage/rules.md +++ b/_security-analytics/usage/rules.md @@ -8,29 +8,29 @@ nav_order: 40 # Working with rules The Rules window lists all security rules and provides options for filtering the list and viewing details for each rule. Further options let you import rules and create new rules by first duplicating a Sigma rule then modifying it. This section covers navigation of the Rules page and description of the actions you can perform. -The Rules page +
The Rules page ## Viewing and filtering rules When you open the Rules page, all rules are listed in the table. Use the search bar to search for specific rules by entering a full or partial name and pressing **Return/Enter** on your keyboard. The list is filtered and displays matching results. -Alternatively, you can use the **Rule type**, **Rule severity**, and **Source** dropdown menus to drill down in the list of alerts and filter for preferred results. You can select multiple options from each menu and use all three menus in combination to narrow results. -Rule menus for filtering results +Alternatively, you can use the **Rule type**, **Rule severity**, and **Source** dropdown lists to drill down in the alerts and filter for preferred results. You can select multiple options from each list and use all three in combination to narrow results. +
Rule menus for filtering results ### Rule details To see rule details, select the rule in the Rule name column of the list. The rule details pane opens. -
The rule details pane +
The rule details pane In Visual view, rule details are arranged in fields, and the links are active. Select **YAML** to display the rule in YAML file format. -
The rule details pane in YAML file view +
The rule details pane in YAML file view * Rule details are formatted as a YAML file according to the Sigma rule specification. * To copy the rule, select the copy icon in the top right corner of the rule. To quickly create a new and customized rule, you can paste the rule into the YAML editor and make any modifications before saving it. See [Customizing rules](#customizing-rules) for details. ## Creating rules There are several ways to create rules on the Rules page. The first is to manually fill in the necessary fields that complete the rule, using either the Visual Editor or YAML Editor. To do this, select the **Create new rule** button in the uppper-right corner of the Rules window. The Create a rule window opens. -
The Create a rule window, which includes the Visual Editor and YAML editor. +
The Create a rule window, which includes the Visual Editor and YAML editor. If you choose to create the rule manually, you can refer to Sigma's [Rule Creation Guide](https://github.com/SigmaHQ/sigma/wiki/Rule-Creation-Guide) to help understand details for each field. * By default, the Visual Editor is displayed. Enter the appropriate content in each field and select **Create** in the lower-right corner of the window to save the rule. @@ -75,6 +75,7 @@ author: - David ANDRE status: experimental ``` +{% include copy.html %} 1. To begin, select the **Import rule** button in the upper-right corner of the page. The Import rule page opens. 1. Either drag a YAML-formatted Sigma rule into the window or browse for the file by selecting the link and opening it. The Import a rule window opens and the rule definition fields are automatically populated in both the Visual Editor and YAML Editor. @@ -84,15 +85,15 @@ status: experimental ## Customizing rules An alternative to importing a rule is duplicating a Sigma rule and then modifying it to create a custom rule. First search for or filter rules in the Rules list to locate the rule you want to duplicate. -Selecting a rule in the Rules name list +
Selecting a rule in the Rules name list 1. To begin, select the rule in the Rule name column. The rule details pane opens. -
Opening the rule details pane +
Opening the rule details pane 1. Select the **Duplicate** button in the upper-right corner of the pane. The Duplicate rule window opens in Visual Editor view and all of the fields are automatically populated with the rule's details. Details are also populated in YAML Editor view. -
Selecting the duplicate button opens the Duplicate rule window +
Selecting the duplicate button opens the Duplicate rule window 1. In either Visual Editor view or YAML Editor view, modify any of the fields to customize the rule. 1. After performing any modifications to the rule, select the **Create** button in the lower-right corner of the window. A new and customized rule is created, and it appears in the list of rules on the main page of the Rules window. -The custom rule now appears in the list of rules. +
The custom rule now appears in the list of rules. You cannot modify the Sigma rule itself. The original Sigma rule always remains in the system. Its duplicate, after modification, becomes the custom rule that is added to the list of rules. {: .note } diff --git a/_security/access-control/index.md b/_security/access-control/index.md index 4a23e9f07b2..48e7b53e2c4 100644 --- a/_security/access-control/index.md +++ b/_security/access-control/index.md @@ -5,7 +5,7 @@ nav_order: 75 has_children: true has_toc: false redirect_from: - - /security/access-control/ + - /security-plugin/access-control/index/ --- # Access control diff --git a/_security/access-control/permissions.md b/_security/access-control/permissions.md index bdfeaf39189..47f2becacc6 100644 --- a/_security/access-control/permissions.md +++ b/_security/access-control/permissions.md @@ -3,6 +3,8 @@ layout: default title: Permissions parent: Access control nav_order: 110 +redirect_from: + - /security-plugin/access-control/permissions/ --- # Permissions diff --git a/_security/configuration/configuration.md b/_security/configuration/configuration.md index 073fd73d16b..a76f18cfa5f 100755 --- a/_security/configuration/configuration.md +++ b/_security/configuration/configuration.md @@ -3,6 +3,8 @@ layout: default title: Configuring the Security backend parent: Configuration nav_order: 5 +redirect_from: + - /security-plugin/configuration/yaml --- # Configuring the Security backend diff --git a/_security/configuration/disable.md b/_security/configuration/disable.md index 036acc95884..b9f9923baf7 100755 --- a/_security/configuration/disable.md +++ b/_security/configuration/disable.md @@ -3,6 +3,8 @@ layout: default title: Disabling security parent: Configuration nav_order: 40 +redirect_from: + - /security-plugin/configuration/generate-certificates/ --- # Disabling security diff --git a/_security/configuration/index.md b/_security/configuration/index.md index 88e535cec5e..81e43b134f5 100644 --- a/_security/configuration/index.md +++ b/_security/configuration/index.md @@ -5,7 +5,8 @@ nav_order: 2 has_children: true has_toc: false redirect_from: - - /security/configuration/ + - /security-plugin/configuration/ + - /security-plugin/configuration/index/ --- # Security configuration diff --git a/_security/configuration/security-admin.md b/_security/configuration/security-admin.md index e3415b93f0d..d88c35d1173 100755 --- a/_security/configuration/security-admin.md +++ b/_security/configuration/security-admin.md @@ -3,6 +3,8 @@ layout: default title: Applying changes to configuration files parent: Configuration nav_order: 25 +redirect_from: + - /security-plugin/configuration/security-admin/ --- # Applying changes to configuration files diff --git a/_security/configuration/tls.md b/_security/configuration/tls.md index 2a332feef6b..8d99cd651d0 100755 --- a/_security/configuration/tls.md +++ b/_security/configuration/tls.md @@ -3,6 +3,8 @@ layout: default title: Configuring TLS certificates parent: Configuration nav_order: 15 +redirect_from: + - /security-plugin/configuration/tls/ --- # Configuring TLS certificates diff --git a/_security/configuration/yaml.md b/_security/configuration/yaml.md index 0138df3f3e2..b37c528fc89 100644 --- a/_security/configuration/yaml.md +++ b/_security/configuration/yaml.md @@ -3,6 +3,8 @@ layout: default title: Modifying the YAML files parent: Configuration nav_order: 10 +redirect_from: + - /security-plugin/configuration/yaml/ --- # Modifying the YAML files diff --git a/_security/index.md b/_security/index.md index 900f7a26cae..a76a3593d85 100755 --- a/_security/index.md +++ b/_security/index.md @@ -6,6 +6,7 @@ has_children: false has_toc: false redirect_from: - /security/ + - /security-plugin/index/ --- # About Security in OpenSearch diff --git a/_tuning-your-cluster/availability-and-recovery/index.md b/_tuning-your-cluster/availability-and-recovery/index.md new file mode 100644 index 00000000000..3a68fd4c22e --- /dev/null +++ b/_tuning-your-cluster/availability-and-recovery/index.md @@ -0,0 +1,9 @@ +--- +layout: default +title: Availability and Recovery +nav_order: 20 +has_children: true +has_toc: true +--- + +The following OpenSearch features help ensure consistent uptime so that your cluster can complete and scale based on your use case, as well as creating snapshots. \ No newline at end of file diff --git a/_opensearch/remote.md b/_tuning-your-cluster/availability-and-recovery/remote.md similarity index 99% rename from _opensearch/remote.md rename to _tuning-your-cluster/availability-and-recovery/remote.md index e39e43334a4..1d2977b8926 100644 --- a/_opensearch/remote.md +++ b/_tuning-your-cluster/availability-and-recovery/remote.md @@ -1,7 +1,10 @@ --- layout: default title: Remote-backed storage -nav_order: 19 +nav_order: 40 +parent: Availability and Recovery +redirect_from: + - /opensearch/remote/ --- # Remote-backed storage diff --git a/_opensearch/search-backpressure.md b/_tuning-your-cluster/availability-and-recovery/search-backpressure.md similarity index 99% rename from _opensearch/search-backpressure.md rename to _tuning-your-cluster/availability-and-recovery/search-backpressure.md index ea45d84c9b6..1133a7d41f9 100644 --- a/_opensearch/search-backpressure.md +++ b/_tuning-your-cluster/availability-and-recovery/search-backpressure.md @@ -1,8 +1,11 @@ --- layout: default title: Search backpressure -nav_order: 63 +nav_order: 60 has_children: false +parent: Availability and Recovery +redirect_from: + - /opensearch/search-backpressure/ --- # Search backpressure diff --git a/_opensearch/segment-replication/configuration.md b/_tuning-your-cluster/availability-and-recovery/segment-replication/configuration.md similarity index 98% rename from _opensearch/segment-replication/configuration.md rename to _tuning-your-cluster/availability-and-recovery/segment-replication/configuration.md index 22aeeb9cfda..b336df6985a 100644 --- a/_opensearch/segment-replication/configuration.md +++ b/_tuning-your-cluster/availability-and-recovery/segment-replication/configuration.md @@ -3,6 +3,7 @@ layout: default title: Segment replication configuration nav_order: 12 parent: Segment replication +grand_parent: Availability and Recovery --- # Segment replication configuration diff --git a/_opensearch/segment-replication/index.md b/_tuning-your-cluster/availability-and-recovery/segment-replication/index.md similarity index 93% rename from _opensearch/segment-replication/index.md rename to _tuning-your-cluster/availability-and-recovery/segment-replication/index.md index b7ff27655ee..b7641f81920 100644 --- a/_opensearch/segment-replication/index.md +++ b/_tuning-your-cluster/availability-and-recovery/segment-replication/index.md @@ -1,10 +1,12 @@ --- layout: default title: Segment replication -nav_order: 64 +nav_order: 70 has_children: true +parent: Availability and Recovery redirect_from: - /opensearch/segment-replication/ + - /opensearch/segment-replication/index/ --- # Segment replication diff --git a/_opensearch/shard-indexing-backpressure.md b/_tuning-your-cluster/availability-and-recovery/shard-indexing-backpressure.md similarity index 94% rename from _opensearch/shard-indexing-backpressure.md rename to _tuning-your-cluster/availability-and-recovery/shard-indexing-backpressure.md index ac58c7d358a..cde2f125cb6 100644 --- a/_opensearch/shard-indexing-backpressure.md +++ b/_tuning-your-cluster/availability-and-recovery/shard-indexing-backpressure.md @@ -3,6 +3,9 @@ layout: default title: Shard indexing backpressure nav_order: 62 has_children: true +parent: Availability and Recovery +redirect_from: + - /opensearch/shard-indexing-backpressure/ --- # Shard indexing backpressure diff --git a/_opensearch/shard-indexing-settings.md b/_tuning-your-cluster/availability-and-recovery/shard-indexing-settings.md similarity index 97% rename from _opensearch/shard-indexing-settings.md rename to _tuning-your-cluster/availability-and-recovery/shard-indexing-settings.md index 8726906352b..88b0ea70b4c 100644 --- a/_opensearch/shard-indexing-settings.md +++ b/_tuning-your-cluster/availability-and-recovery/shard-indexing-settings.md @@ -2,8 +2,10 @@ layout: default title: Settings parent: Shard indexing backpressure -nav_order: 1 -has_children: false +nav_order: 50 +grand_parent: Availability and Recovery +redirect_from: + - /opensearch/shard-indexing-settings/ --- # Settings diff --git a/_opensearch/snapshots/index.md b/_tuning-your-cluster/availability-and-recovery/snapshots/index.md similarity index 75% rename from _opensearch/snapshots/index.md rename to _tuning-your-cluster/availability-and-recovery/snapshots/index.md index 192f6f02ec8..43ceec1d7b9 100644 --- a/_opensearch/snapshots/index.md +++ b/_tuning-your-cluster/availability-and-recovery/snapshots/index.md @@ -1,9 +1,12 @@ --- layout: default title: Snapshots -nav_order: 65 +nav_order: 30 has_children: true -redirect_from: /opensearch/snapshots/ +parent: Availability and Recovery +redirect_from: + - /opensearch/snapshots/ + - /opensearch/snapshots/index/ has_toc: false --- @@ -24,4 +27,4 @@ Snapshots have two main uses: You can take and restore snapshots using the [snapshot API]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore). -If you need to automate taking snapshots, you can use the [Snapshot Management]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-management) feature. +If you need to automate taking snapshots, you can use the [snapshot management]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-management) feature. diff --git a/_opensearch/snapshots/searchable_snapshot.md b/_tuning-your-cluster/availability-and-recovery/snapshots/searchable_snapshot.md similarity index 98% rename from _opensearch/snapshots/searchable_snapshot.md rename to _tuning-your-cluster/availability-and-recovery/snapshots/searchable_snapshot.md index deab138c83b..f7ef3c981da 100644 --- a/_opensearch/snapshots/searchable_snapshot.md +++ b/_tuning-your-cluster/availability-and-recovery/snapshots/searchable_snapshot.md @@ -3,7 +3,9 @@ layout: default title: Searchable snapshots parent: Snapshots nav_order: 40 -has_children: false +grand_parent: Availability and Recovery +redirect_from: + - /opensearch/snapshots/searchable_snapshot/ --- # Searchable snapshots diff --git a/_opensearch/snapshots/sm-api.md b/_tuning-your-cluster/availability-and-recovery/snapshots/sm-api.md similarity index 97% rename from _opensearch/snapshots/sm-api.md rename to _tuning-your-cluster/availability-and-recovery/snapshots/sm-api.md index 70b779ef05a..0db74993d27 100644 --- a/_opensearch/snapshots/sm-api.md +++ b/_tuning-your-cluster/availability-and-recovery/snapshots/sm-api.md @@ -1,14 +1,17 @@ --- layout: default -title: Snapshot Management API +title: Snapshot management API parent: Snapshots nav_order: 30 has_children: false +grand_parent: Availability and Recovery +redirect_from: + - /opensearch/snapshots/sm-api/ --- # Snapshot Management API -Use the [Snapshot Management (SM)]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots) API to automate [taking snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots). +Use the [snapshot management (SM)]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots) API to automate [taking snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots). --- @@ -182,7 +185,7 @@ Parameter | Type | Description `snapshot_config.indices` | String | The names of the indexes in the snapshot. Multiple index names are separated by `,`. Supports wildcards (`*`). Optional. Default is `*` (all indexes). `snapshot_config.repository` | String | The repository in which to store snapshots. Required. `snapshot_config.ignore_unavailable` | Boolean | Do you want to ignore unavailable indexes? Optional. Default is `false`. -`snapshot_config.include_global_state` | Boolean | Do you want to include cluster state? Optional. Default is `true` because of [Security plugin considerations]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore/#security-plugin-considerations). +`snapshot_config.include_global_state` | Boolean | Do you want to include cluster state? Optional. Default is `true` because of [Security plugin considerations]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore#security-considerations). `snapshot_config.partial` | Boolean | Do you want to allow partial snapshots? Optional. Default is `false`. `snapshot_config.metadata` | Object | Metadata in the form of key/value pairs. Optional. `creation` | Object | Configuration for snapshot creation. Required. diff --git a/_tuning-your-cluster/availability-and-recovery/snapshots/sm-dashboards.md b/_tuning-your-cluster/availability-and-recovery/snapshots/sm-dashboards.md new file mode 100644 index 00000000000..9da90a7855f --- /dev/null +++ b/_tuning-your-cluster/availability-and-recovery/snapshots/sm-dashboards.md @@ -0,0 +1,176 @@ +--- +layout: default +title: Snapshot management +parent: Index and snapshot management in OpenSearch Dashboards +nav_order: 17 +--- + +# Snapshot management + +You can set up Snapshot Management (SM) in OpenSearch Dashboards. + +[Snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/index/) are backups of a cluster’s indexes and state. The state includes cluster settings, node information, index metadata (mappings, settings, templates), and shard allocation. + +Snapshots have two main uses: + +1. Recovering from failure + + For example, if cluster health goes red, you might restore the red indexes from a snapshot. + +2. Migrating from one cluster to another + + For example, if you’re moving from a proof of concept to a production cluster, you might take a snapshot of the former and restore it on the latter. + +You can take and restore snapshots using snapshot management in OpenSearch Dashboards. + +If you need to automate snapshots creation, you can use a snapshot policy. + +## Creating a repository + +Before you create an SM policy, you need to set up a repository for snapshots. + +1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**. +1. In the left panel, under **Snapshot Management**, select **Repositories**. +1. Choose the **Create Repository** button. +1. Enter the repository name, type, and location. +1. (Optional) Select **Advanced Settings** and enter additional settings for this repository as a JSON object. Example: + ```json + { + "chunk_size": null, + "compress": false, + "max_restore_bytes_per_sec": "40m", + "max_snapshot_bytes_per_sec": "40m", + "readonly": false + } + ``` +1. Choose the **Add** button. + +## Deleting a repository + +To delete a snapshot repository configuration, select the repository from the **Repositories** list and then choose the **Delete** button. + +## Creating an SM policy + +Create an SM policy to set up automatic snapshots. An SM policy defines an automated snapshot creation schedule and an optional automated deletion schedule. + +1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**. +1. In the left panel, under **Snapshot Management**, select **Snapshot Policies**. +1. Select the **Create Policy** button. +1. In the **Policy settings** section: + 1. Enter the policy name. + 1. (Optional) Enter the policy description. +1. In the **Source and destination** section: + 1. Select or enter source indexes either as a list or as an index pattern. + 1. Select a repository for snapshots. To [create a new repository](#creating-a-repository), select the **Create** button. +1. In the **Snapshot schedule** section: + 1. Select the desired snapshot frequency or enter a custom cron expression for snapshot frequency. + 1. Select the start time and time zone. +1. In the **Retention period** section: + 1. Choose to retain all snapshots or specify retention conditions (the maximum age of retained snapshots). + 1. (Optional) In **Additional settings**, select the minimum and maximum number of retained snapshots, deletion frequency, and deletion start time. +1. In the **Notifications** section, select the snapshot activities you want to be notified about. +1. (Optional) In the **Advanced settings** section, select the desired options: + - **Include cluster state in snapshots** + - **Ignore unavailable indices** + - **Allow partial snapshots** +1. Select the **Create** button. + +## View, edit, or delete an SM policy + +You can view, edit, or delete an SM policy on the policy details page. + +1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**. +1. In the left panel, under **Snapshot Management**, select **Snapshot Policies**. +1. Click on the **Policy name** of the policy you want to view, edit, or delete.
+The policy settings, snapshot schedule, snapshot retention period, notifications, and last creation and deletion are displayed in the policy details page.
If a snapshot creation or deletion fails, you can view information about the failure in the **Last Creation/Deletion** section. To view the failure message, click on the **cause** in the **Info** column. +1. To edit or delete the SM policy, select the **Edit** or **Delete** button. + +## Enable, disable, or delete SM policies + +1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**. +1. In the left panel, under **Snapshot Management**, select **Snapshot Policies**. +1. Select one or more policies in the list. +1. To enable or disable selected SM policies, select the **Enable** or **Disable** button. To delete selected SM policies, in the **Actions** list, select the **Delete** option. + +## View snapshots + +1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**. +1. In the left panel, under **Snapshot Management**, select **Snapshots**. +All automatically or manually taken snapshots appear in the list. +1. To view a snapshot, click on its **Name**. + +## Take a snapshot + +Use the steps below to take a snapshot manually: + +1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**. +1. In the left panel, under **Snapshot Management**, select **Snapshots**. +1. Select the **Take snapshot** button. +1. Enter the snapshot name. +1. Select or enter source indexes either as a list or as an index pattern. +1. Select a repository for the snapshot. +1. (Optional) In the **Advanced options** section, select the desired options: + - **Include cluster state in snapshots** + - **Ignore unavailable indices** + - **Allow partial snapshots** +1. Choose the **Add** button. + +## Deleting a snapshot + +The **Delete** button [deletes]({{site.url}}{{site.baseurl}}/api-reference/snapshots/delete-snapshot/) a snapshot from a repository. + +1. To view a list of your repositories, choose **Repositories** under the **Snapshot Management** section. +2. To view a list of your snapshots, choose **Snapshots** under the **Snapshot Management** section. + +## Restoring a snapshot + +1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**. +1. In the left panel, under **Snapshot Management**, select **Snapshots**. The **Snapshots** tab is selected by default. +1. Select the checkbox next to the snapshot you want to restore, as shown in the following image: + Snapshots{: .img-fluid} + + You can only restore snapshots with the status of `Success` or `Partial`. The status of the snapshot is displayed in the **Snapshot status** column. + {: .note} +1. In the **Restore snapshot** flyout, select the options for restoring the snapshot. + + The **Restore snapshot** flyout lists the snapshot name and status. To view the list of indexes in the snapshot, select the number under **Indices** (for example, `27` in the following image). This number represents the number of indexes in the snapshot. + + Restore Snapshot + + For more information about the options in the **Restore snapshot** flyout, see [Restore snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#restore-snapshots). + + **Ignoring missing indexes** + + If you specify which indexes you want to restore from the snapshot and select the **Ignore unavailable indices** option, the restore operation ignores the indexes that are missing from the snapshot. For example, if you want to restore the `log1` and `log2` indexes, but `log2` is not in the snapshot, `log1` is restored and `log2` is ignored. If you don't select **Ignore unavailable indices**, the entire restore operation fails if an index to be restored is missing from a snapshot. + + **Custom index settings** + + You can choose to customize some settings for the indexes restored from a snapshot:
+  • Select the **Customize index settings** checkbox to provide new values for the specified index settings. All newly restored indexes will use these values instead of the ones in the snapshot.
+  • Select the **Ignore index settings** checkbox to specify the settings in the snapshot to ignore. All newly restored indexes will use the cluster defaults for these settings. + + The examples in the following image set `index.number_of_replicas` to `0`, `index.auto_expand_replicas` to `true`, and `index.refresh_interval` and `index.max_script_fields` to the cluster default values for all newly restored indexes. + + Custom settings + + For more information about index settings, see [Index settings]({{site.url}}{{site.baseurl}}/api-reference/index-apis/create-index/#index-settings). + + For a list of settings that you cannot change or ignore, see [Restore snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#restore-snapshots). + + After choosing the options, select the **Restore snapshot** button. +1. (Optional) To monitor the restore progress, select **View restore activities** in the confirmation dialog. You can also monitor the restore progress at any time by selecting the **Restore activities in progress** tab, as shown in the following image. + + Restore Activities{: .img-fluid} + + You can view the percentage of the job that has been completed in the **Status** column. Once the snapshot restore is complete, the **Status** changes to `Completed (100%)`. + + The **Restore activities in progress** panel is not persistent. It displays only the progress of the current restore operation. If multiple restore operations are running, the panel displays the most recent one. + {: .note } + To view the status of each index being restored, select the link in the **Indices being restored** column (in the preceding image, the `27 Indices` link). The **Indices being restored** flyout (shown in the following image) displays each index and its restore status. + + Restore Indices{: .img-fluid} + + After the restore operation is complete, the restored indexes are listed in the **Indices** panel. To view the indexes, in the left panel, under **Index Management**, choose **Indices**. + +View Indices{: .img-fluid} + \ No newline at end of file diff --git a/_opensearch/snapshots/snapshot-management.md b/_tuning-your-cluster/availability-and-recovery/snapshots/snapshot-management.md similarity index 86% rename from _opensearch/snapshots/snapshot-management.md rename to _tuning-your-cluster/availability-and-recovery/snapshots/snapshot-management.md index 8cedd0b4d51..1a6ddfaf6f2 100644 --- a/_opensearch/snapshots/snapshot-management.md +++ b/_tuning-your-cluster/availability-and-recovery/snapshots/snapshot-management.md @@ -1,14 +1,17 @@ --- layout: default -title: Snapshot Management +title: Snapshot management parent: Snapshots nav_order: 20 has_children: false +grand_parent: Availability and Recovery +redirect_from: + - /opensearch/snapshots/snapshot-management/ --- -# Snapshot Management +# Snapshot management -Snapshot Management (SM) lets you automate [taking snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots). To use this feature, you need to install the [Index Management (IM) Plugin]({{site.url}}{{site.baseurl}}/im-plugin). Snapshots store only incremental changes since the last snapshot. Thus, while taking an initial snapshot may be a heavy operation, subsequent snapshots have minimal overhead. To set up automatic snapshots, you have to create an SM policy with a desired SM schedule and configuration. +Snapshot management (SM) lets you automate [taking snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots). To use this feature, you need to install the [Index Management (IM) Plugin]({{site.url}}{{site.baseurl}}/im-plugin). Snapshots store only incremental changes since the last snapshot. Thus, while taking an initial snapshot may be a heavy operation, subsequent snapshots have minimal overhead. To set up automatic snapshots, you have to create an SM policy with a desired SM schedule and configuration. When you create an SM policy, its document ID is given the name `-sm-policy`. Because of this, SM policies have to obey the following rules: @@ -18,7 +21,7 @@ When you create an SM policy, its document ID is given the name `-s SM-created snapshots have names in the format `--`. Two snapshots created by different policies at the same time always have different names because of the `` prefix. To avoid name collisions within the same policy, each snapshot's name contains a random string suffix. -Each policy has associated metadata that stores the policy status. Snapshot Management saves SM policies and metadata in the system index and reads them from the system index. Thus, Snapshot Management depends on the OpenSearch cluster's indexing and searching functions. The policy's metadata keeps information about the latest creation and deletion only. The metadata is read before running every scheduled job so that SM can continue execution from the previous job's state. You can view the metadata using the [explain API]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#explain). +Each policy has associated metadata that stores the policy status. Snapshot management saves SM policies and metadata in the system index and reads them from the system index. Thus, Snapshot Management depends on the OpenSearch cluster's indexing and searching functions. The policy's metadata keeps information about the latest creation and deletion only. The metadata is read before running every scheduled job so that SM can continue execution from the previous job's state. You can view the metadata using the [explain API]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#explain). An SM schedule is a custom [cron]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/cron) expression. It consists of two parts: a creation schedule and a deletion schedule. You must set up a creation schedule that specifies the frequency and timing of snapshot creation. Optionally, you can set up a separate schedule for deleting snapshots. @@ -44,7 +47,11 @@ We don't recommend setting up the same repository for multiple SM policies with ## Failure management +<<<<<<< HEAD:_opensearch/snapshots/snapshot-management.md If a snapshot operation fails, it is retried a maximum of three times. The failure message is saved in `metadata.latest_execution` and is overwritten when a subsequent snapshot operation starts. You can view the failure message using the [explain API]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#explain). When using OpenSearch Dashboards, you can view the failure message on the [policy details page]({{site.url}}{{site.baseurl}}/dashboards/admin-ui-index/sm-dashboards/#enable-disable-or-delete-sm-policies). Possible reasons for failure include red index status and shard reallocation. +======= +If a snapshot operation fails, it is retried a maximum of three times. The failure message is saved in `metadata.latest_execution` and is overwritten when a subsequent snapshot operation starts. You can view the failure message using the [explain API]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#explain). When using OpenSearch Dashboards, you can view the failure message on the [policy details page]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/snapshots/sm-dashboards/#view-edit-or-delete-an-sm-policy). Possible reasons for failure include red index status and shard reallocation. +>>>>>>> 0767f742 (Create Tuning your cluster section):_tuning-your-cluster/availability-and-recovery/snapshots/snapshot-management.md ## Security diff --git a/_opensearch/snapshots/snapshot-restore.md b/_tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore.md similarity index 99% rename from _opensearch/snapshots/snapshot-restore.md rename to _tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore.md index bfb7a302a1c..8d0d5909fcf 100644 --- a/_opensearch/snapshots/snapshot-restore.md +++ b/_tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore.md @@ -4,6 +4,9 @@ title: Take and restore snapshots parent: Snapshots nav_order: 10 has_children: false +grand_parent: Availability and Recovery +redirect_from: + - /opensearch/snapshots/snapshot-restore/ --- # Take and restore snapshots diff --git a/_opensearch/stats-api.md b/_tuning-your-cluster/availability-and-recovery/stats-api.md similarity index 99% rename from _opensearch/stats-api.md rename to _tuning-your-cluster/availability-and-recovery/stats-api.md index 050d0bb6625..4ac5e4e6ad5 100644 --- a/_opensearch/stats-api.md +++ b/_tuning-your-cluster/availability-and-recovery/stats-api.md @@ -3,7 +3,10 @@ layout: default title: Stats API parent: Shard indexing backpressure nav_order: 2 +grand_parent: Availability and Recovery has_children: false +redirect_from: + - /opensearch/stats-api/ --- # Stats API diff --git a/_tuning-your-cluster/cluster-manager-task-throttling.md b/_tuning-your-cluster/cluster-manager-task-throttling.md new file mode 100644 index 00000000000..a8f6db0a4f4 --- /dev/null +++ b/_tuning-your-cluster/cluster-manager-task-throttling.md @@ -0,0 +1,112 @@ +--- +layout: default +title: Cluster manager task throttling +nav_order: 10 +has_children: false +--- + +# Cluster manager task throttling + +For many cluster state updates, such as defining a mapping or creating an index, nodes submit tasks to the cluster manager. The cluster manager maintains a pending task queue for these tasks and runs them in a single-threaded environment. When nodes send tens of thousands of resource-intensive tasks, like `put-mapping` or snapshot tasks, these tasks can pile up in the queue and flood the cluster manager. This affects the cluster manager's performance and may in turn affect the availability of the whole cluster. + +The first line of defense is to implement mechanisms in the caller nodes to avoid task overload on the cluster manager. However, even with those mechanisms in place, the cluster manager needs a built-in way to protect itself: cluster manager task throttling. + +To turn on cluster manager task throttling, you need to set throttling limits. The cluster manager uses the throttling limits to determine whether to reject a task. + +The cluster manager rejects a task based on its type. For any incoming task, the cluster manager evaluates the total number of tasks of the same type in the pending task queue. If this number exceeds the threshold for this task type, the cluster manager rejects the incoming task. Rejecting a task does not affect tasks of a different type. For example, if the cluster manager rejects a `put-mapping` task, it can still accept a subsequent `create-index` task. + +When the cluster manager rejects a task, the node performs retries with exponential backoff to resubmit the task to the cluster manager. If retries are unsuccessful within the timeout period, OpenSearch returns a cluster timeout error. + +## Setting throttling limits + +You can set throttling limits by specifying them in the `cluster_manager.throttling.thresholds` object and updating the [OpenSearch cluster settings]({{site.url}}{{site.baseurl}}/api-reference/cluster-settings). The setting is dynamic, so you can change the behavior of this feature without restarting your cluster. + +By default, throttling is disabled for all task types. +{: .note} + +The request has the following format: + +```json +PUT _cluster/settings +{ + "persistent": { + "cluster_manager.throttling.thresholds" : { + "" : { + "value" : + } + } + } +} +``` + +The following table describes the `cluster_manager.throttling.thresholds` object. + +Field Name | Description +:--- | :--- +task-type | The task type. See [supported task types](#supported-task-types) for a list of valid values. +value | The maximum number of tasks of the `task-type` type in the cluster manager's pending task queue. Default is `-1` (no task throttling). + +## Supported task types + +The following task types are supported: + +- `create-index` +- `update-settings` +- `cluster-update-settings` +- `auto-create` +- `delete-index` +- `delete-dangling-index` +- `create-data-stream` +- `remove-data-stream` +- `rollover-index` +- `index-aliases` +- `put-mapping` +- `create-index-template` +- `remove-index-template` +- `create-component-template` +- `remove-component-template` +- `create-index-template-v2` +- `remove-index-template-v2` +- `put-pipeline` +- `delete-pipeline` +- `create-persistent-task` +- `finish-persistent-task` +- `remove-persistent-task` +- `update-task-state` +- `put-script` +- `delete-script` +- `put-repository` +- `delete-repository` +- `create-snapshot` +- `delete-snapshot` +- `update-snapshot-state` +- `restore-snapshot` +- `cluster-reroute-api` + +#### Sample request + +The following request sets the throttling threshold for the `put-mapping` task type to 100: + +```json +PUT _cluster/settings +{ + "persistent": { + "cluster_manager.throttling.thresholds": { + "put-mapping": { + "value": 100 + } + } + } +} +``` + +Set the threshold to `-1` to disable throttling for a task type. +<<<<<<< HEAD +{: .note} +======= +{: .note} +<<<<<<< HEAD +>>>>>>> f07c7c3a (Merge main) +======= +>>>>>>> 4b8790cd (Fix links and other things) +>>>>>>> 805aae66 (Fix links and other things) diff --git a/_opensearch/cluster.md b/_tuning-your-cluster/cluster.md similarity index 99% rename from _opensearch/cluster.md rename to _tuning-your-cluster/cluster.md index dc4ecfa1126..0bd08c1fbf9 100644 --- a/_opensearch/cluster.md +++ b/_tuning-your-cluster/cluster.md @@ -1,10 +1,12 @@ --- layout: default -title: Cluster formation -nav_order: 7 +title: Creating a cluster +nav_order: 8 +redirect_from: + - /opensearch/cluster/ --- -# Cluster formation +# Creating a cluster Before diving into OpenSearch and searching and aggregating data, you first need to create an OpenSearch cluster. diff --git a/_replication-plugin/api.md b/_tuning-your-cluster/replication-plugin/api.md similarity index 99% rename from _replication-plugin/api.md rename to _tuning-your-cluster/replication-plugin/api.md index 045a4e65416..60a0a37590a 100644 --- a/_replication-plugin/api.md +++ b/_tuning-your-cluster/replication-plugin/api.md @@ -2,6 +2,9 @@ layout: default title: API nav_order: 50 +parent: Cross-cluster replication +redirect_from: + - /replication-plugin/api/ --- # Cross-cluster replication API diff --git a/_replication-plugin/auto-follow.md b/_tuning-your-cluster/replication-plugin/auto-follow.md similarity index 98% rename from _replication-plugin/auto-follow.md rename to _tuning-your-cluster/replication-plugin/auto-follow.md index f315c1399bc..d3103df91ea 100644 --- a/_replication-plugin/auto-follow.md +++ b/_tuning-your-cluster/replication-plugin/auto-follow.md @@ -2,8 +2,9 @@ layout: default title: Auto-follow nav_order: 20 -has_children: false - +parent: Cross-cluster replication +redirect_from: + - /replication-plugin/auto-follow/ --- # Auto-follow for cross-cluster replication diff --git a/_replication-plugin/get-started.md b/_tuning-your-cluster/replication-plugin/getting-started.md similarity index 98% rename from _replication-plugin/get-started.md rename to _tuning-your-cluster/replication-plugin/getting-started.md index 25ab6215d19..05f515f3c72 100644 --- a/_replication-plugin/get-started.md +++ b/_tuning-your-cluster/replication-plugin/getting-started.md @@ -1,10 +1,13 @@ --- layout: default -title: Get started -nav_order: 10 +title: Getting started +nav_order: 15 +parent: Cross-cluster replication +redirect_from: + - /replication-plugin/get-started/ --- -# Get started with cross-cluster replication +# Getting started with cross-cluster replication With cross-cluster replication, you index data to a leader index, and OpenSearch replicates that data to one or more read-only follower indexes. All subsequent operations on the leader are replicated on the follower, such as creating, updating, or deleting documents. diff --git a/_replication-plugin/index.md b/_tuning-your-cluster/replication-plugin/index.md similarity index 96% rename from _replication-plugin/index.md rename to _tuning-your-cluster/replication-plugin/index.md index 22af0da06b0..46eede78e91 100644 --- a/_replication-plugin/index.md +++ b/_tuning-your-cluster/replication-plugin/index.md @@ -1,10 +1,11 @@ --- layout: default title: Cross-cluster replication -nav_order: 1 -has_children: false +nav_order: 12 +has_children: true redirect_from: - /replication-plugin/ + - /replication-plugin/index/ --- # Cross-cluster replication diff --git a/_replication-plugin/permissions.md b/_tuning-your-cluster/replication-plugin/permissions.md similarity index 98% rename from _replication-plugin/permissions.md rename to _tuning-your-cluster/replication-plugin/permissions.md index e09c9e94603..9c8a8b10b64 100644 --- a/_replication-plugin/permissions.md +++ b/_tuning-your-cluster/replication-plugin/permissions.md @@ -2,6 +2,9 @@ layout: default title: Replication security nav_order: 30 +parent: Cross-cluster replication +redirect_from: + - /replication-plugin/permissions/ --- # Cross-cluster replication security diff --git a/_replication-plugin/settings.md b/_tuning-your-cluster/replication-plugin/settings.md similarity index 94% rename from _replication-plugin/settings.md rename to _tuning-your-cluster/replication-plugin/settings.md index 12edf8657c1..4a5d266d087 100644 --- a/_replication-plugin/settings.md +++ b/_tuning-your-cluster/replication-plugin/settings.md @@ -1,7 +1,10 @@ --- layout: default -title: Settings +title: Replication settings nav_order: 40 +parent: Cross-cluster replication +redirect_from: + - /replication-plugin/settings/ --- # Replication settings diff --git a/images/dashboards/add-panel.png b/images/dashboards/add-panel.png new file mode 100644 index 00000000000..72e76752a85 Binary files /dev/null and b/images/dashboards/add-panel.png differ diff --git a/images/dashboards/add-sample-data.png b/images/dashboards/add-sample-data.png new file mode 100644 index 00000000000..5e4b5579919 Binary files /dev/null and b/images/dashboards/add-sample-data.png differ diff --git a/images/dashboards/dashboard-flight-quickstart.png b/images/dashboards/dashboard-flight-quickstart.png new file mode 100644 index 00000000000..ea45166fc3f Binary files /dev/null and b/images/dashboards/dashboard-flight-quickstart.png differ diff --git a/images/dashboards/discover-quickstart.png b/images/dashboards/discover-quickstart.png new file mode 100644 index 00000000000..a11f4536c64 Binary files /dev/null and b/images/dashboards/discover-quickstart.png differ diff --git a/images/dashboards/dql-interface.png b/images/dashboards/dql-interface.png new file mode 100644 index 00000000000..78ea1d66676 Binary files /dev/null and b/images/dashboards/dql-interface.png differ diff --git a/images/dashboards/dql-query-result.png b/images/dashboards/dql-query-result.png new file mode 100644 index 00000000000..68309793eb9 Binary files /dev/null and b/images/dashboards/dql-query-result.png differ diff --git a/images/dashboards/dql-sample-dataset.png b/images/dashboards/dql-sample-dataset.png new file mode 100644 index 00000000000..98aa69e4c51 Binary files /dev/null and b/images/dashboards/dql-sample-dataset.png differ diff --git a/images/dashboards/dql-search-field.png b/images/dashboards/dql-search-field.png new file mode 100644 index 00000000000..c9ae60256ca Binary files /dev/null and b/images/dashboards/dql-search-field.png differ diff --git a/images/dashboards/filter-data-discover.png b/images/dashboards/filter-data-discover.png new file mode 100644 index 00000000000..2078d4eabc5 Binary files /dev/null and b/images/dashboards/filter-data-discover.png differ diff --git a/images/dashboards/start-end-time.png b/images/dashboards/start-end-time.png new file mode 100644 index 00000000000..97573e72fc9 Binary files /dev/null and b/images/dashboards/start-end-time.png differ diff --git a/images/dashboards/time-filter-popup.png b/images/dashboards/time-filter-popup.png new file mode 100644 index 00000000000..b46c640558a Binary files /dev/null and b/images/dashboards/time-filter-popup.png differ diff --git a/images/dashboards/time-range.png b/images/dashboards/time-range.png new file mode 100644 index 00000000000..b99103c3582 Binary files /dev/null and b/images/dashboards/time-range.png differ diff --git a/images/dashboards/viz-panel-quickstart.png b/images/dashboards/viz-panel-quickstart.png new file mode 100644 index 00000000000..9ccb97dda7f Binary files /dev/null and b/images/dashboards/viz-panel-quickstart.png differ diff --git a/images/viz-type-UI.png b/images/dashboards/viz-type-UI.png similarity index 100% rename from images/viz-type-UI.png rename to images/dashboards/viz-type-UI.png diff --git a/images/discover-analyze.png b/images/discover-analyze.png new file mode 100644 index 00000000000..d2240b1f1c3 Binary files /dev/null and b/images/discover-analyze.png differ diff --git a/images/discover-data-fields.png b/images/discover-data-fields.png new file mode 100644 index 00000000000..c7880c0e74c Binary files /dev/null and b/images/discover-data-fields.png differ diff --git a/images/discover-filter.png b/images/discover-filter.png new file mode 100644 index 00000000000..59fda9de271 Binary files /dev/null and b/images/discover-filter.png differ diff --git a/images/discover-index.png b/images/discover-index.png deleted file mode 100644 index 48f46908749..00000000000 Binary files a/images/discover-index.png and /dev/null differ diff --git a/images/discover-save.png b/images/discover-save.png new file mode 100644 index 00000000000..ab8eb4a9c04 Binary files /dev/null and b/images/discover-save.png differ diff --git a/images/discover-search.png b/images/discover-search.png new file mode 100644 index 00000000000..41f47fcb79f Binary files /dev/null and b/images/discover-search.png differ diff --git a/images/discover-setting-up-data.gif b/images/discover-setting-up-data.gif new file mode 100644 index 00000000000..17eac1b8b72 Binary files /dev/null and b/images/discover-setting-up-data.gif differ diff --git a/images/discover-visualize-2.png b/images/discover-visualize-2.png new file mode 100644 index 00000000000..c670533c0c3 Binary files /dev/null and b/images/discover-visualize-2.png differ diff --git a/images/discover-visualize.png b/images/discover-visualize.png new file mode 100644 index 00000000000..b3aa6432239 Binary files /dev/null and b/images/discover-visualize.png differ diff --git a/images/edit-filter.png b/images/edit-filter.png new file mode 100644 index 00000000000..416a1abce67 Binary files /dev/null and b/images/edit-filter.png differ diff --git a/images/interact-filter-dashboard.png b/images/interact-filter-dashboard.png new file mode 100644 index 00000000000..65ab38073bd Binary files /dev/null and b/images/interact-filter-dashboard.png differ diff --git a/images/last-7--days.png b/images/last-7--days.png new file mode 100644 index 00000000000..79ec2998508 Binary files /dev/null and b/images/last-7--days.png differ diff --git a/version-history.md b/version-history.md index a9bcc6ef881..c69f62c2bc5 100644 --- a/version-history.md +++ b/version-history.md @@ -20,6 +20,7 @@ OpenSearch version | Release highlights | Release date [2.0.1](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.0.1.md) | Includes bug fixes and maintenance updates for Alerting and Anomaly Detection. | 16 June 2022 [2.0.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.0.0.md) | Includes document-level monitors for alerting, OpenSearch Notifications plugins, and Geo Map Tiles in OpenSearch Dashboards. Also adds support for Lucene 9 and bug fixes for all OpenSearch plugins. For a full list of release highlights, see the Release Notes. | 26 May 2022 [2.0.0-rc1](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.0.0-rc1.md) | The Release Candidate for 2.0.0. This version allows you to preview the upcoming 2.0.0 release before the GA release. The preview release adds document-level alerting, support for Lucene 9, and the ability to use term lookup queries in document level security. | 03 May 2022 +[1.3.8](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-1.3.8.md) | Adds OpenSearch security enhancements. Updates tool scripts to run on Windows. Includes maintenance updates and bug fixes for Anomaly Detection and OpenSearch security. | 02 February 2023 [1.3.7](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-1.3.7.md) | Adds Windows support. Includes maintenance updates and bug fixes for error handling. | 13 December 2022 [1.3.6](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-1.3.6.md) | Includes maintenance updates and bug fixes for tenancy in the OpenSearch Security Dashboards plugin. | 06 October 2022 [1.3.5](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-1.3.5.md) | Includes maintenance updates and bug fixes for gradle check and OpenSearch security. | 01 September 2022