diff --git a/docs/management/images/index-lifecycle-policies-create.png b/docs/management/images/index-lifecycle-policies-create.png deleted file mode 100644 index f6d86fa9b4ea5..0000000000000 Binary files a/docs/management/images/index-lifecycle-policies-create.png and /dev/null differ diff --git a/docs/management/images/index_lifecycle_policies_options.png b/docs/management/images/index_lifecycle_policies_options.png deleted file mode 100644 index 184188be181e8..0000000000000 Binary files a/docs/management/images/index_lifecycle_policies_options.png and /dev/null differ diff --git a/docs/management/images/index_management_add_policy.png b/docs/management/images/index_management_add_policy.png deleted file mode 100644 index f0fe493a2e491..0000000000000 Binary files a/docs/management/images/index_management_add_policy.png and /dev/null differ diff --git a/docs/management/index-lifecycle-policies/add-policy-to-index.asciidoc b/docs/management/index-lifecycle-policies/add-policy-to-index.asciidoc deleted file mode 100644 index 0fec62d895754..0000000000000 --- a/docs/management/index-lifecycle-policies/add-policy-to-index.asciidoc +++ /dev/null @@ -1,17 +0,0 @@ -[role="xpack"] -[[adding-policy-to-index]] -=== Adding a policy to an index - -To add a lifecycle policy to an index and view the status for indices -managed by a policy, open the menu, then go to *Stack Management > Data > Index Management*. -This page lists your -{es} indices, which you can filter by lifecycle status and lifecycle phase. - -To add a policy, select the index name and then select *Manage Index > Add lifecycle policy*. -You’ll see the policy name, the phase the index is in, the current -action, and if any errors occurred performing that action. - -To remove a policy from an index, select *Manage Index > Remove lifecycle policy*. - -[role="screenshot"] -image::images/index_management_add_policy.png[][UI for adding a policy to an index] diff --git a/docs/management/index-lifecycle-policies/create-policy.asciidoc b/docs/management/index-lifecycle-policies/create-policy.asciidoc deleted file mode 100644 index 7849ef6b92054..0000000000000 --- a/docs/management/index-lifecycle-policies/create-policy.asciidoc +++ /dev/null @@ -1,93 +0,0 @@ -[role="xpack"] -[[creating-index-lifecycle-policies]] -=== Creating an index lifecycle policy - -An index lifecycle policy enables you to define rules over when to perform -certain actions, such as a rollover or force merge, on an index. Index lifecycle -management automates execution of those actions at the right time. - -When you create an index lifecycle policy, consider the tradeoffs between -performance and availability. As you move your index through the lifecycle, -you’re likely moving your data to less performant hardware and reducing the -number of shards and replicas. It’s important to ensure that the index -continues to have enough replicas to prevent data loss in the event of failures. - -*Index Lifecycle Policies* is automatically enabled in {kib}. Open the menu, then go to -*Stack Management > {es} > Index Lifecycle Policies*. - -NOTE: If you don’t want to use this feature, you can disable it by setting -`xpack.ilm.enabled` to false in your `kibana.yml` configuration file. If you -disable *Index Management*, then *Index Lifecycle Policies* is also disabled. - -[role="screenshot"] -image::images/index-lifecycle-policies-create.png[][UI for creating an index lifecycle policy] - -==== Defining the phases of the index lifecycle - -You can define up to four phases in the index lifecycle. For each phase, you -can enable actions to optimize performance for that phase. - -The four phases in the index lifecycle are: - -* *Hot.* The index is actively being queried and written to. You can -roll over to a new index when the -original index reaches a specified size, document count, or age. When a rollover occurs, a new -index is created, added to the index alias, and designated as the new “hot” -index. You can still query the previous indices, but you only ever write to -the “hot” index. See <>. - -* *Warm.* The index is typically searched at a lower rate than when the data is -hot. The index is not used for storing new data, but might occasionally add -late-arriving data, for example, from a Beat with a network problem that's now fixed. -You can optionally shrink the number replicas and move the shards to a -different set of nodes with smaller or less performant hardware. You can also -reduce the number of primary shards and force merge the index into -smaller {ref}/indices-segments.html[segments]. - -* *Cold.* The index is no longer being updated and is seldom queried, but is -still searchable. If you have a big deployment, you can move it to even -less performant hardware. You might also reduce the number of replicas because -you expect the data to be queried less frequently. To keep the index searchable -for a longer period, and reduce the hardware requirements, you can use the -{ref}/frozen-indices.html[freeze action]. Queries are slower on a frozen index because the index is -reloaded from the disk to RAM on demand. - -* *Delete.* The index is no longer relevant. You can define when it is safe to -delete it. - -The index lifecycle always includes an active hot phase. The warm, cold, and -delete phases are optional. For example, you might define all four phases for -one policy and only a hot and delete phase for another. See {ref}/_actions.html[Actions] -for more information on the actions available in each phase. - -[[setting-a-rollover-action]] -==== Setting a rollover action - -The {ref}/indices-rollover-index.html[rollover] action enables you to automatically roll over to a new index based -on the index size, document count, or age. Rolling over to a new index based on -these criteria is preferable to time-based rollovers. Rolling over at an arbitrary -time often results in many small indices, which can have a negative impact on performance and resource usage. - -When you create an index lifecycle policy, the rollover action is enabled -by default. The default size for triggering the rollover is 50 gigabytes, and -the default age is 30 days. The rollover occurs when any of the criteria are met. - -With the rollover action enabled, you can move to the warm phase on rollover or you can -time the move for a specified number of hours or days after the rollover. The -move to the cold and delete phases is based on the time from the rollover. - -If you are using daily indices (created by Logstash or another client) and you -want to use the index lifecycle policy to manage aging data, you can -disable the rollover action in the hot phase. You can then -transition to the warm, cold, and delete phases based on the time of index creation. - -==== Setting the index priority - -For the hot, warm, and cold phases, you can set a priority for recovering -indices after a node restart. Indices with higher priorities are recovered -before indices with lower priorities. By default, the index priority is set to -100 in the hot phase, 50 in the warm phase, and 0 in the cold phase. -If the cold phase of one index has data that -is more important than the data in the hot phase of another, you might increase -the index priority in the cold phase. See -{ref}/recovery-prioritization.html[Index recovery prioritization]. diff --git a/docs/management/index-lifecycle-policies/intro-to-lifecycle-policies.asciidoc b/docs/management/index-lifecycle-policies/intro-to-lifecycle-policies.asciidoc deleted file mode 100644 index ba1d79710de05..0000000000000 --- a/docs/management/index-lifecycle-policies/intro-to-lifecycle-policies.asciidoc +++ /dev/null @@ -1,30 +0,0 @@ -[role="xpack"] -[[index-lifecycle-policies]] -== Index Lifecycle Policies - -If you're working with time series data, you don't want to continually dump -everything into a single index. Instead, you might periodically roll over the -data to a new index to keep it from growing so big it's slow and expensive. -As the index ages and you query it less frequently, you’ll likely move it to -less expensive hardware and reduce the number of shards and replicas. - -To automatically move an index through its lifecycle, you can create a policy -to define actions to perform on the index as it ages. Index lifecycle policies -are especially useful when working with {beats-ref}/beats-reference.html[Beats] -data shippers, which continually -send operational data, such as metrics and logs, to Elasticsearch. You can -automate a rollover to a new index when the existing index reaches a specified -size or age. This ensures that all indices have a similar size instead of having -daily indices where size can vary based on the number of Beats and the number -of events sent. - -{kib}’s *Index Lifecycle Policies* walks you through the process for creating -and configuring a policy. Before using this feature, you should be familiar -with index lifecycle management: - -* For an introduction, refer to -{ref}/getting-started-index-lifecycle-management.html[Getting started with index -lifecycle management]. -* To dig into the concepts and technical details, see -{ref}/index-lifecycle-management.html[Managing the index lifecycle]. -* To check out the APIs, see {ref}/index-lifecycle-management-api.html[Index lifecycle management API]. diff --git a/docs/management/index-lifecycle-policies/manage-policy.asciidoc b/docs/management/index-lifecycle-policies/manage-policy.asciidoc deleted file mode 100644 index 8e2dc96de4b99..0000000000000 --- a/docs/management/index-lifecycle-policies/manage-policy.asciidoc +++ /dev/null @@ -1,34 +0,0 @@ -[role="xpack"] -[[managing-index-lifecycle-policies]] -=== Managing index lifecycle policies - -Your configured policies appear on the *Index lifecycle policies* page. -You can update an existing index lifecycle policy to fix errors or change -strategies for newly created indices. To edit a policy, select its name. - -[role="screenshot"] -image::images/index_lifecycle_policies_options.png[][UI for viewing and editing an index lifecycle policy] - -In addition, you can: - -* *View indices linked to the policy.* This is important when editing a policy. -Any changes you make affect all indices attached to the policy. The settings -for the current phase are cached, so the update doesn’t affect that phase. This -prevents conflicts when you’re modifying a phase that is currently executing on -an index. The changes takes effect when the next phase in the index lifecycle begins. - -* *Add the policy to an index template.* When an index is automatically -created using the index template, the policy is applied. If the index is rolled -over, the policies for any matching index templates are applied to the newly -created index. For more information, see {ref}/indices-templates.html[Index templates]. - -* *Delete a policy.* You can’t delete a policy that is currently in use or -recover a deleted index. - -[float] -=== Required permissions - -The `manage_ilm` cluster privilege is required to access *Index lifecycle policies*. - -You can add these privileges in *Stack Management > Security > Roles*. - diff --git a/docs/redirects.asciidoc b/docs/redirects.asciidoc index 92e02de89f181..d8c200450d7e5 100644 --- a/docs/redirects.asciidoc +++ b/docs/redirects.asciidoc @@ -113,8 +113,33 @@ This content has moved. See This content has moved. See {ref}/ccr-getting-started.html#ccr-getting-started-remote-cluster[Connect to a remote cluster]. +[role="exclude",id="adding-policy-to-index"] +== Adding a policy to an index + +This content has moved. See +{ref}/set-up-lifecycle-policy.html[Configure a lifecycle policy]. + +[role="exclude",id="creating-index-lifecycle-policies"] +== Creating an index lifecycle policy + +This content has moved. See +{ref}/set-up-lifecycle-policy.html[Configure a lifecycle policy]. + +[role="exclude",id="index-lifecycle-policies"] +== Index Lifecycle Policies + +This content has moved. See +{ref}/index-lifecycle-management.html[ILM: Manage the index lifecycle]. + +[role="exclude",id="managing-index-lifecycle-policies"] +== Managing index lifecycle policies + +This content has moved. See +{ref}/index-lifecycle-management.html[ILM: Manage the index lifecycle]. + [role="exclude",id="tutorial-define-index"] == Define your index patterns This content has moved. See <>. + diff --git a/docs/user/management.asciidoc b/docs/user/management.asciidoc index e0d550a15a907..c371aa695c475 100644 --- a/docs/user/management.asciidoc +++ b/docs/user/management.asciidoc @@ -40,7 +40,7 @@ a| <> flushing, and clearing the cache. Practicing good index management ensures that your data is stored cost effectively. -| <> +| {ref}/index-lifecycle-management.html[Index Lifecycle Policies] |Create a policy for defining the lifecycle of an index as it ages through the hot, warm, cold, and delete phases. Such policies help you control operation costs @@ -180,14 +180,6 @@ include::{kib-repo-dir}/management/alerting/connector-management.asciidoc[] include::{kib-repo-dir}/management/managing-beats.asciidoc[] -include::{kib-repo-dir}/management/index-lifecycle-policies/intro-to-lifecycle-policies.asciidoc[] - -include::{kib-repo-dir}/management/index-lifecycle-policies/create-policy.asciidoc[] - -include::{kib-repo-dir}/management/index-lifecycle-policies/manage-policy.asciidoc[] - -include::{kib-repo-dir}/management/index-lifecycle-policies/add-policy-to-index.asciidoc[] - include::{kib-repo-dir}/management/managing-indices.asciidoc[] include::{kib-repo-dir}/management/ingest-pipelines/ingest-pipelines.asciidoc[]