Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions docs/changelog/83345.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,33 @@ summary: Add min_* conditions to rollover
area: ILM+SLM
type: enhancement
issues: []
highlight:
title: Minimum conditions for the rollover API and ILM actions
body: |-
The rollover API and ILM actions now support minimum conditions for rollover.

Minimum conditions prevent rollover from occuring until they are met. That is, an index
will rollover once one or more max conditions are satisfied and all min conditions are satisfied.

As an example, the following ILM policy would roll an index over if it is at least 7 days old or
at least 100 gigabytes, but only as long as the index is not empty.

```
PUT _ilm/policy/my_policy
{
"policy": {
"phases": {
"hot": {
"actions": {
"rollover" : {
"max_age": "7d",
"max_size": "100gb",
"min_docs": 1
}
}
}
}
}
}
```
notable: true
77 changes: 68 additions & 9 deletions docs/reference/ilm/actions/ilm-rollover.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@

Phases allowed: hot.

Rolls over a target to a new index when the existing index meets one or more of the rollover conditions.
Rolls over a target to a new index when the existing index satisfies
the specified rollover conditions.

IMPORTANT: If the rollover action is used on a <<ccr-put-follow,follower index>>,
policy execution waits until the leader index rolls over (or is
Expand Down Expand Up @@ -45,8 +46,11 @@ PUT my-index-000001
[[ilm-rollover-options]]
==== Options

You must specify at least one rollover condition.
An empty rollover action is invalid.
A rollover action must specify at least one max_* condition, it may include zero
or more min_* conditions. An empty rollover action is invalid.

The index will rollover once any max_* condition is satisfied and all
min_* conditions are satisfied.

// tag::rollover-conditions[]
`max_age`::
Expand Down Expand Up @@ -90,6 +94,32 @@ replicas are ignored.
+
TIP: To see the current shard docs, use the <<cat-shards, _cat shards>> API.
The `docs` value shows the number of documents each shard.

`min_age`::
(Optional, <<time-units, time units>>)
Prevents rollover until after the minimum elapsed time from index creation is reached.
See notes on `max_age`.

`min_docs`::
(Optional, integer)
Prevents rollover until after the specified minimum number of documents is reached.
See notes on `max_docs`.

`min_size`::
(Optional, <<byte-units, byte units>>)
Prevents rollover until the index reaches a certain size.
See notes on `max_size`.

`min_primary_shard_size`::
(Optional, <<byte-units, byte units>>)
Prevents rollover until the largest primary shard in the index reaches a certain size.
See notes on `max_primary_shard_size`.

`min_primary_shard_docs`::
(Optional, integer)
Prevents rollover until the largest primary shard in the index reaches a certain number of documents.
See notes on `max_primary_shard_docs`.

// end::rollover-conditions[]

[[ilm-rollover-ex]]
Expand All @@ -109,7 +139,7 @@ PUT _ilm/policy/my_policy
"hot": {
"actions": {
"rollover" : {
"max_primary_shard_size": "50GB"
"max_primary_shard_size": "50gb"
}
}
}
Expand All @@ -132,7 +162,7 @@ PUT _ilm/policy/my_policy
"hot": {
"actions": {
"rollover" : {
"max_size": "100GB"
"max_size": "100gb"
}
}
}
Expand Down Expand Up @@ -214,8 +244,9 @@ PUT _ilm/policy/my_policy
===== Roll over using multiple conditions

When you specify multiple rollover conditions,
the index is rolled over when _any_ of the conditions are met.
This example rolls the index over if it is at least 7 days old or at least 100 gigabytes.
the index is rolled over when _any_ of the max_* and _all_ of the min_* conditions are met.
This example rolls the index over if it is at least 7 days old or at least 100 gigabytes,
but only as long as the index is not empty.

[source,console]
--------------------------------------------------
Expand All @@ -227,7 +258,35 @@ PUT _ilm/policy/my_policy
"actions": {
"rollover" : {
"max_age": "7d",
"max_size": "100GB"
"max_size": "100gb",
"min_docs": 1
}
}
}
}
}
}
--------------------------------------------------

[ilm-rollover-conditions-ex]]
===== Roll over while maintaining shard sizes

This example rolls the index over when the primary shard size is at least 50gb,
or when the index is at least 30 days old, but only as long as a primary shard is at least 1gb.
For low-volume indices, this prevents the creation of many small shards.

[source,console]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
"policy": {
"phases": {
"hot": {
"actions": {
"rollover" : {
"max_primary_shard_size": "50gb",
"max_age": "30d",
"min_primary_shard_size": "1gb"
}
}
}
Expand All @@ -254,7 +313,7 @@ PUT /_ilm/policy/rollover_policy
"hot": {
"actions": {
"rollover": {
"max_size": "50GB"
"max_size": "50gb"
}
}
},
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ilm/index-rollover.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ On each rollover, the new index becomes the write index.
=== Automatic rollover

{ilm-init} enables you to automatically roll over to a new index based
on the index size, document count, or age. When a rollover is triggered, a new
on conditions like the index size, document count, or age. When a rollover is triggered, a new
index is created, the write alias is updated to point to the new index, and all
subsequent updates are written to the new index.

Expand Down
12 changes: 8 additions & 4 deletions docs/reference/indices/rollover-index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ include::{es-repo-dir}/indices/create-index.asciidoc[tag=index-name-reqs]

`dry_run`::
(Optional, Boolean)
If `true`, checks whether the current index matches one or more specified
If `true`, checks whether the current index satisfies the specified
`conditions` but does not perform a rollover. Defaults to `false`.

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=wait_for_active_shards]
Expand All @@ -132,10 +132,14 @@ include::{es-repo-dir}/indices/create-index.asciidoc[tag=aliases-props]
`conditions`::
(Optional, object)
Conditions for the rollover. If specified, {es} only performs the rollover if
the current index meets one or more of these conditions. If this parameter is
the current index satisfies these conditions. If this parameter is
not specified, {es} performs the rollover unconditionally.
+
IMPORTANT: To trigger a rollover, the current index must meet these conditions
If conditions are specified, at least one of them must be a max_* condition.
The index will rollover if any max_* condition is satisfied and all
min_* conditions are satisfied.
+
IMPORTANT: To trigger a rollover, the current index must satisfy these conditions
at the time of the request. {es} does not monitor the index after the API
response. To automate rollover, use {ilm-init}'s <<ilm-rollover,`rollover`>>
instead.
Expand Down Expand Up @@ -197,7 +201,7 @@ conditions were specified, this is an empty object.
====
`<condition>`::
(Boolean) The key is each condition. The value is its result. If `true`, the
index met the condition at rollover.
index met the condition.
====

[[rollover-index-api-example]]
Expand Down