Add size-based check to rollover API

[clintongormley]

Initially, we added just a doc-count and date-based check to the rollover API. We avoided a size-based check because it could be wildly inaccurate if multiple merges are in progress on different shards of the index.

We could make a more reasonable estimation by converting from segment sizes to # of docs which makes us less prone to merges etc. Could we sample the avg doc size on disk from existing docs and make the decision ultimately based on docs?

[PnPie]

Hi,
I would really like to have a try on this, I made some tests by calculating the size/doc from segments information and it was pretty similar to the actual size per document got from indices stats api after merges etc.
Currently the rollover condition is based on doc number and date, we get these information from indices stats with an async way and put the "rollover action" in an ActionListener:
https://github.com/elastic/elasticsearch/blob/master/core/src/main/java/org/elasticsearch/action/admin/indices/rollover/TransportRolloverAction.java#L117
So if we do as this, we also need to retrieve the segments information through _segments api using the Client interface, it means we need to call the 2 api 2 times to get the segments information and the indices stats, so for retrieving the segments information, we do it before get the indices stats (with a sync way ?) or put it in the indices stats call's listener and do it nestedly and asynchronously ?

[clintongormley]

@dakrone Do you have any thoughts about how to implement this?

[dakrone]

@dakrone Do you have any thoughts about how to implement this?

Definitely!

Since TransportRolloverAction runs on the master node, it should be able to
access ClusterInfoService directly, where we keep periodically-updated sizes
for all shards in the cluster. I'd grab all the shards for a particular index
and average the size of the primary and replica to account for merge overhead.

Personally I think inspecting the /_segments sizes is going to be overkill for
this. The only downside to the ClusterInfoService is that it refreshes every
30 seconds by default, so it might lag a little bit.

[s1monw]

@dakrone I am not sure this is what we want here. We would like to allow the user to specify an index size in order to roll over but in oder to make this work well we need to translate this to # of documents since we would love to know what would be the size of an optimized single shard index since this is the goal here IMO. So ClusterInfoService won't be enough for this unless we add an estimatedDocSize to it as well which we might wanna do. It's an information that we could use elsewhere as well. What do you think about adding this to the DocsStats?

[dakrone]

@s1monw I discussed this with @dnhatn about calculating the average document size and I think we agreed to go forward with that. Adding it to DocsStats would also be useful, especially as I know users have asked for this in the past, and we can definitely use it elsewhere :)

[dnhatn]

@dakrone @s1monw. Thanks for the hints. I will add an estimatedDocSize to the DocsStats and use it to evaluate the max_size condition in the rollover API.