Improve searchable snapshot mount time

[henningandersen]

Reduce the range sizes we fetch during recovery to speed up mount time
until shard started.
On resource constrained setups (rate limiter, disk or network), the time
to mount multiple shards is proportional to the amount of data to fetch
and for most files in a snapshot, we need to fetch only a small piece of
the files to start the shard.

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[tlrx]

LGTM (sorry for the conflicting file)

[DaveCTurner]

LGTM2, I suggested a couple of comments.

[DaveCTurner]

Suggest a comment so we remember why we are doing this:

Suggested change

	public static final Setting<ByteSizeValue> SNAPSHOT_CACHE_RECOVERY_RANGE_SIZE_SETTING = Setting.byteSizeSetting(
	/**
	* Starting up a shard involves reading small parts of some files from the repository, independently of the pre-warming process. If we
	* expand those ranges using {@link CacheService#SNAPSHOT_CACHE_RANGE_SIZE_SETTING} then we end up reading quite a few 32MB ranges. If
	* we read enough of these ranges for the restore throttling rate limiter to kick in then all the read threads will end up waiting on
	* the throttle, blocking subsequent reads. By using a smaller read size during restore we avoid clogging up the rate limiter so much.
	*/
	public static final Setting<ByteSizeValue> SNAPSHOT_CACHE_RECOVERY_RANGE_SIZE_SETTING = Setting.byteSizeSetting(

Also suggest a similar comment on the other setting since this came up as a question in the investigation that led to this PR.

    /**
     * If a search needs data from the repository then we expand it to a larger contiguous range whose size is determined by this setting,
     * in anticipation of needing nearby data in subsequent reads. Repository reads typically have quite high latency (think ~100ms) and
     * the default of 32MB for this setting represents the approximate point at which size starts to matter. In other words, reads of
     * ranges smaller than 32MB don't usually happen much quicker, so we may as well expand all the way to 32MB ranges.
     */
    public static final Setting<ByteSizeValue> SNAPSHOT_CACHE_RANGE_SIZE_SETTING = Setting.byteSizeSetting(

[henningandersen]

Thanks Tanguy and David!