New default merge policy - tiered

[kimchy]

A new merge policy (provided in Lucene 3.2) called tiered which does a better job at merging segments when using near real time.

Provides the following settings:

• index.merge.policy.expunge_deletes_allowed: When expungeDeletes is called, we only merge away a segment if its delete percentage is over this threshold. Default is 10.
• index.merge.policy.floor_segment: Segments smaller than this are "rounded up" to this size, ie treated as equal (floor) size for merge selection. This is to prevent frequent flushing of tiny segments from allowing a long tail in the index. Default is 2mb.
• index.merge.policy.max_merge_at_once: Maximum number of segments to be merged at a time during "normal" merging. Default is 10.
• index.merge.policy.max_merge_at_once_explicit: Maximum number of segments to be merged at a time, during optimize or expungeDeletes. Default is 30.
• index.merge.policy.max_merged_segment: Maximum sized segment to produce during normal merging. This setting is approximate: the estimate of the merged segment size is made by summing sizes of to-be-merged segments (compensating for percent deleted docs). Default is 5gb.
• index.merge.policy.segments_per_tier: Sets the allowed number of segments per tier. Smaller values mean more merging but fewer segments. Default is 10.
• index.reclaim_deletes_weight: Controls how aggressively merges that reclaim more deletions are favored. Higher values favor selecting merges that reclaim deletions. A value of 0.0 means deletions don't impact merge selection. Defaults to 2.0.
• index.compound_format: Should the index be stored in compound format or not. Defaults to false.