leveldb, manualtest: make filterbaselg configurable

[rjl493456442]

This PR makes the log size for filter block to create a new bloom filter configurable.

The main motivation is the extra data of filter block is too large. Let's assume the size of sstable is 2MB, data block size is 4KB and filter size is 2KB(all default value).

Whenever we create a new data block, we will create two bloom filter sections with 2 addition offsets. The size of offset is 4byte(actually we can use binary.Uvarint to reduce the size, but it involves some compatibility issue.

So if the average size of data entry is 200B, then there are 10 entries in this data block. With the BitsByKey as 10, then the bloom filter size is 10 * 10 / 8 = 12.5 bytes. BUT the additional offset size is 8byte(yes, we have 2, 1 totally useless). The overhead is not trivial.

With this option, we can change the size(in compatible way), to reduce the overhead.

[rjl493456442]

Btw @syndtr I don't understand why we need to generate and store the filter separately. Can we just generate a single filter when the whole sstable is finished? Perhaps smaller filter has lowerfalse positive rate?

[syndtr]

BUT the additional offset size is 8byte(yes, we have 2, 1 totally useless)

The other one is offset to the head offsets data. So here we have filter data (variable length), offsets data (variable length), 4-bytes offset of the offsets data and 1-byte baselg.

The overhead is not trivial.

Well, depending on your use case and size of your entry too. It can beneficial if you have slow backing storage that has plenty of space but slow access time as it will reduce disk seeks, regardless of the size overhead.

I don't understand why we need to generate and store the filter separately. Can we just generate a single filter when the whole sstable is finished? Perhaps smaller filter has lowerfalse positive rate?

I'm not really know the reasoning as I stole this from leveldb ;). But, I guess you're right, it has to do with lowering false positive. See: https://github.com/google/leveldb/blob/master/doc/table_format.md

However, I will merge this as it probably beneficial for some use case. Or perhaps to some custom filter that has different characteristic.