Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Enhancement] Dump a new snapshot if persistent index keep too many W…
…AL (StarRocks#22140) Persistent Index for the primary key table will append wal log into `L0` index file if the following conditions are met: 1. The memory usage of `L0` is less than 10% of `L1` usage 3. The memory usage of `L0` is greater than max snapshot size(default is 16MB) So the `L0` index file may keep growing, and keep appending WAL. And this PR(StarRocks#12862) triggers flush or merge compaction if the `L0` file size is greater than 200MB to avoid the growth of `L0` index file. However, if the file size of `L1` is very large(e.g. 4GB) and the `L0` memory usage is less than `L1` 10% usage but greater than 200MB(e.g. 256MB), we will always flush a new snapshot(about 256MB) even if we only update one record which causes a lot of disk IO. So this pr change the check and dump a new snapshot if the `L0` index file keep too many wal logs.
- Loading branch information