history DB compression

[visualDust]

According to @lideming , neetbox backend is storing a lot of duplicate data. For example, when the number of CPU cores is large, the field about the CPU in status will repeat a large number of words "frequency" (one for each core). This is just an example, just to illustrate that we do store a lot of unnecessary duplicate data. Therefore, compressing the data before storing it will be a solution

Therefore, we are planning to add compression on columns which store json data using zstd

Further updates will be posted here.

[visualDust]

according to pyzstd doc, If use pre-trained zstd dictionary, the compression ratio achievable on small data (a few KiB) improves dramatically. We are going to create a table in history db to store zstd dictionary as a very initial try.

[visualDust]

good news: in ed614b8 we have broken down the status table. configs will not be stored more than once. information of hardware monitoring is in another table now. later we would add row limit to the table to keep only recent values.

[visualDust]

#110 added the feature that EventMsg sender can control the length of history. Close this issue before further problems