Give state_col 90% of memory budget and fix other col calculation

[bkchr]

No description provided.

[arkpar]

I suppose it would be good to increase default rocksdb cache size as well, as it appears to affect performance significantly.

[dvdplm]

@arkpar the default is the same 128Mb per column as before – do you suggest a higher value?

[bkchr]

I could just increase the db_cache size in Substrate to 1024.

[arkpar]

@dvdplm I was referring to the default value set by substrate/cli, but apparently it is None which uses DB_DEFAULT_COLUMN_MEMORY_BUDGET_MB. Strangely though we are observing significant performance regression in substrate, even though it is still using this default.

@bkchr Let's set it to 256Mib

[dvdplm]

Strangely though we are observing significant performance regression in substrate

On eth we're seeing some some terrible performance regressions on block import benchmarks too when using this pr: it's 10x slower. It could be related to the config changes and in that case it could impact substrate as well?

[arkpar]

Strangely though we are observing significant performance regression in substrate

On eth we're seeing some some terrible performance regressions on block import benchmarks too when using this pr: it's 10x slower. It could be related to the config changes and in that case it could impact substrate as well?

Looks like it does, as we've merged paritytech/parity-common#257. Increasing cache size seems to help with performance though.