Supported database backends

[tac0turtle]

Would like to have a discussion on which dbs we should support in this repo.

• which DB's are used?

• Which should we deprecate?

[melekes]

rocksdb (fast kv store), boltdb (b+tree - useful for indexing), memdb (useful for testing) are my top picks.

we should probably add badger

not a fan of leveldb

Which should we not maintain anymore?

fsdb?

[tessr]

we should probably add badger

what motivates this? I'd generally like to see requests from users (or prospective users) before adding another database

[melekes]

I'd generally like to see requests from users (or prospective users) before adding another database

I somewhat disagree. From validators (average users) point of view, they usually want stable performant DB with adequate storage requirements. Same for developers building on top of Tendermint, with the exception that they have time / desire sometimes to investigate which DB is the best. In my mind it's not the matter of adding or removing, but providing best possible option for our users.

we should probably add badger
what motivates this?

tendermint/tendermint#1835 (comment)
either badger or bbolt should be the default, not goleveldb in my opinion.

[tac0turtle]

I somewhat disagree. From validators (average users) point of view, they usually want stable performant DB with adequate storage requirements. Same for developers building on top of Tendermint, with the exception that they have time / desire sometimes to investigate which DB is the best. In my mind it's not the matter of adding or removing, but providing best possible option for our users.

I do agree with this, and I am actually in favor of adding badgerdb, but would like to go through some testing and add more testing to this repo before we add more things.

Also, another concern of mine that I found in working on the error handling is that the tests are not run on all the dbs, I will spend time seeing what is leading to this, but would like to see what I can get rid of first so I don't spend time on them.

[tessr]

In my mind it's not the matter of adding or removing, but providing best possible option for our users.

I can agree with this--but I'd like us to make sure we have a manageable maintenance load!

[mvdan]

@melekes are you still interested in adding support for badger/v2? Is https://gist.github.com/melekes/85b8c07dd917828dc2ac6696129b73f2 your latest attempt, or has someone else continued the work since?

I'd like to use badger with the iavl library, so I could work on a pull request here if it would be welcome.

[tac0turtle]

@melekes are you still interested in adding support for badger/v2? Is gist.github.com/melekes/85b8c07dd917828dc2ac6696129b73f2 your latest attempt, or has someone else continued the work since?

I'd like to use badger with the iavl library, so I could work on a pull request here if it would be welcome.

Hey we discussed this in our team call and would welcome a pr with badger. No one has looked into adding it other than that attempt by anton.

[mvdan]

Sounds good, thanks!

[mvdan]

I've also filed #113, which I think should be fixed first. Otherwise depending on tm-db in any way is going to become far heavier, since badger/v2 itself is pretty heavy (note that it imports https://github.com/DataDog/zstd, which takes a good 20-30s to build on my machine due to all the C code).