feat(core/bench): add fs, monoiofs and compfs benchmark #5095
Conversation
|
Thank you very much for your work! I have noticed that Monoio performs well on single reads but not as effectively on concurrent ones. Could this be related to our thread-per-core design? Are there any plans to improve it? |
There was a problem hiding this comment.
How about calling this benchmark fs_alike? By the way, why do we need separate benchmark suites?
There was a problem hiding this comment.
This benchmark puts services in a group, criterion can generate summary graphs of benchmarks that shows difference between them.
There was a problem hiding this comment.
I'm thinking of designing a way to benchmark multiple services, like OPENDAL_TEST=fs,monoiofs,compfs. But it looks like requires some refactor to our current test infrastructure.
There was a problem hiding this comment.
I'm thinking of designing a way to benchmark multiple services, like
OPENDAL_TEST=fs,monoiofs,compfs. But it looks like requires some refactor to our current test infrastructure.
Yep, this makes more sense to me.
| @@ -0,0 +1,13 @@ | |||
| # OpenDAL services fs vs. monoiofs vs. compfs | |||
There was a problem hiding this comment.
We could support more FS interfaces.
|
|
||
| This benchmark compares the performance of OpenDAL services fs, monoiofs and compfs. | ||
|
|
||
| ## Goal |
There was a problem hiding this comment.
Hi, this isn't the benchmark goal. We shouldn't make assumptions about it. In fact, monoiofs can't outperform tokio-based fs in every case.
There was a problem hiding this comment.
In fact, monoiofs can't outperform tokio-based fs in every case.
Actually, monoiofs is expected to be slower since it is currently single threaded. I'm trying to express the final goal we wish to achieve. Did I misunderstood the point of this section?
There was a problem hiding this comment.
Did I misunderstood the point of this section?
Our goal is to measure the performance of various fs services so users can choose for themselves, rather than trying to prove that monoiofs is faster than tokio fs. Please note that OpenDAL does not declare a winner; we simply offer options.
There was a problem hiding this comment.
Thanks for pointing out that!
Monoiofs is currently single threaded ( |
|
Also, PositionWrite is not implemented for monoiofs. |
Part of #4552.
This PR introduces a benchmark that compares the performance of OpenDAL services fs, monoiofs (and compfs, but unfortunately commented out since it is still a work in progress and did not finish the benchmark).
Concurrent benchmarks uses
.chunk(size).concurrent(parallel)rather than polling several independent io tasks (which is the waybench/opsbenchmarks). Not sure which one simulates real world better.Full benchmark result is as follows. I'm going to write a progress report of monoiofs along with a brief analysis of the result on the mailing list, so stay tuned. 😋
Benchmark result