Memory-mapped Utility for Ankerl Hashmap #222
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
N/A
Description of changes:
Added memory-mapped utility for ankerl hashmap.
Ankerl hashmap is extremely fast at retrieval, comparing to usual B-tree based key-value stores, but without the utility of memory-map to support out-of-core lookups.
This PR adds such functionality to make ankerl hashmap a good substitution for the usual key-value stores, if user needs a memory-mappable read-only single-machine key-value store, and cares more about latency.
For 2.5Billion key-value pairs, to randomly retrieve 500 entries, the time cost:
LMDB: 5.2 ms
mmap ankerl (C/C++): 0.05ms
mmap ankerl (Python binding): 0.8ms
Usage (Python-binding): Please refer to the added unittests.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.