Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mmap value store for multi-thread fast retrieval of 2D sub-matrix #257

Merged
merged 3 commits into from
Oct 3, 2023

Conversation

weiliw-amz
Copy link
Contributor

@weiliw-amz weiliw-amz commented Sep 28, 2023

Issue #, if available:
N/A

Description of changes:
Added Numerical (float32) C with Python binding mmap value store (2D matrix).

Accelerates sub-matrix retrieval given rows and cols indices, comparing to for-loop np.array.item(i, j).
For value store of 66M x 100, time cost of retrieving (Python binding) random sub-matrix 500x50, comparing to numpy for-loop is 0.09ms (4 threads) vs. 5ms

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@hallogameboy hallogameboy merged commit b4b817a into amzn:mainline Oct 3, 2023
25 checks passed
@weiliw-amz weiliw-amz deleted the c-value-store branch October 12, 2023 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants