Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Primary cache: range bucketing #4810

Closed
teh-cmc opened this issue Jan 15, 2024 · 1 comment · Fixed by #7033
Closed

Primary cache: range bucketing #4810

teh-cmc opened this issue Jan 15, 2024 · 1 comment · Fixed by #7033
Assignees
@teh-cmc
Labels
blocked can't make progress right now 🚀 performance Optimization, memory use, etc 🔍 re_query affects re_query itself

Comments

@teh-cmc
Copy link
Member

teh-cmc commented Jan 15, 2024

Necessary to prevent OOO write complexity from growing linearly in the range cache.
@teh-cmc teh-cmc added 🔍 re_query affects re_query itself 🚀 performance Optimization, memory use, etc labels Jan 15, 2024
@teh-cmc teh-cmc self-assigned this Jan 18, 2024
@teh-cmc teh-cmc modified the milestone: 0.13 Jan 30, 2024
@teh-cmc
Copy link
Member Author

teh-cmc commented Apr 23, 2024

No point in implementing this in the cache right now while we're moving towards a world where A) the store is natively bucketized and B) the deserialization cache will be replaced by compaction on the store side.

@teh-cmc teh-cmc removed their assignment Apr 23, 2024
@teh-cmc teh-cmc added the blocked can't make progress right now label Apr 23, 2024
@teh-cmc teh-cmc self-assigned this Apr 23, 2024
@teh-cmc teh-cmc mentioned this issue Apr 30, 2024
Merged
5 tasks
teh-cmc added a commit that referenced this issue May 3, 2024
@teh-cmc
Fix missing data for "bridged" range queries (#6177)
a90b3a8 
A cheap fix for #5686, both in terms of added code and runtime
performance, as it only kicks in in very particular circumstances.
The test added in this PR explains the situation better than words ever
could.

The proper solution here would to implement proper bucketing in the
range cache (#4810), but that's a much bigger change that I don't want
to get into until after the upcoming drastic changes to the datastore.

- Fixes #5686
@teh-cmc teh-cmc mentioned this issue Aug 1, 2024
Merged
6 tasks
@teh-cmc teh-cmc closed this as completed in 4d61b04 Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@teh-cmc teh-cmc

Labels
blocked can't make progress right now 🚀 performance Optimization, memory use, etc 🔍 re_query affects re_query itself
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove legacy query engine and promises rerun-io/rerun
1 participant
@teh-cmc