Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

datastore: track bucket count in store stats & mem panel #1555

Merged
merged 2 commits into from
Mar 10, 2023
Merged

datastore: track bucket count in store stats & mem panel #1555

merged 2 commits into from
Mar 10, 2023

Conversation

teh-cmc
Copy link
Member

@teh-cmc teh-cmc commented Mar 10, 2023 •
edited
Loading

image

@teh-cmc teh-cmc added 🧑‍💻 dev experience developer experience (excluding CI) ⛃ re_datastore affects the datastore itself 🚀 performance Optimization, memory use, etc labels Mar 10, 2023
@Wumpf Wumpf self-requested a review March 10, 2023 10:37
Wumpf
Wumpf approved these changes Mar 10, 2023
View reviewed changes
Copy link
Member

@Wumpf Wumpf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

niiice!!

I'd be inclined to call temporal rather regular tho 🤷

crates/re_viewer/src/ui/memory_panel.rs

let label_rows = |ui: &mut egui::Ui, num_rows| {
if num_rows == u64::MAX {
ui.label("+∞")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... well technically not inf :D

@Wumpf
Copy link
Member

Wumpf commented Mar 10, 2023

Actually, what is taking all the memory in that application if it isn't the store?

@teh-cmc teh-cmc mentioned this pull request Mar 10, 2023
Closed
@teh-cmc
Copy link
Member Author

teh-cmc commented Mar 10, 2023

I'd be inclined to call temporal rather regular tho 🤷

This terminology is used everywhere else, so I'd rather stay consistent for now.

Actually, what is taking all the memory in that application if it isn't the store?

I haven't looked in detail, but this program ingests 600k index rows worth of data, so I imagine all of the datastructures in the viewer that scale linearly with the number of messages?

@teh-cmc teh-cmc merged commit 2977a5f into main Mar 10, 2023
@teh-cmc teh-cmc deleted the cmc/datastore_stats_improvements branch March 10, 2023 11:35
@emilk
Copy link
Member

emilk commented Mar 11, 2023

Actually, what is taking all the memory in that application if it isn't the store?

In the clock example, is the LogMsg:es stored in the viewer. Since we don't batch the log calls, we produce one huge ArrowMsg per log call.

You can see this with RERUN_TRACK_ALLOCATIONS=1 cargo r -p rerun -- ../clock_50k.rrd

@emilk emilk mentioned this pull request Mar 27, 2023
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Wumpf Wumpf Wumpf approved these changes

Assignees
No one assigned
Labels
🧑‍💻 dev experience developer experience (excluding CI) 🚀 performance Optimization, memory use, etc ⛃ re_datastore affects the datastore itself
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@teh-cmc @Wumpf @emilk