Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize point clouds #850

Closed
1 of 12 tasks
emilk opened this issue Jan 20, 2023 · 3 comments
Closed
1 of 12 tasks

Optimize point clouds #850

emilk opened this issue Jan 20, 2023 · 3 comments
Labels
🏹 arrow concerning arrow 🚀 performance Optimization, memory use, etc

Comments

@emilk
Copy link
Member

emilk commented Jan 20, 2023

In the nyud example, SceneSpatial::load_objects goes from 8.5ms to 16ms.

Classic:
Screen Shot 2023-01-20 at 14 12 56

Arrow:
Screen Shot 2023-01-20 at 14 23 48

TODO:

  • Improve crates/re_query/benches/query_benchmark.rs with benchmarks of:
    • full primary and full secondary
    • full primary and sparse secondary
    • full primary and splat secondary
    • full primary and missing secondary
  • Realistic point cloud benchmark, similar to nyud and/or colmap, using actual Points3DPart::load
  • Write tests for ComponentJoinedIterator:
    • full primary and full secondary
    • full primary and sparse secondary
    • full primary and splat secondary
    • full primary and missing secondary
  • Optimize ComponentJoinedIterator:
    • Special case for the common case when the instance keys are identical
      (no need to compare them - if they are same length, and starts and end on the same key, with the length as the difference, then they are the same)
@emilk emilk added 🏹 arrow concerning arrow 🚀 performance Optimization, memory use, etc labels Jan 20, 2023
@emilk
Copy link
Member Author

emilk commented Jan 20, 2023

Some of this optimization work will be easier once we remove the "classic" store

@emilk emilk changed the title Arrow store performance regression Arrow store performance regression with many points Jan 25, 2023
@teh-cmc teh-cmc self-assigned this Jan 27, 2023
@rerun-io rerun-io deleted a comment from teh-cmc Feb 3, 2023
@emilk
Copy link
Member Author

emilk commented Feb 4, 2023

Thanks to #1064 we are now down to 9 ms, which is within the noise of the classic store:

image

…but I still think we should spend a few hours making a benchmark and see if there are any low-hanging fruit in ComponentJoinedIterator

@emilk emilk changed the title Arrow store performance regression with many points Optimize point clouds Feb 6, 2023
@teh-cmc teh-cmc assigned teh-cmc and unassigned teh-cmc Feb 15, 2023
@teh-cmc teh-cmc removed their assignment Feb 24, 2023
@emilk
Copy link
Member Author

emilk commented Aug 18, 2023

@emilk emilk closed this as completed Aug 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏹 arrow concerning arrow 🚀 performance Optimization, memory use, etc
Projects
None yet
Development

No branches or pull requests

2 participants