Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically visualize point clouds from depth maps #1333

Closed
teh-cmc opened this issue Feb 16, 2023 · 2 comments
Closed

Automatically visualize point clouds from depth maps #1333

teh-cmc opened this issue Feb 16, 2023 · 2 comments
Assignees
Labels
enhancement New feature or request 🔺 re_renderer affects re_renderer itself

Comments

@teh-cmc
Copy link
Member

teh-cmc commented Feb 16, 2023

Following a discussion with @Wumpf, we would like to experiment with a Renderer that is capable of raytracing through discrete volumes (i.e. an Amanatides & Woo kinda thing).

This renderer should ideally be capable of using either a 3D texture + bounding box or a 2D texture (interpreted as depth) + bounding box as input and render that in a spatial view as needed.

Our first real use-case would be to try and render depth maps in spatial views.

Today, the only way we have at our disposal to do that is to project the depth map back into 3D on the client-side, build a point cloud out of that, and upload the result to Rerun (see nyud example).
Of course this is pretty costly and doesn't scale nicely with the resolution: a 4K depth map would result in a 8M+ point cloud each frame 😰

@teh-cmc teh-cmc added enhancement New feature or request 🔺 re_renderer affects re_renderer itself labels Feb 16, 2023
@teh-cmc teh-cmc self-assigned this Feb 17, 2023
@teh-cmc
Copy link
Member Author

teh-cmc commented Feb 19, 2023

Here's a sneak peek of the first experiments on that front.

As it stands right now it is completely unusable in real conditions, not to mention the math that barely makes any sense at all... but it does show that it can look both fairly fancy and much more "readable" than a point cloud indeed!

23-02-19_17.51.49.patched.mp4

@teh-cmc teh-cmc changed the title Depth maps as discrete raytraced volumes Automatically visualize point clouds from depth maps Mar 3, 2023
@teh-cmc
Copy link
Member Author

teh-cmc commented Mar 3, 2023

Implemented in #1421

@teh-cmc teh-cmc closed this as completed Mar 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request 🔺 re_renderer affects re_renderer itself
Projects
None yet
Development

No branches or pull requests

1 participant