Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow adding a 2D image to a 3D space view #3932

Open
emilk opened this issue Oct 19, 2023 · 2 comments
Open

Allow adding a 2D image to a 3D space view #3932

emilk opened this issue Oct 19, 2023 · 2 comments
Labels
enhancement New feature or request 📺 re_viewer affects re_viewer itself user-request This is a pressing issue for one of our users

Comments

@emilk
Copy link
Member

emilk commented Oct 19, 2023

Based on https://discord.com/channels/1062300748202921994/1164158709103083612

Sometimes you want to show a 2D image with some 3D data next to it, like at 1:34 in this video: https://youtu.be/bKUAZ7IQcf0?si=x6VGZTB8usL90I2v&t=94

To accomplish this one should be able to do rr.log("3d/image", rr.Image(…)) + rr.log("3d/paths", rr.LineStrips3D(paths)), and then add both entities to the same 3D space view. However, this is currently not allowed:

image

Repro:

import numpy as np
import rerun as rr

rr.init("rerun_example_test", spawn=True)
image = np.zeros((8, 12, 3), dtype=np.uint8)
rr.log("world/image", rr.Image(image))
rr.log("world/points", rr.Points3D([[0, 0, 0], [1, 1, 1], [2, 2, 2]]))
@emilk emilk added enhancement New feature or request 👀 needs triage This issue needs to be triaged by the Rerun team 📺 re_viewer affects re_viewer itself user-request This is a pressing issue for one of our users and removed 👀 needs triage This issue needs to be triaged by the Rerun team labels Oct 19, 2023
@jleibs
Copy link
Member

jleibs commented Oct 20, 2023

Interestingly we only recently prevented this, as of: #3822

Rather than treating images as optionally being 3D, I'm inclined to require the explicit addition of an orthographic projection:

rr.log("world/image", rr.Image(image), rr.Orthographic.IDENTITY)
rr.log("world/points", rr.Points3D([[0, 0, 0], [1, 1, 1], [2, 2, 2]]))

Which would have the added bonus of also allowing us to control the mapping between 3D and 2D coordinates in cases where pixel coordinates don't make sense for the 3d content.

@jleibs jleibs closed this as completed Oct 20, 2023
@jleibs
Copy link
Member

jleibs commented Oct 20, 2023

Github misinterpreted my comment as a resolution

@jleibs jleibs reopened this Oct 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request 📺 re_viewer affects re_viewer itself user-request This is a pressing issue for one of our users
Projects
None yet
Development

No branches or pull requests

2 participants