Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix presence of segmentation images and tensor causing generation of root space view #3681

Merged
merged 6 commits into from
Oct 5, 2023

Conversation

Wumpf
Copy link
Member

@Wumpf Wumpf commented Oct 5, 2023

What

Definitely need to come up with something better to steer the heuristics - we made good strides in space view refactor and recent advances in System heuristics, but there's still a lot to be desired!

In the test scene there's still the issue of a unexplicably large accumulated bounding box which causes the camera to zoom out a lot. This is most likely caused by an initially bad heuristic for the frustum length. I suspect this is a problem we already had in 0.8 and might be fairly hard to solve until we fixup our blueprint & object property story.

Before:
image

After:
image

Test code:

"""Log a pinhole and a random image."""
import numpy as np
import rerun as rr

rr.init("rerun_example_pinhole", spawn=True)
rng = np.random.default_rng(12345)

image = rng.uniform(0, 255, size=[3, 3, 3])
rr.log("world/camera/image", rr.Pinhole(focal_length=3, width=3, height=3))
rr.log("world/camera/image", rr.Image(image))

image = rng.uniform(0, 255, size=[3, 3, 1])
rr.log("seg", rr.SegmentationImage(image))

Also made sure this doesn't regress the fix in #3583 (which touched this code last!)

Checklist

@Wumpf Wumpf added 🪳 bug Something isn't working 📺 re_viewer affects re_viewer itself exclude from changelog PRs with this won't show up in CHANGELOG.md labels Oct 5, 2023
Copy link
Member

@emilk emilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is_interesting_space_view_at_root used to return true for segmentation images, but I don't understand how that would put it in the same spaceview as an entity with a different root?

Comment on lines 148 to 151
// If there are any images directly under the root, don't create root space either.
// -> For images we want more fine grained control and resort to child-of-root spaces only.
for entity_path in &candidate.contents.root_group().entities {
if contains_any_image(entity_path, data_store) {
if contains_tensor_data(entity_path, data_store) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this mess things up for non-2D tensors?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit concerned now. I could do the 2dness check on top and be mostly with the old behavior but this has slight regression potential on the fix from #3583
Well, let's try it out!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it would be probably best then to check for Image and SegmentationImage indicators.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. This seems safest, easiest and passes both repro scripts

@Wumpf
Copy link
Member Author

Wumpf commented Oct 5, 2023

but I don't understand how that would put it in the same spaceview as an entity with a different root?

Root is the root here, i.e. /. If we decide to create a Space View there we end up deciding it must be 3D and then things go awry

@emilk
Copy link
Member

emilk commented Oct 5, 2023

scripts/run_all.py looks good

@emilk emilk merged commit b26ca17 into main Oct 5, 2023
30 of 31 checks passed
@emilk emilk deleted the andreas/fix-spaceview-heuristic branch October 5, 2023 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🪳 bug Something isn't working exclude from changelog PRs with this won't show up in CHANGELOG.md 📺 re_viewer affects re_viewer itself
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bad heuristic results in adding inappropriate segmentation to 3D scenes
2 participants