Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bad heuristic results in adding inappropriate segmentation to 3D scenes #3677

Closed
jleibs opened this issue Oct 4, 2023 · 2 comments · Fixed by #3681
Closed

Bad heuristic results in adding inappropriate segmentation to 3D scenes #3677

jleibs opened this issue Oct 4, 2023 · 2 comments · Fixed by #3681
Assignees
Labels
🪳 bug Something isn't working 📺 re_viewer affects re_viewer itself

Comments

@jleibs
Copy link
Member

jleibs commented Oct 4, 2023

Repro:

"""Log a pinhole and a random image."""
import numpy as np
import rerun as rr

rr.init("rerun_example_pinhole", spawn=True)
rng = np.random.default_rng(12345)

image = rng.uniform(0, 255, size=[3, 3, 3])
rr.log("world/camera/image", rr.Pinhole(focal_length=3, width=3, height=3))
rr.log("world/camera/image", rr.Image(image))

image = rng.uniform(0, 255, size=[3, 3, 1])
rr.log("seg", rr.SegmentationImage(image))

image

Note this only happens with rr.SegmentationImage but not with rr.Image so there's probably some place we need to do a more complete check.

@jleibs jleibs added 🪳 bug Something isn't working 👀 needs triage This issue needs to be triaged by the Rerun team labels Oct 4, 2023
@jleibs jleibs added 📺 re_viewer affects re_viewer itself and removed 👀 needs triage This issue needs to be triaged by the Rerun team labels Oct 4, 2023
@emilk
Copy link
Member

emilk commented Oct 4, 2023

so seg and world/camera/image are added to the same spaceview, despite them not having a common ancestor O.o that's really strange

@Wumpf Wumpf self-assigned this Oct 5, 2023
@Wumpf
Copy link
Member

Wumpf commented Oct 5, 2023

that part isn't that strange since as of 0.8 (or was it 0.7 already?) we sometimes spawn a space view at the root

emilk added a commit that referenced this issue Oct 5, 2023
…root space view (#3681)

### What

* Fixes #3677

Definitely need to come up with something better to steer the heuristics
- we made good strides in space view refactor and recent advances in
System heuristics, but there's still a lot to be desired!

In the test scene there's still the issue of a unexplicably large
accumulated bounding box which causes the camera to zoom out a lot. This
is most likely caused by an initially bad heuristic for the frustum
length. I suspect this is a problem we already had in 0.8 and might be
fairly hard to solve until we fixup our blueprint & object property
story.


Before:

![image](https://github.com/rerun-io/rerun/assets/1220815/fa3e3867-aa52-47da-b0a9-83eae9436429)

After:

![image](https://github.com/rerun-io/rerun/assets/1220815/2dd20446-bc68-4c22-99ab-0465f64262dc)


Test code:
```py
"""Log a pinhole and a random image."""
import numpy as np
import rerun as rr

rr.init("rerun_example_pinhole", spawn=True)
rng = np.random.default_rng(12345)

image = rng.uniform(0, 255, size=[3, 3, 3])
rr.log("world/camera/image", rr.Pinhole(focal_length=3, width=3, height=3))
rr.log("world/camera/image", rr.Image(image))

image = rng.uniform(0, 255, size=[3, 3, 1])
rr.log("seg", rr.SegmentationImage(image))
```

Also made sure this doesn't regress the fix in #3583 (which touched this
code last!)

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3681) (if
applicable)

- [PR Build Summary](https://build.rerun.io/pr/3681)
- [Docs
preview](https://rerun.io/preview/a29d7764be26c9cc746a726c589a284ef7efad8f/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/a29d7764be26c9cc746a726c589a284ef7efad8f/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://ref.rerun.io/dev/bench/)
- [Wasm size tracking](https://ref.rerun.io/dev/sizes/)

---------

Co-authored-by: Emil Ernerfeldt <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🪳 bug Something isn't working 📺 re_viewer affects re_viewer itself
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants