Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add how-to guide for clearing entities #3211

Merged
merged 10 commits into from
Sep 6, 2023
92 changes: 92 additions & 0 deletions docs/content/howto/short-lived-entities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
title: Log short lived data
order: 5
description: How to log data that isn't valid for the whole recording
---
In order to create coherent views of streaming data, the Rerun Viewer shows the latest values for each visible entity at the current timepoint. But some data may not be valid for the entire recoding even if there are no updated values. How do you tell Rerun that something you've logged should no longer be shown?

## Log entities as cleared
The most straight forward option is to explicitly log that an entity has been cleared. Rerun allows you to do this by logging a special `ClearEntity` to any path. The timepoint at which the `ClearEntity` is logged is the time point after which that entity will no longer be visible in your views.

For example, if you have an object tracking application, your code might look something like this:
```python
...
for frame in sensors.read():
# Associate the following logs with `frame == frame.id`
rr.set_time_sequence("frame", frame.id)
# Do the actual tracking update
tracker.update(frame)
if tracker.is_lost:
# Clear everything on or below `tracked/{tracker.id}`
# and that happened on or before `frame == frame.id`
rr.log(f"tracked/{tracker.id}", rr.ClearEntity(recursive=True))
else:
# Log data to the main entity and a child entity
rr.log(f"tracked/{tracker.id}", rr.Rect2D(tracker.bounds))
rr.log(f"tracked/{tracker.id}/cm", rr.Point2D(tracker.cm))

```
## Clarify data meaning
In some cases, the best approach may be to rethink how you log data to better express what is actually happening. Take the following example where update frequencies don't match:

```python
...
for frame in sensors.read():
# Associate the following logs with `frame = frame.id`
rr.set_time_sequence("frame", frame.id)
# Log every image that comes in
rr.log("input/image", rr.Image(frame.image))
if frame.id % 10 == 0:
# Run detection every 10 frames
detection = detector.detect(frame)

# Woops! These detections will not update at the
# same frequency as the input data and thus look strange
rr.log("input/detections", rr.Rect2D(detection.bounds))
```
You could fix this example by logging `rr.ClearEntity`, but in this case it makes more sense to change what you log to better express what is happening. Re-logging the image to another namespace on only the frames where the detection runs makes it explicit which frame was used as the input to the detector. This will create a second view in the viewer that always allows you to see the frame that was used for the current detection input.

Here is an example fix:
```python
class Detector:
...
def detect(self, frame):
downscaled = self.downscale(frame.image)
# Log the downscaled image
rr.log("detections/source", rr.Image(downscaled))
result = self.model(downscaled)
detection = self.post_process(result)
# Log the detections together with the downscaled image
# Image and detections will update at the same frequency
rr.log("downscaled/detections", rr.Rect2D(detection.bounds))
return detection
...
for frame in sensors.read():
# Associate the following logs with `frame = frame.id`
rr.set_time_sequence("frame", frame.id)
# Log every image that comes in
rr.log("input/image", rr.Image(frame.image))
if frame.id % 10 == 0:
# Run detection every 10 frames
# Logging of detections now happens inside the detector
detected = detector.detect(frame)
```

## Log data with spans instead of timepoints
In some cases you already know how long a piece of data will be valid at the time of logging. Rerun does **not yet support** associating logged data with spans like `(from_timepoint, to_timepoint)` or `(timepoint, time-to-live)`.
nikolausWest marked this conversation as resolved.
Show resolved Hide resolved

Follow the issue [here](https://github.com/rerun-io/rerun/issues/3008).

### Workaround by manually clearing entities
For now the best workaround is to manually clear data when it is no longer valid.
```python
# Associate the following data with `start_time` on the `time` timeline
rr.set_time_seconds("time", start_time)
# Log the data as usual
rr.log("short_lived", rr.Tensor(one_second_tensor))
# Associate the following clear with `start_time + 1.0` on the `time` timeline
rr.set_time_seconds("time", start_time + 1.0)
nikolausWest marked this conversation as resolved.
Show resolved Hide resolved
# Set the time back so other data isn't accidentally logged in the future.
rr.set_time_seconds("time", start_time)
rr.log("short_lived", rr.ClearEntity())
```
2 changes: 2 additions & 0 deletions docs/cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@
"consts",
"devel",
"dicom",
"downscaled",
"dtype",
"egui",
"emilk",
Expand Down Expand Up @@ -100,6 +101,7 @@
"RGBA",
"scipy",
"scrollwheel",
"timepoints",
"trackpad",
"usize",
"venv",
Expand Down