Skip to content

Commit

Permalink
Static data 2: static-aware Python SDK (#5536)
Browse files Browse the repository at this point in the history
Just exposing all the new static stuff to the Python SDK, and trying to
kill the "timeless" terminology in the process.

---

Part of a PR series that removes the concept of timeless data in favor
of the much simpler concept of static data:
- #5534
- #5535
- #5536
- #5537
- #5540
  • Loading branch information
teh-cmc authored Apr 5, 2024
1 parent 08109da commit 800f409
Show file tree
Hide file tree
Showing 71 changed files with 257 additions and 178 deletions.
2 changes: 1 addition & 1 deletion crates/re_query_cache/tests/latest_at.rs
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,7 @@ fn invalidation() {

// Test the following scenario:
// ```py
// rr.log("points", rr.Points3D([1, 2, 3]), timeless=True)
// rr.log("points", rr.Points3D([1, 2, 3]), static=True)
//
// # Do first query here: LatestAt(+inf)
// # Expected: points=[[1,2,3]] colors=[]
Expand Down
2 changes: 1 addition & 1 deletion crates/re_query_cache/tests/range.rs
Original file line number Diff line number Diff line change
Expand Up @@ -439,7 +439,7 @@ fn invalidation() {

// Test the following scenario:
// ```py
// rr.log("points", rr.Points3D([1, 2, 3]), timeless=True)
// rr.log("points", rr.Points3D([1, 2, 3]), static=True)
//
// # Do first query here: LatestAt(+inf)
// # Expected: points=[[1,2,3]] colors=[]
Expand Down
2 changes: 1 addition & 1 deletion docs/content/concepts/spaces-and-transforms.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ Note that none of the names in the paths are special.

You can use [`rr.ViewCoordinates`](https://ref.rerun.io/docs/python/stable/common/archetypes/#rerun.archetypes.ViewCoordinates) to set your preferred view coordinate systems, giving semantic meaning to the XYZ axes of the space.

For 3D spaces it can be used to log what the up-axis is in your coordinate system. This will help Rerun set a good default view of your 3D scene, as well as make the virtual eye interactions more natural. This can be done with `rr.log("world", rr.ViewCoordinates(up="+Z"), timeless=True)`.
For 3D spaces it can be used to log what the up-axis is in your coordinate system. This will help Rerun set a good default view of your 3D scene, as well as make the virtual eye interactions more natural. This can be done with `rr.log("world", rr.ViewCoordinates(up="+Z"), static=True)`.

You can also use this `log_view_coordinates` for pinhole entities, but it is encouraged that you instead use [`rr.log(…, rr.Pinhole(camera_xyz=…))`](https://ref.rerun.io/docs/python/stable/common/archetypes/#rerun.archetypes.Pinhole) for this. The default coordinate system for pinhole entities is `RDF` (X=Right, Y=Down, Z=Forward).

Expand Down
12 changes: 6 additions & 6 deletions docs/content/concepts/timelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ An _event_ refer to an instance of logging one or more component batches to one
</picture>


## Timeless data
## Static data

The [`rr.log()`](https://ref.rerun.io/docs/python/stable/common/logging_functions/#rerun.log) function has a `timeless=False` default argument.
If `timeless=True` is used instead, the entity become *timeless*. Timeless entities belong to all timelines (existing ones, and ones not yet created) and are shown leftmost in the time panel in the viewer.
This is useful for entities that aren't part of normal data capture, but set the scene for how they are shown.
For instance, if you are logging cars on a street, perhaps you want to always show a street mesh as part of the scenery, and for that it makes sense for that data to be timeless.
The [`rr.log()`](https://ref.rerun.io/docs/python/stable/common/logging_functions/#rerun.log) function has a `static=False` default argument.
If `static=True` is used instead, the data logged becomes *static*. Static data belongs to all timelines (existing ones, and ones not yet created) and shadows any temporal data of the same type on the same entity.
This is useful for data that isn't part of normal data capture, but sets the scene for how it should be shown.
For instance, if you are logging cars on a street, perhaps you want to always show a street mesh as part of the scenery, and for that it makes sense for that data to be static.

Similarly, [coordinate systems](spaces-and-transforms.md) or [annotation context](annotation-context.md) are typically timeless.
Similarly, [coordinate systems](spaces-and-transforms.md) or [annotation context](annotation-context.md) are typically static.
2 changes: 1 addition & 1 deletion docs/content/howto/ros2-nav-turtlebot.md
Original file line number Diff line number Diff line change
Expand Up @@ -421,7 +421,7 @@ def urdf_callback(self, urdf_msg: String) -> None:
rerun_urdf.log_scene(scene=scaled,
node=urdf.base_link,
path="map/robot/urdf",
timeless=True)
static=True)
```

Back in `rerun_urdf.log_scene` all the code is doing is recursively walking through
Expand Down
4 changes: 2 additions & 2 deletions docs/content/reference/data-loaders/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,9 @@ When the viewer and/or SDK executes an external loader, it will pass to it a set

Recommended prefix to prepend to all entity paths.

* `--timeless` (optional)
* `--static` (optional)

The data is expected to be logged timelessly.
The data is expected to be logged as static.

* `--time <timeline1>=<time1> <timeline2>=<time2> …` (optional)

Expand Down
4 changes: 2 additions & 2 deletions docs/snippets/all/annotation-context/example.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
rr.AnnotationInfo(id=1, label="Person", color=(255, 0, 0)),
],
),
timeless=True,
static=True,
)

# Annotation context with simple keypoints & keypoint connections.
Expand All @@ -20,5 +20,5 @@
keypoint_annotations=[rr.AnnotationInfo(id=i, color=(0, 255 / 9 * i, 0)) for i in range(10)],
keypoint_connections=[(i, i + 1) for i in range(9)],
),
timeless=True,
static=True,
)
2 changes: 1 addition & 1 deletion docs/snippets/all/annotation_context_connections.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
)
]
),
timeless=True,
static=True,
)

rr.log(
Expand Down
2 changes: 1 addition & 1 deletion docs/snippets/all/annotation_context_rects.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
rr.init("rerun_example_annotation_context_rects", spawn=True)

# Log an annotation context to assign a label and color to each class
rr.log("/", rr.AnnotationContext([(1, "red", (255, 0, 0)), (2, "green", (0, 255, 0))]), timeless=True)
rr.log("/", rr.AnnotationContext([(1, "red", (255, 0, 0)), (2, "green", (0, 255, 0))]), static=True)

# Log a batch of 2 rectangles with different `class_ids`
rr.log("detections", rr.Boxes2D(mins=[[-2, -2], [0, 0]], sizes=[[3, 3], [2, 2]], class_ids=[1, 2]))
Expand Down
2 changes: 1 addition & 1 deletion docs/snippets/all/annotation_context_segmentation.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,6 @@
image[100:180, 130:280] = 2

# Log an annotation context to assign a label and color to each class
rr.log("segmentation", rr.AnnotationContext([(1, "red", (255, 0, 0)), (2, "green", (0, 255, 0))]), timeless=True)
rr.log("segmentation", rr.AnnotationContext([(1, "red", (255, 0, 0)), (2, "green", (0, 255, 0))]), static=True)

rr.log("segmentation/image", rr.SegmentationImage(image))
2 changes: 1 addition & 1 deletion docs/snippets/all/asset3d_out_of_tree.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

rr.init("rerun_example_asset3d_out_of_tree", spawn=True)

rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, timeless=True) # Set an up-axis
rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True) # Set an up-axis

rr.set_time_sequence("frame", 0)
rr.log("world/asset", rr.Asset3D(path=sys.argv[1]))
Expand Down
2 changes: 1 addition & 1 deletion docs/snippets/all/asset3d_simple.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@

rr.init("rerun_example_asset3d", spawn=True)

rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, timeless=True) # Set an up-axis
rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True) # Set an up-axis
rr.log("world/asset", rr.Asset3D(path=sys.argv[1]))
8 changes: 4 additions & 4 deletions docs/snippets/all/scalar_multiple_plots.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@
lcg_state = np.int64(0)

# Set up plot styling:
# They are logged timeless as they don't change over time and apply to all timelines.
# They are logged as static as they don't change over time and apply to all timelines.
# Log two lines series under a shared root so that they show in the same plot by default.
rr.log("trig/sin", rr.SeriesLine(color=[255, 0, 0], name="sin(0.01t)"), timeless=True)
rr.log("trig/cos", rr.SeriesLine(color=[0, 255, 0], name="cos(0.01t)"), timeless=True)
rr.log("trig/sin", rr.SeriesLine(color=[255, 0, 0], name="sin(0.01t)"), static=True)
rr.log("trig/cos", rr.SeriesLine(color=[0, 255, 0], name="cos(0.01t)"), static=True)
# Log scattered points under a different root so that they show in a different plot by default.
rr.log("scatter/lcg", rr.SeriesPoint(), timeless=True)
rr.log("scatter/lcg", rr.SeriesPoint(), static=True)

# Log the data on a timeline called "step".
for t in range(0, int(tau * 2 * 100.0)):
Expand Down
2 changes: 1 addition & 1 deletion docs/snippets/all/segmentation_image_simple.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,6 @@
rr.init("rerun_example_segmentation_image", spawn=True)

# Assign a label and color to each class
rr.log("/", rr.AnnotationContext([(1, "red", (255, 0, 0)), (2, "green", (0, 255, 0))]), timeless=True)
rr.log("/", rr.AnnotationContext([(1, "red", (255, 0, 0)), (2, "green", (0, 255, 0))]), static=True)

rr.log("image", rr.SegmentationImage(image))
6 changes: 3 additions & 3 deletions docs/snippets/all/series_line_style.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@
rr.init("rerun_example_series_line_style", spawn=True)

# Set up plot styling:
# They are logged timeless as they don't change over time and apply to all timelines.
# They are logged as static as they don't change over time and apply to all timelines.
# Log two lines series under a shared root so that they show in the same plot by default.
rr.log("trig/sin", rr.SeriesLine(color=[255, 0, 0], name="sin(0.01t)", width=2), timeless=True)
rr.log("trig/cos", rr.SeriesLine(color=[0, 255, 0], name="cos(0.01t)", width=4), timeless=True)
rr.log("trig/sin", rr.SeriesLine(color=[255, 0, 0], name="sin(0.01t)", width=2), static=True)
rr.log("trig/cos", rr.SeriesLine(color=[0, 255, 0], name="cos(0.01t)", width=4), static=True)

# Log the data on a timeline called "step".
for t in range(0, int(tau * 2 * 100.0)):
Expand Down
6 changes: 3 additions & 3 deletions docs/snippets/all/series_point_style.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
rr.init("rerun_example_series_point_style", spawn=True)

# Set up plot styling:
# They are logged timeless as they don't change over time and apply to all timelines.
# They are logged as static as they don't change over time and apply to all timelines.
# Log two point series under a shared root so that they show in the same plot by default.
rr.log(
"trig/sin",
Expand All @@ -17,7 +17,7 @@
marker="circle",
marker_size=4,
),
timeless=True,
static=True,
)
rr.log(
"trig/cos",
Expand All @@ -27,7 +27,7 @@
marker="cross",
marker_size=2,
),
timeless=True,
static=True,
)

# Log the data on a timeline called "step".
Expand Down
2 changes: 1 addition & 1 deletion docs/snippets/all/view_coordinates_simple.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

rr.init("rerun_example_view_coordinates", spawn=True)

rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, timeless=True) # Set an up-axis
rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True) # Set an up-axis
rr.log(
"world/xyz",
rr.Arrows3D(
Expand Down
12 changes: 6 additions & 6 deletions examples/python/arkit_scenes/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,10 @@
vertex_colors=mesh.visual.vertex_colors,
indices=mesh.faces,
),
timeless=True,
static=True,
)
```
Here, the mesh is logged to the [world/mesh entity](recording://world/mesh) and is marked as timeless, since it does not
Here, the mesh is logged to the [world/mesh entity](recording://world/mesh) and is marked as static, since it does not
change in the context of this visualization.
### 3D bounding boxes
Expand Down Expand Up @@ -121,7 +121,7 @@ def log_annotated_bboxes(annotation: dict[str, Any]) -> None:
rotations=rr.Quaternion(xyzw=rot.as_quat()),
labels=label,
),
timeless=True,
static=True,
)


Expand Down Expand Up @@ -213,7 +213,7 @@ def log_arkit(recording_path: Path, include_highres: bool) -> None:
None
"""
rr.log("description", rr.TextDocument(DESCRIPTION, media_type=rr.MediaType.MARKDOWN), timeless=True)
rr.log("description", rr.TextDocument(DESCRIPTION, media_type=rr.MediaType.MARKDOWN), static=True)

video_id = recording_path.stem
lowres_image_dir = recording_path / "lowres_wide"
Expand Down Expand Up @@ -242,7 +242,7 @@ def log_arkit(recording_path: Path, include_highres: bool) -> None:
timestamp = f"{round(float(timestamp), 3):.3f}"
camera_from_world_dict[timestamp] = camera_from_world

rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, timeless=True)
rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True)
ply_path = recording_path / f"{recording_path.stem}_3dod_mesh.ply"
print(f"Loading {ply_path}…")
assert os.path.isfile(ply_path), f"Failed to find {ply_path}"
Expand All @@ -255,7 +255,7 @@ def log_arkit(recording_path: Path, include_highres: bool) -> None:
vertex_colors=mesh.visual.vertex_colors,
indices=mesh.faces,
),
timeless=True,
static=True,
)

# load the obb annotations and log them in the world frame
Expand Down
4 changes: 2 additions & 2 deletions examples/python/clock/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,12 @@ def rotate(angle: float, len: float) -> tuple[float, float, float]:
0.0,
)

rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Y_UP, timeless=True)
rr.log("world", rr.ViewCoordinates.RIGHT_HAND_Y_UP, static=True)

rr.log(
"world/frame",
rr.Boxes3D(half_sizes=[LENGTH_S, LENGTH_S, 1.0], centers=[0.0, 0.0, 0.0]),
timeless=True,
static=True,
)

for step in range(steps):
Expand Down
8 changes: 4 additions & 4 deletions examples/python/controlnet/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,8 @@ def run_canny_controlnet(image_path: str, prompt: str, negative_prompt: str) ->
canny_image = np.concatenate([canny_image, canny_image, canny_image], axis=2)
canny_image = PIL.Image.fromarray(canny_image)

rr.log("input/raw", rr.Image(image), timeless=True)
rr.log("input/canny", rr.Image(canny_image), timeless=True)
rr.log("input/raw", rr.Image(image), static=True)
rr.log("input/canny", rr.Image(canny_image), static=True)

controlnet = ControlNetModel.from_pretrained(
"diffusers/controlnet-canny-sdxl-1.0",
Expand All @@ -95,8 +95,8 @@ def run_canny_controlnet(image_path: str, prompt: str, negative_prompt: str) ->

pipeline.enable_model_cpu_offload()

rr.log("positive_prompt", rr.TextDocument(prompt), timeless=True)
rr.log("negative_prompt", rr.TextDocument(negative_prompt), timeless=True)
rr.log("positive_prompt", rr.TextDocument(prompt), static=True)
rr.log("negative_prompt", rr.TextDocument(negative_prompt), static=True)

images = pipeline(
prompt,
Expand Down
6 changes: 3 additions & 3 deletions examples/python/detect_and_track_objects/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
The color and label for each class is determined by the
[rr.AnnotationContext archetype](https://www.rerun.io/docs/reference/types/archetypes/annotation_context) which is
logged to the root entity using `rr.log("/", …, timeless=True` as it should apply to the whole sequence and all
logged to the root entity using `rr.log("/", …, static=True` as it should apply to the whole sequence and all
entities that have a class id.
### Detections
Expand Down Expand Up @@ -369,7 +369,7 @@ def track_objects(video_path: str, *, max_frame_count: int | None) -> None:
class_descriptions = [
rr.AnnotationInfo(id=cat["id"], color=cat["color"], label=cat["name"]) for cat in coco_categories
]
rr.log("/", rr.AnnotationContext(class_descriptions), timeless=True)
rr.log("/", rr.AnnotationContext(class_descriptions), static=True)

detector = Detector(coco_categories=coco_categories)

Expand Down Expand Up @@ -460,7 +460,7 @@ def main() -> None:

setup_logging()

rr.log("description", rr.TextDocument(DESCRIPTION, media_type=rr.MediaType.MARKDOWN), timeless=True)
rr.log("description", rr.TextDocument(DESCRIPTION, media_type=rr.MediaType.MARKDOWN), static=True)

video_path: str = args.video_path
if not video_path:
Expand Down
2 changes: 1 addition & 1 deletion examples/python/dicom_mri/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ def list_dicom_files(dir: Path) -> Iterable[Path]:


def read_and_log_dicom_dataset(dicom_files: Iterable[Path]) -> None:
rr.log("description", rr.TextDocument(DESCRIPTION, media_type=rr.MediaType.MARKDOWN), timeless=True)
rr.log("description", rr.TextDocument(DESCRIPTION, media_type=rr.MediaType.MARKDOWN), static=True)

voxels_volume, _ = extract_voxel_data(dicom_files)

Expand Down
2 changes: 1 addition & 1 deletion examples/python/dna/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@


def log_data() -> None:
rr.log("description", rr.TextDocument(DESCRIPTION, media_type=rr.MediaType.MARKDOWN), timeless=True)
rr.log("description", rr.TextDocument(DESCRIPTION, media_type=rr.MediaType.MARKDOWN), static=True)

rr.set_time_seconds("stable_time", 0)

Expand Down
9 changes: 5 additions & 4 deletions examples/python/external_data_loader/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,8 @@
parser.add_argument("--application-id", type=str, help="optional recommended ID for the application")
parser.add_argument("--recording-id", type=str, help="optional recommended ID for the recording")
parser.add_argument("--entity-path-prefix", type=str, help="optional prefix for all entity paths")
parser.add_argument(
"--timeless", action="store_true", default=False, help="optionally mark data to be logged as timeless"
)
parser.add_argument("--timeless", action="store_true", default=False, help="deprecated: alias for `--static`")
parser.add_argument("--static", action="store_true", default=False, help="optionally mark data to be logged as static")
parser.add_argument(
"--time",
type=str,
Expand Down Expand Up @@ -77,7 +76,9 @@ def main() -> None:
with open(args.filepath) as file:
body = file.read()
text = f"""## Some Python code\n```python\n{body}\n```\n"""
rr.log(entity_path, rr.TextDocument(text, media_type=rr.MediaType.MARKDOWN), timeless=args.timeless)
rr.log(
entity_path, rr.TextDocument(text, media_type=rr.MediaType.MARKDOWN), static=args.static or args.timeless
)


def set_time_from_args() -> None:
Expand Down
8 changes: 4 additions & 4 deletions examples/python/face_tracking/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ def __init__(self, video_mode: bool = False):
rr.ClassDescription(
info=rr.AnnotationInfo(id=0), keypoint_connections=[(0, 1), (1, 2), (2, 0), (2, 3), (0, 4), (1, 5)]
),
timeless=True,
static=True,
)

def detect_and_log(self, image: npt.NDArray[np.uint8], frame_time_nano: int) -> None:
Expand Down Expand Up @@ -223,11 +223,11 @@ def __init__(self, video_mode: bool = False, num_faces: int = 1):
)
)

rr.log("video/landmarker", rr.AnnotationContext(class_descriptions), timeless=True)
rr.log("reconstruction", rr.AnnotationContext(class_descriptions), timeless=True)
rr.log("video/landmarker", rr.AnnotationContext(class_descriptions), static=True)
rr.log("reconstruction", rr.AnnotationContext(class_descriptions), static=True)

# properly align the 3D face in the viewer
rr.log("reconstruction", rr.ViewCoordinates.RDF, timeless=True)
rr.log("reconstruction", rr.ViewCoordinates.RDF, static=True)

def detect_and_log(self, image: npt.NDArray[np.uint8], frame_time_nano: int) -> None:
height, width, _ = image.shape
Expand Down
6 changes: 3 additions & 3 deletions examples/python/gesture_detection/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Meanwhile, the 3D points allows the creation of a 3D model of the hand for a mor

The 2D and 3D points are logged through a combination of two archetypes.
For the 2D points, the Points2D and LineStrips2D archetypes are utilized. These archetypes help visualize the points and connect them with lines, respectively.
As for the 3D points, the logging process involves two steps. First, a timeless [`ClassDescription`](https://www.rerun.io/docs/reference/types/datatypes/class_description) is logged, that contains the information which maps keypoint ids to labels and how to connect
As for the 3D points, the logging process involves two steps. First, a static [`ClassDescription`](https://www.rerun.io/docs/reference/types/datatypes/class_description) is logged, that contains the information which maps keypoint ids to labels and how to connect
the keypoints. Defining these connections automatically renders lines between them. Mediapipe provides the `HAND_CONNECTIONS` variable which contains the list of `(from, to)` landmark indices that define the connections.
Second, the actual keypoint positions are logged in 3D [`Points3D`](https://www.rerun.io/docs/reference/types/archetypes/points3d) archetype.

Expand All @@ -73,10 +73,10 @@ rr.log(
keypoint_connections=mp.solutions.hands.HAND_CONNECTIONS,
)
),
timeless=True,
static=True,
)

rr.log("Hand3D", rr.ViewCoordinates.LEFT_HAND_Y_DOWN, timeless=True)
rr.log("Hand3D", rr.ViewCoordinates.LEFT_HAND_Y_DOWN, static=True)
```

### 2D Points
Expand Down
Loading

0 comments on commit 800f409

Please sign in to comment.