-
Notifications
You must be signed in to change notification settings - Fork 373
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Document annotation context in manual (#2453)
<!-- Open the PR up as a draft until you feel it is ready for a proper review. Do not make PR:s from your own `main` branch, as that makes it difficult for reviewers to add their own fixes. Add any improvements to the branch as new commits to make it easier for reviewers to follow the progress. All commits will be squashed to a single commit once the PR is merged into `main`. Make sure you mention any issues that this PR closes in the description, as well as any other related issues. To get an auto-generated PR description you can put "copilot:summary" or "copilot:walkthrough" anywhere. --> ### What Fixes #1337 (😎) * #1337 ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) <!-- This line will get updated when the PR build summary job finishes. --> PR Build Summary: https://build.rerun.io/pr/2453 <!-- pr-link-docs:start --> Docs preview: https://rerun.io/preview/9711742/docs Examples preview: https://rerun.io/preview/9711742/examples <!-- pr-link-docs:end -->
- Loading branch information
Showing
5 changed files
with
144 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# Annotation context with two classes, using two labeled classes, of which ones defines a color. | ||
rr.log_annotation_context( | ||
"masks", # Applies to all entities below "masks". | ||
[ | ||
rr.AnnotationInfo(id=0, label="Background"), | ||
rr.AnnotationInfo(id=1, label="Person", color=(255, 0, 0)), | ||
], | ||
) | ||
|
||
# Annotation context with simple keypoints & keypoint connections. | ||
rr.log_annotation_context( | ||
"detections", # Applies to all entities below "detections". | ||
rr.ClassDescription( | ||
info=rr.AnnotationInfo(label="Snake"), | ||
keypoint_annotations=[rr.AnnotationInfo(id=i, color=(0, 255 / 9 * i, 0)) for i in range(10)], | ||
keypoint_connections=[(i, i + 1) for i in range(9)], | ||
), | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
// Annotation context with two classes, using two labeled classes, of which ones defines a color. | ||
MsgSender::new("masks") // Applies to all entities below "masks". | ||
.with_timeless(true) | ||
.with_component(&[AnnotationContext { | ||
class_map: [ | ||
ClassDescription { | ||
info: AnnotationInfo { | ||
id: 0, | ||
label: Some(Label("Background".into())), | ||
color: None, | ||
}, | ||
..Default::default() | ||
}, | ||
ClassDescription { | ||
info: AnnotationInfo { | ||
id: 0, | ||
label: Some(Label("Person".into())), | ||
color: Some(ColorRGBA(0xFF000000)), | ||
}, | ||
..Default::default() | ||
}, | ||
] | ||
.into_iter() | ||
.map(|class| (ClassId(class.info.id), class)) | ||
.collect(), | ||
}])? | ||
.send(rec_stream)?; | ||
|
||
// Annotation context with simple keypoints & keypoint connections. | ||
MsgSender::new("detections") // Applies to all entities below "detections". | ||
.with_timeless(true) | ||
.with_component(&[AnnotationContext { | ||
class_map: std::iter::once(( | ||
ClassId(0), | ||
ClassDescription { | ||
info: AnnotationInfo { | ||
id: 0, | ||
label: Some(Label("Snake".into())), | ||
color: None, | ||
}, | ||
keypoint_map: (0..10) | ||
.map(|i| AnnotationInfo { | ||
id: i, | ||
label: None, | ||
color: Some(ColorRGBA::from_rgb(0, (255 / 9 * i) as u8, 0)), | ||
}) | ||
.map(|keypoint| (KeypointId(keypoint.id), keypoint)) | ||
.collect(), | ||
keypoint_connections: (0..9) | ||
.map(|i| (KeypointId(i), KeypointId(i + 1))) | ||
.collect(), | ||
}, | ||
)) | ||
.collect(), | ||
}])? | ||
.send(rec_stream)?; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
--- | ||
title: Annotation Context | ||
order: 4 | ||
--- | ||
|
||
## Overview | ||
|
||
Any visualization that assigns an identifier ("Class ID") to an instance or entity can benefit from using Annotations. | ||
By using an Annotation Context, you can associate labels and colors with a given class and then re-use | ||
that class across entities. | ||
|
||
<!-- Example link should point to `latest` but at the time of writing the samples just got renamed --> | ||
This is particularly useful for visualizing the output of classifications algorithms | ||
(as demonstrated by the [Detect and Track Objects](https://github.com/rerun-io/rerun/tree/main/examples/python/detect_and_track_objects) example), | ||
but can be used more generally for any kind of reoccurring categorization within a Rerun recording. | ||
|
||
![class_ids](https://static.rerun.io/5508e3fd5b2fdc020eda0bd545ccb97d26a01303_classids.png) | ||
|
||
|
||
### Keypoints & Keypoint Connections | ||
|
||
Rerun allows you to define keypoints *within* a class. | ||
Each keypoint can define its own properties (colors, labels, etc.) that overwrite its parent class. | ||
|
||
A typical example usage of keypoints is annotating the joints of a skeleton within a pose detection: | ||
In that case, the entire detected pose/skeleton is assigned a Class ID and each joint within gets a Keypoint ID. | ||
|
||
To help you more with this (and similar) use-case(s), you can also define connections between keypoints | ||
as part of your annotation class description: | ||
The viewer will draw the connecting lines for all connected keypoints whenever that class is used. | ||
Just as with labels & colors this allows you to use the same connection information on any instance that class in your scene. | ||
|
||
Keypoints are currently only applicable to 2D and 3D points. | ||
|
||
![keypoints](https://static.rerun.io/a8be4dff9cf1d2793d5a5f0d5c4bb058d1430ea8_keypoints.png) | ||
|
||
|
||
### Logging an Annotation Context | ||
|
||
Annotation Context is typically logged as [timeless](timelines.md#timeless-data) data, but can change over time if needed. | ||
|
||
The Annotation Context is defined as a list of Class Descriptions that define how classes are styled | ||
(as well as optional keypoint style & connection). | ||
|
||
Annotation contexts are logged with: | ||
|
||
* Python: [`log_annotation_context`](https://ref.rerun.io/docs/python/latest/common/annotations/#rerun.log_annotation_context) | ||
* Rust: [`AnnotationContext`](https://docs.rs/rerun/latest/rerun/external/re_log_types/component_types/context/struct.AnnotationContext.html) | ||
|
||
code-example: annotation-context | ||
|
||
|
||
## Affected Entities | ||
|
||
Each entity that uses a Class ID component (and optionally Keypoint ID components) will look for | ||
the nearest ancestor that in the [entity path hierarchy](entity-path.md#path-hierarchy-functions) that has an Annotation Context defined. | ||
|
||
|
||
## Segmentation images | ||
|
||
Segmentation images are single channel integer images/tensors where each pixel represents a class id. | ||
By default, Rerun will automatically assign colors to each class id, but by defining an Annotation Context, | ||
you can explicitly determine the color of each class. | ||
|
||
* Python: [`log_segmentation_image`](https://ref.rerun.io/docs/python/latest/common/images/#rerun.log_segmentation_image) | ||
* Rust: Log a [`Tensor`](https://docs.rs/rerun/latest/rerun/components/struct.Tensor.html) with [`TensorDataMeaning::ClassId`](https://docs.rs/rerun/latest/rerun/components/enum.TensorDataMeaning.html#variant.ClassId) | ||
|
||
![segmentation image](https://static.rerun.io/7c47738b791a7faaad8f0221a78c027300d407fc_segmentation_image.png) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
c75ec58
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold
1.25
.datastore/num_rows=1000/num_instances=1000/packed=false/insert/default
3836563
ns/iter (± 49017
)2815756
ns/iter (± 5430
)1.36
datastore/num_rows=1000/num_instances=1000/packed=false/latest_at/default
404
ns/iter (± 3
)309
ns/iter (± 4
)1.31
datastore/num_rows=1000/num_instances=1000/packed=false/latest_at_missing/primary/default
298
ns/iter (± 3
)225
ns/iter (± 1
)1.32
datastore/num_rows=1000/num_instances=1000/packed=false/range/default
3954464
ns/iter (± 53453
)2856450
ns/iter (± 8882
)1.38
datastore/num_rows=1000/num_instances=1000/gc/default
2602652
ns/iter (± 31228
)1729278
ns/iter (± 3254
)1.51
mono_points_arrow/encode_log_msg
223522994
ns/iter (± 2534389
)176586531
ns/iter (± 932130
)1.27
mono_points_arrow_batched/generate_message_bundles
24639172
ns/iter (± 1360185
)18605463
ns/iter (± 124567
)1.32
mono_points_arrow_batched/generate_messages
5584000
ns/iter (± 315096
)3643465
ns/iter (± 28319
)1.53
mono_points_arrow_batched/encode_log_msg
625539
ns/iter (± 5753
)419995
ns/iter (± 1491
)1.49
mono_points_arrow_batched/encode_total
29051937
ns/iter (± 978623
)23206851
ns/iter (± 138981
)1.25
batch_points_arrow/encode_log_msg
69833
ns/iter (± 916
)48759
ns/iter (± 97
)1.43
batch_points_arrow/decode_log_msg
48626
ns/iter (± 293
)38636
ns/iter (± 91
)1.26
arrow_mono_points/insert
2802171376
ns/iter (± 25309002
)1815602553
ns/iter (± 12143193
)1.54
arrow_mono_points/query
1300268
ns/iter (± 21229
)944816
ns/iter (± 1916
)1.38
arrow_batch_points/query
16486
ns/iter (± 247
)12394
ns/iter (± 20
)1.33
arrow_batch_vecs/query
458262
ns/iter (± 5986
)322117
ns/iter (± 617
)1.42
This comment was automatically generated by workflow using github-action-benchmark.