Update docs about transforms

[abey79]

What

Release 0.7 introduced a new rr.log_transform3d() and deprecated rr.log_rigid3(). This PR updates the documentation to reflect this.

• improved the wording in the space and transform section
• updated code examples
• marked log_rigid3 as deprecated in python's doc

Checklist

• I have read and agree to Contributor Guide and the Code of Conduct
• I've included a screenshot or gif (if applicable)

PR Build Summary: https://build.rerun.io/pr/2496

Docs preview: https://rerun.io/preview/bf70108/docs
Examples preview: https://rerun.io/preview/bf70108/examples

[abey79]

Here is a write up of how I approached the original text and the ways it confused me (and made me feel dumb).

Pre-existing mental model

• My main mental model adjacent to this topic is the scene graph, where many/most nodes have transforms w.r.t their parent (as such, they are typically not considered "split" in multiple "coordinate systems" by their transforms).
• Coordinates systems and transforms really are standard stuff I know about. This chapter isn't going to teach me much.

Initial instincts

Because of what precedes, my first read wasn't thorough, and I quickly developed these initial instincts:

• One space view contains/displays one space.
• Transforms === 3D to 3D affine transforms.

This proved sufficient to make useful use of the Python API, working on the example, etc. It wasn't sufficient to understand how/why everything worked, but who cares if it works, right?

Confusions

Cue actually working on the "Spaces and Transforms" docs. Soon enough it appears that my mental model of it all is broken and I need to thoroughly read this stuff (there was a bit grief curve process with some cursing at the docs for a bit). Here is a somewhat chronological of what went on in my mind:

Every Entity in Rerun exists in some Space. This is at the core of how Rerun organizes the visualizations of the data
that you have logged. In the Rerun Viewer you view data by configuring a "Space View," which is a view
of a set of entities as seen from a particular Space.

It took me many reads to realise that this didn't mean "one space view <-> one space".

A "Space" is, very loosely, a generalization of the idea of a "Coordinate System" (sometimes known as a "Coordinate Frame") to arbitrary data. If a collection of
entities are part of the same Space, it means they can be rendered together in the same view, using the same
"coordinates."

Why is that we're not just call them "coordinate systems" then?

As some examples:

• For 2d and 3d geometric primitives this means they share the same origin and coordinate system.
• For scalar plots it means they share the same plot axes.
• For text logs, it means they share the same conceptual stream.

Okaaay. That's kind of an edge case, feels kinda weird to shoehorn that into the Space concept, but ok. I'm going to ignore that text stuff and continue thinking that "transform === 3D to 3D affine transform".

Which entities belong to which Spaces is a function of the Transform system, which uses the following rules to define
the connectivity of Spaces:

1. Every unique Entity Path defines a potentially unique space.
2. Unless otherwise specified, every path is trivially connected to its parent by the Identity transform.
3. Logging a transform to a path defines the relationship between that path and its parent (replacing the Identity
   connection).
4. Only paths which are connected by the Identity transform are effectively considered to be part of the same
   Space. All others are considered to be disjoint.

Oh, so we can't have two entities with a transform displayed in the same space view. That's really stupid. (Proceeds to realise that this clearly contradicts experience)

Note that in the absence of transforms, all entity paths are fully connected by the Identity transform, and therefore
share the same Space. However, as soon as you begin to log transforms, you can end up with additional spaces.

They do insist on this stuff...

Consider the following scenario:

rr.log_points("world/mapped_keypoints", ...)
rr.log_points("world/robot/observed_features", ...)
rr.log_rigid3("world/robot", ...)

There are 4 parent/child entity relationships represented in this hierarchy.

• (root) -> world
• world -> world/mapped_keypoints
• world -> world/robot
• world/robot -> world/robot/observed_features

The call: rr.log_rigid3("world/robot", ...) only applies to the relationship: world -> world/robot because the
logged transform (world/robot) describes the relationship between the entity and its parent (world). All of the
other relationships are considered to be an identity transform.

This leaves us with two spaces. In one space, we have the entities world, and world/mapped_keypoints. In the other
space we have the entities world/robot and world/robot/observed_features.

At this point, I really read that bit hoping to get some insight. I even drew a little scene graph thing:

So we have tree. Then a transform splits the tree into two sub-trees for some reason (although there are transforms everywhere, just identity, right?). And for some reason I supposedly can't put both tree in the same space view. Long story short, that whole example (which took some effort to replay in my head) just confirmed my previous (flawed) understanding.

Practically speaking, this means that the position values of the points from world/mapped_keypoints and the points
from world/robot/observed_features are not directly comparable. If you were to directly draw these points in a single
coordinate system the results would be meaningless.

Wuut?! I mean, yeah of course, if you disregard the transforms but why would one even consider this when we're in the middle of discussing them.

Fortunately, Rerun can still put these entities in the same Space View because it is able to automatically transform data between different spaces.

Oh I knew it! That doc is contradicting itself. Pitchforks! Github Issues!

in the next section:

Rerun transforms are currently limited to connections between Spatial views of 2D or 3D data. There are 3 types of
transforms that can be logged:

• Rigid3D transforms define a pure 3D translation + rotation relationship between two paths.
  rerun.log_rigid3)
• Pinhole transforms define a 3D -> 2D camera projection. (See:
  rerun.log_pinhole)

Some realisations going on here, out of the willingness to now learn some stuff given the current level of confusion:

• Transforms can be something else than "3D to 3D affine transform".
• pinhole isn't just a glorified 3D object to represent a camera in a 3D space, it's actually a 2D to 3D transform. That's actually kind of cool, wonder if there are other such generalised transforms?

• Unknown transforms specify that the data cannot be transformed. In this case it will not be possible to combine the
  data into a single view and you will need to create two separate views to explore the data. (See:
  rerun.log_unknown_transform)

Ok so now we're splitting scene graphs, but for real? Does it mean we really can't put both sub-trees in the same Space View, this time?

Conclusion

By now, my mental model (state? 😅) probably doesn't allow me to make substantial improvements how things are presented here (beyond the probably naive changes currently included in this PR). Here are a few random thoughts though:

• Do we really need an (explicit) concept of "Space"? What would it take to remove it entirely from user-facing stuff docs/SDKs?
• Lots of concepts here are adjacent, but not fully overlapping, with "common stuff" (scene graphs, etc.)—sometime reusing the common names (transform comes to mind) in a way that is slightly uncommon. It's probably a bad thing, but if we want to keep doing that, we should very clearly dispel any a priori intuition the reader might have, and that should happen very early in the doc.
• After yet another read, I now realise that "For scalar plots it means they share the same plot axes" doesn't just mean "2D scene graph stuff / just like 3D but in 2D", without realising there there might be plot-specific semantic content there too (which I probably not fully understand yet). Shoehorning something that very much look like "regular" spatial transform into splitting plot and text streams feels weird. How does that even work? Will a transform3d really split my text log into two views?

[teh-cmc]

Drafting so it gets out of the review queue