Mesh3D archetype

[teh-cmc]

Implements the Mesh3D archetype:

table Mesh3D {
  vertex_positions: [rerun.components.Position3D] ("attr.rerun.component_required", order: 1000);
  vertex_normals: [rerun.components.Vector3D] ("attr.rerun.component_optional", nullable, order: 3000);
  vertex_colors: [rerun.components.Color] ("attr.rerun.component_optional", nullable, order: 3100);

  mesh_material: rerun.components.Material ("attr.rerun.component_optional", nullable, order: 3200);
  mesh_properties: rerun.components.MeshProperties ("attr.rerun.component_recommended", nullable, order: 2000);

  class_ids: [rerun.components.ClassId] ("attr.rerun.component_optional", nullable, order: 3300);
  instance_keys: [rerun.components.InstanceKey] ("attr.rerun.component_optional", nullable, order: 3400);
}

An interesting facet of the Mesh3D archetype is that it's composed of two kinds of data:

• Per-vertex data, which uses our usual query system with joins and everything, allowing the mesh's components to be viewed in different ways, partially updated, etc.
• Per-mesh data, which is essentially global data that applies to the mesh as a whole and is kind of a splat but not really.

Because the vertices becomes the instances, meshes cannot be batched.

There's a bit of a mess here and there because the ancestors of Mesh3D & Asset3D used to be tightly coupled.
This will be fixed in the next PR that implements the next-gen Asset3D archetype.

No dedicated roundtrip tests: API examples already cover everything.

Fixes #2788

---

Here's an example of partial updates:

rr.init("rerun_example_mesh3d_partial_updates", spawn=True)

vertex_positions = np.array([[-1.0, 0.0, 0.0], [1.0, 0.0, 0.0], [0.0, 1.0, 0.0]], dtype=np.float32)

# Log the initial state of our triangle
rr.set_time_sequence("frame", 0)
rr2.log(
    "triangle",
    rr2.Mesh3D(
        vertex_positions,
        vertex_normals=[0.0, 0.0, 1.0],
        vertex_colors=[[255, 0, 0], [0, 255, 0], [0, 0, 255]],
    ),
)

# Only update its vertices' positions each frame
factors = np.abs(np.sin(np.arange(1, 300, dtype=np.float32) * 0.04))
for i, factor in enumerate(factors):
    rr.set_time_sequence("frame", i)
    rr2.log_components("triangle", [rr2.cmp.Position3DArray.from_similar(vertex_positions * factor)])

23-09-19_15.45.08.patched.mp4

---

Checklist

• I have read and agree to Contributor Guide and the Code of Conduct
• I've included a screenshot or gif (if applicable)
• I have tested demo.rerun.io (if applicable)

• PR Build Summary
• Docs preview
• Examples preview
• Recent benchmark results
• Wasm size tracking

[teh-cmc]

Python colors are broken now that I've fixed the nullability crash... Looking... fixed

[emilk]

Why not call this mesh_indices: rerun.components.MeshIndices instead?

[teh-cmc]

Feels likely like we're gonna want to stash a few more things in there sooner than later (winding order, index format, ...) 🤷‍♂️

[emilk]

MeshTriangles maybe? "properties" is very vague. I would expect there to be color stuff in there, not fundamental stuff like indices.

[teh-cmc]

Im not necessarily a fan of MeshProperties but I like MeshTriangles even less tbh.

• This really has nothing to do with triangles at this point (it's now vertex_indices)
• Indices are not "fundamental", quite the contrary they're optional :/

Anyway, time to ship this; feel free to change it in another PR.

[teh-cmc]

Added vertex colors support to raw_mesh.py while I'm here fixing colors... 🌈

[teh-cmc]

One thing to keep in mind: this will make large meshes (like hundreds of thousands of vertices and more) quite slower than before, since meshes are now more akin to point clouds: all vertex properties are joined etc according to their instance keys.

We could easily bypass that join (this PR actually used to do that) to regain that performance, at the cost of violating our query model and therefore user expectations.
Considering the kind of slowdown we're talking about (e.g. handful of milliseconds for ARKit scene), and the fact that #3107 would remove the join cost entirely in 99.9% use cases, I vote to keep this as-is.

For reference, arkit before:

and arkit after:

[emilk]

Amazing!

[emilk]

Now that we have as_component_batches, can't try_to_arrow just call self.as_component_batches and then call try_to_arrow() on each component batch? A lot less code needs generating. I guess we get an additional dyn call per component, but that seems worth it

[teh-cmc]

• Reimplement Archetype::try_to_arrow in terms of as_component_batches #3386

[teh-cmc]

One thing to keep in mind: this will make large meshes (like hundreds of thousands of vertices and more) quite slower than before, since meshes are now more akin to point clouds: all vertex properties are joined etc according to their instance keys.

We could easily bypass that join (this PR actually used to do that) to regain that performance, at the cost of violating our query model and therefore user expectations. Considering the kind of slowdown we're talking about (e.g. handful of milliseconds for ARKit scene), and the fact that #3107 would remove the join cost entirely in 99.9% use cases, I vote to keep this as-is.

For reference, arkit before: and arkit after:

#3377 wins us back a millisecond:

The only way to do better at this point would be to not deserialize at all when we take the optimized path, instead working with raw ArrowBuffers in that case (which is effectively what the legacy view did):

• Expose ArchetypeView API to deal with raw arrow buffers when join is bypassed #3379

[emilk]

There's almost 2ms not covered by puffin scopes

[teh-cmc]

There's almost 2ms not covered by puffin scopes

(Yes, this is much faster than previous screenshots... because this is a debug build (???!!!!))

It makes no sense why we're spending so much time where we're spending it but whatever, that's a problem for another day.