Support images in BGR and BGRA

[abey79]

Describe the annoyance

There is generally no consensus between RGB and BGR image formats. For example, Pillow's PIL.Image.open() yields RGB format (including when the image is converted to a Numpy array). OpenCV's cv2.imread() or VideoCapture instead yield BGR numpy arrays.

rr.log_image() expects RGB and yields false colours otherwise. At the very least, this should be documented. Ideally, some optional format argument should be added so alternative formats may be correctly ingested.

To Reproduce
This code yields false color (e.g. blue skins):

import cv2
import rerun as rr

rr.init("demo")
rr.spawn()
image = cv2.imread("image.jpg")
rr.log_image("image", image)

This is fixed by adding explicit format conversion:

# ... 
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
rr.log_image("image", image)

Ideally, something like that would be possible:

rr.log_image("image", image, format=rr.BGR)

[abey79]

FWIW np.array(mediapipe.Image.create_from_file("image.jpg").numpy_view()) is RGB

[emilk]

This is related to adding more semantics to color tensors in general:

• RGB, BGR, RGBA, …
• Linear vs gamma
• Data range (0-255, 0-65535, …) (Allow to specify a tensor's data range via component #2341)
• Color map

[emilk]

The quick-fix is just to reorder the data into RGB on the SDK side.

The deeper fix is adding a "ColorSemantics" component or similar.

[abey79]

rr.log_image is no more, but the same applies for the rr.Image archetype.

[jleibs]

Note some places in the code indicate we should change our defaults to BGR.

I do NOT think we should do this. BGR is a bad, legacy choice from opencv. It should not be our default.

[emilk]

The simple way to implement BGR and BGRA right now is to upload them as RGB(A) textures, and add a swap_red_blue: bool flag to the rectangle.wgsl shader

[emilk]

Blocked on

• Create a UI for viewing/editing the new ImageFormat component #7100

[Wumpf]

not blocked by it, we can have bgr ofc without it being editable