Document pixel format naming convention and surface format

[smcv]

Description

• pixels: Document the naming convention

  Whenever I have to fix something endianness-related, I always get
  confused about whether the byte-oriented format that guarantees to put
  red in byte 0 is RGBA8888 or RGBA32. (The answer is that it's RGBA32.)

• surface: Document the in-memory layout of the pixels

  I don't think we ever explicitly said this.

Existing Issue(s)

Prompted by #8315.

[fdwr]

I think I'm being confused by SDL's terminology for image formats again.

@smcv 😉 I was confused too. I'm currently accumulating a large collection of pixel formats across various hardware and graphics API's (like afrantzis's list but more comprehensive to include older bitplane formats too, like Amiga and 2D video game consoles), which includes SDL pixel formats naturally. Below is a subset of what I have. It's important to me that a pixel format of a given name unambiguously conveys the layout, and that reading data from a file across different devices is consistent, where a b8g8r8a8 data file means exactly that (b8,g8,r8,a8 - blue in byte 0, green, red, and finally alpha in byte 3) regardless of endianness, which it sounds from your documentation update that these are the stable formats which satisfy that:

struct SDL_PIXELFORMAT_RGB24
{
    uint8_t redUnorm8;
    uint8_t greenUnorm8;
    uint8_t blueUnorm8;
};

struct SDL_PIXELFORMAT_BGR24
{
    uint8_t blueUnorm8;
    uint8_t greenUnorm8;
    uint8_t redUnorm8;
};

struct SDL_PIXELFORMAT_BGRA32
{
    uint8_t blueUnorm8;
    uint8_t greenUnorm8;
    uint8_t redUnorm8;
    uint8_t alphaUnorm8;
};

struct SDL_PIXELFORMAT_RGBA32
{
    uint8_t redUnorm8;
    uint8_t greenUnorm8;
    uint8_t blueUnorm8;
    uint8_t alphaUnorm8;
};

struct SDL_PIXELFORMAT_ARGB32
{
    uint8_t alphaUnorm8;
    uint8_t redUnorm8;
    uint8_t greenUnorm8;
    uint8_t blueUnorm8;
};

...

And these are less stable formats, where the meaning flips depending on the architecture:

// Note for the following format, RGBA is actually stored as ARGB
// on the vast majority of machines today (little endian).
// On big endian machines, RGBA is stored as RGBA.
// So a suffix is appended here for clarity.
struct SDL_PIXELFORMAT_RGBA8888_LE
{
    uint8_t alphaUnorm8;
    uint8_t blueUnorm8;
    uint8_t greenUnorm8;
    uint8_t redUnorm8;
};

struct SDL_PIXELFORMAT_RGBA8888_BE
{
    uint8_t redUnorm8;
    uint8_t greenUnorm8;
    uint8_t blueUnorm8;
    uint8_t alphaUnorm8;
};

Correct?

Reading through documentation of many other formats, it sounds like PFNC, DXGI, Metal, and WebGPU all decided to go with memory order of nomenclature, and that WebGPU's bgra8unorm always means {b,g,r,a} regardless of architecture endianness (though for rgb10a2uint, I bet WebGPU treats it as a single uint32 meaning endianness matters). So it's good to have this clarity - thanks for updating the docs.

[smcv]

It's important to me that ... reading data from a file across different devices is consistent

In SDL world, formats with names like RGBA32 have this property, but formats with names like RGBA8888 do not.

less stable formats, where the meaning flips depending on the architecture

If the format is described in terms of a byte sequence (as is common in a file on disk), the meaning of RGBA32 is stable between architectures, and the meaning of RGBA8888 flips depending on the architecture. But, if the format is thought of as being a sequence of 32-bit words, the opposite is true: RGBA32 could be said not to be stable between architectures, because it has the alpha in either the most or least significant 8 bits of the first 32-bit word depending on architecture; while RGBA8888 could be said to be stable between architectures, because it always has the alpha in the least significant 8 bits (with the way those bits happen to be arranged in memory treated as an implementation detail). Neither way of looking at a format is necessarily wrong, so I think it's important to be specific about which one you mean.

I think your descriptions in terms of structs are accurate, but I am not a SDL maintainer and what I say is not necessarily authoritative.

rgb10a2uint

This would presumably be described as either RGBA1010102 or ABGR2101010 in SDL, depending whether endianness conventions are the same or reversed. It's difficult to see how a format like this, where the channels do not line up neatly with bytes, could be described in terms of byte-sequences in a machine-independent way, so I suspect that endianness will matter for such formats - it certainly does in SDL.

[fdwr]

Thanks again Simon for confirming (at least as far as you know) and improving the documentation.

[slouken]

@smcv is correct. :)