Use WEBGL_render_shared_exponent for HDR textures

[elalish]

Description

This should give better GPU memory usage than our current half-float textures, which could make a significant difference to render speeds since the environment map is a bit of a cache-killer. What do you think @WestLangley, @mrdoob?

Solution

Use WEBGL_render_shared_exponent, which apparently already has Chrome and Safari support - we should check on the rest of the browsers.

Alternatives

Leave it as is, I suppose.

Additional context

No response

[WestLangley]

Conformance test.
https://registry.khronos.org/webgl/sdk/tests/conformance2/extensions/webgl-render-shared-exponent.html

[WestLangley]

More info: see RGB9_E5 https://www.khronos.org/opengl/wiki/Small_Float_Formats

[Mugen87]

What type of typed array are you use and how do you prepare the data if you want to use RGB9_E5 e.g. with DataTexture?

Is there some sort of reference implementation that we can use similar to the code of DataUtils.toHalfFloat().

I understand the engine has to support the RGB9_E5 format as a constant first so you can configure it. But I'm interested in the actual data so we can easier test the format.

[Mugen87]

Is there some sort of reference implementation that we can use similar to the code of DataUtils.toHalfFloat().

Maybe from https://registry.khronos.org/OpenGL/extensions/EXT/EXT_texture_shared_exponent.txt:

rgb9e5 float3_to_rgb9e5(const float rgb[3])
{
  rgb9e5 retval;
  float maxrgb;
  int rm, gm, bm;
  float rc, gc, bc;
  int exp_shared;
  double denom;

  rc = ClampRange_for_rgb9e5(rgb[0]);
  gc = ClampRange_for_rgb9e5(rgb[1]);
  bc = ClampRange_for_rgb9e5(rgb[2]);

  maxrgb = MaxOf3(rc, gc, bc);
  exp_shared = Max(-RGB9E5_EXP_BIAS-1, FloorLog2(maxrgb)) + 1 + RGB9E5_EXP_BIAS;
  assert(exp_shared <= RGB9E5_MAX_VALID_BIASED_EXP);
  assert(exp_shared >= 0);
  /* This pow function could be replaced by a table. */
  denom = pow(2, exp_shared - RGB9E5_EXP_BIAS - RGB9E5_MANTISSA_BITS);

  maxm = (int) floor(maxrgb / denom + 0.5);
  if (maxm == MAX_RGB9E5_MANTISSA+1) {
    denom *= 2;
    exp_shared += 1;
    assert(exp_shared <= RGB9E5_MAX_VALID_BIASED_EXP);
  } else {
    assert(maxm <= MAX_RGB9E5_MANTISSA);
  }

  rm = (int) floor(rc / denom + 0.5);
  gm = (int) floor(gc / denom + 0.5);
  bm = (int) floor(bc / denom + 0.5);

  assert(rm <= MAX_RGB9E5_MANTISSA);
  assert(gm <= MAX_RGB9E5_MANTISSA);
  assert(bm <= MAX_RGB9E5_MANTISSA);
  assert(rm >= 0);
  assert(gm >= 0);
  assert(bm >= 0);

  retval.field.r = rm;
  retval.field.g = gm;
  retval.field.b = bm;
  retval.field.biasedexponent = exp_shared;

  return retval;
}

@elalish Does that look right?

[elalish]

I have no idea - figured it would be easier to construct the data in a shader and let the GPU driver handle the format. Might be nice in conjunction with the UltraHDR support @mrdoob is working on, since that'll involve multiplying two SDR textures to get an HDR texture.

[donmccurdy]

KTX Software can generate E5B9G9R9 from float32 EXRs as well:

ktx create --format E5B9G9R9_UFLOAT_PACK32 --zstd 18 input.exr output.ktx2

ktx create --format B10G11R11_UFLOAT_PACK32 --zstd 18 input.exr output.ktx2

Size on disk looks similar to a compressed EXR, but the memory cost would be lower. We'd probably also need to add support to detect E5B9G9R9 in THREE.KTX2Loader.

[Mugen87]

Just wanted to copy over my comment from #28012 since it might be important for this discussion:

I have tested the change with a modified version of RGBELoader and webgl_loader_texture_hdr. I did not add these changes to this PR since RGBELoader might not be the best spot for using RGB9E5. That's because RGB9E5 is not a color renderable format so we can't convert a RGB9E5 equirectangular environment map to a RGB9E5 cube map. This is required though since we don't support equirectangular textures in the shaders. If we switch to half float in the conversion process, it would not make sense to use RGB9E5 in the first place. The same issue exists with PMREM as well.

[elalish]

Uh, I thought that was the whole point of WEBGL_render_shared_exponent, which says this:

When this extension is enabled:
The RGB9_E5 floating-point internal format becomes color-renderable.

[donmccurdy]

Note that browser support is necessary but not sufficient for support on any given device, the graphics drivers also need to support it. Unfortunately overall support is currently 0.07% according to https://web3dsurvey.com/webgl2/extensions/WEBGL_render_shared_exponent. 😞

[Mugen87]

Sorry, I should have written my comment more clear. I just wanted to note that I couldn't get the RTT use case to work on any of my devices and yeah the reason is the missing WEBGL_render_shared_exponent support. Hope it improves over time.

[elalish]

Ugh, sorry, I should have checked driver support. I made the error of thinking Safari and Chrome supporting it meant it was already widely available. Well, it'll be nice someday...

[mrdoob]

Should we revert #28012?

[Mugen87]

@mrdoob Would you be okay if we add the routines in the addons folder somewhere? As mentioned in the PR RGB9E5 might be interesting for certain projects and the packing/unpacking routines aren't trivial. So it could be good to have them around especially when WEBGL_render_shared_exponent is better supported.

[Mugen87]

Never mind, I put the code into a Gist. That should be sufficient for now^^.

https://gist.github.com/Mugen87/459649eece614c4e6d57177c138757e1

[Mugen87]

Closing. We can reconsider producing RGB9E5 textures in loaders when the support for the WEBGL_render_shared_exponent extension gets better.