[Android] Support HDR output

[DarkKilauea]

Based on #94496
Implements godotengine/godot-proposals#10817 for Android
Testing/Sample project: https://github.com/DarkKilauea/godot-hdr-output

Adds support for rendering HDR output on Android devices, should only be merged after #94496. See the last commit for android specific details.

HDR 10 support is added back in via this PR due to my Android devices requiring it in order to show an HDR output. Looking at https://developer.android.com/reference/android/view/Display.HdrCapabilities it doesn't look like there is any support for HDR output via a linear Rec 709 color space.

Known issues:

• Forward Mobile seems to squash the dynamic range too much to really show the "HDR effect".
• SDR white level is a guess due to the lack of APIs to get the actual value. Works well on the Pixel phones I have.
• Not sure if Window.setColorMode is actually necessary to get HDR output to show.
• Binding Java methods in Godot is new to me, so they may not be correct or may be bound in the wrong classes.

[Calinou]

• Forward Mobile seems to squash the dynamic range too much to really show the "HDR effect".

godotengine/godot-proposals#7878 might help with this, but with a performance cost.

[DarkKilauea]

Rebased on the latest HDR code

[DarkKilauea]

Added back in support for HDR10 in order to get HDR working on my Google Pixel devices. It seems to be working now.

Luminance does not currently adjust when the screen brightness is changed, so that still needs to be added. There is also a bug where HDR must be turned off, then back on, in order to work.

[DarkKilauea]

Added support for adjusting dynamically to HDR/SDR ratio changes (for devices that support that), falling back to checking every 500 ms for devices that do not.

I think this is ready to be reviewed now.

[DarkKilauea]

@Calinou I believe the contrast issues may be fixed now, could you try again when you get a chance?

Thanks.

[m4gr3d]

This should include a p_window_id parameter that defaults to MAIN_WINDOW_ID.
I'm working on adding multi-window support to Android, so I'd imagine we want to update the hdr output per window.

[m4gr3d]

Per my comment above, this should be a Map<WindowID, bool>

[m4gr3d]

Per my comment above, this should be a Map<WindowID, float>

[m4gr3d]

Per my comment above, this should be a Map<WindowID, float>

[Calinou]

@Calinou I believe the contrast issues may be fixed now, could you try again when you get a chance?

Thanks.

I've tested it again at various brightness levels on the device, and the issue still appears to be here. It seems the black level is raised over the entire scene in both 2D and 3D (even pure black in the 2D HDR tester seems to be more like a shade of dark gray).

Is it possible to take HDR screenshots on an Android device so I can (try to) send an example of what it looks like? If not, I can try to take photos with another device with fixed settings.

[allenwp]

Thanks for your work on this, DarkKilauea!

I've finally had a chance to try this PR with a Google Pixel 9a that I picked up a few weeks back. Here are some notes:

1. HDR luminance values do not appear to exactly match SDR luminance values: When I toggle HDR off, I see that SDR is brighter.
2. Reported reference luminance increases from 125 up to a maximum of 250 when the user decreases their screen brightness setting to its lowest setting. The maximum luminance does not increase, so this results in a decreasing output_max_linear_value as they turn their brightness down. A medium screen brightness has a reference luminance of 125 and a dark screen brightness has a reference luminance of 250 (backwards from what it should be). This backwards behaviour does not happen when the user turns up their screen brightness and the reference luminance reasonably increases up to a maximum of 507.
3. When the user changes their screen brightness setting to be dark, but not so dark that it begins to exhibit the previous problem, Godot continues to report an output_max_linear_value of 8.0, but the phone doesn't actually present this high of a dynamic range. Here is a photo comparing the Android (top) with the expected behaviour from an iPhone (bottom) with a similar brightness setting:

(Notice that SDR parts of the image are a similar brightness, but parts of the image that should be up to 8 times brighter than SDR reference white are not that bright.)

I also have an EXR HDR version of this photo. You can turn up the exposure on this HDR image to see that SDR parts of the image are approximately the same brightness: https://drive.google.com/file/d/15HD_B3B3iOZ7dxFizZyEjtntWsBEzzQB/view?usp=sharing

Additionally, the max brightness magenta colour on the right side of the colour sweep looks especially dark compared to the other colours in the colour sweep on Android HDR. This does not happen when running in SDR mode on Android. My suspicion is that this is related to some sort of tonemapping that is being applied to the image.

I have not yet tested how this Pixel 9a behaves when presenting through ASurfaceControl as described in the previous reviews. It's possible that all of these issues may be resolved by changing to that approach. I personally am not able to say for sure without trying it first.

I've tried opening the Sponza scene as described by Calinou... But I am not able to reproduce any contrast issues. The only issues with the Sponza scene are those I described above. It's possible this contrast issue is specific to how the Samsung Galaxy S25 Ultra handles tonemapping for HDR10, but I couldn't say for sure.

The next step in understanding these issues will likely to be trying a prototype of presenting via ASurfaceControl instead of the vulkan swapchain to see if that resolves the issues...

[K1aymore]

I've finally had a chance to try this PR with a Google Pixel 9a that I picked up a few weeks back.

Nice tests! I also have a Pixel 9a, was this done with the Enhanced HDR Brightness setting enabled or disabled? That may be a reason for strange reported values, although since it was enabled by default for me, it would be best to handle both with it off and with it on at various intensities.

There is also an Extra Dim setting (and Night Light and Color Correction) which might affect it too. Color Correction seems to be passed to the application somehow, because some apps restart after it's toggled, but we may not need to handle it in any special way.

[DarkKilauea]

I implemented as a second commit support for setting a maximum output value for HDR.

Unfortunately it appears that the HDR10 buffer is still required, trying to switch back to the linear buffer leads to a loss of color depth, even if the brightness does increase to the set limit.

I took some screenshots, though they don't show the entire range:

Linear	HDR10

[allenwp]

I've finally had a chance to try this PR with a Google Pixel 9a that I picked up a few weeks back.

Nice tests! I also have a Pixel 9a, was this done with the Enhanced HDR Brightness setting enabled or disabled? That may be a reason for strange reported values, although since it was enabled by default for me, it would be best to handle both with it off and with it on at various intensities.

There is also an Extra Dim setting (and Night Light and Color Correction) which might affect it too. Color Correction seems to be passed to the application somehow, because some apps restart after it's toggled, but we may not need to handle it in any special way.

Strangely, my Google Pixel 9a does not have an "Enhanced HDR Brightness" mode; it only has an "Adaptive brightness" mode. Those photos and tests were done with Adaptive brightness, Night Light, and Extra dim turned off and Colors set to Natural.

I implemented as a second commit support for setting a maximum output value for HDR.

This seems to have introduced a new issue where get_output_max_linear_value is now returning a different value than reported by the "Show HDR/SDR ratio" developer setting when the screen brightness is turned up: the Godot reported value is now fixed at 4.0, regardless of the device screen brightness setting and device HDR/SDR ratio.

[m4gr3d]

Is this needed if the above setting is only implemented on Android?

[allenwp]

I strongly feel that for 4.7 the Android Android "desired HDR headroom" should be hardcoded to something like 8.0 (this is the highest that my Google Pixel is able to go and also the highest my iPhone 13 Pro is able to go, so this makes sense as a good number to hardcode) and not exposed to the user in any way.

Godot 4.8 will introduce a new Window.output_linear_value_limit property which will be available on all platforms. For most platforms it will probably be best to default to 128.0 (for reasons I will describe in my upcoming proposal) and for Android and iOS it will probably be best to default to 8.0.

In 4.8, this new property may be linked directly to the Android "desired HDR headroom" and equivalent on iOS.

I am focused on finishing 4.7 for now, but once that's wrapped up, I will start work on the user-facing Window.output_linear_value_limit property and all of the related platform-specific features.

[allenwp]

We chatted about how to address the issues described in the comments here and here in the last rendering meeting. There are a few things left to look into.

From my understanding, these two issues are caused by GPU drivers (or some code that is outside of our control) performing some sort of tonemapping or gamut mapping to bring an HDR10 colour space (a.k.a. VK_COLOR_SPACE_HDR10_ST2084_EXT) into the range that the device is capable of handling. If this is the root problem, the solution is to not use VK_COLOR_SPACE_HDR10_ST2084_EXT at all anywhere in Godot because no matter what way we swing it, some bit of code outside of our control will be converting this HDR10 colour space and performing this tonemapping that we want to avoid.

I really don't know what is possible because I haven't worked much with Vulkan or Android APIs, but from what I've read and discussed with others, I expect the solution is to do something like this...

1. Render in Vulkan as normal like we currently do in master (VK_FORMAT_R16G16B16A16_SFLOAT)
2. Set the destination for Vulkan to be VK_FORMAT_A2R10G10B10_UNORM_PACK32 with VK_COLOR_SPACE_SRGB_NONLINEAR_KHR (nonlinear sRGB-encoded 10 bit buffer)
   • Or maybe we need to use a different colour space like VK_COLOR_SPACE_PASS_THROUGH_EXT, but I'm not sure if anything like that is available on all devices that support HDR, so maybe we're forced to VK_COLOR_SPACE_SRGB_NONLINEAR_KHR.
3. Scale values based on the Window.output_max_linear_value (a.k.a Android HDR ratio) just before writing to this A2R10G10B10 buffer. This could be done as follows:
   • Set the reference_multiplier used in blit.glsl to equal Window.output_max_linear_value
   • Add color.rgb /= data.reference_multiplier; to blit.glsl
4. Do some magic to pipe Vulkan result to the ASurfaceControl to be interpreted as an extended dynamic range image that is all ready to send out to the display as-is according to the correctly configured HDR ratios without any further transformations.

If this approach doesn't work, the HDR mode will appear too dark. So it will be pretty easy to test.

Regarding step 3, this where it makes sense to add color.rgb /= data.reference_multiplier for a quick prototype:

} else if (data.target_color_space == COLOR_SPACE_REC709_NONLINEAR_SRGB) {
	// Negative values and values above 1.0 will be clipped by the target,
	// so no need to clip them here.
	if (data.source_is_srgb == false) {
		// Android HDR scaling (make sure the reference_multiplier equals 1.0 when not using Android HDR)
		color.rgb /= data.reference_multiplier;
		
		// linear -> sRGB conversion
		color.rgb = linear_to_srgb(color.rgb);

		// Even if debanding was applied earlier in the rendering process, it must
		// be reapplied after the linear_to_srgb floating point operations.
		// When the linear_to_srgb operation was not performed, the source is
		// already an 8-bit format and debanding cannot be effective. In this
		// case, GPU driver rounding error can add noise so debanding should be
		// skipped entirely.
		if (data.use_debanding) {
			color.rgb += screen_space_dither(gl_FragCoord.xy);
		}
	}
}

For the final PR, if this approach works, we need to add a new 10-bit screen_space_dither function to blit.glsl, just like what is done in tonemap_mobile.glsl. And so maybe it makes sense to have something like this instead with a whole new COLOR_SPACE_REC709_NONLINEAR_SRGB_HDR colour space:

Rough sketch of what a blit could look like with a new colour space and debanding updated

// From https://alex.vlachos.com/graphics/Alex_Vlachos_Advanced_VR_Rendering_GDC2015.pdf
// and https://www.shadertoy.com/view/MslGR8 (5th one starting from the bottom)
// NOTE: `frag_coord` is in pixels (i.e. not normalized UV).
// This dithering must be applied after encoding changes (linear/nonlinear) have been applied
// as the final step before quantization from floating point to integer values.
vec3 screen_space_dither(vec2 frag_coord, float bit_alignment_diviser) {
	// Iestyn's RGB dither (7 asm instructions) from Portal 2 X360, slightly modified for VR.
	// Removed the time component to avoid passing time into this shader.
	vec3 dither = vec3(dot(vec2(171.0, 231.0), frag_coord));
	dither.rgb = fract(dither.rgb / vec3(103.0, 71.0, 97.0));

	// Subtract 0.5 to avoid slightly brightening the whole viewport.
	// Use a dither strength of 100% rather than the 37.5% suggested by the original source.
	return (dither.rgb - 0.5) / bit_alignment_diviser;
}

...

// Colorspace conversion for final blit.
if (data.target_color_space == COLOR_SPACE_REC709_LINEAR) {
	(unchanged)
} else if (data.target_color_space == COLOR_SPACE_REC709_NONLINEAR_SRGB) {
	// Negative values and values above 1.0 will be clipped by the target,
	// so no need to clip them here.
	if (data.source_is_srgb == false) {
		// linear -> sRGB conversion
		color.rgb = linear_to_srgb(color.rgb);

		// Even if debanding was applied earlier in the rendering process, it must
		// be reapplied after the linear_to_srgb floating point operations.
		// When the linear_to_srgb operation was not performed, the source is
		// already an 8-bit format and debanding cannot be effective. In this
		// case, GPU driver rounding error can add noise so debanding should be
		// skipped entirely.
		if (data.use_debanding) {
			// Divide by 255 to align to 8-bit quantization.
			color.rgb += screen_space_dither(gl_FragCoord.xy, 255.0); // <-- New parameter
		}
}  else if (data.target_color_space == COLOR_SPACE_REC709_NONLINEAR_SRGB_HDR) { // <-- New COLOR_SPACE
	// Negative values and values above output_max_value will be clipped by the target,
	// so no need to clip them here.

	// Android HDR scaling
	color.rgb /= data.reference_multiplier;
		
	// linear -> sRGB conversion
	color.rgb = linear_to_srgb(color.rgb);

	if (data.use_debanding) {
		// Divide by 1023 to align to 10-bit quantization.
		color.rgb += screen_space_dither_10_bit(gl_FragCoord.xy, 1023.0);
	}
}