Add a debanding property to Viewport

[Calinou]

It can be enabled in the Project Settings (rendering/quality/screen_filters/use_debanding). It's disabled by default as it has a small performance impact and can make PNG screenshots much larger (due to how dithering works).

Debanding will also slightly brighten the scene's dark areas – we should try to fix this if possible. Help is welcome in that area 🙂

Like MSAA, toggling the debanding setting in the Project Settings will apply to the editor viewport immediately, without having to restart the editor.

This closes #17006.

Preview

View slider comparisons on Imgsli

[Calinou]

Strangely enough, I can notice much more banding when debanding is enabled on my laptop monitor (when looking at the screenshots at 1:1 scale). I don't know why exactly.

Maybe this is because the laptop monitor's panel is 6-bit + FRC instead of true 8-bit, and that doesn't play well with checkerboard dithering? My desktop monitor is an AOC CQ32G1, for reference.

[SaracenOne]

This looks like a good start, though I've recently learned that there are much more sophisticated dithering algorithms out there which can result in much better performance and image quality
http://media.steampowered.com/apps/valve/2015/Alex_Vlachos_Advanced_VR_Rendering_GDC2015.pdf
https://www.shadertoy.com/view/Md3XRf
https://www.shadertoy.com/view/MslGR8

Thanks to @s-ilent for bringing this to my attention

[clayjohn]

@SaracenOne yea, we should probably use triangular noise instead of a fixed pattern to dither.

[Calinou]

I reworked this pull request to use the Portal 2 VR debanding algorithm. See the new slider comparison: https://imgsli.com/MjA0NjI

[akien-mga]

This shouldn't have been merged while CI doesn't pass... it's actually bogus:

-	ClassDB::bind_method(D_METHOD("viewport_set_use_debanding" "viewport", "enable"), &RenderingServer::viewport_set_use_debanding);
+	ClassDB::bind_method(D_METHOD("viewport_set_use_debanding"
+								  "viewport",
+								 "enable"),
+			&RenderingServer::viewport_set_use_debanding);

This makes all subsequent commits fail CI checks too.

I'll push a fix (7480a7d).

[hoontee]

It's gonna require an algorithm change, but I don't like how obvious the dithering is due to its uniform distribution (brightened for visibility:)

Is it possible to inexpensively make this pseudorandom?

(I can't tell you how usable this code may be, but it appears to get the job done: https://www.shadertoy.com/view/MdVfz3)

[clayjohn]

@hoontee there is indeed a bit of a repeating pattern. But under normal viewing circumstances it is not really detectable. Note, the technique was developed for VR where people would be very sensitive to such things. That being said, I experimented a bit and using a gradient noise indeed resulted in much more natural looking dithering.

Note: youll want to turn your brightness up and open these screenshots in a tab with a dark background to really see the difference

No Dither

Normal

Artificially brightened

This PR (looks great unless you zoom very close, or increase contrast)

Normal

Artificially brightened

Gradient noise (looks slightly more natural)

Normal

Artificially brightened

Code for procedural gradient noise

float remap_tri(float v) {
	// Convert uniform distribution into triangle-shaped distribution.
	float orig = v * 2.0 - 1.0;
	v = orig / sqrt(abs(orig));
	v = max((-1.0), v); // Nerf the NaN generated by 0*rsqrt(0). Thanks @FioraAeterna!
	v = v - sign(orig);

	return v;
}

vec2 hash22(vec2 p) {
	vec3 p3 = fract(vec3(p.xyx) * vec3(.1031, .1030, .0973));
	p3 += dot(p3, p3.yzx + 33.33);
	return -1.0 + 2.0 * fract((p3.xx + p3.yz) * p3.zy);
}

float noise(in vec2 p) {
	vec2 i = floor(p);
	vec2 f = fract(p);

	vec2 u = f * f * (3.0 - 2.0 * f);

	return mix(mix(dot(hash22(i + vec2(0.0, 0.0)), f - vec2(0.0, 0.0)),
					   dot(hash22(i + vec2(1.0, 0.0)), f - vec2(1.0, 0.0)), u.x),
				   mix(dot(hash22(i + vec2(0.0, 1.0)), f - vec2(0.0, 1.0)),
						   dot(hash22(i + vec2(1.0, 1.0)), f - vec2(1.0, 1.0)), u.x),
				   u.y) /
		   sqrt(0.5);
}
// From http://alex.vlachos.com/graphics/Alex_Vlachos_Advanced_VR_Rendering_GDC2015.pdf
// and https://www.shadertoy.com/view/MslGR8 (5th one starting from the bottom)
// NOTE: `frag_coord` is in pixels (i.e. not normalized UV).
vec3 screen_space_dither(vec2 frag_coord) {
	return vec3(remap_tri(noise(frag_coord) * 0.5 + 0.5)) / 1024.0;
}

[hoontee]

@clayjohn That gradient noise would be much better for the engine. If your debanding code causes a different type of banding, then it doesn't matter how fast it is; it's useless.

Going a step further, would it make sense to incorporate TIME into the noise function?

[Calinou]

it's useless.

Keep in mind that in most games, banding isn't that noticeable. It might more noticeable during the prototyping stage, but as long as you do your work on the art direction side, most banding will probably no longer be noticeable by the time your game is ready to ship. Perfect is the enemy of good 🙂

Many times, you might not need a debanding shader at all in a game with photorealistic materials. Banding usually becomes more noticeable in games that use a low-poly art style and simple, cartoony textures.

I believe that in real world use cases, the current debanding shader is more than adequate. Performance matters for this kind of thing, as you generally want to always enable debanding no matter the target hardware.

PS: Remember that when you artificially brighten an image using an image editor, it'll always have noticeable banding unless the image was saved with an HDR color depth (> 8 bpc).

[hoontee]

It was noticeable at original resolution on my monitor. Maybe you can add a debanding quality project setting? (Off / Performance / Quality drop-down)

[Calinou]

Maybe you can add a debanding quality project setting? (Off / Performance / Quality drop-down)

That means we have more code to maintain (and possibly backport, since I plan to backport this to the 3.2 branch).

I'd prefer seeing significant demand from players to improve the debanding quality if they manage to notice the current dithering pattern in real gameplay 🙂

If we have enough demand for this, then we can consider adding an option to improve debanding quality.