DirectionalLight3D affecting FogVolume in all VisualInstance3D Layers

[xShader1374]

Godot version

Godot v4.2.dev5.mono - Windows 10.0.22621 - Vulkan (Forward+) - dedicated NVIDIA GeForce GTX 1660 (NVIDIA; 31.0.15.3734) - AMD Ryzen 7 3700X 8-Core Processor (16 Threads)

System information

Windows 11 - Godot 4.2.dev5.mono

Issue description

DirectionalLight3D affects FogVolume (and fog in general) despite them having different layers on VisualInstance3D, dunno why the behavior happens, hope this will get solved cus i really wanted to use the functionality to use light in different layers without it affecting the main world's fog in a different layer, but i noticed it got affected despite them having different layers, as said before.

(Tested both on Godot and Godot mono, noticed that fog gets affected even when a material from a different layer emits light, so it's probably a fog's problem)

Here's a minimal reproduction:

Steps to reproduce

Create a new Node3D, add a new DirectionalLight and change its VisualInstance3D's layer to 2, add a WorldEnvironment node, activate fog and put density to 0, now add a new FogVolume node, change its VisualInstance3D's later to 3 or leave it to 1 and see the light affecting the fog even if they have different layers

Minimal reproduction project

FogLightBug.zip

[Calinou]

Are you referring to the DirectionalLight Cull Mask property?

[xShader1374]

Yes, exactly that, sorry

[ecmjohnson]

This is actually quite a bit trickier than I thought it would be. The way that fog is rendered combines the individual fog volume instances prior to computing the effects of lighting in a separate shader.

I'm assuming this design is to improve performance since now the light scattering calculations are done once taking into account the combined fogs in each froxel. In order to support the light's cull mask property, I'm thinking along the lines of splitting the fog rendering into multiple passes: one for each unique layer mask. This wouldn't impact the performance when all fog volumes have the same layer mask, but would allow for identifying each light + fog layer mask so the light can be skipped if it culls that layer.

For working geometry instances, the removal of the light contribution is done in the shader. For example, for directional lights in the forward clustered renderer:

godot/servers/rendering/renderer_rd/shaders/forward_clustered/scene_forward_clustered.glsl

Lines 1920 to 1922 in bf8dd73

	if (!bool(directional_lights.data[i].mask & instances.data[instance_index].layer_mask)) {
	continue; //not masked
	}

There is some commented out code in the process fog shader similar to this; however, the referenced variable doesn't exist (draw_call) and it wouldn't work here since all the fog volumes have already been merged:

godot/servers/rendering/renderer_rd/shaders/environment/volumetric_fog_process.glsl

Lines 486 to 488 in bf8dd73

	//if (!bool(omni_omni_lights.data[light_index].mask & draw_call.layer_mask)) {
	// continue; //not masked
	//}

As mentioned, the way fog rendering works is that the individual fog volume instances are combined in volumetric_fog.glsl into the emissive_only_map, density_only_map, and light_only_map buffers. Then, volumetric_fog_process.glsl reads these combined maps and computes the contributions of each light. Integration is also done in by another pass by a different configuration of volumetric_fog_process.glsl. It looks like the second shader needs to be run once per layer mask and we will need one copy of the maps from the first shader for each layer mask.

[clayjohn]

Lights don't interact with FogVolumes. They interact with Volumetric fog, which doesn't have a layer property. So it's expected that you can't use the layer property of fog volumes to mask out certain lights.