[DRAFT] Implement motion vectors in compatibility renderer

[devloglogan]

This PR implements motion vectors in the compatibility renderer, primarily for use with the XR_FB_space_warp OpenXR extension. For testing purposes, this PR currently implements this extension as well. Eventually this logic will be handled via GDExtension in godot_openxr_vendors. This PR will remain as a draft until that's the case.

Technical details

I've tried to emulate how motion vectors are implemented in the Forward+ renderer where possible, but it's worth noting that space warp requires 3D motion vectors. Whereas the Forward+ renderer is rendering 2D motion vectors for TAA.

Motion vectors are rendered in a new, separate render pass. Why this was done:

• Since motion vectors can be rendered at a much smaller resolution this extra pass should be quite fast (e.g. the Quest 3 recommends a resolution of around 420x420).
• This will make it easier to allow developers to include or exclude specific objects from motion vectors, which can be helpful in minimizing the artifacts from space warp. (Note: this isn’t part of the current PR, but something planned for the future.)
• Combining motion vectors with the color pass would hinder future compatibility between Application Space Warp and the XR_META_recommended_layer_resolution OpenXR extension, since all attachments would then have to be of the same resolution.

The vertex shader code in scene.glsl has been placed in a new vertex_shader() function. When motion vectors are used, it will run the vertex shader on both previous and current vertex data. The scene state data and multiview data have been duplicated to hold the previous frame’s data. The vertex shader outputs both the previous and current clip positions, which are used to calculate motion vectors in the fragment shader.

For skeletons, GLES::MeshInstance::Surface now stores two vertex buffers, generating the second one on demand when motion vectors are enabled. The VBOs will alternate between current/previous, and the previous VBO is used to populate new scene.glsl vertex attribs at locations 16/17.

Similarly, for multimeshes/particles, GLES::MultiMesh now also stores two vertex buffers with the secondary one generated on demand. New scene.glsl vertex attribs (location 18 to 21) store the previous data, and the setup for both the current/previous vertex attribs is now done in a new GLES3::MeshStorage::multimesh_vertex_attrib_setup() function.

Issues that need to be addressed

There are a couple things that need to be ironed out that I would really appreciate some input on from those who have a bit more rendering know how.

• This implementation of motion vectors requires the GPU to have a max vertex attribute value of at least 22. This requirement is met on Quest hardware (or any other headset which uses the Qualcomm XR-series chips), and we check it before running the motion vector pass. But when opening the editor on desktop some errors might be thrown from ShaderGLES3::_initialize_version(), which attempts to compile all shader variants at startup, including the motion vector variant of scene.glsl. What is the best way to prevent it from compiling this variant if there aren’t enough vertex attributes?

• In non-one-shot particle systems, it's possible for a particle instance to go from the end of its lifetime one frame to the beginning of its lifetime the next frame. At the moment this causes an undesired result on the motion vector texture (we should draw nothing for the particle on these frames). Is there a way we can determine if this is the case in scene.glsl?

• As mentioned above the vertex shader in scene.glsl has moved into a new vertex_shader() function. For multiview data, I'm currently passing in a boolean value indicating whether to use multiview_data_block.prev_data or multiview_data_block.data. I'd rather just pass in the MultiviewData structs themselves rather than the boolean, similar to how the SceneData structs are passed in. However, doing this seems to cause anything using multiview data to just not be rendered. What I have in the PR works, so maybe it's not a big deal, but if anybody has ideas on how to fix this I'd appreciate it!

Contributed with ❤️ by W4Games

[clayjohn]

This implementation of motion vectors requires the GPU to have a max vertex attribute value of at least 22. This requirement is met on Quest hardware (or any other headset which uses the Qualcomm XR-series chips), and we check it before running the motion vector pass. But when opening the editor on desktop some errors might be thrown from ShaderGLES3::_initialize_version(), which attempts to compile all shader variants at startup, including the motion vector variant of scene.glsl. What is the best way to prevent it from compiling this variant if there aren’t enough vertex attributes?

Yes, motion vectors should be a specialization for now. Then the version will only be compiled when it is actually used.

In non-one-shot particle systems, it's possible for a particle instance to go from the end of its lifetime one frame to the beginning of its lifetime the next frame. At the moment this causes an undesired result on the motion vector texture (we should draw nothing for the particle on these frames). Is there a way we can determine if this is the case in scene.glsl?

Not from scene.glsl, no. We will have to do something in the actual particle system. I think that we handled this case already in the RD renderers. Most likely we will need to duplicate the particle buffer to effectively clear the history

As mentioned above the vertex shader in scene.glsl has moved into a new vertex_shader() function. For multiview data, I'm currently passing in a boolean value indicating whether to use multiview_data_block.prev_data or multiview_data_block.data. I'd rather just pass in the MultiviewData structs themselves rather than the boolean, similar to how the SceneData structs are passed in. However, doing this seems to cause anything using multiview data to just not be rendered. What I have in the PR works, so maybe it's not a big deal, but if anybody has ideas on how to fix this I'd appreciate it!

I can't think of any reason why one struct would work but not another. We should try to get it working though as the bool will likely create a dynamic branch which we really don't want