Add option to bake blendshape Meshes with their current blendshape mix

[smix8]

Describe the project you are working on

Godot things.

Describe the problem or limitation you are having in your project

The feature is frequently required for gameplay or visual effects.

Prominently character editors can have a large amount of blend shapes loaded on a mesh for customization. In this situation the blend shapes are so numbers and performance taxing that they are not feasible for actual gameplay. When the edit is done the result gets baked to a "normal" mesh without blend shapes and the heavy mesh with all the blend shapes for the character editor gets unloaded.

Outside of this there are plenty of gameplay and engine features that require mesh surfaces to sample but also only work with a "normal" mesh, ignoring the "real" visuals of a blend shape changed mesh entirely. They all require that the blend shape result gets baked to a "normal" mesh in order to function.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Since the current blend shape values of a mesh are mostly stored and edited on a MeshInstance3D Godot could provide the functionality there (or the RenderingServer instance API).

This feature can be seen as a supplement to skinned mesh baking from proposal #8120. The two in tandem can be chained to bake a mesh that is both the animated current pose as well as an accurate representation of the blend shape modified mesh. As a bonus the baked mesh also enables MultiMesh to work with both. This enables options for far more responsive crowd actors compared to vertex animation textures that can not be changed or controlled easily.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The MeshInstance3D gets a new function e.g. bake_mesh_from_current_blend_shape_mix().

This function collects the current blend shape array from the Mesh the MeshInstance3D uses.
Then it loops over the surfaces of the mesh that the MeshInstance3D uses and xforms the required mesh arrays by the current blend value of the blend shape.
The result gets applied to the "bake" Mesh that is provided as a parameter.

var bake_mesh: ArrayMesh = ArrayMesh.new()

MeshInstance3D.bake_mesh_from_current_blend_shape_mix(bake_mesh)

There was an old attempt to add something like this that could be updated godotengine/godot#76725.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, and baking meshes like this is also performance heavy so scripts are not the right place.

Is there a reason why this should be core and not an add-on in the asset library?

Rendering is part of core.
Scripts are too slow for this with more complex meshes.
Plenty of common gameplay and visual effects require to bake the current visual result of blend shapes to a new mesh.

[fire]

Your proposed solution to bake a mesh that represents both the current animated pose and the blend shape modified mesh seems good.

It is important that the performance is acceptable and so meshes should be bake-able with their rigged meshing and with their blend-shapes. Because of performance constraints and the relative difficulty, I and Lyuma also agree it needs to be a core feature, not an add-on.

Furthermore for our own projects, we want to set up an Inverse Kinematics (IK) system for animations using different configurations of the same solver and data-based testing that is testable using this baking system.

[smix8]

The primary performance bottleneck at the moment is that the mesh arrays and blend shape arrays need to be received from RenderingServer and the GPU and there is no function or API to reuse cached mesh arrays.

If you do the baking at runtime with a complex mesh you have unavoidable frame rate issues from requesting the arrays, especially when the RenderingServer runs with threads. This is also a problem when baking with a thread, because the RenderingServer gets stalled by it regardless.

I think a good solution for that is to add a bool toggle on the MeshInstance3D or Mesh resource for a cache option that allows to cache those mesh arrays and blend shapes arrays in advance for later use. This would allow to run the baking in a background thread without stalling the RenderingServer with the array request. Preferably the cache would be directly on the Mesh resource so a single read-only copy can be shared with all resource users.

[akien-mga]

Implemented by godotengine/godot#76725.