[3.x] Additive Animation Fix (and Sub2 node feature)

[Riordan-DC]

Adds the Add2 node option: Add directly.
Default behavior is current behavior to preserve backwards compatibility.

Demonstration:

Issue premise:
Additive animation is where two animations are added together.
Normally animations are blended together which means interpolation between poses.
Additive animation adds poses. We can control how much we add by blending the pose being added
between the identity pose and itself.
Godot has had an Add2 node in its animation tree for years however this does not add poses, it blends them.

Why this matters:
Additive animation is how your animated character plays two animations at once.
For example:
In every shooter your character can look up and down, the body moves its aim up and down.
How this works under the hood is that we have an aiming forward pose and we are adding the aim up/down pose.
Why not blend? Imagine the player is running, and breathing, and reloading, while looking up. To accomplish this with blends
you could filter the blends per bone but that looks rubbish. In practice these animations are added, sometimes referred to as layered.
In our example we play the running animation and blend the torso into the aiming pose, we then add the aiming pose (Unreal calls these aim offsets), then we add the reload animation if it is playing, then we add the breathing loop.
All these adds combine to a single animation that is expressive, dynamic, and natural.

About this commit:
This commit adds a new option to the Add2 node allowing users to add directly which does add poses as expected.
It also adds a new animation tree node called Sub2, for the creation of delta animations, often used in conjunction with Add2.

delete_me.mp4

In this commit:

• Animation states contain a mix mode parameter allowing unlimited custom mixing options
• Added the add directly mix mode
• New node Sub2 subtracts pose. (WARNING: This is a new feature and Im sure Godot managers will want to revise and discuss this new feature)

Why add a new animation node called Sub2?
If we directly add two animations we add every pose directly. So even poses that are the same in both animations get added!
Instead we add the delta between two animations by first subtracting a mutual pose from the animation being added.
Reference:
http://guillaumeblanc.github.io/ozz-animation/samples/additive/
EDIT: The term rest pose has a meaning already and I don't want to confuse the use of Sub2. I also added a helpful reference.

Notes:

• Note: This doesn't change any existing nodes or setups.
• This is a messy commit and news some polish but is is a good first draft.
• Use blender NLA to test what additive animations should look like.
• I understand the Sub2 node is a new feature. If accepted I hope we can add it for 3.6!
• I promise Ill shine this commit up real nice

edit: removed 'draft'.

[TokageItLab]

It is a known problem that Add2 in 3.x is broken, but I am skeptical that Sub2 is needed for 3.x because 3.x pose already stores the result of the subtracted by rest. In 4.x, this is no longer done. See also Animation data rework for 4.0.

So, in 4.x, it adequate makes sense for Sub2 to exist for pre-calculation of Add2, but for 3.x, a filter may be sufficient. The reason this is necessary in 3.x is, as mentioned the above article, for skeletons that do not have a rest (they have a T-pose separate from the rest) such as the model exported Maya.

Then, if you want to implement Sub2, we need to decide if Sub2 should be implemented in 4.x at first.

[Riordan-DC]

It is a known problem that Add2 in 3.x is broken, but I am skeptical that Sub2 is needed for 3.x because 3.x pose already stores the result of the subtracted by rest. In 4.x, this is no longer done. See also Animation data rework for 4.0.

Oops typo. I meant to write reference pose not rest pose. I've edited the article to remove the reference to rest pose and also included this helpful resource:
http://guillaumeblanc.github.io/ozz-animation/samples/additive/

Then, if you want to implement Sub2, we need to decide if Sub2 should be implemented in 4.x at first.

That is outside of my level of understanding. I've never used 4.x and am unfamiliar with that project. I am just trying to fix an old pain point in Godot 3, Im not sure what 4.x has to do with my commit.

My promotion of Sub2:

• As per the above linked resource, additive blending is more often used to add delta animations. Delta animations are created by finding the delta between two animations. In a blend tree structure this either requires the use of a Sub2 node or by cramming that functionality into another node.
• Having to make your own delta animations with a separate Sub2 node encapsulates functionality (as seen in the screenshot in the article). This makes blend trees easier to understand as all the functionality is shown in the tree, not buried within nodes. We want to avoid creating any single node which does too much as then the node tree becomes less expressive.
• Sub2 is a new node and therefore wont break any existing node trees
• I think because it is new it needs to be test, a lot.

@TokageItLab

[TokageItLab]

Sub2 was discussed once in the following but it has been limbo. godotengine/godot-proposals#2700 (comment)

However, in 3.x, it should not be necessary, at least in the case of importing a model with rest.

I have explained the following matters several times:

[Godot 3]
Bone Pose Initial Value = Transform(Quat(0,0,0,1), Vector3(0,0,0))
Visible Bone Pose = BoneRest * BonePose

[Godot 4]
Bone Pose Initial Value = BoneRest 
Visible Bone Pose = BonePose

And in fact, the animation stored in glTF has no concept of rest, and stores the same value as the Pose in Godot 4.

But in Godot 3, if the skeleton bones have data that can be read as BoneRests, Godot 3 glTF importer subtracts the rests from the animation data and imports the processed values as BonePoses.

[Godot 3]
Godot BonePose = glTF BoneRest^(-1) * glTF AnimationData

[Godot 4]
Godot BonePose = glTF AnimationData

This means that Godot 3.x glTF importer has already done the work that Sub2 should have done. (But it may be useful in cases like Maya where you do not export data as a BoneRest to glTF ≈ the default pose and rest are different, or if you want to subtract additional data from it ..well the latter is a corner case)

In any case, at this time, we need the step to implement the new features in Godot 4 first and then follow the procedure of backporting to Godot 3, so if you are in a hurry to implement Add2 Directly option, Sub2 should be removed from this commit PR.

If not hurry, we need a PR to implement Sub2 to Godot 4. I think that it would make sense to implement it to Godot 4. (Although it looks like the amount parameter is lacking. And also implementing enum constants for Sub as a special case looks like a not good way)

[Riordan-DC]

@TokageItLab
Ah ok I didnt know it was procedure to add new feature to 4.x and backport to 3.x.

I'm not sure what rest poses have to do with Sub2. You are correct rest poses are not at risk of being added twice.
The point of Sub2 is to subtract ANY reference pose.
I made a short video to demonstrate its use.
https://youtu.be/yacsDJ64c_4

Let me know what you think.

[TokageItLab]

In other words, it is convenient to be able to use any pose as a reference, but in Godot 3, Sub2 should not be necessary if the desired pose is imported as a rest.

I don't think there are many cases where a model has multiple reference poses and switches between them. In other words, in Godot 3, the case of needed it is basically a corner case where an incorrect rest is imported (means something is wrong in the glTF), which makes testing difficult.

And, you might intend to animate and synchronize reference poses, but in the first place, Godot 3's blend calculation and synchronization breaks down with complicated chained AnimationNodes, so that is also difficult to test.

However, as noted above, in Godot 4, the importer does not consider rests, so the expectation is that you basically need to use Sub2 to precompute Add2. This case would make the test result and what it want to do more clear.

[Riordan-DC]

In other words, it is convenient to be able to use any pose as a reference, but in Godot 3, Sub2 should not be necessary if the desired pose is imported as a rest.

Sub2 gives users the ability to add as many reference poses as they like with as many Add2 nodes as they like without relying on the whims of whatever 3D file format or whichever 3D editing software.

I don't think there are many cases where a model has multiple reference poses and switches between them. In other words, in Godot 3, the case of needed it is basically a corner case where an incorrect rest is imported (means something is wrong in the glTF), which makes testing difficult.

I disagree that a character needing multiple reference (also called base) poses would be a corner case.

In my example I use 1 reference pose which is the "Pistol Aim". That is, using one arm to aim a pistol and the other arm is posed on the characters hip.
What if I want my character to also hold a rifle, or a bow, or a melee weapon?
These all pose the arms in different ways. This means they all need their own Add2
branch in the blend tree with their own reference pose. Right there that is 4 reference poses required.
I can't export multiple rest poses can I? Every model has only 1 rest pose.

[TokageItLab]

What do you want to subtract from each weapon pose? If you don't need the lower body animation, then the filter is sufficient, and if the addition of the upper body for each animation doesn't work correctly, then your glTF rest may be wrong. At least that should be the case with Godot 3.

As I have said many times, the Bone Pose in Godot 3 is already the result of the Rest subtracted.

If rest is equal to the reference pose, then the result of blending the weapon animation with the Directly option in Add2 to the reference pose will be equal to Blend2.

The mainly need for Add2 is not for such weapon switching, but for inertial transitions such as adding a knockback animation to the original animation, or swinging the arm up when switching weapons, which would be unnatural with Blend2.

However, the problem here is that when creating the animation of the weapon's recoil, the arms of each weapon face a different direction from the rest, and the difference between the rest and the direction of the arm holding the weapon will be added by Add2. That would be understandable.

[Riordan-DC]

No issues with rests being added twice. Lets about forget rest poses.

You are correct I filter the lower half out anyway with a blend. In the video I shared I mistakenly mentioned the rests are not being applied twice. This was never an error. I should have said the aim up/down poses

I think I have been bad at explaining this whole issue.

The need for Sub2 is simply the need for being able to create a delta animation.

When an artist creates an additive animation they create the animation they want layered.

They do NOT themselves create the delta animation as visually it is an unnatural looking pose
created with math, it only looks natural when added to an animation.

Why create a delta animation?

An artist is tasked with creating an aim pistol pose (lets call that A).
Next they are tasked with adding a reload or recoil animation for that aim pistol pose.
How does the artist proceed? They first copy the aim pistol pose, then they animate their animation
from that aim pistol pose (lets call that B).
Ok fantastic, now we have an aim pistol pose and when a player fires we add the recoil animation over top?

Our output animation is:

O = A + B

Uh oh, poses in animations A and B are largely the same (B being derived from A's pose), when we add directly
we are applying those same poses twice!
So we need a delta animation. We only want to add the difference between animations A and B.
So we add our delta animation (D) to A instead of B.

D = (B - A)
O = A + D

Now our animation looks correct. Computing D requires Sub2, computing O requires Add2.
EDIT:
What if we use Add2 without Sub2? That is also fine for additive animations that created relative to the rest pose.
For example: a breathing animation or hit reaction.

[TokageItLab]

If not hurry, we need a PR to implement Sub2 to Godot 4. I think that it would make sense to implement it to Godot 4.

This is all I want to say.

You may have misunderstood, I am not against adding of Sub2. Rather, I am in favor of adding it.

But it is not possible to add it to Godot 3.x first.

If the new feature is added to Godot 3.x first, the 3.x Project will be incompatible with the 4.0 Project and will generate a new issue of breaks compatibility.

So, the only case to add a new feature directly to Godot 3.x should be if it is something that is only a problem in 3.x.

The lack of Sub2 is still a problem in Godot 4, and Sub2 is rather more often needed in Godot 4 than in Godot 3.x.

---

No issues with rests being added twice. Lets about forget rest poses.

At least in Godot 4, it is completely wrong. However, Godot 3.x does not include Rest in Pose, so that is not likely to occur.

In Godot 3, if the animation to be subtracted by Sub2 is not Rest, the Pose will contain a value other than Delta, which can cause problems.

For example, if you want to take a Delta animation from an idle animation, even if the first frame of the idle animation is the same as Rest, the Delta animation after the second frame is not correct, which is a problem.

Conversely, if the idle animation is always stationary and it is equal to Rest, it means that the Godot 3.x importer has already extracted the Delta animation.

Godot 4 considers Rest to be just an initial value of Pose, so the importer imports the animation value without any processing.

Concluded:

Godot 3

• The value of the animation (Pose) same as Rest should be Transform3D(Quat(0, 0, 0, 1), Vector3(0,0,0))

Godot 4

• The value of the animation (Pose) same as Rest should be equal to Rest

This means that in Godot 4, there are more cases where the Delta animation needs to be extracted. Sorry, this was a mistake, Godot 4 does not consider Rest on import, but does consider Rest on blending process, so Rest is not added twice. In other words, the need for Sub2 is equivalent between Godot 4 and Godot 3.

---

Also, Godot 3.x cannot create complex AnimationTree because the calculation and Sync are completely broken before the need for Sub2 and AddDirectly. The broken calculations were not just an AddDirectly problem.

For example, there were strange calculations that produced different results depending on how the nodes were connected, even though the same blend calculations were being performed.

• Bugfix: Fix Anim Tree blending inconsistency #34134 (comment)

That fix includes huge breaking compatibility, so it can't be backported to 3.x.

• Fix blend animation to solve TRS track bug & blend order inconsistency #57675

Also, sync was broken for some nodes, and broken Sync meant that the Delta animation calculation was also broken.

• Add sync to NodeTransition and BlendSpace1D/2D and refactor sync in AnimationTree #62623

For these reasons, it is difficult to perform complex AnimationTree tests with Godot 3.x.

---

So Sub2 needs to be added to Godot 4 first. If you want, I will send a PR for Godot 4 later.

[Riordan-DC]

You may have misunderstood, I am not against adding of Sub2. Rather, I am in favor of adding it.

But it is not possible to add it to Godot 3.x first.

If the new feature is added to Godot 3.x first, the 3.x Project will be incompatible with the 4.0 Project and will generate a new issue of breaks compatibility.

I misunderstood, sorry. I understand the issue a little better now thank you.

This means that in Godot 4, there are more cases where the Delta animation needs to be extracted.

So Sub2 needs to be added to Godot 4 first. If you want, I will send a PR for Godot 4 later.

That sounds good thanks.
Hopefully adding Sub2 to 4.x will be easier than 3.x.

[TokageItLab]

I sent #76616, please check it to know whether it satisfies what you intended it to do.

[TokageItLab]

#76616 has been merged and can be backported.

However, this #76310 PR contains some distinctive implementations and will need to be as consistent with #76616 as possible.

For example, the enum MixMode is not needed, and the bool value add_directly should be sufficient. However, it would probably be better to name it use_blend as an inverted add_directly flag. Because, as you can see from the implementation of #76616, Sub2 can be calculated in the same way as add_directly by changing the blend order and inverting the blend amount.

This means that in Godot 3.x, you need to implement the add_directly option for Sub2 as well, but the word "add" is strange, so an option like use_blend is recommended. I know that it is nonsense in using use_blend = true in Sub2, but the same can be said for Add2. It is simply for consistency.

[Riordan-DC]

@TokageItLab Sorry I have been dragging my feet adding these changes. I did not do the math but now I've done my homework haha

Here is my understanding for critique and documentation.

3.X GENERIC BLEND (CURRENT)

t->loc = t->loc.linear_interpolate(loc, blend);
if (t->rot_blend_accum == 0) {
	t->rot = rot;
	t->rot_blend_accum = blend;
} else {
	float rot_total = t->rot_blend_accum + blend;
	t->rot = rot.slerp(t->rot, t->rot_blend_accum / rot_total).normalized();
	t->rot_blend_accum = rot_total;
}
t->scale = t->scale.linear_interpolate(scale, blend);

PROPOSED: ADD DIRECT

// NOT the same as generic with positive blend
t->loc += loc * blend;
// same as a generic with positive blend
t->scale = t->scale.linear_interpolate(scale, blend);
// maybe same as generic with positive blend?
Quat q = Quat().slerp(rot.normalized(),  blend).normalized();
t->rot = (t->rot * q).normalized();

PROPOSED: SUBTRACT

// NOT the same as generic with negative blend
// same as ADD direction with -blend!
t->loc -= loc * blend;
// same as generic with negative blend
t->scale = t->scale.linear_interpolate(scale, blend);
// maybe same as generic with negative blend?
Quat q = Quat().slerp(rot.normalized().inverse(),  blend).normalized();
t->rot = (t->rot * q).normalized();

Solution explanation:

The location is the only variable that needs special attention for add direct and sub.

We add a single flag. Called use_blend which if true will perform the generic 3.x location blend, the default:

t->loc = t->loc.linear_interpolate(loc, blend); 
// underlying function: t->loc = (t->loc - loc) * blend

// If use_blend is false we will use this proposed location blend:

t->loc += loc * blend; 	
// underlying function: t->loc = t->loc + loc * blend;

PROPOSED UPDATED 3.X GENERIC BLEND

if (use_blend) {
	t->loc = t->loc.linear_interpolate(loc, blend);
} else {
	t->loc += loc * blend;
}
if (t->rot_blend_accum == 0) {
	t->rot = rot;
	t->rot_blend_accum = blend;
} else {
	float rot_total = t->rot_blend_accum + blend;
	t->rot = rot.slerp(t->rot, t->rot_blend_accum / rot_total).normalized();
	t->rot_blend_accum = rot_total;
}
t->scale = t->scale.linear_interpolate(scale, blend);

I hope I have understood everything correctly. I think the most annoying part of all this is finding out a nice way to get the use_blend flag into the _process_graph function. The recursion hurts my head.

EDIT:

We also need to do rotation different. This becomes:

if (as.use_blend) {
        blend = Math::absf(blend); // If negative rot_total could be 0, causing divide by 0 errors, inf numbers.
        t->loc = t->loc.linear_interpolate(loc, blend);
        if (t->rot_blend_accum == 0) {
	        t->rot = rot;
	        t->rot_blend_accum = blend;
        } else {
	        float rot_total = t->rot_blend_accum + blend;
	        t->rot = rot.slerp(t->rot, t->rot_blend_accum / rot_total).normalized();
	        t->rot_blend_accum = rot_total;
        }
} else {
        // Direct addition / subtraction
        t->loc += loc * blend;
        Quat q = Quat();
        q = q.slerp(rot.normalized(), blend).normalized();
        t->rot = (t->rot * q).normalized();
}
t->scale = t->scale.linear_interpolate(scale, Math::absf(blend));

[Riordan-DC]

I hate this hack. If anyone can find a better way, please make a PR / comment.
I am trying to pass a variable from an animation node to _process_graph, where the blending occurs.
How this works:

1. sub/add2 animation node calls blend_input
2. blend_input calls blend_node
3. blend_node sets use_blend in the input animation node
4. animation node leaf (animation/blendspace) adds animation state to the blend tree, adding its use_blend variable also.

The issue: In between steps 3. and 4. we may travel through many other nodes and the use_blend flag will be overwritten. This code segment will avoid overwriting use_blend with true if it is already false. Allowing our modified use_blend to pass down the tree and into _process_graph.

There has to be an easier way. I tried this way because I tried to modify the least amount of code.

[AThousandShips]

Instead just use:

p_node->use_blend = p_use_blend && use_blend;

(No this needed either)

[Riordan-DC]

Your rebase went wrong and you included a lot of unrelated commits

Oops. How do I fix it?

[AThousandShips]

You would need to do a rebase and drop the merge commit you made and instead rebase on top of 3.x, like so: git rebase -i 3.x, you then need to push the chagnes using git push -f

If it doesn't work I can help in a bit but have to go

[AThousandShips]

There, fixed it up 🙂

[AThousandShips]

Instead just use:

p_node->use_blend = p_use_blend && use_blend;

(No this needed either)

[TokageItLab]

use_blend should not be added to AnimationNode but to AnimationNodeAdd2, AnimationNodeAdd3 and AnimationNodeSub2 each as a replacement for add_directly.

[Riordan-DC]

use_blend should not be added to AnimationNode but to AnimationNodeAdd2, AnimationNodeAdd3 and AnimationNodeSub2 each as a replacement for add_directly.

@TokageItLab
This solution requires use_blend be added to AnimationNode.

Explanation: use_blend is used to pass information down to the AnimationState struct which is used by AnimationTree::_process_graph to do animating. Because of the tree structure the AnimationNodes are explored recursively and I am using use_blend to pass information through recursive calls.

The use is seen here:
#

Personally I don't like using a variable like this because I avoid recursion and the code above seems hacky but currently I have not figured out a better way to set AnimationState's use_blend variable.

[TokageItLab]

I cannot understand your intention at all, but it definitely should not be necessary to have both use_blend and add_directly. At least in your previous commit, only add_directly was there and it worked.

By the way, Godot4 added a Deterministic option recently that reproduces Godot3's add2 (when Deterministic = false), so I think the best solution would be to backport that if you are possible.

[Riordan-DC]

Godot 4 has a completely different animation system. It's solutions are not compatible with Godot 3.

There is no commit where AnimationNode does not require an extra variable to pass down a preference for additive blending. If the issue is the naming I don't mind changing it to add_directly but it still must be a member of AnimationNode to work.

[TokageItLab]

Yeah, that is confusing.

Indeed, there have been additional members as arguments in past commits, but I'm not sure if it really needs to be an argument.

Maybe one way to do it would be to add a Normalized property to the AnimationTree. Perhaps it could be migratable to some extent to the Deterministic option (as Deterministic = !Normalized). Without implementing Normalized option, the missing options for Add3 and Sub2 need to be fixed for consistency.