Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rmodels] Incorrect animation of glTF model #4055

Closed
4 tasks done
paulmelis opened this issue Jun 10, 2024 · 30 comments
Closed
4 tasks done

[rmodels] Incorrect animation of glTF model #4055

paulmelis opened this issue Jun 10, 2024 · 30 comments

Comments

@paulmelis
Copy link
Contributor

paulmelis commented Jun 10, 2024

Please, before submitting a new issue verify and check:

  • I tested it on latest raylib version from master branch
  • I checked there is no similar issue already reported
  • I checked the documentation on the wiki
  • My code has no errors or misuse of raylib

By the way, the FAQ currently reads:

GLTF: This file format supports animation but raylib is not able to load them at this moment. GLTF animations loading was implemented in the past but GLTF is a quite complex file format and could implement animations in many ways, raylib used to fail a lot on loading them and support was removed.

But there is LoadModelAnimations() which supports GLTF? So seems the FAQ entry is out of date?

Issue description

While trying to figure out how to make #4037 work well together with animated models, I noticed current support for animations isn't entirely bug-free (hence making the glTF transform work on top of it somewhat challenging).

For example, with https://github.com/KhronosGroup/glTF-Sample-Models/blob/main/2.0/BrainStem/glTF-Binary/BrainStem.glb (which needs #4053 to handle the missing bone names) the initial load of the model, without applying UpdateModelAnimation() is okay (which is not surprising, as there's only a single mesh):

animation

But when applying UpdateModelAnimation() in the render loop, the transforms based on the armature make parts fly all over the place incorrectly:

animation

But then a very similar model like https://github.com/KhronosGroup/glTF-Sample-Models/blob/main/2.0/CesiumMan/glTF-Binary/CesiumMan.glb does animate correctly, and I don't immediately see the big difference between the two files causing the issues.

Environment

Arch Linux, GTX TITAN

Code Example

#include <string>

#define SUPPORT_FILEFORMAT_JPG
#include "raylib.h"

#define PLATFORM_DESKTOP

#if defined(PLATFORM_DESKTOP)
#define GLSL_VERSION            330
#else   // PLATFORM_ANDROID, PLATFORM_WEB
#define GLSL_VERSION            120
#endif

int main(int argc, char *argv[])
{
    std::string modelfile("../models/robot.glb");
    if (--argc == 1)
        modelfile = argv[1];

    const int screenWidth = 950;
    const int screenHeight = 540;

    SetConfigFlags(FLAG_MSAA_4X_HINT);
    InitWindow(screenWidth, screenHeight, "play animation");

    Model model = LoadModel(modelfile.c_str());

    BoundingBox bbox = GetModelBoundingBox(model);
    printf("Model bbox min: %.6f, %.6f, %.6f\n", bbox.min.x, bbox.min.y, bbox.min.z);
    printf("Model bbox max: %.6f, %.6f, %.6f\n", bbox.max.x, bbox.max.y, bbox.max.z);

    int animCount = 0;
    ModelAnimation* modelAnimations = LoadModelAnimations(modelfile.c_str(), &animCount);
    printf("Loaded %d animation(s)\n", animCount);

    Camera3D cam = (Camera3D){ 0 };
    //cam.position = (Vector3) { bbox.max.x*2.5f, bbox.max.y*2.5f, bbox.max.z*2.5f };
    //cam.target = (Vector3) { 0.5f*(bbox.min.x+bbox.max.x), 0.5f*(bbox.min.y+bbox.max.y), 0.5f*(bbox.min.z+bbox.max.z) };
    cam.position = (Vector3){ 3.0f, 3.0f, 3.0f };
    cam.projection = CAMERA_PERSPECTIVE;
    cam.up = (Vector3){ 0.0f, 1.0f, 0.0f };
    cam.fovy = 45.0f;

    bool animating = false;

    SetTargetFPS(60);
    
    int fc = 0;

    // Main game loop
    while (!WindowShouldClose())
    {
        float dt = GetFrameTime();

        UpdateCamera(&cam, CAMERA_ORBITAL);

        if (modelAnimations && animating)
        {
            fc++;
            fc %= (modelAnimations[0].frameCount);
            UpdateModelAnimation(model, modelAnimations[0], fc);
        }

        BeginDrawing();
            ClearBackground(RAYWHITE);
            BeginMode3D(cam);
                DrawModelEx(model, (Vector3) { 0.0f, 0.5f, 0.0f }, (Vector3) { 0.0f, 1.0f, 0.0f }, 0.0f, (Vector3) { 1.0f, 1.0f, 1.0f }, WHITE);
                DrawGrid(10, 1.0f);    
            EndMode3D();
        EndDrawing();

        if (IsKeyPressed(KEY_S)) TakeScreenshot("animation.png");
        if (IsKeyPressed(KEY_A)) animating = !animating;
    }

    UnloadModel(model);
    CloseWindow();

    return 0;
}

Edit: first linked glb was wrong, should be BrainStem.glb, now fixed

@raysan5
Copy link
Owner

raysan5 commented Jun 12, 2024

@paulmelis Thanks for reporting about the FAQ, just reviewed it.

About the different models animations working or not, I don't know the reason but I can guess is related to the bones hierarchy and its transformations. Is it possible to export a GLTF file with the transformations "baked" by bone?

@ed-mart
Copy link

ed-mart commented Jun 13, 2024

Were these models exported by blockbench?

@paulmelis
Copy link
Contributor Author

Were these models exported by blockbench?

I don't know. They're part of the glTF sample repo, but no idea how they were created.

@paulmelis
Copy link
Contributor Author

Hmm, one difference I see that might be of influence is that CesiumMan.glb only has one mesh with one primitive, while BrainStem.glb also only has one mesh but that mesh has a few dozen primitives (each with their own JOINT_0 and WEIGHTS_0 arrays). And primitives are converted into separate raylib Mesh's during loading.

@Arcod7
Copy link

Arcod7 commented Jun 15, 2024

Were these models exported by blockbench?

Hello, I tested exported models by blockbench of the cobblemon's pokemon. In general, feet and body are placed correctly during animation, but the head and other bones like leaves on a ivysaur go right in the center. I have faced a similar issue with the Fox of the glTF sample repo.

@raysan5
Copy link
Owner

raysan5 commented Jun 16, 2024

@paulmelis @Arcod7 thanks for the further investigation of this issue! It seems issue could be related how meshes are exported by blockbench but also how they are loaded by raylib...

@Arcod7
Copy link

Arcod7 commented Jun 16, 2024

Here's an illustrated example : The Torterra model in blockbench exported to glb works great in blender, but breaks in raylib (on the latest github dev version).
Video

Here's a short clip

Screenshot 2024-06-16 at 12 19 50
In Blockbench, we can see that the parts breaking are the ones indented more than 1 in the body. For example the head is affected (body->torso->head) and the feet too while the leg is well placed (legs are not part of torso).
During the animation, I noticed that while the head and others are placed in the center of the model, the feet seems to be at the center of the leg and not the body.

I'm highly motivated to try to solve the bug, do you have any idea that could help me ?

@raysan5
Copy link
Owner

raysan5 commented Jun 16, 2024

@Arcod7 Mmmh... interesting, it seems that node transformations are only considered for the first hierarchy level but not applied down to the other hierarchy levels.

@Arcod7
Copy link

Arcod7 commented Jun 16, 2024

Okay, do you think you will have time for this soon ? If there's something else I can provide to help, you can tell me !

@raysan5
Copy link
Owner

raysan5 commented Jun 16, 2024

@Arcod7 I'm afraid I don't have time to review it. I neither implemented current loader (only vertex data loading) so it would take me a lot of time that unfortunately I don't have at the moment.

@VitoTringolo
Copy link
Contributor

VitoTringolo commented Jun 25, 2024

Hey I think I quite fixed this one! I need some more time to check it but it seems to work quite well!
The only thing that is not managed yet is the right rotation for the models with the Z-UP

Before
fox_old
After
fox_new

Before
simple_old
After
simple_new

Before
simple_cesium
After
simple_cesium_new

I'll create a merge request soon :)

@raysan5
Copy link
Owner

raysan5 commented Jun 25, 2024

@VitoTringolo Wow! It looks great!!!

@MrScautHD
Copy link
Contributor

MrScautHD commented Jun 25, 2024

@VitoTringolo

Can you maybe try this model to?
Player-Female.zip

@Arcod7
Copy link

Arcod7 commented Jun 25, 2024

Hello @VitoTringolo, you did great, it looks stunning !
I have a project with raylib that ends in 2 days and I would love to try out your fix, could you push it on your fork ?

Also did you only touched the translation part or you also fixed rotations ?

On the model provided by @MrScautHD (and a lot of my models), parts of the body disappear with the rotation applied.
For exemple, the head of the Player-Female model is only present at the frame first and last frame of the run animation. It's the same thing with the arm of the player with the idle animation.

Run animation: (only scale and rotation applied, I disabled transformations)
image
image

Idle animation: (only scale and rotation applied, I disabled transformations)
image
image

@raysan5
Copy link
Owner

raysan5 commented Jun 25, 2024

@MrScautHD @Arcod7 is this model created with Blockbench?

@MrScautHD
Copy link
Contributor

@MrScautHD @Arcod7 is this model created with Blockbench?

yea

@Arcod7
Copy link

Arcod7 commented Jun 25, 2024

@MrScautHD @Arcod7 is this model created with Blockbench?

My models are also coming from blockbench (bbmodel exported to glb). They are working great with blender (not a single issue).

@VitoTringolo
Copy link
Contributor

@VitoTringolo

Can you maybe try this model to? Player-Female.zip

Sure thing!
This is the result that seems to be correct!
If you have other models to test, please share :)

player_female

@VitoTringolo
Copy link
Contributor

Hello @VitoTringolo, you did great, it looks stunning ! I have a project with raylib that ends in 2 days and I would love to try out your fix, could you push it on your fork ?

Also did you only touched the translation part or you also fixed rotations ?

On the model provided by @MrScautHD (and a lot of my models), parts of the body disappear with the rotation applied. For exemple, the head of the Player-Female model is only present at the frame first and last frame of the run animation. It's the same thing with the arm of the player with the idle animation.

Run animation: (only scale and rotation applied, I disabled transformations) image image

Idle animation: (only scale and rotation applied, I disabled transformations) image image

Before we were completely not considering the transform matrix when the animation channel was NULL! This was the main issue! So to answer your question, yes I am now considering the translation the rotation and also the scaling factor for each joint

@Arcod7
Copy link

Arcod7 commented Jun 25, 2024

It looks so nice 😊 !
Can you try the second and third animations of this model ? This is where I was getting issues with head and arms which disappear

My models are those from cobblemon
These are .bbmodel, it can be exported to .glb by blockbench (make sure to check the "Export Groups as Armature" when exporting)

If you want a sample of exported models I can provide it.

@VitoTringolo
Copy link
Contributor

It seems to work well also for these animations.

player_female_idle

player_female_run

@Arcod7
Copy link

Arcod7 commented Jun 26, 2024

This fixes almost everything on the cobblemon models, thanks a lot !
There is juste one last thing, sometimes a bbmodel animation has an animation length of 0 which means that there's no end to the animation but it still playable.
Do you have an idea of what's going on with raylib ?

@VitoTringolo
Copy link
Contributor

This fixes almost everything on the cobblemon models, thanks a lot ! There is juste one last thing, sometimes a bbmodel animation has an animation length of 0 which means that there's no end to the animation but it still playable. Do you have an idea of what's going on with raylib ?

I'm not sure I understand what you described, what do you mean with "animation has an animation length of 0 "? Can you share a model with these properties?

@Arcod7
Copy link

Arcod7 commented Jun 27, 2024

Sure ! Here's a model with a ground_walk working at index 1 (animation length is set to 2 in blockbench) and a ground_idle that's not working as expected at index 0 (animation length is set to 0 in blockbench)
torterra.glb.zip

https://github.com/raysan5/raylib/assets/77203393/cb7a6135-09f9-4f23-98ae-6356668612f9
I link a video to illustrate the issue

Have a good day !

@VitoTringolo
Copy link
Contributor

Oh OK now I understood, sorry 😅

We have the same behavior of Blender with this model.
In my opinion the solution for this is to set a value different then zero before exporting the model from Blockbench.
I tried it and the result looks good

torterra_idle

@raysan5
Copy link
Owner

raysan5 commented Jun 28, 2024

In my opinion the solution for this is to set a value different then zero before exporting the model from Blockbench.

I think that is the right solution. Otherwise, it should be checked if the glTF official specs allow other behaviours.

@raysan5 raysan5 closed this as completed Jun 28, 2024
@raysan5
Copy link
Owner

raysan5 commented Jun 28, 2024

@MrScautHD What is the license of the Player-Female.glb model? Is it free?

@MrScautHD
Copy link
Contributor

MrScautHD commented Jun 28, 2024

@MrScautHD What is the license of the Player-Female.glb model? Is it free?

Its mine, so feel free to use it. 😊

@MrScautHD
Copy link
Contributor

MrScautHD commented Jun 29, 2024

@VitoTringolo and @raysan5 What about this error?
It really spamms the console like 1000 lines and the model still work.

image

It is coming from:
image

@VitoTringolo
Copy link
Contributor

@VitoTringolo and @raysan5 What about this error? It really spamms the console like 1000 lines and the model still work.

image

It is coming from: image

Oh yeah! That's because of this line of code:

if (FloatEquals(tend, tstart)) return false;

I think I can change to return true, without any problem to avoid flooding!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants