Textures / UV mapping incorrect after exporting FBX with GLTFExporter

[danielealbano]

Description

I wrote a simple application to merge a number of FBX, where the first FBX is used for the model, materials and textures, and the additional FBXs for the animations, and generate a GLB / GLTF to be used in Godot.

The FBX are generated by Mixamo where the first is exported with the skin and the subsequent without it (as it's not really needed).

When I export the model using the GLTFExporter the textures are totally messed up, as the textures look correct.

Just in case, I tried with and without animation, the problem is the same.

I have also tried converting the Phong material generated by the FBXLoader to a Standard material but nothing changed.

I also tried to fly the y and/or the x of the texture but it didn't fix the issue.

I can provide the source model in FBX, with the related animations (although they are not part of the problem).

I "think" there might be an issue with the UV mapping but I am not really sure of how to check it.

Reproduction steps

1. Upload an FBX (textures must be made available to the loader)
2. Export via the GLTF exporter

Code

Code to load the FBX and merge the animations

function loadFBX(name : string, reader : FileReader) {
    const loader = new FBXLoader()
    let fbxModel = loader.parse(reader.result, "/resources/")

    if (mainModel == null) {
        mixer = new THREE.AnimationMixer(fbxModel)
        fbxModel.animations = []

        //Center model in view
        const box = new THREE.Box3().setFromObject(fbxModel)
        const center = box.getCenter(new THREE.Vector3())
        console.log(fbxModel)
        scene.add(fbxModel)

        mainModel = fbxModel
        mainModelAnimationMixer = mixer
    } else {
        fbxModel.animations.forEach((a: THREE.AnimationClip, i) => {
            a.name = name
            mainModel.animations.push(a)
        })
    }
}

Code to generate the GLTF and download it

download.addEventListener('click', (e: Event) => {
    const exporter = new GLTFExporter();

    // Parse the input and generate the gltf output
    exporter.parse(
        mainModel,
        // called when the gltf has been generated
        function(gltf) {
            // download the gltf file
            const blob = new Blob([JSON.stringify(gltf)], {type: 'text/plain'});
            const link = document.createElement('a');
            link.download = 'model.gltf';
            link.href = URL.createObjectURL(blob);
            link.click();
        },
        // called when there is an error in the generation
        function ( error ) {
            console.log( 'An error happened' );
            console.log(error);
        },
        {
            binary: false,
            animations: mainModel.animations,
            maxTextureSize: 4096,
            includeCustomExtensions: true
        }
    );

Live example

The code is available here but it's not working on jsfiddle
 https://jsfiddle.net/rcdgqtwf/6/

Screenshots

Here a preview of the models in blender, source FBX on the left and generated GLTF on the right

I have tried also other tools apart from blender and the result (the right one) is the same.

Version

r158

Device

Desktop

Browser

Chrome

OS

Windows, Linux

[Mugen87]

Is this issue somehow reproducible in the three.js editor? https://threejs.org/editor/

It supports importing FBX and exporting glTF assets. A reproduction in the editor would make it easier to investigate the issue on our side.

Do you mind sharing one of your FBX assets in this thread?

[danielealbano]

Is this issue somehow reproducible in the three.js editor? https://threejs.org/editor/

It supports importing FBX and exporting glTF assets. A reproduction in the editor would make it easier to investigate the issue on our side.

Do you mind sharing one of your FBX assets in this thread?

It happens although a bit differently although disclaimer: as it's an FBX and it's a local file, textures (map and normalMap) aren't loaded automatically, so I had to fiddle with the console to import them.

Here the code I used from the console to manually load the textures (the UI doesn't refresh but the exported file has them)

let textureLoader = new THREE.TextureLoader()

editor.scene.children[0].children[1].material[0].map = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Skin_MAT_BC_V1.png")
editor.scene.children[0].children[1].material[0].normalMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Skin_MAT_N_V1.png")

editor.scene.children[0].children[1].material[1].map = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Clothes_MAT_BC_V1.png")
editor.scene.children[0].children[1].material[1].normalMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Clothes_MAT_N_V1.png")

Here the result

I also tried to swap the material to a MeshStandardMaterial and this is the result

code

let textureLoader = new THREE.TextureLoader()

// first material
newMaterial = new THREE.MeshStandardMaterial()
newMaterial.copy(editor.scene.children[0].children[1].material[0])
newMaterial.map = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Skin_MAT_BC_V1.png")
newMaterial.normalMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Skin_MAT_N_V1.png")
newMaterial.roughness = 0.5
newMaterial.metalness = 0.5
newMaterial.metalnessMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Skin_MAT_AO_R_M_V1.png")
newMaterial.aoMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Skin_MAT_AO_R_M_V1.png")
newMaterial.roughnessMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Skin_MAT_AO_R_M_V1.png")
editor.scene.children[0].children[1].material.splice(0, 1, newMaterial)

// second material
newMaterial = new THREE.MeshStandardMaterial()
newMaterial.copy(editor.scene.children[0].children[1].material[1])
newMaterial.roughness = 0.5
newMaterial.metalness = 0.5
newMaterial.metalnessMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Clothes_MAT_AO_R_M_V1.png")
newMaterial.aoMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Clothes_MAT_AO_R_M_V1.png")
newMaterial.roughnessMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Clothes_MAT_AO_R_M_V1.png")
newMaterial.map = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Clothes_MAT_BC_V1.png")
newMaterial.normalMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Clothes_MAT_N_V1.png")
editor.scene.children[0].children[1].material.splice(1, 1, newMaterial)

It's mangled similarly although there are some black patches, I guess because I didn't set any wrapping.

And talking about wrapping, I wonder if the issue is related to that: I noticed that the model has a set of UV coords outside the canonical 0...1, most likely because the material is set to use wrapping.

Here it's what I mean

Now that I have noticed this discrepancy I will try to do some more testing locally.

Does the editor do anything special to load the FBX? Does it use a custom FBX loader that is not from the github repo?

[danielealbano]

Sorry, forgot to provide the FBX
https://we.tl/t-uIBz0g8uGu

It's via wetransfer as there are the textures as well that are fairly large

[danielealbano]

Errata - I was able to reload the textures manually using the code above, just issuing the commands very slowly, here the result

let textureLoader = new THREE.TextureLoader()

editor.scene.children[0].children[1].material[0].map = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Skin_MAT_BC_V1.png")
editor.scene.children[0].children[1].material[0].normalMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Skin_MAT_N_V1.png")

editor.scene.children[0].children[1].material[1].map = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Clothes_MAT_BC_V1.png")
editor.scene.children[0].children[1].material[1].normalMap = textureLoader.load("https://192.168.178.124:5173/resources/T_BigZombie_Clothes_MAT_N_V1.png")

I updated the previous comment

[danielealbano]

SUPER interesting, I was playing with the console further as I was recalling I saw the model with correct textures at some point and discovered that if I apply the texture of the material 0 to the material 1, the two parts of the model they get the same texture and infact the body is correct meanwhile the shorts are messed up.

Is there any chance that the exporter has issues with multiple materials? Or maybe is it using only the "last" material for everything?

[GitHubDragonFly]

I can help you a little bit with this issue.

Try to use my ASSIMP Viewer to load one of your FBX models together with all the textures, I tried your SK_BigZombie-mixamo.fbx, and then export it to GLB format. Be patient since ASSIMP is a little bit slow to load.

It did work on my end producing rather large GLB file (~110MB).

My FBX Viewer on the other hand did load your files but exported them pretty much in the same way as you are getting it (see the attached picture below).

I cannot be smart and tell you what might be causing the issue.

[danielealbano]

Thanks for the hint @GitHubDragonFly but the main reason for which I am trying to go down this path is to build a tool that will easily merge together the Mixamo animation and import the final result in Godot (or similar game engine).

Thr resulting GLTF generated has all the animations working just fine, which is my biggest problem, I prefer to spend a bit of time debugging the code to figure out where the issue is, I have pin pointed it to triangles being assigned the wrong materials so I will delve deep in the exporter to see if I can find a pointer.

Merging the animation manually is extremely painful and with threejs is really straightforward to merge them, I initially tried in C++ with the Autodesk FBX sdk but the documentation is really terrible and although using it I got to a point where the multiple animations are in the same FBX I can't get them to be attached to the model 🤦

So I prefer the threejs way if I can find where the issue is.

[Mugen87]

It happens although a bit differently although disclaimer: as it's an FBX and it's a local file, textures (map and normalMap) aren't loaded automatically, so I had to fiddle with the console to import them.

@danielealbano Tip: You should not overwrite the textures with new instances loaded via TextureLoader. When you do this, you overwrite the texture settings embedded in the FBX. However, the goal is to get the asset correctly loaded and exported without any further adjustments.

When I drag'n'drop the FBX asset with its four textures into the editor, the asset is loaded as expected like in your first screenshot:

After a closer investigation of the asset in the editor, I can confirm there is an issue with GLTFExporter. The problem is that the asset uses a specific multi-material setup which the exporter does not support, see #21538.

Marking this issue as a duplicate.

[danielealbano]

@Mugen87 thanks for sharing the link to the other issue, I was able to use createMeshesFromMultiMaterialMesh from SceneUtils to fix the issue, thanks for implementing it!

Said that though, I had to do a small change to properly support skinned meshes as the generated messes are not linked to the skeleton.

In the function createMeshesFromMultiMaterialMesh, in examples/jsm/utils/SceneUtils.js, I simply changed

const newMesh = new Mesh( newGeometry, newMaterial );
object.add( newMesh );

to

const newMesh = new THREE.SkinnedMesh( newGeometry, newMaterial );
object.add( newMesh );
newMesh.bind( mesh.skeleton );

And now the animations play nicely!

Do you think it's worth to make a small PR to automatically detect if it's a SkinnedMesh, instead of a Mesh, and attach the skeleton?
Meanwhile additional operations might be required (I am not sure if a Skeleton might be a child of a SkinnedMesh and, in that case, the skeleton should be moved out and probably added, together with the new meshes, under a parent node), having returned the skinned meshes directly might avoid to manually re-re-creating them.

Also, do you think it might be worth to add a warning with a link to the documentaton if the GLTFExporter detects multiple materials on the same mesh?

Let me know if it's ok for you, I can make the PRs later on

[Mugen87]

I think it's better if we try to fix the issue in GLTFExporter first. I'll file a PR.