Fbx Import for Camera Tracking

[MarMar54321]

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like

It would be great if you could make it possible to import fbx animations from blender for example to nerfstudio as camera path. Some other Nerf generating application is able todo so and this makes things easier.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[tancik]

This would be a nice feature to include. Do you know of other nerf libraries that have this support that we can reference?

[MarMar54321]

Hi,Yes there is one that allow that and you can export an animation From Blender as fbx and import directly to there. It is https://lumalabs.aiNot sure if this answers your question.BRMarkusAm 28.11.2022 um 00:39 schrieb Matthew Tancik ***@***.***>: This would be a nice feature to include. Do you know of other nerf libraries that have this support that we can reference? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[tancik]

Thanks for the extra info! Luma is unfortunately closed source so there isn't much we can gleam from them. There doesn't seem to be an easy to use python FBX library that I can find. Autodesk FBX library doesn't have a pypi package and py-FBX appears to be an unfinished project. I'll keep searching...

[vincepapaix]

just an idea here, instead of working on a import/export of FBX files in nerfstudio if it's too complex. An alternative will be to find a way to convert a FBX camera data into a camera_path.json, this can be achieved in python in various softwares.

the main issue though is that we need to have a correlation between the 3d world space in blender/maya/etc and nerfstudio.

[tancik]

Yea, that could work, can you provide examples of other python based software doing this?

[vincepapaix]

A software like the foundry nuke can import fbx/abc camera system and in the attributes of the camera it converts xyz position rotation to world matrix
from there you can use the scritping panels to write some code to export out a camera_path.json that can be imported in nerfstudio

there is non commercial nuke here
https://www.foundry.com/products/nuke-family/non-commercial

Hope it helps

[tancik]

We will add a blender workflow which will hopefully capture this workflow in #1218

[bhack]

A software like the foundry nuke can import fbx/abc camera system and in the attributes of the camera it converts xyz position rotation to world matrix
from there you can use the scritping panels to write some code to export out a camera_path.json that can be imported in nerfstudio

Can we re-open this? A direct fbx/abc export it will have a more universal integration coverage.

[tancik]

We weren't sure how to add this functionality in practice. Do you have any ideas?

[bhack]

You can use the python API https://docs.alembic.io/python/examples.html