Creating metadata.json file for my own dataset

[AliKaramiFBK]

Hi
I want to run this net on my own dataset
I have images and masks
also the calibration file that I got from metashep including interior and exterior orientations
but I dont know how to create metadata.json file as it is required to run your code

would you please let me know how can I create this file

best regards
Ali

[jasonyzhang]

Hi Ali,

To use NeRS out of the box, you will need to set initial cuboid dimensions and the camera extrinsics via azimuth/elevation representation in degrees. The metadata.json is just an easier way to store this information. I would recommend that you start with the demo notebook which you can also run on colab. The notebook will show how these values are used.

To set the camera parameters using your calibration data, you will need to convert the extrinsics to rotation matrices. What is the format of your extrinsics data?

Also, how many images are you using? Typically, for sparse views, automatically recovered camera poses are highly inaccurate.

Best,
Jason

[AliKaramiFBK]

Hi Jason
thanks for your reply
I have the calibration file in Metashape and also calmap
so I can have many different formats but not metadata.json
do you know how can I convert it ? any libraries or available scripts to do that.
Here are some formats that I can get from Metashape.

Best
Ali

[jasonyzhang]

Hi,

You will need to convert your camera poses to the Pytorch3D camera convention.

Given the 4x4 camera extrinsics tensor from COLMAP, you should be able to convert it to the Pytorch3D camera convention by doing something like this (based on facebookresearch/pytorch3d#1120):

T = np.array(colmap_camera)  # 4x4
torch3d_T_colmap = np.array([[-1, 0, 0], [0, -1, 0], [0, 0, 1]])
R, t = T[:3, :3], T[:3, 3]
R = (torch3d_T_colmap @ R).T
t = torch3d_T_colmap @ t

If you want to use the COLMAP cameras, you should no longer being using metadata.json, but rather write your own loader that reads these camera poses.

How many images are you using? What is the output when using the demo notebook with hand selected poses?

[eric-yim]

Can you link sample code for loading camera poses from COLMAP? I'm lost with how to start.