It success if we revert back the version

[ichsan2895]

As the developers said in this thread HERE, The developers use old version of Nerfstudio. And I try it.

This ERROR also disappears when I revert back to previous version

>> ns-train nerfgs --data path/to/my_dataset --max-num-iterations 1000 \
    --pipeline.model.ply-file-path path/to/nerfgs.ply

Step (% Done)       Train Iter (time)    ETA (time)           Train Rays / Sec                       
-----------------------------------------------------------------------------------                  
500 (50.00%)        99.006 ms            49 s, 503.237 ms     6.92 M                                 
510 (51.00%)        103.206 ms           50 s, 570.830 ms     6.64 M                                 
520 (52.00%)        106.473 ms           51 s, 106.916 ms     6.31 M                                 
530 (53.00%)        111.375 ms           52 s, 346.087 ms     5.98 M                                 
540 (54.00%)        99.696 ms            45 s, 860.255 ms     6.78 M                                 
550 (55.00%)        103.981 ms           46 s, 791.480 ms     6.60 M                                 
560 (56.00%)        110.463 ms           48 s, 603.667 ms     6.37 M                                 
570 (57.00%)        89.986 ms            38 s, 693.966 ms     7.90 M                                 
580 (58.00%)        89.734 ms            37 s, 688.363 ms     7.59 M                                 
590 (59.00%)        104.310 ms           42 s, 767.154 ms     6.53 M                                 
---------------------------------------------------------------------------------------------------- 
Viewer running locally at: http://localhost:7007/ (listening on 0.0.0.0)                              
[03:30:22] Splitting 0.0006363746341970631 gaussians: 1273/2000394                                     �]8;id=652483;file:///workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/models/splatfacto.py�\splatfacto.py�]8;;�\:�]8;id=560069;file:///workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/models/splatfacto.py#550�\550�]8;;�\
           Duplicating 0.0013257388294505982 gaussians: 2652/2000394                                   �]8;id=695205;file:///workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/models/splatfacto.py�\splatfacto.py�]8;;�\:�]8;id=28418;file:///workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/models/splatfacto.py#588�\588�]8;;�\
Printing profiling stats, from longest to shortest duration in seconds
Trainer.train_iteration: 0.0746              
VanillaPipeline.get_train_loss_dict: 0.0589              
Traceback (most recent call last):
  File "/usr/local/bin/ns-train", line 8, in <module>
    sys.exit(entrypoint())
  File "/workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/scripts/train.py", line 262, in entrypoint
    main(
  File "/workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/scripts/train.py", line 247, in main
    launch(
  File "/workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/scripts/train.py", line 189, in launch
    main_func(local_rank=0, world_size=world_size, config=config)
  File "/workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/scripts/train.py", line 100, in train_loop
    trainer.train()
  File "/workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/engine/trainer.py", line 265, in train
    callback.run_callback_at_location(
  File "/workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/engine/callbacks.py", line 115, in run_callback_at_location
    self.run_callback(step=step)
  File "/workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/engine/callbacks.py", line 100, in run_callback
    self.func(*self.args, **self.kwargs, step=step)
  File "/workspace/NERFSTUDIO_ABSGS/nerfstudio/nerfstudio/models/splatfacto.py", line 463, in refinement_after
    torch.zeros_like(dup_params["scales"][:, 0]),
KeyError: 'scales'

What the version that I use:

# clean up previous version
pip uninstall gsplat nerfstudio -y

pip install gsplat==0.1.3
pip install git+https://github.com/nerfstudio-project/[email protected]

# force to install previous version
git clone https://github.com/grasp-lyrl/NeRFtoGSandBack --recursive
cd NeRFtoGSandBack/
git checkout c3f72e269b1d36c81a7908733353e3bf92864820
pip install -e nerfsh
pip install -e nerfgs
ns-install-cli

Now, the result is much better with 10k iterations of ns-train nerfgs. Here is the rendered frame of the video:

Additional info:

Ubuntu 22.04.5 LTS
Python 3.10.12
Torch 2.0.1+cu118

[MrDieAlot]

Can confirm that the commands above works also on Windows. Trained 30000 iterations and seem to get good results.

[ichsan2895]

My side notes:

1. Nerfstudio is evolving change every time. Sometimes new features gets new bugs.
2. Recently, there are big overhaul of splatfacto and gsplat. There are called Abs Grad in nerfstudio==1.1.0 and Fast implementation of SH & New method of camera poses optimization in gsplat==0.1.11
3. So, if the developers use old nerfstudio for this repo, I think it must test again in newest commit of nerfstudio & gsplat.

[Siming-He]

Thanks for the notes and test!

I'll revert back the version and edit the README so that people can use the code with older nerfstudio.

I'll create a new branch, and change the code for newer nerfstudio & gsplat when I have time.

[gradeeterna]

Hey, does this work with latest version of nerfstudio yet, or do we still need to use the older one?

Also I was hoping that I could train nerfacto on fisheye images and then convert to gs, but when I tried it the nerf2gs part required pinhole like regular splatfacto. Is there any workaround for that? Thanks!