Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Poor performance on NeRF Blender Synthetic Data & potential bugs #806

Closed
dragonlong opened this issue Oct 19, 2022 · 4 comments
Closed

Poor performance on NeRF Blender Synthetic Data & potential bugs #806

dragonlong opened this issue Oct 19, 2022 · 4 comments

Comments

@dragonlong
Copy link

Describe the bug
Hi, when I was trying to test nerfstudio-0.1.5 on the original NeRF dataset by using ./scripts/benchmarking/launch_train_blender.sh -m nerfacto -g 0, it seems the training loss for some datasets will suddenly jump into NaN for some cases, or the losses will not converge well after 2k steps in my case. Another concern is that the testing performance doesn't seem to match with expected performance, the psnr should reach > 30dB after some training. Is there anything special that I need to take care of when doing this testing? I have attached the screenshots below to help diagnosis. Thanks!

To Reproduce
Use nerfstudio-0.1.5, and run ./scripts/benchmarking/launch_train_blender.sh -m nerfacto -g 0 on NeRF blender dataset

Expected behavior
The performance should at least match with the performance reported with previous NeRF papers, like >23dB for drums, and >30dB for others

Screenshots
image
image
@tancik
Copy link
Contributor

tancik commented Oct 19, 2022

It looks like you are running the nerfacto model. This model is designed for real world unbounded scenes, as such the configuration is not ideal for unbounded synthetic scenes like the blender dataset. For example the following flags should be set: --pipeline.model.near-plane 2. --pipeline.model.far-plane 6. --pipeline.datamanager.camera-optimizer.mode off --pipeline.model.use-average-appearance-embedding False.
These alone aren't sufficient, other methods hardcode into the model that the background is white which is not done in nerfacto and other method uniformly sample the space which is also not done in nerfacto.

@dragonlong
Copy link
Author

Hi Matthew,

Thanks for the feedback! I have launched again the experiments with both instant-ngp and nerfacto, now the training profile looks normal to me at least from the training side. Will close this ticket afterwards
image

@tancik tancik mentioned this issue Nov 22, 2022
Closed
@devernay devernay mentioned this issue Nov 22, 2022
Merged
tancik pushed a commit that referenced this issue Nov 28, 2022
@devernay
Fix benchmarking scripts (#1005)
8fc7950 
* launch_eval_blender.sh: fix script

- add shebang
- add -s option to launch a single job per GPU
- last GPU was ignored
- kill all subprocesses when script is terminated

* launch_train_blender.sh: fix script

- add -s option to launch a single job per GPU
- add -v option to use tensorboard instead of wandb
- set nerfacto options according to #806 (comment)
- use a single timestamp for all training jobs
- last GPU was ignored
- kill all subprocesses when script is terminated
- print the eval script command-line

* launch_eval_blender.sh: fix script

* Update benchmarking.md

* launch_train_blender.sh: add -s to eval command-line

* Update launch_train_blender.sh

* Update benchmarking.md
@tancik tancik mentioned this issue Dec 7, 2022
Closed
tancik pushed a commit to dozeri83/nerfstudio that referenced this issue Jan 20, 2023
@devernay @tancik
Fix benchmarking scripts (nerfstudio-project#1005)
1adb6b5 
* launch_eval_blender.sh: fix script

- add shebang
- add -s option to launch a single job per GPU
- last GPU was ignored
- kill all subprocesses when script is terminated

* launch_train_blender.sh: fix script

- add -s option to launch a single job per GPU
- add -v option to use tensorboard instead of wandb
- set nerfacto options according to nerfstudio-project#806 (comment)
- use a single timestamp for all training jobs
- last GPU was ignored
- kill all subprocesses when script is terminated
- print the eval script command-line

* launch_eval_blender.sh: fix script

* Update benchmarking.md

* launch_train_blender.sh: add -s to eval command-line

* Update launch_train_blender.sh

* Update benchmarking.md
@iSach iSach mentioned this issue Feb 16, 2023
Merged
chris838 pushed a commit to chris838/nerfstudio that referenced this issue Apr 22, 2023
@devernay
Fix benchmarking scripts (nerfstudio-project#1005)
611c153 
* launch_eval_blender.sh: fix script

- add shebang
- add -s option to launch a single job per GPU
- last GPU was ignored
- kill all subprocesses when script is terminated

* launch_train_blender.sh: fix script

- add -s option to launch a single job per GPU
- add -v option to use tensorboard instead of wandb
- set nerfacto options according to nerfstudio-project#806 (comment)
- use a single timestamp for all training jobs
- last GPU was ignored
- kill all subprocesses when script is terminated
- print the eval script command-line

* launch_eval_blender.sh: fix script

* Update benchmarking.md

* launch_train_blender.sh: add -s to eval command-line

* Update launch_train_blender.sh

* Update benchmarking.md
@tancik tancik mentioned this issue Apr 24, 2023
Closed
@annci62 annci62 mentioned this issue Apr 26, 2023
Open
@iszihan iszihan mentioned this issue Sep 18, 2023
Open
@lehduong lehduong mentioned this issue Nov 22, 2023
Closed
@dubrovin-sudo
Copy link

--pipeline.model.near-plane 2. --pipeline.model.far-plane 6. --pipeline.datamanager.camera-optimizer.mode off --pipeline.model.use-average-appearance-embedding False

This command give an error:
Unrecognized or misplaced arguments: --pipeline.datamanager.camera-optimizer.mode

@samadbarrikhojasteh
Copy link

Hi,
You need to put them before the --data and --output-dir like:

ns-train nerfacto --vis viewer --pipeline.model.predict-normals True --pipeline.model.near-plane 2. --pipeline.model.far-plane 6. --pipeline.datamanager.camera-optimizer.mode off --pipeline.model.use-average-appearance-embedding False --data /.../output_colmap --output-dir /.../output_result nerfstudio-data

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tancik @dragonlong @samadbarrikhojasteh @dubrovin-sudo