Add option for refining the TSDF using the mesh's AABB #3459
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I found the TSDF mesh quality to be highly dependent on the TSDF grid location and resolution. Currently, the user is responsible for providing a tight bounding box around the object to ensure good TSDF grid resolution in the areas of interest. However, this is often inconvenient and slow. It might also be hard to achieve when the NeRF is trained on a server for which the viewer isn't available.
In this PR, I suggest a simple solution for refining a rough bounding box estimate using the computed bounding box of the original mesh. In practice, I found this to lead to big improvements.
Note that this assumes that the density estimates are good enough to prevent floaters far from the object. I set the option to False by default, as this will be an issue if no special care is taken.
I trained
nerfacto
with alpha-transparent training on images of a mustard bottle:Exporting with rough estimate + no TSDF refinement:
Exporting with rough estimate + TSDF refinement:
The refined mesh is much smoother.