Computing and predicting normals #905
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* documented coordinate conventions without a figure for now * another sentence * update frames docs * explain intrinsics and extrinsics * more comments * fix build
Start viewer with more accurate speed calc
ethanweber
force-pushed
the
ethan/adding_normals
branch
from
0016d84 to
89a2c94
Compare
ethanweber
changed the title
|
|
ethanweber
changed the title
ethanweber
commented
Nov 19, 2022
There was a problem hiding this comment.
Nice work. Left some comments that you can address before we merge @terrancewang.
nerfstudio/fields/nerfacto_field.py
Outdated
| @@ -260,9 +289,19 @@ def get_outputs(self, ray_samples: RaySamples, density_embedding: Optional[Tenso | |||
| dim=-1, | |||
| ) | |||
| # print(semantics_input) | |||
There was a problem hiding this comment.
delete (although not a change you made)
nerfstudio/models/nerfacto.py
Outdated
| @@ -281,6 +334,12 @@ def get_image_metrics_and_images( | |||
|
|
|||
| images_dict = {"img": combined_rgb, "accumulation": combined_acc, "depth": combined_depth} | |||
|
|
|||
| # normals to RGB for visualization. TODO: use a colormap | |||
| if "normals" in outputs: | |||
| images_dict["normals"] = outputs["normals"] | |||
There was a problem hiding this comment.
Does this not need to be scaled into the range [0, 1]? Normals would be in the range [-1, -1].
ethanweber
commented
Nov 20, 2022
nerfstudio/models/tensorf.py
Outdated
| @@ -219,7 +223,9 @@ def get_outputs(self, ray_bundle: RayBundle): | |||
| ray_samples_pdf = self.sampler_pdf(ray_bundle, ray_samples_uniform, weights) | |||
|
|
|||
| # fine field: | |||
| field_outputs_fine = self.field.forward(ray_samples_pdf, acc_mask, colors.WHITE.to(weights.device)) | |||
| field_outputs_fine = self.field.forward( | |||
| ray_samples_pdf, self.predict_normals, acc_mask, colors.WHITE.to(weights.device) | |||
There was a problem hiding this comment.
Will TensoRF work with predicted normals?
ethanweber
commented
Nov 20, 2022
| self, ray_samples: RaySamples, mask: Optional[TensorType] = None, bg_color: Optional[TensorType] = None | ||
| self, | ||
| ray_samples: RaySamples, | ||
| compute_normals: bool = False, |
There was a problem hiding this comment.
Or raise on error if compute_normals is True.
tancik
approved these changes
Nov 20, 2022
chris838
pushed a commit
to chris838/nerfstudio
that referenced
this pull request
Apr 22, 2023
* staring normals PR * code runs but visualization looks bad * Camera and world coordinate conventions (nerfstudio-project#889) * documented coordinate conventions without a figure for now * another sentence * update frames docs * explain intrinsics and extrinsics * more comments * fix build * Start viewer with more accurate speed calculation (nerfstudio-project#899) Start viewer with more accurate speed calc * Minor doc updates (nerfstudio-project#903) * fix conflicts * code runs but visualization looks bad * normals implemented and vis fixed. still debugging quality * minor name change * comment * fix black * lint * predicting normals * adding detach * predicted normals * improved normals prediction performance * cleaning code * cleaning code * cleaning code * adding type to function param * fixing tensorf forward func * reverting tensorf changes * fix style with tensorf field * adding normals check to tensorffield Co-authored-by: Matthew Tancik <[email protected]> Co-authored-by: terrance <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR gives fields the options to compute and predict normals.