NuScenes dataparser #939
Merged
NuScenes dataparser #939
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
November 10, 2022 00:25
tancik
reviewed
Nov 11, 2022
| from pathlib import Path | ||
| import torch | ||
|
|
||
| from nuscenes.nuscenes import NuScenes as NuScenesDatabase |
There was a problem hiding this comment.
This package is missing from the requirements.
scripts/process_nuscenes_masks.py
Outdated
| @@ -0,0 +1,136 @@ | |||
| from dataclasses import dataclass | |||
There was a problem hiding this comment.
Can you create a new folder scripts/datasets/ to put this file in.
|
|
||
| @dataclass | ||
| class NuScenesDataParserConfig(DataParserConfig): | ||
| """Nerfstudio dataset config""" |
There was a problem hiding this comment.
Can you add a little more information about what the nuscenes dataset it, including a mention of the mask script. I worry people won't know the mask script exists.
There was a problem hiding this comment.
Hi,
How to generate the mask of the Nuscene dataset.
[23:33:03] Saving config to: outputs/scene-0061/nerfacto/2023-02-01_233303/config.yml experiment_config.py:124
[23:33:03] Saving checkpoints to: outputs/scene-0061/nerfacto/2023-02-01_233303/nerfstudio_models trainer.py:120
logging events to: outputs/scene-0061/nerfacto/2023-02-01_233303
Setting up training dataset...
Caching all 36 images.
Printing profiling stats, from longest to shortest duration in seconds
Traceback (most recent call last):
File "/home/zongmu/anaconda3/envs/nerfstudio/bin/ns-train", line 8, in <module>
sys.exit(entrypoint())
File "/DATA2/xiaoquan.wang/nerfstudio/scripts/train.py", line 247, in entrypoint
main(
File "/DATA2/xiaoquan.wang/nerfstudio/scripts/train.py", line 233, in main
launch(
File "/DATA2/xiaoquan.wang/nerfstudio/scripts/train.py", line 172, in launch
main_func(local_rank=0, world_size=world_size, config=config)
File "/DATA2/xiaoquan.wang/nerfstudio/scripts/train.py", line 86, in train_loop
trainer.setup()
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/engine/trainer.py", line 145, in setup
self.pipeline = self.config.pipeline.setup(
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/configs/base_config.py", line 57, in setup
return self._target(self, **kwargs)
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/pipelines/base_pipeline.py", line 222, in __init__
self.datamanager: VanillaDataManager = config.datamanager.setup(
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/configs/base_config.py", line 57, in setup
return self._target(self, **kwargs)
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/data/datamanagers/base_datamanager.py", line 327, in __init__
super().__init__()
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/data/datamanagers/base_datamanager.py", line 150, in __init__
self.setup_train()
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/data/datamanagers/base_datamanager.py", line 360, in setup_train
self.train_image_dataloader = CacheDataloader(
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/data/utils/dataloaders.py", line 80, in __init__
self.cached_collated_batch = self._get_collated_batch()
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/data/utils/dataloaders.py", line 117, in _get_collated_batch
batch_list = self._get_batch_list()
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/data/utils/dataloaders.py", line 111, in _get_batch_list
batch_list.append(res.result())
File "/home/zongmu/anaconda3/envs/nerfstudio/lib/python3.8/concurrent/futures/_base.py", line 437, in result
return self.__get_result()
File "/home/zongmu/anaconda3/envs/nerfstudio/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/home/zongmu/anaconda3/envs/nerfstudio/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/data/datasets/base_dataset.py", line 120, in __getitem__
data = self.get_data(image_idx)
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/data/datasets/base_dataset.py", line 101, in get_data
data["mask"] = get_image_mask_tensor_from_path(filepath=mask_filepath, scale_factor=self.scale_factor)
File "/DATA2/xiaoquan.wang/nerfstudio/nerfstudio/data/utils/data_utils.py", line 29, in get_image_mask_tensor_from_path
pil_mask = Image.open(filepath)
File "/home/zongmu/anaconda3/envs/nerfstudio/lib/python3.8/site-packages/PIL/Image.py", line 3227, in open
fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/DATA/xinchao.gou/nuscenes/masks/CAM_FRONT/n015-2018-07-24-11-22-45+0800__CAM_FRONT__1532402930612460.png
|
tancik
approved these changes
Nov 12, 2022
|
Can you run |
done |
Closed
|
@zmurez Is there an option to use all the cameras instead of just the front camera, or should I add that myself? |
|
Just saw the comment here: |
|
@zmurez Hi! Thanks for your great work! I wonder about the difference between world coordinates in nuScenes and that in nerfstudio. (I've tried to find it by myself but failed, still confused after reading the docs.) Thanks a lot! |
|
@zmurez Thanks for your contribution! Have you tested nuScenes dataset's other cameras? I have tested it with other cameras and all of them failed otherwise front camera. Thanks! |
chris838
pushed a commit
to chris838/nerfstudio
that referenced
this pull request
Apr 22, 2023
* nuscenes dataparser * cleanup * style * cleanup * cleanup * cleanup Co-authored-by: zmurez <[email protected]>
|
Is there any way to use multiple cameras yet? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds support for the NuScenes dataset #921. Optionally you can use the 3D cuboid labels to mask dynamic objects.