Data pre-processing

[liruikangyk]

Thanks a lot for your excellent work, but I have some difficulties downloading the processed dataset you kindly provided (due to the complex network reachability in my country... ToT). I wonder that do you have the plan to share the data processing code of Waymo/Nuscenes datasets? Thank u very much!

[ShengyuH]

hi,

Thanks for your interest in our work. Let me try to upload the dataset to an alternative server(BaiduYun) which is more accessible to mainland China. In the meantime, I do plan to release the data preprocessing code for both Waymo and nuScenes datasets. However, currently I am busy with another project, I might only be able to release that part by December. Sorry for the inconvenience.

Best,
Shengyu

[Kin-Zhang]

Also looking forward to code in data preprocessing.

[wuqianliang]

Also looking forward to code in data preprocessing.

[RaymondByc]

Hi, would you like to release the pre-process code this month?

Are there differences with FastFlow3D?

[ShengyuH]

hi,

I apologize for the delayed response. I recently uploaded a folder named dataset_toolbox with two subfolders, waymo and nuscene, containing scripts for exploring and preprocessing the respective datasets. The folder prep_nuscene_waymo_sf contains scripts for generating the final data. Although they may not be directly executable, I hope the functions provided will serve your needs.

Best Regards,
Shengyu

[yanconglin]

Hi @ShengyuH ,
Thank you for releasing the code.
I do notice that there are several different waymo/nuscenes -variants for scene flow estimation.
Could you please clarify:

1. Is your version the same as the official version released by Waymo (implemented there, Fastflow3D)?

2. NuScenes: There is another preprocessed version for scene flow estimation (SLIM, ICCV2021). What are the differences between the SLIM version and yours?

I would greatly appreciate it if you can kindly comment on this. Many thanks.

[ShengyuH]

hi,

Thank you for your interest.

1. I didn't check the codebase as it's not the official one. I also didn't compare to the scene flows released by Waymo. I prepared the dataset before their release.
2. They didn't release the codebase so I can't really judge that.

I am aware that there are several dataset preparations available there, I think we share the same principle but might differ slightly in implementation.

I hope this clarifies.

best,
Shengyu

[yanconglin]

Thank you for the clarification. it would be a tough decision on which version to adopt as sota methods are not benchmarking on the data.

[ShengyuH]

Yes, and please be careful that how different methods crop and split the scenes is also different. I hope our released dataset and the dataloader can help a bit in that sense.

[yanconglin]

Hi, again, a question on ground removal.
visualized some examples and found out that the ground removal does not work nicely. Can you please share some hints on the threshold adopted in the paper?
Attached please find an example from waymo using the given threshold (black indicates ground):