Ground Truth aid for Semantic Segmentation of Point Cloud

[v-prgmr]

This PR provides the aid for semantic segmentation of the pointcloud obtained from the LIDAR. The getLidarData() API will provide a fourth attribute called labels which refers to the label of every point in the pointcloud.

The labels are the names of actors in the environment. So after the obtaining the pointcloud with the labels, one can postprocess the labels to the desired classes of their choice. i.e. The Blocks environment has an orange ball and every point falling on this orange ball will be named "OrangeBall". So the labels can be changed as per someone's requirement.

-----Python API-----
Add the following lines to parse_lidarData function to create a pandas dataframe of the point cloud with labels and then save it to a .csv file. You also need to install and import pandas in your script.

import pandas as pd

def parse_lidarData(self, data):
        # reshape array of floats to array of [X,Y,Z]  
        lidarTimeStamp = data.time_stamp  
        points = numpy.array(data.point_cloud, dtype=numpy.dtype('f8'))  
        points = numpy.reshape(points, (int(points.shape[0] / 3), 3))
        labels = data.labels
        x = pd.DataFrame(data = numpy.float_(points), columns = ["x", "y", "z"])
        se = pd.Series(labels)
        x['labels'] = se
        x.to_csv("C:/test/{}.csv".format(lidarTimeStamp), index= None)

People familiar with the blocks environment know the cone and the orangeball. In the image below you can see the corresponding points of the pointcloud along with their labels. (Software used for visualizing: CloudCompare )

[msftclas]

All CLA requirements met.

[v-prgmr]

@madratman, I have no clue why the CI failed. Any insight?

[madratman]

You can see the travis build logs by clicking "Details" on the right hand side of " continuous-integration/travis-ci/pr — The Travis CI build failed " above.
this link - https://travis-ci.org/microsoft/AirSim/builds/530704928?utm_source=github_status&utm_medium=notification

tl;dr

In file included from /build/AirLib/src/api/RpcLibClientBase.cpp:37:
/build/AirLib/include/api/RpcLibAdapatorsBase.hpp:491:15: error: no member named
      'labels' in 'msr::airlib::LidarData'
                        labels = s.labels;
                                 ~ ^
/build/AirLib/include/api/RpcLibAdapatorsBase.hpp:501:6: error: no member named
      'labels' in 'msr::airlib::LidarData'
                        d.labels = labels;
                        ~ ^
2 errors generated.
AirLib/CMakeFiles/AirLib.dir/build.make:62: recipe for target 'AirLib/CMakeFiles/AirLib.dir/build/AirLib/src/api/RpcLibClientBase.cpp.o' failed
make[2]: *** [AirLib/CMakeFiles/AirLib.dir/build/AirLib/src/api/RpcLibClientBase.cpp.o] Error 1
CMakeFiles/Makefile2:159: recipe for target 'AirLib/CMakeFiles/AirLib.dir/all' failed
make[1]: *** [AirLib/CMakeFiles/AirLib.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 68%] Building CXX object MavLinkCom/MavLinkTest/CMakeFiles/MavLinkTest.dir/build/MavLinkCom/MavLinkTest/main.cpp.o
[ 69%] Linking CXX executable ../../output/bin/MavLinkTest

I fail to see the point of this PR is filling in the label_ member is left to end user.

[v-prgmr]

@madratman , my bad that i didnt scroll to the end. Added the new labels member to the CommonStructs.hpp now. Will try the PR again now.
Btw, I could not understand your last comment. label_ will hold the labels of everypoint in the pointcloud.

[v-prgmr]

Added Comments to explain this implementation

[v-prgmr]

labels list for receiving the labels from the c++

[v-prgmr]

variable to hold the actor name (i,e, label)

[v-prgmr]

pushing the name of the actors (actor_string) into the labels vector

[v-prgmr]

these two lines (153 and 154) get the name of the actor

[madratman]

I think this PR is not that valuable, as it's asking the user to specify the segmentation ID.
#2011 gets the segmentation ID from unreal engine.
Closing this in favor of #2011