Part segmentation format

The STK support several different types of segmentation formats.

The main types are given below:

  TRIMESH:      'trimesh',                 // Segmentation is specified using mesh index and triangle index (used for wss surfaces)
  TRIMESH_HIER: 'trimeshHier',
  SEG_GROUP:    'segmentGroups',           // Segmentation is specified as array of elements (either vertices or triangles) to segment index
                                           // During annotation, segments are grouped together (used for ScanNet, Matterport3D)
  INDEXED_SEG:   'indexedSegmentation'

After annotation, the annotated segments can be exported as

annotated-segment-triindices - annotation is specified as triangle indices (triIndices)
annotated-segment-ids - annotation is specified as segment ids 

Associated with a well-formed segmentation is a metadata field with following fields:

• method - How was the segmentation created
• format - What format was used to create the segmentation
• condenseFaceIndices - Boolean indicating whether the face indices are condensed, where instead of storing all the face indices, ranges of consecutive indices are stored as array of start and end index (not including the end index). Range of size one are still stored as a singleton element.

trimesh

Segmentation is specified by the meshIndex and the triangle index within the mesh. This is the format used for wss surfaces and hssd connectivity segs.

Segmentation is stored as an array of segments, with segIndex giving the index in the array, and meshIndex and triIndex specifying the mesh and triangle indices (within that mesh).

Example:

{
  "modelId": "fpModel.00258bed0c6e87a14e33c3eebffc48c898135698",
  "metadata": {
    "method": "connectivity",
    "format": "trimesh",
    "condenseFaceIndices": true
  },
  "segmentation": [
    {
      "meshIndex": 0,
      "triIndex": [                              # Triangle indices from 0 to 9936 (exclusive, not including 9936)
        [
          0,
          9936
        ]
      ],
      "segIndex": 0
    },
    ...
  ]
}

Old example (where the segmentation is called surface, and there is no metadata specifying the format):

{
  "modelId": "wss.room24",
  "surface": [
    {
      "surfaceIndex": 137,
      "meshIndex": 3,
      "triIndex": [
        445,
        446
      ]
    },
    ...
  ]
}

indexedSegmentation

Segmentation format used used for hierarchical part segmentation.

• The elementType specifies what the segmentation is over (triangles or vertices) and the elementCount specifies how many triangles/vertices there are.

• Segmentation is given as array of numbers indicating the segment index corresponding to each triangle (i.e. it should be an array of elementCount with the ith element indicating the segment index for the ith triangle)

• Multiple segmentations can be provided, each with a different name. There doesn't have to be any relationship between the different segmentations. Actual segmentations can be arbitrary.

• Labels for segments are provided as an array of strings (typically with 0 = 'unknown').

Example:

{
  "id": "xxx.yyy",                       # xxx indicate the asset group name, yyy is the asset id         
  "elementType": "triangles",
  "elementCount": 4837,
  "segmentation": [
    {
      "name": "faces",                   # Faces (in js/obj file), there are often quads
      "index": [ 0, 0, 1, 1, ...]
    },
    {
      "name": "materials",               # segmented by material
      "index": [ 0, 0, 0, 0, ...]
    },
    {
      "name": "meshes",                  # same as for material for original model (some models have multiple meshes)
      "index": [ 0, 0, 0, 0, ...]
    },
    {
      "name": "surfaces",                # result of normal based segmentation (respecting original mesh/material boundaries), mostly flattish, but can be curved
      "index": [ 0, 0, 0, 0, ...]
    },
    {
      "name": "pieces",                  # result of less aggressive (don't care about normal) segmentation (still respecting original mesh/material boundaries)      
      "index": [ 0, 0, 0, 0, ...]
    },
    {
      "name": "components",              # intended to be connected components (not the greatest), across different meshes/materials
      "index": [ 0, 0, 0, 0, ...]
    },
    {
      "name": "parts",                   # Annotated parts by some person!  
      "labels": [
        "unknown",
        "mattress",
        "bedcover",
        "pillow",
        "pillow",
        "bedframe"
      ],
      "index": [ 1, 1, 1, 1, ...]
    }
  ],
  "annId": 15044
}