validation dataset only bboxes are loaded, no keypoints

[smidm]

Hi,

I'm training on a custom dataset (COCO foot) with a config file based on mspn50_coco_256x192.py. I've expected that the loader merges keypoint annotations (ann_file) and detected bboxes (bbox_file).

Unlike the expectation only the bboxes are loaded with all keypoints set to zero. The logic is quite clear in

mmpose/mmpose/datasets/datasets/top_down/topdown_coco_dataset.py

Line 98 in c1460e4

"""Load dataset."""

You can get either detected bboxes or keypoint annotations.

How should be the validation dataset correctly loaded? Thanks!

[jin-s13]

Yes, during validation, the keypoints will be set to zero. The boxes are either from gt json (ann_file) or detection file (bbox_file). The trained model will predict the keypoints given the bbox.

We will NOT merge keypoint annotations & detected bboxes.

After that, during evaluation, we will reload the predictions and the gt annotations.

[smidm]

How can then the validation step with bbox_file work when there are no validation keypoints loaded? This is the default (at least) for body/topdown/coco models.

[jin-s13]

The whole process is split into two parts: inference & evaluation.
During inference, the model makes keypoint predictions given the boxes, and save the predictions into pred.json.
During evaluation, we load the pred.json (obtained during inference) and gt.json (where both gt bbox and gt keypoints are available) to compute the metric.