Subsetting person class

[Steffgroe]

Hi! First of all thank you for this working implementation of SSD in pytorch.
I am trying to build a person detector with this implementation for my bachelor thesis. I removed the other classes in voc_dataset.py, but I run into trouble because your code is loading all the annotations classes by default. What do I need to change to make it train only on person and background for pascal VOC?
When I change the _get_annotation function to only include class_name == 'person' I get the following error:
ndexError: Traceback (most recent call last):
File "/home/steff/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 106, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/steff/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 106, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/steff/.local/lib/python3.6/site-packages/torch/utils/data/dataset.py", line 81, in getitem
return self.datasets[dataset_idx][sample_idx]
File "/home/steff/bachelor-project/pytorch-ssd-master/vision/datasets/voc_dataset.py", line 37, in getitem
image, boxes, labels = self.transform(image, boxes, labels)
File "/home/steff/bachelor-project/pytorch-ssd-master/vision/ssd/data_preprocessing.py", line 34, in call
return self.augment(img, boxes, labels)
File "/home/steff/bachelor-project/pytorch-ssd-master/vision/transforms/transforms.py", line 55, in call
img, boxes, labels = t(img, boxes, labels)
File "/home/steff/bachelor-project/pytorch-ssd-master/vision/transforms/transforms.py", line 275, in call
overlap = jaccard_numpy(boxes, rect)
File "/home/steff/bachelor-project/pytorch-ssd-master/vision/transforms/transforms.py", line 30, in jaccard_numpy
inter = intersect(box_a, box_b)
File "/home/steff/bachelor-project/pytorch-ssd-master/vision/transforms/transforms.py", line 13, in intersect
max_xy = np.minimum(box_a[:, 2:], box_b[2:])
IndexError: too many indices for array

How can I subset the class person properly?
Thanks in advance!

[qfgaohao]

Hi @Steffgroe , currently there is no easy way to use a subset of VOC datasets. However, you can easily use a subset of the OpenImages dataset.

[cyscn]

@Steffgroe you just need to change the 47 line of .../pytorch-ssd/vision/datasets/voc_dataset.py to
self.class_names = ('BACKGROUND', 'person')

[kurianbenoy]

@Steffgroe you just need to change the 47 line of .../pytorch-ssd/vision/datasets/voc_dataset.py to
self.class_names = ('BACKGROUND', 'person')
This doesn't work out @cyscn . @qfgaohao is it the right way to solve the problem?

[ChetanPatil28]

@kurianbenoy For this u actually need to change the trainval.txt and test.txt files which contains the list of all the .xml files.
During training or testing, there is likely chance that one of .xml file might not contain 'person' at all and due to this the boxes are returned empty and hence the IndexError.
So you simply need to put those .xml file_names (in trainval.txt and test.txt) which contain atleast one instance of 'person' so that u will always have len(boxes)>0. This way u wont get any IndexError.
After this, however, there is likely to get IndexError again because there might be one .xml file which will have just one instance of person and is_difficult='1' which again will return empty box. line69,70 in voc_dataset.py
Possible ways to eliminate this error is to set keep_difficult=True in line11,voc_dataset.py
OR
Only use the .xml files where there is atleast one instance of 'person' and its attribute is_difficult='0'

[cyscn]

Make sure there are no uppercase letters in your voc dataset ，like “PERSON”，“DOG”
in pytorch-ssd/vision/datasets/voc_dataset.py line 99,
class_name = object.find('name').text.lower().strip()
Only lowercase is allowed by default
OR，change to
class_name = object.find('name').text.strip()

[kurianbenoy]

@Steffgroe you just need to change the 47 line of .../pytorch-ssd/vision/datasets/voc_dataset.py to
self.class_names = ('BACKGROUND', 'person')

Actually only this is required for creating a subset of PASCALVOOC dataset on using someother dataset like StandfordAction40, merely changing class names and file_paths to the folder in voc_dataset.py doesn't work.

cc: @qfgaohao @cyscn

[liuyizhuang]

i have the same problem ,can you help me@Steffgroe

Traceback (most recent call last):
File "train_ssd.py", line 323, in
device=DEVICE, debug_steps=args.debug_steps, epoch=epoch)
File "train_ssd.py", line 116, in train
for i, data in enumerate(loader):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 582, in next
return self._process_next_batch(batch)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
IndexError: Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 99, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 99, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataset.py", line 85, in getitem
return self.datasets[dataset_idx][sample_idx]
File "/data1/liuyizhuang/pytorch-ssd-master/vision/datasets/voc_dataset.py", line 65, in getitem
image, boxes, labels = self.transform(image, boxes, labels)
File "/data1/liuyizhuang/pytorch-ssd-master/vision/ssd/data_preprocessing.py", line 35, in call
return self.augment(img, boxes, labels)
File "/data1/liuyizhuang/pytorch-ssd-master/vision/transforms/transforms.py", line 55, in call
img, boxes, labels = t(img, boxes, labels)
File "/data1/liuyizhuang/pytorch-ssd-master/vision/transforms/transforms.py", line 344, in call
boxes[:, :2] += (int(left), int(top))
IndexError: too many indices for array

[liuyizhuang]

@Steffgroe you just need to change the 47 line of .../pytorch-ssd/vision/datasets/voc_dataset.py to
self.class_names = ('BACKGROUND', 'person')

Actually only this is required for creating a subset of PASCALVOOC dataset on using someother dataset like StandfordAction40, merely changing class names and file_paths to the folder in voc_dataset.py doesn't work.

cc: @qfgaohao @cyscn

but, then how should we do? i have tried the above methods. it don't work

[ChetanPatil28]

@liuyizhuang , Adding to my previous comment, i have created a script which will create two new files (trainval_new.txt and test_new.txt) from the original (trainval.txt and test.txt) files. You can modify the subset (eg. ['person']) in the script.
Once script is run, you need to make two changes in line vision/datasets/voc_dataset.py 19 and line 21 ie specify the new files (trainval_new.txt and test_new.txt) instead of (trainval.txt and test.txt).
Make sure the class_names in line 25 is also same as the subset specified in the following script.
Here's the link to the script
https://drive.google.com/open?id=1E2rcEWi4L956yB7LYbnGvj-Aa4NQ90Pf

I have tried it and worked for me.

[liuyizhuang]

@liuyizhuang , Adding to my previous comment, i have created a script which will create two new files (trainval_new.txt and test_new.txt) from the original (trainval.txt and test.txt) files. You can modify the subset (eg. ['person']) in the script.
Once script is run, you need to make two changes in line vision/datasets/voc_dataset.py 19 and line 21 ie specify the new files (trainval_new.txt and test_new.txt) instead of (trainval.txt and test.txt).
Make sure the class_names in line 25 is also same as the subset specified in the following script.
Here's the link to the script
https://drive.google.com/open?id=1E2rcEWi4L956yB7LYbnGvj-Aa4NQ90Pf

I have tried it and worked for me.

thank you , this may not be my problem .because my dataset is made by myself , so not work

[liuyizhuang]

@liuyizhuang , Adding to my previous comment, i have created a script which will create two new files (trainval_new.txt and test_new.txt) from the original (trainval.txt and test.txt) files. You can modify the subset (eg. ['person']) in the script.
Once script is run, you need to make two changes in line vision/datasets/voc_dataset.py 19 and line 21 ie specify the new files (trainval_new.txt and test_new.txt) instead of (trainval.txt and test.txt).
Make sure the class_names in line 25 is also same as the subset specified in the following script.
Here's the link to the script
https://drive.google.com/open?id=1E2rcEWi4L956yB7LYbnGvj-Aa4NQ90Pf

I have tried it and worked for me.

hello ,must my dataset format be same as the VOC dataset? my dataset don't have val.txt ,segmentation,Classification Task and Detection Task. And i put all dataset in a folder.

[ChetanPatil28]

As far as i know, this repo supports format like VOC and OpenImages. In case of VOC, the annotations will be in the form of xml, if the annotations are in the csv format, then its like Open_images format.

[liuyizhuang]

As far as i know, this repo supports format like VOC and OpenImages. In case of VOC, the annotations will be in the form of xml, if the annotations are in the csv format, then its like Open_images format.

thank you ,i have xml ,but "too many indices for array"

[phamkhactu]

i still want to detect subsetting person, but when I load pre-train weight 'mb2-ssd-lite-mp-0_686.pth'. I get problem
size mismatch for classification_headers.0.3.weight: copying a param with shape torch.Size([126, 576, 1, 1]) from checkpoint, the shape in current model is torch.Size([12, 576, 1, 1]
I get it because origin has 21 classes, but I have 2 classes. Can you help me to fix it?

[ChetanPatil28]

The error basically means that you are trying to load weights trained for 20+1 classes, and your model has 1+1 classes.
It's clearly indicated in the number in the first index.
There are two fixed for this.

1. Change the model classes to 20 classes, and later you catch the person class.
   Or
2. Train a new model with 1(person) class.

[phamkhactu]

@ChetanPatil28 thank you, I fix it by pruning weight. First I load pre-train model weight. Second I get weight from index, such as I want to detect car, so I only get car branch (id= 7 in 21 classes), so I concatenate([weight[0:6], weight[42:48]) in classification_head weight. Is it right for pruning?. And I get another problem relate to priors box. I use origin priors box, because it has 1:1, 2:1, 1:2, 1:3, 3:1(aspect ratio), I think it good for car detection. But when I testing with my weight, it very bad for detection. May be it cause by priors box generate? If it True, Can you help to modify SSDSpecs ? Thanks in advance.

`
It returns the center, height and width of the priors. The values are relative to the image size

Args:

    specs: SSDSpecs about the shapes of sizes of prior boxes. i.e.

       specs = [

SSDSpec(19, 16, SSDBoxSizes(60, 105), [2, 3]),

SSDSpec(10, 32, SSDBoxSizes(105, 150), [2, 3]),

SSDSpec(5, 64, SSDBoxSizes(150, 195), [2, 3]),

SSDSpec(3, 100, SSDBoxSizes(195, 240), [2, 3]),

SSDSpec(2, 150, SSDBoxSizes(240, 285), [2, 3]),

SSDSpec(1, 300, SSDBoxSizes(285, 330), [2, 3])
 ]


    image_size: image size.

    clamp: if true, clamp the values to make fall between [0.0, 1.0]

Returns:

    priors (num_priors, 4): The prior boxes represented as [[center_x, center_y, w, h]]. All the values

        are relative to the image size.



priors = []

for spec in specs:

    scale = image_size / spec.shrinkage

    for j, i in itertools.product(range(spec.feature_map_size), repeat=2):

        x_center = (i + 0.5) / scale

        y_center = (j + 0.5) / scale



        # small sized square box

        size = spec.box_sizes.min

        h = w = size / image_size

        priors.append([

            x_center,

            y_center,

            w,

            h

        ])



        # big sized square box

        size = math.sqrt(spec.box_sizes.max * spec.box_sizes.min)

        h = w = size / image_size

        priors.append([

            x_center,

            y_center,

            w,

            h

        ])



        # change h/w ratio of the small sized box

        size = spec.box_sizes.min

        h = w = size / image_size

        for ratio in spec.aspect_ratios:

            ratio = math.sqrt(ratio)

            priors.append([

                x_center,

                y_center,

                w * ratio,

                h / ratio

            ])

            priors.append([

                x_center,

                y_center,

                w / ratio,

                h * ratio

            ])

`
Here is result, maybe cause by priors box generate.