Accuracy reported is different from accuracy through reproduced models

[aishwarya-rm]

I notice that when I run the vgg and mobilenet v2 models that you have already trained I get very similar accuracy to what you've reported, but when I try to reproduce those models through the method you suggest, I get significantly lower (20%) accuracy. Do you have any suggestions about how to improve that?

[pannaf]

Same. For mobilenet v2, with the default suggestion, I get something like:

Average Precision Per-class:
aeroplane: 0.5657933307950505
bicycle: 0.5958365508753679
bird: 0.4086400403629856
boat: 0.39042670180981476
bottle: 0.2186384020784879
bus: 0.6301307106013779
car: 0.6580358771149338
cat: 0.6643675101095109
chair: 0.3050131169713594
cow: 0.3497012449605594
diningtable: 0.5554262138275705
dog: 0.5491393700338869
horse: 0.6335805483851406
motorbike: 0.6147627261899834
person: 0.5939436865972345
pottedplant: 0.2141154264353295
sheep: 0.4522329331115968
sofa: 0.5716547884678509
train: 0.7024842940363902
tvmonitor: 0.5006772289276247

Average Precision Across All Classes:0.5087300350846028

This is probably because the line to train it lists only 20 epochs, which I don't think is enough. At 160 epochs, it's approaching the reported performance:

Average Precision Per-class:
aeroplane: 0.6717632263511579
bicycle: 0.773783494220913
bird: 0.6021018186509035
boat: 0.5321030529004017
bottle: 0.34351326631029755
bus: 0.7863310851386212
car: 0.7419784863918485
cat: 0.8146225528095588
chair: 0.5089493935903733
cow: 0.6304121683980828
diningtable: 0.7302754329890575
dog: 0.768516489115439
horse: 0.8268834591628732
motorbike: 0.7902032886946462
person: 0.7071659350335838
pottedplant: 0.3983321045913091
sheep: 0.6145165868654735
sofa: 0.766619031563188
train: 0.8006663148748322
tvmonitor: 0.6389221108619328

Average Precision Across All Classes:0.6723829649257247

[qfgaohao]

@eyeshoe @pannaf Have you used the pretrained mobilenetv2 models?

[pannaf]

No, I have not. I may have misunderstood, but I thought OP was asking about how to get the same accuracy with re-training from the ImageNet weights.

[aishwarya-rm]

I have used the pretrained models, but I'm curious how you actually trained them- I was confused because I thought that the process you explain to train is actually how you trained those models.

[STARK1234]

I used the pretrained model, and I set num_epochs 200. Only reach 0.66

[pannaf]

Have you tried running it again? Could have just been an unlucky initialization.

[STARK1234]

I will give a try. Have you reached the reported performance? How many epochs?

[pannaf]

Yes, by 200 epochs I reached the reported performance.

[STARK1234]

Okay, thx.