EfficientNet-ES/EL pruned models

Dense Models

EfficientNet-ES (EdgeTPU-Small) and EfficientNet-ES (EdgeTPU-Large) are trained with 8 Quadro RTX 8000 using pytorch-image-models repo.
The training scripts with hyper-params are

./distributed_train.sh 8 /imagenet --model efficientnet_es -b 128 --sched step --epochs 450 --decay-epochs 2.4 --decay-rate .97 --opt rmsproptf --opt-eps .001 -j 8 --warmup-lr 1e-6 --weight-decay 1e-5 --drop 0.2 --drop-connect 0.2 --aa rand-m9-mstd0.5 --remode pixel --reprob 0.2 --amp --lr .064

./distributed_train.sh 8 /imagenet --model efficientnet_el -b 128 --sched step --epochs 450 --decay-epochs 2.4 --decay-rate .97 --opt rmsproptf --opt-eps .001 -j 8 --warmup-lr 1e-6 --weight-decay 1e-5 --drop 0.2 --drop-connect 0.2 --aa rand-m9-mstd0.5 --remode pixel --reprob 0.2 --amp --lr .064

Pruned Models

The pruning is done by use of the DG_Prune submodule. The code is provided at my forked pytorch-image-models in DG branch.
The pruning is done using the lottery ticket hypothesis (LTH) algorithm. The pruning hyperparameters are provided in the json files attached. The training hyperparameters are exactly the same as the dense training.

Results

Model	Top1 Acc	Top5 Acc
EfficientNet-ES	77.906	94.038
EfficientNet-ES Pruned	75.060	92.438
EfficientNet-EL	81.296	95.562
EfficientNet-EL Pruned	80.318	95.212