Skip to content

Latest commit

 

History

History
44 lines (36 loc) · 2.41 KB

README.md

File metadata and controls

44 lines (36 loc) · 2.41 KB
Raw

neuron-selectivity-transfer

This code is a re-implementation of the imagenet classification experiments in the paper Like What You Like: Knowledge Distill via Neuron Selectivity Transfer .

Citation

If you use our code in your research or wish to refer to the baseline results, please use the following BibTeX entry.

@article{NST2017
  author =   {Zehao Huang and Naiyan Wang},
  title =    {Like What You Like: Knowledge Distill via Neuron Selectivity Transfer},
  journal =  {arXiv preprint arXiv:1707.01219},
  year =     {2017}
}

Implementation

This code is implemented by a modified MXNet which supports ResNeXt-like augmentation. (This version of MXNet does not support cudnn7)

ImageNet data preparation

Download the ImageNet dataset and create pass through rec (following tornadomeet's repository but using unchange mode)

Run

  • bash init.sh
  • modify config/cfgs.py
  • python train.py

Results on ImageNet-1k

Single Model, Single Crop Validation Error:

Method Model Top-1 Top-5 Download
Student Inception-BN 25.74 8.07 Dropbox
NST (Poly kernel) Inception-BN 24.81 7.55 Dropbox
NST (Poly kernel) + KD Inception-BN 24.34 7.11 Dropbox
NST (Poly kernel) + KD A modified ResNet-50 21.05 5.56 Dropbox

Note: The symbol of our modified ResNet-50 (following SENet Appendix B) is available in Dropbox. The mean RGB for our modified ResNet-50 is [123.68, 116.28, 103.53], std is [58.395, 57.12, 57.375]. For Inception-BN, we don't need to do data pre-processing since we add a BN layer in the beginning of the network.