Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

densenet and shufflenetv1 model low accuracy #124

Closed
aceduce opened this issue Dec 6, 2018 · 1 comment
Closed

densenet and shufflenetv1 model low accuracy #124

aceduce opened this issue Dec 6, 2018 · 1 comment

Comments

@aceduce
Copy link

aceduce commented Dec 6, 2018

Hi, I ran most CNN models in the model zoo for onnx and achieve similar/reasonable accuracy for the imagenet test data.

  1. but when it comes to densenet and shufflenet, I saw the accuracy is very very low. For ShuffleNet, I always observe same results labels almost regardless of what input images I give. This is tested under Caffe2, (and MXNET, TVM, when feasible),. They all show similar issues. Is it because some preprocessing is missed? in the doc, it doesn't say any preprocessing is needed. Or it's due to some model issues.

  2. As some previous issues reported (Resnet50, SqueezeNet and VGG19 represented twice from different source frameworks #82), models such as resnet in onnx/models is not as good as in onnx/models/models/image_classification/, what's the cause for that? in general, I don't see preprocessing specified in models under onnx/models. I also observe point 3 as follows in Pytorch, which could be a hint, I guess (?)

  3. It seems in Pytorch torchvision, there are base models (such as resnet, densenet, squeezenet) and related specific models (such as resnet18, densenet121 etc), I think they are the ones exported to onnx/models and /models/models/image_classification/ respectively. Does anyone know the background why there are separate models in torchvision? Are the 'base' model ready for use also, or just the specific models? Through the test I did, I find out that most models under https://github.com/onnx/models/tree/master/models/image_classification can reach the good level of accuracy.

Just to recap, the core questions are how to deploy the densenet and shufflenet under onnx/models/ to reasonable accuracy. Please help!

Thank you!

@prasanthpul prasanthpul transferred this issue from onnx/onnx Dec 6, 2018
@prasanthpul
Copy link
Member

dupe of #123

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants