Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resnet50, SqueezeNet and VGG19 represented twice from different source frameworks #82

Open
crahrig opened this issue Jul 24, 2018 · 4 comments

Comments

@crahrig
Copy link

crahrig commented Jul 24, 2018

The model zoo currently contains three models (Resnet50, SqueezeNet and VGG19) that each have two variants which is confusing to end consumers.

Ideally these should be de-duplicated. If that doesn't make sense then they should state their differences outside of origin framework and be organized in a way that places them all in the same sub-folder/path on the repo.

Following are pointers to the existing duplicated models:

@lupesko
Copy link

lupesko commented Jul 26, 2018

Thanks for the note @crahrig ! Fair point.
Labeled and you are welcomed to submit a PR to merge into models listed here where there is more detailed documentation.

@idilsulo
Copy link

For the case of SqueezeNet models, there are not only two different models, but also one of the models results in poor accuracy values when I execute it.

For the first SqueezeNet model the link is given as the following:
https://github.com/onnx/models/tree/master/models/image_classification/squeezenet

The probabilities that I get after executing this model is given below:

class=n02123045 tabby, tabby cat ; probability=0.411851
class=n02124075 Egyptian cat ; probability=0.295258
class=n02123159 tiger cat ; probability=0.251817
class=n02127052 lynx, catamount ; probability=0.037665
class=n02114367 timber wolf, grey wolf, gray wolf, Canis lupus ; probability=0.001155

Whereas, for the second model, the probabilities predicted for each class is pretty inaccurate. The link to this model is given as the following:
https://github.com/onnx/models/tree/master/squeezenet

The probabilities are also given below:

class=n15075141 toilet tissue, toilet paper, bathroom tissue ; probability=1.000000
class=n02319095 sea urchin ; probability=1.000000
class=n02395406 hog, pig, grunter, squealer, Sus scrofa ; probability=1.000000
class=n02391049 zebra ; probability=1.000000
class=n02389026 sorrel ; probability=1.000000

@snnn
Copy link
Contributor

snnn commented Nov 15, 2018

Right now, I think, the top priority thing is getting them correct. All models here should conform to ONNX standard, and have a reasonable accuracy. I don't think it's a good time point to dedup models. Let's close issues like: #90 first.

@idilsulo
Copy link

@snnn What I mentioned above is not related to models being duplicate but rather one model having a really low accuracy value (not being correct) compared to the other one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants