Doubt around the Inception Score #457
-
Hi! After training a GAN and computing the Inception Score (IS) to evaluate the diversity, I'm confused with the result the metric gives. If I understand correctly IS output should be between 1 and 1000 (because the inception network trained on ImageNet has 1000 classes) and a higher value means better quality/diversity. However, the value I obtain for the KL mean is very close to 1, while the KL std is almost 0. This confuses me because the images generated are pretty diverse from each other. I would like to ask why the result is so close to 1. I've checked the documentation for the IS metric but the message under Does this mean that I need to override the |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 5 replies
-
Hi, |
Beta Was this translation helpful? Give feedback.
Hi,
So you should not override your own compute method. The reason that docstring is in the documenting is that it has not been overriden by the inception metric class:
https://github.com/PyTorchLightning/metrics/blob/00f0256db37406df09c18ddc18f4eca99ef7d59e/torchmetrics/image/inception.py#L158-L159
which means that it just defaults to the docstring of the metric class.
https://github.com/PyTorchLightning/metrics/blob/00f0256db37406df09c18ddc18f4eca99ef7d59e/torchmetrics/metric.py#L375-L378
That said, I am not sure why you are getting a score around 1. If you could provide a file with a number of generated images we could maybe try to debug it for you?