Why image shape different between Image.open and torchvision.io.read_image

[kero-ly]

🐛 Describe the bug

EXIF image:

I have a JPEG image above with EXIF information and I tried to load this image into pytorch for augmentation.

1. try with opencv

import cv2
img = cv2.imread("1.jpg")
print(img.shape[0], img.shape[1])

the result is

201 151

2. try with pillow

from PIL import Image
img3 = Image.open("1.jpg")
print(img3.size)

the result is

(201, 151)

3. try with torchvison.io

import torchvision as tv
img4 = tv.io.read_image("1.jpg")
print(img4.shape)

the result is

torch.Size([3, 151, 201])

The result of torchvison.io is in [image_channels, image_height, image_width] format, which means the image is not rotated. However, opencv and pillow will deal with the EXIF information and rotate the image to the correct orientation.

I wonder if torchvision.io.read_image misses the EXIF information in jpeg or not?

Versions

Name: torchvision
Version: 0.9.1
Summary: image and video datasets and models for torch deep learning
Home-page: https://github.com/pytorch/vision

Name: Pillow
Version: 9.4.0
Summary: Python Imaging Library (Fork)
Home-page: https://python-pillow.org

[pmeier]

from PIL import Image
img3 = Image.open("1.jpg")
print(img3.size)

In PIL, size means (width, height), while in PyTorch we use (height, width):

assert img3.size == (img3.width, img3.height)
assert img4.shape[-2:] == (img3.height, img3.width)

So nothing is missing. This is just different convention.

[kero-ly]

Thanks for your reply! I wonder if PyTorch will support dealing with the EXIF info in future versions? I prefer to use torch API or PIL API in my project, otherwise I could not process image with EXIF info @pmeier

[pmeier]

@kero-ly Please open a dedicated issue for this.

[NicolasHug]

Just to add a bit to #7947 (comment)

Looking at the image with an image viewer, it's fairly clear that H, W == 201, 151.

Opencv reports .shape as H W, just like in torchvision. So here:

• PIL and torchvision think that H, W == 151, 201
• OpenCV thinks that H, W == 201, 151, which is "more correct"