Unable to convert heic to jpg taken by iPhone 15 pro iOS18

[tza17313]

I found that the heic format photos taken by iPhone15 pro under iOS18 cannot be converted to jpg by heif-convert

libheif version: 1.17.6

Here are some logs：

nero@d2c24395c402:/upload# heif-convert IMG_1159.HEIC 1.jpg
Could not read HEIF/AVIF file: Invalid input: Unspecified: Too many auxiliary image references

nero@d2c24395c402:/upload# heif-info IMG_1159.HEIC
MIME type: image/heic
main brand: heic
compatible brands: tmap, heic, mif1
Could not read HEIF/AVIF file: Invalid input: Unspecified: Too many auxiliary image references

When the iPhone 15 Pro system is still iOS17, the HEIC photos taken can be converted to JPG using the above command.

The file can be downloaded via the link below：

IMG_1159.zip

https://drive.google.com/file/d/1kEBvALrnM0EUwkQzsg8YL0Pwd8ZpRWWO/view?usp=sharing

[bradh]

Can you try with this branch: https://github.com/strukturag/libheif/tree/develop-v1.18.0 ?

I think I have that resolved by 58ba30d

[farindk]

With develop-v1.18.0, I get:

Could not read HEIF/AVIF file: Invalid input: Non-existing item ID referenced: Non-existing aux image referenced

Maybe the problem is that a referenced image is a tmap image.

[fxthomas]

For reference: the tmap image is a derived image (more info about that HEIF feature here) that implements the upcoming ISO/CD 21496-1 standard which is meant to replace all the current incompatible/proprietary HDR gain map implementations.

It sounds like it's ignored by m_all_images in libheif/context.cc because the derived image type is not yet supported, but some other items reference it and make parsing fail.

I used GPAC/mp4box on one of the samples I have and I think at least the EXIF block reference both the primary image and the tmap -- I could be wrong, the structure is difficult to parse (mentally) as I'm not used to it.

[farindk]

Yes, we even have a pull request to add tmap support, but I didn't integrate it yet as nothing about the standard is public yet. Or I didn't find it. Adobe has removed all specific information from their website.

On the other hand, we should probably ignore all references that we cannot parse because there might be other extensions in the future that give rise to the same issue.

[farindk]

Fixes this. Gain map is not read yet.

[Ashok-Varma]

hi @farindk will this be added in next release cut?
any rough estimates on the next release cut?

[farindk]

Yes, it will be in the v1.18.0 release.
This will be released very soon.
(I'm still waiting for some other pieces of information on another part that I want to include in that release. )

[luohuide]

Please release it as soon as possible, IOS 18 photos require it.

[farindk]

v1.18.0 is released

[lastzero]

@farindk Awesome, thanks a lot! I've tested heif-dec in version 1.18.1 (presumably the same as heif-convert in earlier versions), however the image orientation now seems to be wrong even for images taken with an iPhone 15 Pro that has iOS 17 installed:

• HEIC: Compatibility with iOS 18 image files photoprism/photoprism#4439 (comment)

Based on previous feedback, the orientation issues we observed earlier were supposed to be caused by a bug in an old iOS version well before iOS 17 - so this is unexpected:

• Handle image rotation for JPEG output #227 (comment)

It seems possible that libheif itself is working properly and the issue is instead caused by heif-dec, e.g. because it copies the metadata 1:1 without changing the Exif Orientation value, even though the image is already rotated?

If so, should using libheif through libvips solve this? The only downside would be losing compatibility with Linux < 3.17 and 32-bit operating systems, as libvips is not compatible with these:

• Synology: Fatal glibc error "cannot get entropy for arc4random" photoprism/photoprism#4339
• Config: Disable the use of libvips on 32-bit operating systems photoprism/photoprism#4299

We would be very grateful for any advice on how we can solve the HEIC orientation issues for our users! 🙏

[lastzero]

It seems possible that libheif itself is working properly and the issue is instead caused by heif-dec, e.g. because it copies the metadata 1:1 without changing the Exif Orientation value, even though the image is already rotated?

Note that I was able to fix the JPEG image orientation by simply setting the value of the Exif Orientation tag to 1 after new files have been generated with heif-dec. Since the behavior was different with the heif-convert command included in v1.17.1, we added a new config option to our software that allows users of older versions to change this. This is not ideal, but a workable solution.

Based on my tests, there is still one HEIC image file where the generated JPEG has an incorrect orientation. However, since it is from an iPhone 7 and the camera software is displayed as "ProCam 10.5.8", I believe it is not a problem with libheif but with the file itself:

• https://dl.photoprism.app/samples/Formats/Image/HEIC/iphone_7.heic

Hopefully this information will help other developers who come across the same issue! 🤗

[farindk]

Issue might be fixed with e7c5649.

What happened: heif-dec (or heif-convert) should reset the Exif orientation tag to "normal" when converting to JPEG. However, we added a bounds check to avoid the reader accessing memory beyond the Exif block in case the data is faulty. Unfortunately, the check was in the wrong direction such that it did not read the Exif block completely and thus did not reset the orientation tag. (I.e. what you have now done manually.)

I'll release an updated v1.18.2 in the coming days.
Maybe you can check that it works again in the v1.18.x branch.

BTW: I also noticed that there are some image width/height fields in the Exif that should also be swapped for rotated images. I've added that too.

PS: thanks for your support.

[lastzero]

I'm glad my report helped find and fix this issue! I can confirm that it also occurred with v1.17.6, which I believe is the version containing the vulnerability fix:

• https://dl.photoprism.app/dist/libheif/

Setting width and height based on the actual JPEG dimensions is also a welcome improvement. From what I remember, we worked around that by getting width and height from the JPEG file headers rather than trusting the Exif metadata.

I am traveling until August 19 and will therefore not be able to test the changes right now. However, I trust your own testing, so please feel free to go ahead and release these improvements as a new version! 👍

[birdman002]

Is there any chance of getting this fix applied to older versions this package?

[farindk]

Is there any chance of getting this fix applied to older versions this package?

What would be the difference to simply install a v1.18.x or v1.19.x version?

[birdman002]

Is there any chance of getting this fix applied to older versions this package?

What would be the difference to simply install a v1.18.x or v1.19.x version?

Due to circumstances of having to stay on a certain version of alpine we can't do 1.18 or 1.19. We may have to manually download and install this package but curious if there is any plans to fix previous versions say 1.16 or 1.17

[farindk]

I didn't plan this because I don't see a reason to release another v1.17.x version when v1.18.2 would do the same and it's also backwards compatible.

If you really want to stay with the v1.17.x, you can cherry pick 4d76bae for the fix.

[birdman002]

I didn't plan this because I don't see a reason to release another v1.17.x version when v1.18.2 would do the same and it's also backwards compatible.

If you really want to stay with the v1.17.x, you can cherry pick 4d76bae for the fix.

Why thank you :)