🐛 maxPhotoDimensions must match one of the AVVideoDimension found in the video source device's active format's supportedMaxPhotoDimensions

[Dingenis]

What's happening?

When the camera prop enableHighQualityPhotos is set to true, the photo prop is set to true and a format is passed via the format prop, the app will crash on start.

Reproduceable Code

const device = useCameraDevice("back");
const format = useCameraFormat(device, []);

<Camera
style={StyleSheet.absoluteFill}
isActive={true} 
device={device}
format={format}
photo={true}
enableHighQualityPhotos={true}
/>

Relevant log output

2024-03-11 14:44:29.299001+0100 otofmobile[590:69909] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[AVCapturePhotoOutput setMaxPhotoDimensions:] maxPhotoDimensions must match one of the AVVideoDimension found in the video source device's active format's supportedMaxPhotoDimensions'

Camera Device

{
  "position": "back",
  "minFocusDistance": 10,
  "sensorOrientation": "landscape-right",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "isMultiCam": false,
  "supportsRawCapture": false,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "neutralZoom": 1,
  "minExposure": -8,
  "name": "Back Camera",
  "hasFlash": true,
  "maxZoom": 16,
  "minZoom": 1,
  "hardwareLevel": "full",
  "hasTorch": true,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "formats": []
}

Device

iPhone SE (16.3.1)

VisionCamera Version

4.0.0-beta.5

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[mrousavy]

I'm not sure about maxPhotoDimensions - I don't know if I even need to set that? Maybe I can just leave it out, as I think the capture pipeline uses the highest available resolution by default anyways.... trying that in #2659

Would be great if you could also help with testing that - I wanna know if there are any differences between setting maxPhotoDimensions to the highest possible resolution of the format, and just not setting it at all.

[Dingenis]

hey @mrousavy, thank you for looking into this! And sure thing, I would love to help! How can I best test it for you? I started off by running the example of v4 branch and the fix/maxresolution branch and noting down the format for both. These don't seem to differ, does that then mean setting maxPhotoDimensions isn't needed? Futhermore, the crash indeed goes away with the fix/maxresolution branch.

Format on v4
{ minISO: 22, photoHeight: 2160, fieldOfView: 65.70668029785156, videoHeight: 2160, autoFocusSystem: 'phase-detection', supportsVideoHdr: false, pixelFormats: [ 'yuv', 'yuv', 'rgb' ], videoStabilizationModes: [ 'auto', 'cinematic', 'off', 'standard' ], videoWidth: 3840, minFps: 3, supportsPhotoHdr: false, photoWidth: 3840, supportsDepthCapture: false, maxISO: 880, maxFps: 60 }

Format on fix/maxresolution
{ minISO: 22, photoHeight: 2160, fieldOfView: 65.70668029785156, videoHeight: 2160, autoFocusSystem: 'phase-detection', supportsVideoHdr: false, pixelFormats: [ 'yuv', 'yuv', 'rgb' ], videoStabilizationModes: [ 'auto', 'cinematic', 'off', 'standard' ], videoWidth: 3840, minFps: 3, supportsPhotoHdr: false, photoWidth: 3840, supportsDepthCapture: false, maxISO: 880, maxFps: 60 }

[mrousavy]

thanks - does the resulting photo have the same resolution?

[Dingenis]

Yes!

Photofile on fix/maxresolution
{ "isRawPhoto":false, "thumbnail":null, "path":"file:///private/var/mobile/Containers/Data/Application/C3519339-ECC7-46D5-B0DB-E6322504865E/tmp/28245647-7C3A-4B2C-B956-D4F1A161F853.jpeg", "height":2160, "metadata":{ "{MakerApple}":{ "1":14, "2":null, "3":{ "timescale":1000000000, "epoch":0, "value":3216802336416, "flags":1 }, "4":0, "5":167, "6":123, "7":1, "8":[ -0.06240756809711456, -0.9795665740966797, -0.23674972355365753 ], "12":[ 0.31640625, 1.4296875 ], "13":0, "14":0, "15":3, "16":1, "20":1, "23":0, "25":0, "31":0, "32":"2F939D99-18E5-4EC4-B9F7-4987194D6090", "35":[ 25, 268435531 ], "37":40, "38":1, "39":12.925505638122559, "43":"65A8E65E-6D11-438A-81AB-C293AC08B1B8", "45":5365, "46":1, "47":114, "54":80, "55":4, "59":0, "60":4, "65":false, "74":2 }, "{TIFF}":{ "ResolutionUnit":2, "Software":"16.3.1", "Make":"Apple", "DateTime":"2024:03:18 17:40:31", "XResolution":72, "HostComputer":"iPhone SE (2nd generation)", "Model":"iPhone SE (2nd generation)", "YResolution":72 }, "DPIHeight":72, "Orientation":6, "{Exif}":{ "DateTimeOriginal":"2024:03:18 17:40:31", "ExposureTime":0.01, "FNumber":1.8, "LensSpecification":[ 3.99, 3.99, 1.8, 1.8 ], "ExposureBiasValue":0, "ColorSpace":1, "FocalLenIn35mmFilm":32, "BrightnessValue":0.15357126268642995, "ExposureMode":0, "LensModel":"iPhone SE (2nd generation) back camera 3.99mm f/1.8", "SceneType":1, "PixelXDimension":3840, "ShutterSpeedValue":6.645877377994082, "SensingMethod":2, "SubjectArea":[ 1920, 1079, 2112, 950 ], "ApertureValue":1.6959938131099002, "SubsecTimeDigitized":"693", "FocalLength":3.99, "LensMake":"Apple", "SubsecTimeOriginal":"693", "OffsetTimeDigitized":"+01:00", "PixelYDimension":2160, "ISOSpeedRatings":[ 800 ], "WhiteBalance":0, "DateTimeDigitized":"2024:03:18 17:40:31", "OffsetTimeOriginal":"+01:00", "ExifVersion":"0232", "OffsetTime":"+01:00", "Flash":16, "ExposureProgram":2, "MeteringMode":5 }, "DPIWidth":72 }, "isMirrored":false, "orientation":"landscape-right", "width":3840 }

[mrousavy]

Okay then I think it makes sense to just remove that prop.

[mrousavy]

Okay I updated the PR #2659 - it now works like a charm!

[mrousavy]

Hey - I think this issue has been fixed in VisionCamera 4.0.0. 🥳

Please try V4 and let me know if you still experience this issue;

• if not, please consider 💖 sponsoring me on GitHub 💖 to support the development of VisionCamera and thank me for my time spent on fixing bugs and building new features.
• if you still see this issue, please comment and we can re-open this. But please update your native logs with the native (Xcode/Android Studio) logs from running VisionCamera V4 so I can investigate this.