Cannot focus up close on newer phones with 3+ cameras like iPhone 13 Pro

[Quaybe]

Hello,

As the title states, we can't get the camera to focus on barcodes if they are close enough to the phone where the phone camera would normally switch to the "macro" lens, or the lens designed to capture images closer to the phone. The iPhone 13 Pro, for example, has 3 lenses. A wide-angle lens, a "normal" lens, and a "macro" lens. The normal lens is used for most things, but it can't focus up close, so it switches to the macro lens. However, quaggle2 does not switch to the macro lens, leaving barcodes out of focus and unable to scan when you need to get close to them.

Is this something that can be fixed? I've attached a picture below. In many cases, if I move the phone away, it is then too far away for Quaggle to recognize the barcode. This issue seems to occur on any phone with 3+ lenses.

?

[github-actions]

Thank you for filing an issue! Please be patient. :-)

[ericblade]

Hi there! Frequent problem actually. You'll probably need to implement a way for the user to select a specific camera -- many phones such as Samsung and LG have multiple logical cameras, and the default selected by just giving "environment" as the device constraint is often the wide-angle lens, which is no good for barcode identification thanks to the fisheye and focus.

You can use Quagga.CameraAccess.enumerateVideoDevices to get a list of valid camera devices, and their deviceIds, and then you can pass a deviceId in as a constraint to the inputStream configuration .. make sure you do not pass in a facingMode in that case.

Unfortunately, mobile devices tend not to give useful human readable names to their devices, so it may be some trial and error (although i've heard some people have been pretty successful at matching up specific deviceIds on specific models). When you call enumerateVideoDevices() on a desktop, you'll get something like [ { deviceId: 'unique device identifier', name: 'Mfg/Model of Camera' } ] but on mobiles,you tend to get [ { deviceId: 'device identifier', name: 'camera' } ] ... at best, you get something with a name like 'camera - back' or 'camera - front'. I have yet to see a phone that returns anything more useful than that.

SO, what I do is I have a form select that the user can use to select which deviceId to use, and then when they change it, it stops Quagga if running, and then restarts it with the new deviceId.

I don't know specifically of a better way. I feel like there might be something that can be done with constraints to get the browser environment to automatically select the right one, but i haven't come up with it yet.

[ericblade]

Writing a note to myself to update the documentation to include all of this information, considering it's quite important to know.

Oh, yes, there has also been recent updates (or maybe it's just proposals?? pretty sure someone's implemented it) in some browsers that allow the browser to control the auto-focus on a camera, which would be fantastic -- unfortunately, i have yet to run into a mobile phone that actually supports it yet, though.

[Quaybe]

Eric,

Thank you for that information sir! I do have a couple of follow up questions.

We tried implementing this, but it only shows an option for "Front camera" and "back camera", with no way to specify which lens to use (as in wide angle vs. close up/macro).

We also made some tweaks and seem to have it working much better for iPhones, but on my coworker's Samsung S21, it decided it was only going to use the wide angle lens for some reason. As you can imagine, this does not scan barcodes very well.

So, we're still trying to figure out how to force it to either the normal lens or the close up lens. Any thoughts appreciated!

[Quaybe]

That brings me to another question, is there a way to implement zooming in using Quaggle?

[ericblade]

One of my users had reported that getUserMedia returns at least? 3-4 cameras on a S20, I'd assume it's similar for S21..

As far as zoom goes, I'm pretty sure it is possible, depending on the device's capabilities and the browser version in use. Zoom is outside the scope of Quagga, but I'm pretty sure that it is possible to implement it at least in Chrome webview.

[suzukieng]

Just chiming in here, as I've faced the same problem. On iOS, Safari only ever reports a "Front camera" and a "Back camera". Samsung and other Android devices report a whole range of cameras with non-human-readable names, so it's trial and error there.

I wish browsers would give more detailed information about available cameras.

[ericblade]

whole range of cameras with non-human-readable names

my LG G6 and Motorola Stylus report things like "camera1 front" "camera2 back" "camera3 back", which is readable, but not exactly helpful :|

[Quaybe]

Just chiming in here, as I've faced the same problem. On iOS, Safari only ever reports a "Front camera" and a "Back camera". Samsung and other Android devices report a whole range of cameras with non-human-readable names, so it's trial and error there.

I wish browsers would give more detailed information about available cameras.

Interesting, I wonder if Chrome on iPhone would be any different. EDIT: It's not, Chrome on iPhone still only shows one front and one back camera.

I appreciate the info!

It surprises me that neither OS have a more integrated way to use the camera when called by a web site. I wonder if dedicated apps on the devices are able to use the automatic lens-switching like the Apple Camera app does, for example?

[ericblade]

TODO: Document better how to enumerate and select a device, with some version of the above information about mobile devices and their related issues.

Also TODO: Test to see if anyone is presently supporting actually setting auto-focus and documenting that if so

(autofocus just works on my Motorola G Stylus, on my several previous phones, autofocus was not available to the browser)

[Quaybe]

I'd just like to note that the camera does focus. I see it focus on objects far away, and if I put my hand in front, it focuses on my hand for example, up to the limit of the "main" lens. It just doesn't switch to the lens meant for close-ups.

Seems like on my colleague's S21, it defaults to the wide angle lens which is even worse. What's interesting is, it seems like a lot of apps that utilize the camera are facing the same issue. For example, if I open my banking app and try to get the camera to focus up close, it will not. My colleague's S21 again uses the wide angle lens which makes it nearly impossible to deposit checks via the mobile app. Just some more information for you... seems to be quite the issue!

[nitelite]

In case this helps anyone else struggling with focus issues:

Some device+browser-combinations will allow you to control the focus manually. I ended up adding a focus range slider, that uses something like the following snippet to change the camera's focus. This allows users to do manual adjustments in edge cases where the autofocus is off.

function setFocus(focusDistance) {
   const track = Quagga.CameraAccess.getActiveTrack();
   const capabilities = track.getCapabilities();
   if (!capabilities.focusDistance) {
      console.warn("Camera does not support focus control");
      return;
   }

   const focusContraints = {
      advanced: [
         {
            focusMode: "manual",
            focusDistance
         }
      ]
   };

   track
      .applyConstraints(focusContraints)
      .then(() => {
         console.log("Focus updated to", focusContraints);
      })
      .catch((err) => {
         console.warn("Unable to set focus to specified distance", err);
      });
}   

capabilities.focusDistance will contain the min, max and step to be able to set the proper value for the focusDistance.

[ericblade]

Thank you @nitelite .. I was not aware of a proposed "focusDistance" field.

[lehko0]

Since iOS 16.4 we now have access to all Virtual Camera of the iPhone. So if you want Quagga to choose the camera with an auto-focus, you can had the constraint focusMode: "continuous"

I tested it with an iPhone 14 Pro Max

Here is an exemple:

Quagga.init({
    inputStream : {
      name : "Live",
      type : "LiveStream",
      constraint : {
        focusMode: "continuous",
        facingMode: "environment"
      },
      target: document.querySelector('#yourElement')    // Or '#yourElement' (optional)
    },
    decoder : {
      readers : ["code_128_reader"]
    }
  }, function(err) {
      if (err) {
          console.log(err);
          return
      }
      console.log("Initialization finished. Ready to start");
      Quagga.start();
  });

if you are curious, here is the link to the release note of the Webkit 16.4 that talk about the changes

[ericblade]

That's great to know! I wonder if there's a way to toggle it on/off once you have a camera track, and if so, is it a standard way that would work on any device that supports it?

[lehko0]

So after digging more, I found out that Apple since iOS 16.4 changes the camera order, and the first rear camera that now gives, when requesting a facingMode: "environment", is the virtual camera that does auto-focus on iPhones that have 3 cameras. So the "fix" I said earlier is wrong, focusMode: "continuous" does nothing. By simply asking to have facingMode: "environment", we will now have a camera that does autofocus.

Also, unfortunately, we don't seem to have access to whether a camera is auto-focusing or not. This link talks about exactly this problem and there was not really a solution.

At least, this issue seem now resolve by changing nothing, we just need to ask for the environment camera and have the devices with iOS 16.4 or more.

[ericblade]

it kind of sounds like they are leaning towards "put what you want in constraints, and let the OS figure out how to do it", while everyone else is like "here's your choices of devices, use whichever one you think is best" :| :| :|

[ericblade]

According to that bug, you're going to get the Ultra-Wide camera, which is probably not at all what you want, though.

I wish I had access to a device to really run some tests on. :|