🐛 react native vision camera don't scan pdf-417 from real documents

[fmoraless]

What's happening?

I have implemented react native vision camera in my react native 0.75 application.

my objective is to scan tickets, which have the code pdf-417

Following the documentation, and doing some tests, I have been able to verify that the pdf417 scanning works correctly with ios. But when scanning the same document with android it does not read the pdf417 of the sale receipts.

I was doing some tests to solve this problem and it turns out that when creating a pdf417 with few characters, it is possible to read it (page to create pdf417 cognex).

But the problem is that the voucher or ticket ,has a standard that has a lot of characters (it is an Xml with all the data of the ballot, social reason, trade rut, amount, date, etc).

I appreciate your support
Tks

Reproduceable Code

import {Alert, Pressable, StyleSheet, Text, View} from 'react-native';
import React, {useCallback, useEffect, useRef, useState} from 'react';
import {
  Camera,
  Templates,
  useCameraDevice,
  useCameraFormat,
  useCameraPermission,
  useCodeScanner,
} from 'react-native-vision-camera';
import {useIsFocused} from '@react-navigation/native';
import {Loader} from '../../components/Loader';
import type {Code} from 'react-native-vision-camera';
import {Icon, IconButton} from 'react-native-paper';
import {useAppState} from '../../hooks/useAppState';

export const ScannerScreen = ({navigation}) => {
  //  const [mode, setMode] = useState('camera');

  const device = useCameraDevice('back');
  //console.log('device', device?.hardwareLevel);

  //const devices = Camera.getAvailableCameraDevices();
  //console.log(JSON.stringify(devices, null, 2));

  const format = useCameraFormat(device, Templates.Instagram);

  const showCodeAlert = (value: string, onDismissed: () => void): void => {
    console.log('showCodeAlert', value);
    isShowingAlert.current = true;
    Alert.alert('Code Scanned', value, [
      {
        text: 'OK',
        onPress: () => {
          isShowingAlert.current = false;
          onDismissed();
        },
      },
    ]);
  };

  const isShowingAlert = useRef(false);

  const onCodeScanned = useCallback((codes: Code[]) => {
    console.log('onCodeScanned', codes);
    console.log(`Scanned ${codes.length} codes!`);
    const value = codes[0].value;
    console.log('Scanned value:', value);
    if (value == null) {
      return;
    }

    if (isShowingAlert.current) {
      return;
    }

    showCodeAlert(value, () => {
      isShowingAlert.current = false;
    });
    isShowingAlert.current = true;
  }, []);

  /*  const codeScanner = useCodeScanner({
    codeTypes: ['qr', 'ean-13', 'pdf-417', 'aztec', 'codabar'],
    onCodeScanned: onCodeScanned,
  }); */
  const codeScanner = useCodeScanner({
    codeTypes: ['qr', 'ean-13', 'code-128', 'code-39', 'pdf-417'],
    onCodeScanned: onCodeScanned,
  });

  const isFocused = useIsFocused();
  const appState = useAppState();
  const isActive = isFocused && appState === 'active';
  const {hasPermission, requestPermission} = useCameraPermission();

  const camera = useRef<Camera>(null);

  console.log('ScannerScreen::Application::STATUS::AppState', appState);
  console.log('ScannerScreen::Application::STATUS::isActive', isActive);

  //console.log('cameraScreen::MODE', mode);

  useEffect(() => {
    if (!hasPermission) {
      requestPermission();
    }
  }, [hasPermission]);

  //console.log('has Permission?', hasPermission);

  const onCodeScanPress = () => {
    console.log('code scan pressed');
  };

  if (!hasPermission) {
    return <Loader />;
  }

  if (device == null) {
    return (
      <View style={StyleSheet.absoluteFill}>
        <Text>device error</Text>
      </View>
    );
  }

  return (
    <View style={{flex: 1}}>
      {device != null && (
        <Camera
          style={StyleSheet.absoluteFill}
          device={device}
          isActive={isActive}
          codeScanner={codeScanner}
          enableZoomGesture={true}
          format={format}
        />
      )}

      <View>
        <IconButton
          onPress={() => {
            navigation.goBack();
          }}
          icon="arrow-left"
          size={30}
          iconColor="white"
          style={{
            position: 'absolute',
            top: 30,
            left: 20,
            padding: 5,
            borderRadius: 10,
            backgroundColor: 'rgba(0, 0, 0, 0.5)',
          }}
        />
      </View>

      <View
        style={{
          position: 'absolute',
          top: 50,
          right: 20,
          padding: 5,
          borderRadius: 10,
          backgroundColor: 'rgba(0, 0, 0, 0.5)',
        }}>
        <IconButton icon="barcode-scan" iconColor="white" size={30} />
      </View>
      {/* <Pressable
        onPress={onCodeScanPress}
        style={{
          position: 'absolute',
          alignSelf: 'center',
          bottom: 50,
          width: 75,
          height: 75,
          borderRadius: 75,
          backgroundColor: 'white',
        }}
      /> */}
    </View>
  );
};

Relevant log output

Android studio logCat

_64  I  [Exynos][EDEN][v1.6.20][RT::EdenRuntime] ExecuteReq:353: (+) modelId=106416265, mode=0 req=0x71c03b4c80
2024-10-21 01:19:58.230  3797-3810  CameraLigh...or_Service com.samsung.adaptivebrightnessgo     I  MSG_SERVICE_CAMERA_LOOP
2024-10-21 01:19:58.231  3797-3810  CameraLigh...or_Service com.samsung.adaptivebrightnessgo     I  startBackgroundThread : mBackgroundHandler is set
2024-10-21 01:19:58.231  3797-3810  CameraLigh...or_Service com.samsung.adaptivebrightnessgo     I  Camera ID 3 is available
2024-10-21 01:19:58.231  1226-6949  CameraService           cameraserver                         I  this package can access to hidden camera ids.
2024-10-21 01:19:58.231  3797-3810  CameraManager           com.samsung.adaptivebrightnessgo     I  registerAvailabilityCallback: Is device callback = false
2024-10-21 01:19:58.231  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 0 status STATUS_NOT_AVAILABLE
2024-10-21 01:19:58.231  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 1 status STATUS_PRESENT
2024-10-21 01:19:58.231  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 2 status STATUS_PRESENT
2024-10-21 01:19:58.231  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 3 status STATUS_PRESENT
2024-10-21 01:19:58.232  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 20 status STATUS_PRESENT
2024-10-21 01:19:58.232  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 20 status STATUS_NOT_PRESENT
2024-10-21 01:19:58.232  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 20 status STATUS_NOT_PRESENT
2024-10-21 01:19:58.232  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 21 status STATUS_PRESENT
2024-10-21 01:19:58.232  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 21 status STATUS_NOT_PRESENT
2024-10-21 01:19:58.232  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 23 status STATUS_PRESENT
2024-10-21 01:19:58.232  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 23 status STATUS_NOT_PRESENT
2024-10-21 01:19:58.232  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 23 status STATUS_NOT_PRESENT
2024-10-21 01:19:58.232  3797-3810  CameraManagerGlobal     com.samsung.adaptivebrightnessgo     I  postSingleUpdate device: camera id 50 status STATUS_PRESENT
2024-10-21 01:19:58.232  3797-28548 CameraLigh...or_Service com.samsung.adaptivebrightnessgo     I  Camera 3 is available
2024-10-21 01:19:58.232  3797-28548 CameraLigh...or_Service com.samsung.adaptivebrightnessgo     I  Camera open callback start
2024-10-21 01:19:58.234  1226-6949  CameraService           cameraserver                         I  CameraService::connect call (PID -1 "com.samsung.adaptivebrightnessgo", camera ID 3) and Camera API version 2
2024-10-21 01:19:58.235  1226-6949  CameraService           cameraserver                         I  isUidActiveLocked E: uid 5021, callingPackage com.samsung.adaptivebrightnessgo, isRegistered true
2024-10-21 01:19:58.235  1226-6949  CameraService           cameraserver                         I  isUidActiveLocked X: return true.
2024-10-21 01:19:58.235  1226-6949  CameraService           cameraserver                         I  this package is a adaptive brightness app.
2024-10-21 01:19:58.235  1226-6949  CameraService           cameraserver                         I  CameraService::validateClientPermissionsLocked is ok : calling pid 3797, calling uid 5021, client com.samsung.adaptivebrightnessgo , cameraservice pid=1226, device user 0, currently allowed device users: 0
2024-10-21 01:19:58.235  1226-6949  CameraService           cameraserver                         I  CameraService::handleEvictionsLocked
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  handleEvictionsLocked: priorityMap[0] : pid=27944, priorityScore=0, state=2
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  this package is a adaptive brightness app.
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  wouldEvictLocked: [incoming client] owner = 3797, priority-score=999, cost=51, returnIncompatibleClients=false
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  wouldEvictLocked: [cost info] current cost=51, adding cost=51, total cost=102, max cost=100
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  wouldEvictLocked: [highestPriority] owner=27944, score=0
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  wouldEvictLocked: [existing client] curOwner=27944, curPriority-score=0, conflicting=false
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         E  CameraService::connect X (PID 3797) rejected (existing client(s) with higher priority).
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  wouldEvictLocked: [incoming client] owner = 3797, priority-score=999, cost=51, returnIncompatibleClients=true
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  wouldEvictLocked: [cost info] current cost=51, adding cost=51, total cost=102, max cost=100
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  wouldEvictLocked: [highestPriority] owner=27944, score=0
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         I  wouldEvictLocked: [existing client] curOwner=27944, curPriority-score=0, conflicting=false
2024-10-21 01:19:58.236  1226-6949  CameraService           cameraserver                         E     Conflicts with: Device 0, client package com.myApp.rindegastos_rn (PID 27944, score 0, state 2)
2024-10-21 01:19:58.236  3797-28548 CameraLigh...or_Service com.samsung.adaptivebrightnessgo     I  Open camera failed: error 2
2024-10-21 01:19:58.236  3797-28548 CameraLigh...or_Service com.samsung.adaptivebrightnessgo     I  stopTask called, isCameraStopPending : false
2024-10-21 01:19:58.237  3797-28548 CameraLigh...or_Service com.samsung.adaptivebrightnessgo     I  abort camera exception
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  java.lang.IllegalStateException: Session has been closed; further changes are illegal.
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at android.hardware.camera2.impl.CameraCaptureSessionImpl.checkNotClosed(CameraCaptureSessionImpl.java:887)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at android.hardware.camera2.impl.CameraCaptureSessionImpl.abortCaptures(CameraCaptureSessionImpl.java:431)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at com.samsung.adaptivebrightnessgo.CameraLightSensorService.stopTask(CameraLightSensorService.java:610)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at com.samsung.adaptivebrightnessgo.CameraLightSensorService.access$800(CameraLightSensorService.java:54)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at com.samsung.adaptivebrightnessgo.CameraLightSensorService$2.onError(CameraLightSensorService.java:250)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at android.hardware.camera2.impl.CameraDeviceImpl$8.run(CameraDeviceImpl.java:383)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at android.os.Handler.handleCallback(Handler.java:942)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at android.os.Handler.dispatchMessage(Handler.java:99)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at android.os.Looper.loopOnce(Looper.java:226)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at android.os.Looper.loop(Looper.java:313)
2024-10-21 01:19:58.237  3797-28548 System.err              com.samsung.adaptivebrightnessgo     W  	at android.os.HandlerThread.run(HandlerThread.java:67)
2024-10-21 01:19:58.237  3797-28548 CameraLigh...or_Service com.samsung.adaptivebrightnessgo     I  stopBackgroundThread : mBackgroundHandler is set to null
2024-10-21 01:19:58.296  1468-3796  MotionReco...ionService system_server                        D  values = 50.0
2024-10-21 01:19:58.297  1468-3796  MotionReco...ionService system_server                        D  onReceive lux: 50.0  action: 
2024-10-21 01:19:58.297  1468-3796  MotionReco...ionService system_server                        D  Inject brightness form Cam 50.0

Camera Device

LOG  {
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "full",
  "maxZoom": 8,
  "minZoom": 1,
  "maxExposure": 20,
  "supportsLowLightBoost": true,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 10,
  "minExposure": -20,
  "name": "0 (BACK) androidx.camera.camera2",
  "hasFlash": true,
  "hasTorch": true,
  "position": "back",
  "id": "0"
}

Device

Samsung Galaxy s20FE

VisionCamera Version

4.5.3

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[maintenance-hans]

Guten Tag, Hans here! 🥨🍻

Thanks for your detailed report. It’s great to see you've followed ze documentation and tried to troubleshoot. However, to help mrousavy resolve this issue more effectively, it would be helpful to gather additional logs while reproducing ze problem in Android.

The logs you provided are good, but could you please ensure you are using adb logcat for capturing logs specifically when ze scanning fails? This will help us get more context about what might be going wrong.

Also, please keep in mind that mrousavy maintains this in his free time, so if you appreciate his efforts, consider supporting ze project by sponsoring him here.

Let’s keep this issue open for now as it seems you are facing a valid situation!

Note: If you think I made a mistake, please ping @mrousavy to take a look.

[fmoraless]

Here there is a log, while vision camera app it was running.

I'm using react native vision camera 4.5.3
"react-native": "0.75.3",

Thanks

logcat vision camera.txt

[EdzonBolivar11]

Same issue here, trying to scan a barcode (pdf-417) from license driver of USA, going deeper seems like when the barcode has more than 250 characters it doesn't scan

@mrousavy

[EdzonBolivar11]

Is there any update with this bug?

[fmoraless]

I've looked in the documentation page , in frame processor plugins, and I've tried with some listed plugins and I have read barcodes, 6 of 10 pdf-417 barcodes were read. I guess is an issue about the processor performance. I need to do more tests about it, but is the beginning.

Regards

[EdzonBolivar11]

Which plugins did you used? @fmoraless

[xulihang]

You can try other frame processor plugins:

• vision-camera-zxing
• vision-camera-dynamsoft-barcode-reader