🐛 Camera formats are not working consistently on Android #1064
Labels
🐛 bug
Something isn't working
Comments
Because no code is perfect and I am providing this software without warranty. I built all of it in my free time. I know that formats on Android are broken, this is because the native CameraX library just sucks. It provides minimal configuration options, and sometimes doesn't work - yet Android docs mention that you should use it and it will be the future of Camera libs on Android, when in reality Camera2 is much more powerful. I'll gladly work on this issue to refactor the entire API and make it work consistently on both iOS and Android if someone pays me for it, but I won't work hundreds of non-paid hours just because some people complain on GitHub yet offer no actual help/PRs :) |
mrousavy
changed the title
|
You are absolutely right and sorry for my sort-of harsh expression. I would be working on this too, but I doubt my knowledge of native code will be sufficient for anything :( |
|
Photo & video sizes not configurable neither with presets nor with format props. (With format you can actually change size - but it will be random on android, not even close to width/height of specified format). I had to hardcode desired resolution with patch: #868 (comment) |
|
Hey @garmoshka-mo! That patch on iOS is not safe and might crash on a few devices. On Android; yes, I left a few comments in the codebase warning that this is not accurate on Android due to CameraX's design. I will rewrite the android part to Camera2 in the V3 PR, sponsor me on GitHub to support this effort. I need a lot of free time to do that. |
|
Hey! I've rewritten the entire Android codebase of VisionCamera from CameraX to Camera2 in the efforts of ✨ VisionCamera V3. I just now completed the Camera2 rewrite and I believe the core structure is running, but there might be some edge cases to iron out. Can you try and test the PR #1674 for me to see if you can still reproduce this issue here? Here's an instruction on how you can test that: #1674 (comment) If the issue cannot be reproduced with that version/PR anymore, then hoorayy, I fixed it! 🎉 Thank you! |
Merged
austinbh1003
pushed a commit
to austinbh1003/vision-camera
that referenced
this issue
Jul 11, 2024
See mrousavy/react-native-vision-camera#1376 ## Breaking Changes * Frame Processors are now **synchronous**. Previously they ran on a separate Thread. If you want to run something on a separate Thread now, use `runAsync` inside a Frame Processor * Frame Processor Plugins are no longer in the global object with the `__` prefix, but rather stored directly in the `FrameProcessorPlugins` object exported by react-native-vision-camera. (e.g. replace `__scanQRCodes(frame)` with `FrameProcessorPlugins.scanQRCodes(frame)`) * `frameProcessorFps` no longer exists. Use `runAtTargetFps` inside a Frame Processor to throttle some calls. * `onFrameProcessorPerformanceSuggestionAvailable` no longer exists. Use the FPS display (`enableFpsGraph={true}`) to see how your Frame Processor performs over time. This is more in-line with how React Native works (Dev Tools / Perf Monitor) * VisionCamera V3 will not work on RN 0.70 or below. You need to use RN 0.71. This is because the build script got way simpler and smaller, making it faster to build and way less error prone. Backwards compatibility is just too complex here. * Reanimated is no longer used as a Worklet Runtime. Instead, VisionCamera now uses [react-native-worklets-core](https://github.com/margelo/react-native-worklets-core). ## Progress You can test the latest V3 release by creating a new RN project with RN 0.71 and installing VisionCamera + RNWorklets: ```sh yarn add [email protected] yarn add react-native-worklets-core yarn add @shopify/react-native-skia ``` Things to test: * TensorFlow Lite plugin to load any `.tflite` model!! ✨ (see [this PR for more info](mrousavy/react-native-vision-camera#1633), will be a separate library soon) * Drawing onto a Frame using Skia!! 🎉 * Using `frame.toArrayBuffer()` to get the Frame's byte content in JS * New Android build script. This should drastically speed up the build time! 💨 * New Worklet library. This replaces Reanimated Worklets. Should be faster and more stable :) * New synchronous Frame Processors. Should be faster :) * `runAtTargetFps` and `runAsync` in Frame Processors * Using HostObjects or HostFunctions (like models from PyTorch) inside a Frame Processor. This will probably require a few native bindings on PyTorch's end to make the integration work (cc @raedle) Overall V3 is close to completion. I have a few things to do the coming days so not sure how much work I can put into this. **If anyone wants to support the development of v3, I'd appreciate donations / sponsors: https://github.com/sponsors/mrousavy** ❤️ :) ## Related issues features - resolves mrousavy/react-native-vision-camera#1376 - fixes mrousavy/react-native-vision-camera#281 - resolves mrousavy/react-native-vision-camera#211 - resolves mrousavy/react-native-vision-camera#130 - resolves mrousavy/react-native-vision-camera#117 - fixes mrousavy/react-native-vision-camera#76 - resolves mrousavy/react-native-vision-camera#75 - resolves mrousavy/react-native-vision-camera#562 - resolves mrousavy/react-native-vision-camera#565 - fixes mrousavy/react-native-vision-camera#570 - fixes mrousavy/react-native-vision-camera#287 - resolves mrousavy/react-native-vision-camera#311 - fixes mrousavy/react-native-vision-camera#315 - resolves mrousavy/react-native-vision-camera#323 - fixes mrousavy/react-native-vision-camera#340 - fixes mrousavy/react-native-vision-camera#354 - resolves mrousavy/react-native-vision-camera#420 - fixes mrousavy/react-native-vision-camera#434 - fixes mrousavy/react-native-vision-camera#452 - fixes mrousavy/react-native-vision-camera#496 - fixes mrousavy/react-native-vision-camera#497 - resolves mrousavy/react-native-vision-camera#499 - fixes mrousavy/react-native-vision-camera#516 - fixes mrousavy/react-native-vision-camera#527 - fixes mrousavy/react-native-vision-camera#542 - fixes mrousavy/react-native-vision-camera#548 - fixes mrousavy/react-native-vision-camera#561 - fixes mrousavy/react-native-vision-camera#740 - fixes mrousavy/react-native-vision-camera#770 ...and then pretty much every Android issue lol - fixes mrousavy/react-native-vision-camera#1675 (**maybe**, please test @PrernaBudhraja) - fixes mrousavy/react-native-vision-camera#1671 .. maybe also (not tested): - fixes mrousavy/react-native-vision-camera#1698 - fixes mrousavy/react-native-vision-camera#1687 - fixes mrousavy/react-native-vision-camera#1685 - fixes mrousavy/react-native-vision-camera#1681 - fixes mrousavy/react-native-vision-camera#1650 - fixes mrousavy/react-native-vision-camera#1646 - fixes mrousavy/react-native-vision-camera#1635 - fixes mrousavy/react-native-vision-camera#1631 - fixes mrousavy/react-native-vision-camera#1621 - fixes mrousavy/react-native-vision-camera#1615 - fixes mrousavy/react-native-vision-camera#1612 - fixes mrousavy/react-native-vision-camera#1605 - fixes mrousavy/react-native-vision-camera#1599 - fixes mrousavy/react-native-vision-camera#1585 - fixes mrousavy/react-native-vision-camera#1581 - fixes mrousavy/react-native-vision-camera#1569 - fixes mrousavy/react-native-vision-camera#1568 - fixes mrousavy/react-native-vision-camera#1565 - fixes mrousavy/react-native-vision-camera#1561 - fixes mrousavy/react-native-vision-camera#1558 - fixes mrousavy/react-native-vision-camera#1554 - fixes mrousavy/react-native-vision-camera#1551 - fixes mrousavy/react-native-vision-camera#1547 - fixes mrousavy/react-native-vision-camera#1543 - fixes mrousavy/react-native-vision-camera#1538 - fixes mrousavy/react-native-vision-camera#1536 - fixes mrousavy/react-native-vision-camera#1534 - fixes mrousavy/react-native-vision-camera#1528 - fixes mrousavy/react-native-vision-camera#1520 - fixes mrousavy/react-native-vision-camera#1498 - fixes mrousavy/react-native-vision-camera#1489 - fixes mrousavy/react-native-vision-camera#1477 - fixes mrousavy/react-native-vision-camera#1474 - fixes mrousavy/react-native-vision-camera#1463 - fixes mrousavy/react-native-vision-camera#1462 - fixes mrousavy/react-native-vision-camera#1449 - fixes mrousavy/react-native-vision-camera#1443 - fixes mrousavy/react-native-vision-camera#1437 - fixes mrousavy/react-native-vision-camera#1431 - fixes mrousavy/react-native-vision-camera#1429 - fixes mrousavy/react-native-vision-camera#1427 - fixes mrousavy/react-native-vision-camera#1423 - fixes mrousavy/react-native-vision-camera#1416 - fixes mrousavy/react-native-vision-camera#1407 - fixes mrousavy/react-native-vision-camera#1403 - fixes mrousavy/react-native-vision-camera#1402 - fixes mrousavy/react-native-vision-camera#1398 - fixes mrousavy/react-native-vision-camera#1396 - fixes mrousavy/react-native-vision-camera#1395 - fixes mrousavy/react-native-vision-camera#1379 - fixes mrousavy/react-native-vision-camera#1377 - fixes mrousavy/react-native-vision-camera#1374 - fixes mrousavy/react-native-vision-camera#1373 - fixes mrousavy/react-native-vision-camera#1365 - fixes mrousavy/react-native-vision-camera#1356 - fixes mrousavy/react-native-vision-camera#1353 - fixes mrousavy/react-native-vision-camera#1352 - fixes mrousavy/react-native-vision-camera#1351 - fixes mrousavy/react-native-vision-camera#1343 - fixes mrousavy/react-native-vision-camera#1340 - fixes mrousavy/react-native-vision-camera#1334 - fixes mrousavy/react-native-vision-camera#1330 - fixes mrousavy/react-native-vision-camera#1322 - fixes mrousavy/react-native-vision-camera#1296 - fixes mrousavy/react-native-vision-camera#1283 - fixes mrousavy/react-native-vision-camera#1260 - fixes mrousavy/react-native-vision-camera#1253 - fixes mrousavy/react-native-vision-camera#1251 - fixes mrousavy/react-native-vision-camera#1245 - fixes mrousavy/react-native-vision-camera#1238 - fixes mrousavy/react-native-vision-camera#1227 - fixes mrousavy/react-native-vision-camera#1226 - fixes mrousavy/react-native-vision-camera#1225 - fixes mrousavy/react-native-vision-camera#1222 - fixes mrousavy/react-native-vision-camera#1211 - fixes mrousavy/react-native-vision-camera#1208 - fixes mrousavy/react-native-vision-camera#1193 - fixes mrousavy/react-native-vision-camera#1191 - fixes mrousavy/react-native-vision-camera#1184 - fixes mrousavy/react-native-vision-camera#1164 - fixes mrousavy/react-native-vision-camera#1143 - fixes mrousavy/react-native-vision-camera#1128 - fixes mrousavy/react-native-vision-camera#1122 - fixes mrousavy/react-native-vision-camera#1120 - fixes mrousavy/react-native-vision-camera#1110 - fixes mrousavy/react-native-vision-camera#1097 - fixes mrousavy/react-native-vision-camera#1081 - fixes mrousavy/react-native-vision-camera#1080 - fixes mrousavy/react-native-vision-camera#1064 - fixes mrousavy/react-native-vision-camera#1053 - fixes mrousavy/react-native-vision-camera#1047 - fixes mrousavy/react-native-vision-camera#1044 - fixes mrousavy/react-native-vision-camera#1032 - fixes mrousavy/react-native-vision-camera#1026 - fixes mrousavy/react-native-vision-camera#1023 - fixes mrousavy/react-native-vision-camera#1015 - fixes mrousavy/react-native-vision-camera#1012 - fixes mrousavy/react-native-vision-camera#997 - fixes mrousavy/react-native-vision-camera#960 - fixes mrousavy/react-native-vision-camera#959 - fixes mrousavy/react-native-vision-camera#954 - fixes mrousavy/react-native-vision-camera#946 - fixes mrousavy/react-native-vision-camera#945 - fixes mrousavy/react-native-vision-camera#922 - fixes mrousavy/react-native-vision-camera#908 - fixes mrousavy/react-native-vision-camera#907 - fixes mrousavy/react-native-vision-camera#868 - fixes mrousavy/react-native-vision-camera#855 - fixes mrousavy/react-native-vision-camera#834 - fixes mrousavy/react-native-vision-camera#793 - fixes mrousavy/react-native-vision-camera#779 - fixes mrousavy/react-native-vision-camera#746 - fixes mrousavy/react-native-vision-camera#740 - fixes mrousavy/react-native-vision-camera#727 - fixes mrousavy/react-native-vision-camera#671 - fixes mrousavy/react-native-vision-camera#613 - fixes mrousavy/react-native-vision-camera#595 - fixes mrousavy/react-native-vision-camera#588 - fixes mrousavy/react-native-vision-camera#570 - fixes mrousavy/react-native-vision-camera#569 - fixes mrousavy/react-native-vision-camera#542 - fixes mrousavy/react-native-vision-camera#516 - fixes mrousavy/react-native-vision-camera#515 - fixes mrousavy/react-native-vision-camera#434 - fixes mrousavy/react-native-vision-camera#354 - fixes mrousavy/react-native-vision-camera#323 - fixes mrousavy/react-native-vision-camera#315 - fixes mrousavy/react-native-vision-camera#281 - fixes mrousavy/react-native-vision-camera#211 - fixes mrousavy/react-native-vision-camera#76
max71126
added a commit
to max71126/react-native-vision-camera
that referenced
this issue
Sep 19, 2024
See mrousavy/react-native-vision-camera#1376 ## Breaking Changes * Frame Processors are now **synchronous**. Previously they ran on a separate Thread. If you want to run something on a separate Thread now, use `runAsync` inside a Frame Processor * Frame Processor Plugins are no longer in the global object with the `__` prefix, but rather stored directly in the `FrameProcessorPlugins` object exported by react-native-vision-camera. (e.g. replace `__scanQRCodes(frame)` with `FrameProcessorPlugins.scanQRCodes(frame)`) * `frameProcessorFps` no longer exists. Use `runAtTargetFps` inside a Frame Processor to throttle some calls. * `onFrameProcessorPerformanceSuggestionAvailable` no longer exists. Use the FPS display (`enableFpsGraph={true}`) to see how your Frame Processor performs over time. This is more in-line with how React Native works (Dev Tools / Perf Monitor) * VisionCamera V3 will not work on RN 0.70 or below. You need to use RN 0.71. This is because the build script got way simpler and smaller, making it faster to build and way less error prone. Backwards compatibility is just too complex here. * Reanimated is no longer used as a Worklet Runtime. Instead, VisionCamera now uses [react-native-worklets-core](https://github.com/margelo/react-native-worklets-core). ## Progress You can test the latest V3 release by creating a new RN project with RN 0.71 and installing VisionCamera + RNWorklets: ```sh yarn add [email protected] yarn add react-native-worklets-core yarn add @shopify/react-native-skia ``` Things to test: * TensorFlow Lite plugin to load any `.tflite` model!! ✨ (see [this PR for more info](mrousavy/react-native-vision-camera#1633), will be a separate library soon) * Drawing onto a Frame using Skia!! 🎉 * Using `frame.toArrayBuffer()` to get the Frame's byte content in JS * New Android build script. This should drastically speed up the build time! 💨 * New Worklet library. This replaces Reanimated Worklets. Should be faster and more stable :) * New synchronous Frame Processors. Should be faster :) * `runAtTargetFps` and `runAsync` in Frame Processors * Using HostObjects or HostFunctions (like models from PyTorch) inside a Frame Processor. This will probably require a few native bindings on PyTorch's end to make the integration work (cc @raedle) Overall V3 is close to completion. I have a few things to do the coming days so not sure how much work I can put into this. **If anyone wants to support the development of v3, I'd appreciate donations / sponsors: https://github.com/sponsors/mrousavy** ❤️ :) ## Related issues features - resolves mrousavy/react-native-vision-camera#1376 - fixes mrousavy/react-native-vision-camera#281 - resolves mrousavy/react-native-vision-camera#211 - resolves mrousavy/react-native-vision-camera#130 - resolves mrousavy/react-native-vision-camera#117 - fixes mrousavy/react-native-vision-camera#76 - resolves mrousavy/react-native-vision-camera#75 - resolves mrousavy/react-native-vision-camera#562 - resolves mrousavy/react-native-vision-camera#565 - fixes mrousavy/react-native-vision-camera#570 - fixes mrousavy/react-native-vision-camera#287 - resolves mrousavy/react-native-vision-camera#311 - fixes mrousavy/react-native-vision-camera#315 - resolves mrousavy/react-native-vision-camera#323 - fixes mrousavy/react-native-vision-camera#340 - fixes mrousavy/react-native-vision-camera#354 - resolves mrousavy/react-native-vision-camera#420 - fixes mrousavy/react-native-vision-camera#434 - fixes mrousavy/react-native-vision-camera#452 - fixes mrousavy/react-native-vision-camera#496 - fixes mrousavy/react-native-vision-camera#497 - resolves mrousavy/react-native-vision-camera#499 - fixes mrousavy/react-native-vision-camera#516 - fixes mrousavy/react-native-vision-camera#527 - fixes mrousavy/react-native-vision-camera#542 - fixes mrousavy/react-native-vision-camera#548 - fixes mrousavy/react-native-vision-camera#561 - fixes mrousavy/react-native-vision-camera#740 - fixes mrousavy/react-native-vision-camera#770 ...and then pretty much every Android issue lol - fixes mrousavy/react-native-vision-camera#1675 (**maybe**, please test @PrernaBudhraja) - fixes mrousavy/react-native-vision-camera#1671 .. maybe also (not tested): - fixes mrousavy/react-native-vision-camera#1698 - fixes mrousavy/react-native-vision-camera#1687 - fixes mrousavy/react-native-vision-camera#1685 - fixes mrousavy/react-native-vision-camera#1681 - fixes mrousavy/react-native-vision-camera#1650 - fixes mrousavy/react-native-vision-camera#1646 - fixes mrousavy/react-native-vision-camera#1635 - fixes mrousavy/react-native-vision-camera#1631 - fixes mrousavy/react-native-vision-camera#1621 - fixes mrousavy/react-native-vision-camera#1615 - fixes mrousavy/react-native-vision-camera#1612 - fixes mrousavy/react-native-vision-camera#1605 - fixes mrousavy/react-native-vision-camera#1599 - fixes mrousavy/react-native-vision-camera#1585 - fixes mrousavy/react-native-vision-camera#1581 - fixes mrousavy/react-native-vision-camera#1569 - fixes mrousavy/react-native-vision-camera#1568 - fixes mrousavy/react-native-vision-camera#1565 - fixes mrousavy/react-native-vision-camera#1561 - fixes mrousavy/react-native-vision-camera#1558 - fixes mrousavy/react-native-vision-camera#1554 - fixes mrousavy/react-native-vision-camera#1551 - fixes mrousavy/react-native-vision-camera#1547 - fixes mrousavy/react-native-vision-camera#1543 - fixes mrousavy/react-native-vision-camera#1538 - fixes mrousavy/react-native-vision-camera#1536 - fixes mrousavy/react-native-vision-camera#1534 - fixes mrousavy/react-native-vision-camera#1528 - fixes mrousavy/react-native-vision-camera#1520 - fixes mrousavy/react-native-vision-camera#1498 - fixes mrousavy/react-native-vision-camera#1489 - fixes mrousavy/react-native-vision-camera#1477 - fixes mrousavy/react-native-vision-camera#1474 - fixes mrousavy/react-native-vision-camera#1463 - fixes mrousavy/react-native-vision-camera#1462 - fixes mrousavy/react-native-vision-camera#1449 - fixes mrousavy/react-native-vision-camera#1443 - fixes mrousavy/react-native-vision-camera#1437 - fixes mrousavy/react-native-vision-camera#1431 - fixes mrousavy/react-native-vision-camera#1429 - fixes mrousavy/react-native-vision-camera#1427 - fixes mrousavy/react-native-vision-camera#1423 - fixes mrousavy/react-native-vision-camera#1416 - fixes mrousavy/react-native-vision-camera#1407 - fixes mrousavy/react-native-vision-camera#1403 - fixes mrousavy/react-native-vision-camera#1402 - fixes mrousavy/react-native-vision-camera#1398 - fixes mrousavy/react-native-vision-camera#1396 - fixes mrousavy/react-native-vision-camera#1395 - fixes mrousavy/react-native-vision-camera#1379 - fixes mrousavy/react-native-vision-camera#1377 - fixes mrousavy/react-native-vision-camera#1374 - fixes mrousavy/react-native-vision-camera#1373 - fixes mrousavy/react-native-vision-camera#1365 - fixes mrousavy/react-native-vision-camera#1356 - fixes mrousavy/react-native-vision-camera#1353 - fixes mrousavy/react-native-vision-camera#1352 - fixes mrousavy/react-native-vision-camera#1351 - fixes mrousavy/react-native-vision-camera#1343 - fixes mrousavy/react-native-vision-camera#1340 - fixes mrousavy/react-native-vision-camera#1334 - fixes mrousavy/react-native-vision-camera#1330 - fixes mrousavy/react-native-vision-camera#1322 - fixes mrousavy/react-native-vision-camera#1296 - fixes mrousavy/react-native-vision-camera#1283 - fixes mrousavy/react-native-vision-camera#1260 - fixes mrousavy/react-native-vision-camera#1253 - fixes mrousavy/react-native-vision-camera#1251 - fixes mrousavy/react-native-vision-camera#1245 - fixes mrousavy/react-native-vision-camera#1238 - fixes mrousavy/react-native-vision-camera#1227 - fixes mrousavy/react-native-vision-camera#1226 - fixes mrousavy/react-native-vision-camera#1225 - fixes mrousavy/react-native-vision-camera#1222 - fixes mrousavy/react-native-vision-camera#1211 - fixes mrousavy/react-native-vision-camera#1208 - fixes mrousavy/react-native-vision-camera#1193 - fixes mrousavy/react-native-vision-camera#1191 - fixes mrousavy/react-native-vision-camera#1184 - fixes mrousavy/react-native-vision-camera#1164 - fixes mrousavy/react-native-vision-camera#1143 - fixes mrousavy/react-native-vision-camera#1128 - fixes mrousavy/react-native-vision-camera#1122 - fixes mrousavy/react-native-vision-camera#1120 - fixes mrousavy/react-native-vision-camera#1110 - fixes mrousavy/react-native-vision-camera#1097 - fixes mrousavy/react-native-vision-camera#1081 - fixes mrousavy/react-native-vision-camera#1080 - fixes mrousavy/react-native-vision-camera#1064 - fixes mrousavy/react-native-vision-camera#1053 - fixes mrousavy/react-native-vision-camera#1047 - fixes mrousavy/react-native-vision-camera#1044 - fixes mrousavy/react-native-vision-camera#1032 - fixes mrousavy/react-native-vision-camera#1026 - fixes mrousavy/react-native-vision-camera#1023 - fixes mrousavy/react-native-vision-camera#1015 - fixes mrousavy/react-native-vision-camera#1012 - fixes mrousavy/react-native-vision-camera#997 - fixes mrousavy/react-native-vision-camera#960 - fixes mrousavy/react-native-vision-camera#959 - fixes mrousavy/react-native-vision-camera#954 - fixes mrousavy/react-native-vision-camera#946 - fixes mrousavy/react-native-vision-camera#945 - fixes mrousavy/react-native-vision-camera#922 - fixes mrousavy/react-native-vision-camera#908 - fixes mrousavy/react-native-vision-camera#907 - fixes mrousavy/react-native-vision-camera#868 - fixes mrousavy/react-native-vision-camera#855 - fixes mrousavy/react-native-vision-camera#834 - fixes mrousavy/react-native-vision-camera#793 - fixes mrousavy/react-native-vision-camera#779 - fixes mrousavy/react-native-vision-camera#746 - fixes mrousavy/react-native-vision-camera#740 - fixes mrousavy/react-native-vision-camera#727 - fixes mrousavy/react-native-vision-camera#671 - fixes mrousavy/react-native-vision-camera#613 - fixes mrousavy/react-native-vision-camera#595 - fixes mrousavy/react-native-vision-camera#588 - fixes mrousavy/react-native-vision-camera#570 - fixes mrousavy/react-native-vision-camera#569 - fixes mrousavy/react-native-vision-camera#542 - fixes mrousavy/react-native-vision-camera#516 - fixes mrousavy/react-native-vision-camera#515 - fixes mrousavy/react-native-vision-camera#434 - fixes mrousavy/react-native-vision-camera#354 - fixes mrousavy/react-native-vision-camera#323 - fixes mrousavy/react-native-vision-camera#315 - fixes mrousavy/react-native-vision-camera#281 - fixes mrousavy/react-native-vision-camera#211 - fixes mrousavy/react-native-vision-camera#76
isaaccolson
pushed a commit
to isaaccolson/deliveries-mobile
that referenced
this issue
Oct 30, 2024
See mrousavy#1376 ## Breaking Changes * Frame Processors are now **synchronous**. Previously they ran on a separate Thread. If you want to run something on a separate Thread now, use `runAsync` inside a Frame Processor * Frame Processor Plugins are no longer in the global object with the `__` prefix, but rather stored directly in the `FrameProcessorPlugins` object exported by react-native-vision-camera. (e.g. replace `__scanQRCodes(frame)` with `FrameProcessorPlugins.scanQRCodes(frame)`) * `frameProcessorFps` no longer exists. Use `runAtTargetFps` inside a Frame Processor to throttle some calls. * `onFrameProcessorPerformanceSuggestionAvailable` no longer exists. Use the FPS display (`enableFpsGraph={true}`) to see how your Frame Processor performs over time. This is more in-line with how React Native works (Dev Tools / Perf Monitor) * VisionCamera V3 will not work on RN 0.70 or below. You need to use RN 0.71. This is because the build script got way simpler and smaller, making it faster to build and way less error prone. Backwards compatibility is just too complex here. * Reanimated is no longer used as a Worklet Runtime. Instead, VisionCamera now uses [react-native-worklets-core](https://github.com/margelo/react-native-worklets-core). ## Progress You can test the latest V3 release by creating a new RN project with RN 0.71 and installing VisionCamera + RNWorklets: ```sh yarn add [email protected] yarn add react-native-worklets-core yarn add @shopify/react-native-skia ``` Things to test: * TensorFlow Lite plugin to load any `.tflite` model!! ✨ (see [this PR for more info](mrousavy#1633), will be a separate library soon) * Drawing onto a Frame using Skia!! 🎉 * Using `frame.toArrayBuffer()` to get the Frame's byte content in JS * New Android build script. This should drastically speed up the build time! 💨 * New Worklet library. This replaces Reanimated Worklets. Should be faster and more stable :) * New synchronous Frame Processors. Should be faster :) * `runAtTargetFps` and `runAsync` in Frame Processors * Using HostObjects or HostFunctions (like models from PyTorch) inside a Frame Processor. This will probably require a few native bindings on PyTorch's end to make the integration work (cc @raedle) Overall V3 is close to completion. I have a few things to do the coming days so not sure how much work I can put into this. **If anyone wants to support the development of v3, I'd appreciate donations / sponsors: https://github.com/sponsors/mrousavy** ❤️ :) ## Related issues features - resolves mrousavy#1376 - fixes mrousavy#281 - resolves mrousavy#211 - resolves mrousavy#130 - resolves mrousavy#117 - fixes mrousavy#76 - resolves mrousavy#75 - resolves mrousavy#562 - resolves mrousavy#565 - fixes mrousavy#570 - fixes mrousavy#287 - resolves mrousavy#311 - fixes mrousavy#315 - resolves mrousavy#323 - fixes mrousavy#340 - fixes mrousavy#354 - resolves mrousavy#420 - fixes mrousavy#434 - fixes mrousavy#452 - fixes mrousavy#496 - fixes mrousavy#497 - resolves mrousavy#499 - fixes mrousavy#516 - fixes mrousavy#527 - fixes mrousavy#542 - fixes mrousavy#548 - fixes mrousavy#561 - fixes mrousavy#740 - fixes mrousavy#770 ...and then pretty much every Android issue lol - fixes mrousavy#1675 (**maybe**, please test @PrernaBudhraja) - fixes mrousavy#1671 .. maybe also (not tested): - fixes mrousavy#1698 - fixes mrousavy#1687 - fixes mrousavy#1685 - fixes mrousavy#1681 - fixes mrousavy#1650 - fixes mrousavy#1646 - fixes mrousavy#1635 - fixes mrousavy#1631 - fixes mrousavy#1621 - fixes mrousavy#1615 - fixes mrousavy#1612 - fixes mrousavy#1605 - fixes mrousavy#1599 - fixes mrousavy#1585 - fixes mrousavy#1581 - fixes mrousavy#1569 - fixes mrousavy#1568 - fixes mrousavy#1565 - fixes mrousavy#1561 - fixes mrousavy#1558 - fixes mrousavy#1554 - fixes mrousavy#1551 - fixes mrousavy#1547 - fixes mrousavy#1543 - fixes mrousavy#1538 - fixes mrousavy#1536 - fixes mrousavy#1534 - fixes mrousavy#1528 - fixes mrousavy#1520 - fixes mrousavy#1498 - fixes mrousavy#1489 - fixes mrousavy#1477 - fixes mrousavy#1474 - fixes mrousavy#1463 - fixes mrousavy#1462 - fixes mrousavy#1449 - fixes mrousavy#1443 - fixes mrousavy#1437 - fixes mrousavy#1431 - fixes mrousavy#1429 - fixes mrousavy#1427 - fixes mrousavy#1423 - fixes mrousavy#1416 - fixes mrousavy#1407 - fixes mrousavy#1403 - fixes mrousavy#1402 - fixes mrousavy#1398 - fixes mrousavy#1396 - fixes mrousavy#1395 - fixes mrousavy#1379 - fixes mrousavy#1377 - fixes mrousavy#1374 - fixes mrousavy#1373 - fixes mrousavy#1365 - fixes mrousavy#1356 - fixes mrousavy#1353 - fixes mrousavy#1352 - fixes mrousavy#1351 - fixes mrousavy#1343 - fixes mrousavy#1340 - fixes mrousavy#1334 - fixes mrousavy#1330 - fixes mrousavy#1322 - fixes mrousavy#1296 - fixes mrousavy#1283 - fixes mrousavy#1260 - fixes mrousavy#1253 - fixes mrousavy#1251 - fixes mrousavy#1245 - fixes mrousavy#1238 - fixes mrousavy#1227 - fixes mrousavy#1226 - fixes mrousavy#1225 - fixes mrousavy#1222 - fixes mrousavy#1211 - fixes mrousavy#1208 - fixes mrousavy#1193 - fixes mrousavy#1191 - fixes mrousavy#1184 - fixes mrousavy#1164 - fixes mrousavy#1143 - fixes mrousavy#1128 - fixes mrousavy#1122 - fixes mrousavy#1120 - fixes mrousavy#1110 - fixes mrousavy#1097 - fixes mrousavy#1081 - fixes mrousavy#1080 - fixes mrousavy#1064 - fixes mrousavy#1053 - fixes mrousavy#1047 - fixes mrousavy#1044 - fixes mrousavy#1032 - fixes mrousavy#1026 - fixes mrousavy#1023 - fixes mrousavy#1015 - fixes mrousavy#1012 - fixes mrousavy#997 - fixes mrousavy#960 - fixes mrousavy#959 - fixes mrousavy#954 - fixes mrousavy#946 - fixes mrousavy#945 - fixes mrousavy#922 - fixes mrousavy#908 - fixes mrousavy#907 - fixes mrousavy#868 - fixes mrousavy#855 - fixes mrousavy#834 - fixes mrousavy#793 - fixes mrousavy#779 - fixes mrousavy#746 - fixes mrousavy#740 - fixes mrousavy#727 - fixes mrousavy#671 - fixes mrousavy#613 - fixes mrousavy#595 - fixes mrousavy#588 - fixes mrousavy#570 - fixes mrousavy#569 - fixes mrousavy#542 - fixes mrousavy#516 - fixes mrousavy#515 - fixes mrousavy#434 - fixes mrousavy#354 - fixes mrousavy#323 - fixes mrousavy#315 - fixes mrousavy#281 - fixes mrousavy#211 - fixes mrousavy#76
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What were you trying to do?
I want to adjust coordinates returned from frame processor plugin to draw points on screen correctly
Reproduceable Code
No response
What happened instead?
Camera formats and everything related to it are completely wrong and broken.
photoWidthandphotoHeightofuseCameraFormathook are totally wrong, since the width and height of frame in frameprocessor plugin is for example 640x640 while current cameraformat is 2000x4000 :|and #910 should be open too
how all of this is even possible :|
Relevant log output
No response
Device
Android devices
VisionCamera Version
2.13.0
Additional information
The text was updated successfully, but these errors were encountered: