feat: Basic read-only Frame Processors (ImageAnalysis Use-Case)
#2664
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
I like the use of ImageAnalysis 🎉 |
mrousavy
added a commit
that referenced
this pull request
Apr 22, 2024
* feat: Give V4 🫴 * Update to compile SDK 34 * fix kotlin error * fix ui thread * Run all on Main Thread * Update lifecycle * Take Photo? * Type Camera Errors * Unbind before starting bind * Create Recording base? * Add `photoQualityBalance` to iOS * feat: Add `takeSnapshot()` to `PreviewView` * Add new API * Low Light Boost throttles * feat: Add CodeScanner pipeline (`ImageAnalysis`) * Format * Remove outputs * Update CameraSession.kt * Call back with Video result * Delete `RecordingSession` * Configure zoom and torch * Only update torch, zoom or exposure if needed * Create `takeSnapshot()` * Use `compressionQuality` * Format * Catch CameraErrors directly * Use new API for Snapshot * Throw `.fileError` * fix temp filename * fix: Implement `takeSnaphot` on iOS * Update Camera.tsx * Don't re-initialize outputs if only inputId changes * Update CameraSession.kt * Update CameraSession.kt * Use photo/video size selectors * fix: Use runOnUiThread only if needed * Code org * Update CameraSession.kt * feat: Use HDR or NIGHT extensions if available * fix: Check for ImageAnalysis support * feat: Shutter sound and stabilization * Update CameraSession.kt * restructure Threading a bit * Restructure Threading in CameraSession, configure is UI Thread * Update CameraSession.kt * Update CameraSession.kt * Better `mainExecutor` * Implement `focus()` * Use file type * Make `VideoPipeline` a CameraX use-case * Update CameraQueues.kt * Update VideoPipeline.kt * feat: Implement native HardwareBuffer rendering for `VideoPipeline` * Try use VideoPipeline * Try extending ImageAnalysis * Create `FrameProcessorEffect` as camera middleman * Use PREVIEW output * fix not running FP for PREVIEW * Update FrameProcessorEffect.kt * feat: Allow setting custom format in ImageWriter * Restructure code a bit * Make everything synchronized * Make `FrameProcessorEffect` closeable * chore: Clean up Codebase * Don't re-create Recorder if actively recording * docs: Update docs for `qualityBalance` * Update CameraProps.ts * Remove OpenGL VideoPipeline (now CameraX does that) * Properly disable CameraSession when `isActive=false` * Remove a few Errors * chore: release 4.0.0-beta.0 * chore: release 4.0.0-beta.1 * fix: Rename to `qualityBalance` * Update package.json * Small changes * Update CameraView.kt * Throw focus-canceled * Implement `onStarted` and `onStopped` for Android again * Always activate Camera lifecycle, ignore `isAttachedToWindow` * feat: Add `onShutter` event * chore: release 4.0.0-beta.2 * fix: Fix optional delegate * Update CameraError.kt * feat: Add `cancelRecording` * chore: release 4.0.0-beta.3 * Add `Preview` docs * feat: Add `androidPreviewViewType` prop * feat: Use CameraX for device details (#2624) * feat: Use CameraX device details * Remove `maxZoom` from format * More * feat: Implement the rest (some Camera2 fallbacks) * Properly await ProcessCameraProvider and ExtensionsManager * Initialize earlier * Update CameraDeviceDetails.kt * feat: Properly get orientation and isMirrored in FP * fix: Properly close `ImageWriter` * fix: Fix transform matrix * fix: Use image transformation info from input surface * fix: Capture directly in lambda * fix: Fix focus not running on UI Thread * chore: release 4.0.0-beta.4 * fix: Fix calling `sendAvailableDevicesChangedEvent` before module is initialized * fix: Allow re-creating if session changed from outside * fix: Don't play shutter sound unless explicitly enabled * fix: Allow catching errors in `.await()` * feat: Remove `enableAutoStabilization` prop as this was deprecated in iOS 13 * fix: Export `Snapshot.ts` * feat: Add `type` to `onShutter` (`photo` or `snapshot`) * fix: Only unbind current outputs, not all * chore: Lint/Fix * chore: release 4.0.0-beta.5 * fix: Use dummy metering factory instead of display-oriented * fix: Remove unneeded context * fix: Fix rotation getting * Update Podfile.lock * fix: Check format requirements in outputs * fix: Remove `not-compatible-with-outputs` error on Android since CameraX supports StreamSharing * fix: Remove `didSessionChangeFromOutside` * fix: Catch Objective-C errors in iOS Frame Processor Plugins * feat: Catch Objective-C init errors on plugin init * fix: Fix `TimeoutException` by using `STARTED` lifecycle * chore: release 4.0.0-beta.6 * add todo about hdr extension bug * fix: Remove focus listeners after resetting * fix: Fix capture session configuration error by manually layouting PreviewView * feat: Write proper photo metadata (orientation & isMirrored) (#2660) * feat: Write proper photo metadata (orientation & isMirrored) * Format * fix: Fix Camera not starting because PreviewView isn't laid out * fix: Use `CREATED` as default lifecycle state * fix: Fix 10-bit HDR in combination with HDR Extension * Update CameraSession.kt * fix: Throw correct errors * fix: Properly check SDR * fix: Update outputs if video stabilization or low-light-boost changes * fix: Also dont enable if other extension is enabled * chore: release 4.0.0-beta.7 * Update package.json * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) (#2664) * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) * feat: Set target resolution * Update CameraSession.kt * Refactor * fix: Fix Frame Processor not being unbound * fix: Properly set `maxPhotoDimensions` on PhotoOutput (#2659) * fix: Properly set `maxPhotoDimensions` on PhotoOutput * fix: Remove `maxPhotoDimensions` since it is the highest value by default anyways I think * fix: Remove `enableHighQualityPhotos` entirely * fix: Update RN Worklets * Revert "fix: Remove `enableHighQualityPhotos` entirely" This reverts commit 7110b76. * fix: Use high res capture / maxPhotoDimensions always. * Update CameraSession+Configuration.swift * chore: release 4.0.0-beta.8 * Update CameraSession+Photo.swift * fix: Fix focus on Android by using PX instead of DP (#2666) * fix: focus on Android (v4) * format --------- Co-authored-by: Marc Rousavy <[email protected]> * Optimised images with calibre/image-actions * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) (#2665) * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) * feat: Also add location tag to video * fix: Fix location tagging for videos * fix: Write proper location to video * Create VisionCamera branding * fix: Write branding * Use quicktime location instead of common location * Finish metadata track * Update Podfile.lock * Try writing metadata * Write video metadata * Throw error with cause * fix: Properly write metadata at start * Properly convert to ISO 6709 String * Use quicktime metadata location again * fix: Put branding into QuickTime information field * Format * Update MetadataProvider.swift * Only stream location if camera is active * fix: Set location * Format * fix: Fix changing object reference in `usePermission` * fix: Use proper location * Update MetadataProvider.kt * fix: Use last location as default * Add some logs * Only update location once every 5s * fix: Also allow coarse location * feat: Add location docs and `enableLocation` to expo config plugin * Optimised images with calibre/image-actions --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Optimised images with calibre/image-actions * fix: Remove HardwareBufferUtils * fix: Fix CodeScanner not working * chore: release 4.0.0-beta.9 * fix: Fix location permission not resolving on iOS (#2672) * fix: Fix location permission not resolving on iOS * fix: Remove unused code * fix: Ignore first call * Use `GlobalReferenceHolder` * chore: Remove RN Fast Image (we dont have web images anyways) * Update package.json * chore: release 4.0.0-beta.10 * fix: Fix wrong orientation enum being send to ImageCapture (#2683) * perf: Add `@FastNative` flags to critical Frame Processor methods * fix: Fix `RCTDeviceEventEmitter` import for RN 0.71.X (#2692) fix: build for rn 0.71.X * chore: release 4.0.0-beta.11 * feat: Disable shutter sound if in silent * fix: Remove `enablePrecapture` * chore: release 4.0.0-beta.12 * fix: Remove `mustPlayShutterSound()` check * Update Podfile.lock * chore: release 4.0.0-beta.13 * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * fix: Use `ResolutionSelector`s with fallback values * fix: Fall-back to H.264 if custom codec is not supported * Update Camera.tsx * feat: Add `hardware-cost-too-high` error for iOS 16 * chore: release 4.0.0-beta.14 * fix: Use 1280x720 for QR scanner * feat: Skia for V4! 🥳 (#2727) * feat: Add Skia support (`toSkImage()`) * feat: Add `frame.getPlaformBuffer()` * Try use getPlatformBuffer * Update Frame.ts * fix: Fix `getPlatformBuffer()` * fix: Fix buffer access * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * feat: Build Skia example renderer * fix: Render a bit better * fix: Use atomic updates now * fix: Remove SK flags * fix: Clear and copy Frame * fix: Use bigint * fix: Use `ImageProxy` and close it to avoid stalling * Only do copy on iOS * fix: Add throw method signature * feat: Box into `DrawableFrame` * feat: Create `SkiaCameraCanvas` * feat: Allow disabling `preview` use-case * feat: Catch some errors * Update FrameHostObject.cpp * fix: Use proper layouts * Update useSkiaFrameProcessor.ts * fix: Use main Coroutine Scope * fix: Fix wrong changes because we now properly abort changes * chore: Format * fix: Upgrade react-native-worklets-core Now includes shift() * fix: Add `focusRequiresPreview` error to iOS * chore: Use latest react-native-worklets-core * fix: Fix buffer orientation on iOS (it's currently always `.up` * fix: Only pass function into native, no object * fix: Format code * fix: Fix styling * fix: Fix pointer-events * Update Camera.tsx * Update useSkiaFrameProcessor.ts * Update Worklets * Draw radial face blur * circle that spins * fix: Proxify * fix: Also use worklets proxy lazily * fix: Try multiple error handlers * fix: Also use Skia Proxy * Add docs to the objects * fix: Use `createModuleProxy` * fix: Also re-create surface if height is different * perf: Remove unneeded mutex locks as Frame only locks refCount * Update FRAME_PROCESSORS_CREATE_OVERVIEW.mdx * fix: Remove unneeded CPU copy * fix: Catch PixelFormat error and fallback to default format * fix: Throw proper PixelFormat error with more info * Update CameraError.swift * Update Frame.java * fix: Use `jsi::Scope` In Hermes, this will probably not do anything. * chore: Update react-native-worklets-core * fix: Dispose previous Surface * Update useSkiaFrameProcessor.ts * fix: Fix surface re-creating each time * chore: Bump to latest RN Skia with the features we need * feat: Clean everything on unmount * Add CPU copy again. On iOS, we need it. * fix: Fix Android build * fix: Fix Android build again * Update ModuleProxy.ts * fix: Use type imports if possible * fix: Add RN Skia patch * Add react-native-fast-tflite * fix: Configure for tflite files as well * Revert "fix: Configure for tflite files as well" This reverts commit 37db0f5. * Revert "Add react-native-fast-tflite" This reverts commit 264dd4b. * Update @Shopify+react-native-skia+1.2.0.patch * Add `useSkiaFrameProcessor` to ESLint * Update yarn.lock * fix: Upgrade to latest Skia to use NativeBuffer API now * fix: Use `CVPixelBuffer` instead of `CMSampleBuffer` * fix: Fix package.json * chore: Update to latest react-native-worklets-core * fix: Use thread-local storage for SkSurface * Update Podfile.lock * fix: Migrate over to new Worklets API * Update FRAME_PROCESSORS.mdx * Update useSkiaFrameProcessor.ts * fix: Run destroy in interaction manager * fix: Safely remove surfaces * fix: Update RNWC to fix terminated threads using dead `thread_local` storage * fix: Install hierarchy fitter in `CameraView` * chore: Remove Skia example from example app * Update CameraPage.tsx * fix: Simplify default pixel format * fix: Rename `ReactLogger` to `VisionLogger` * chore: Format * fix: Fix ResolutionSelector `difference` abs (#2746) * chore: release 4.0.0-beta.15 * Update CameraPage.tsx * chore: Fix `type` imports in TS * fix: Remove `enableGpuBuffers` * fix: Make dependencies (`ModuleProxy`) actually optional (#2750) fix: Make dependencies actually optional * feat: Optionally compile out Location APIs (#2751) * fix: Rename `disableFrameProcessors` to `enableFrameProcessors` * add podfile writer * Create vc_example_cocoapod_utils.rb * feat: Make Location compile optionally * fix: Only compile out location related permissio * Update Podfile * Use `$(inherited)` * fix: Fix `minFps` being larger in `Range` than `maxFps` (#2755) * fix: Fix `minFps` being larger in `Range` than `maxFps` * import * fix: Also override `onProviderDisabled` to prevent abstract method crash * fix: Clean up Skia resources on Worklet context to avoid race condition (#2754) * fix: Clean up Skia resources on Worklet context to avoid race condition * Update FrameProcessorPlugins.ts * Update useSkiaFrameProcessor.ts * docs: Update docs for V4 (#2747) * fix: Update prism-react-renderer * fix: Fix docs * fix: Remove `enableGpuBuffers` * feat: Update Docusaurus to latest 3 * Update docusaurus.config.js * update languages * update languages * rename getting_started * Update some docs * Update CODE_SCANNING.mdx * update some docs * Update docusaurus.config.js * files * starting * fix: Properly display video * lots of updates * more docs * fix: Fix errors causing memory-leaks by not closing Frame * Add example * more * almost * fix broken links * update skia fp docs * Update FRAME_PROCESSORS_SKIA.mdx * Update FRAME_PROCESSORS_SKIA.mdx * fix broken link * chore: release 4.0.0-beta.16 * fix: Fix `Promise` resolve wrong name * fix: Fix build for Android because of Worklet usage * feat: Separate VisionCamera into two subspecs (#2761) * feat: Separate VisionCamera into two subspecs * Swift * update podspec * fix: Refactor * some fixes * hmm * fix: Only export ObjC/Swift visible headers * fix: Also drop C++ sources * fix: Move to correct folder * fix: Fix any remaining C++ includes * chore: Format * fix: Fix missingg Swift include * fix: Add PluginRegistry Swift include * Update FrameProcessorPluginRegistry.h * fix: Disable location * fix: Use `public_header_files` instead of `preserve_paths` * Disable FP if needed * fix Podfile * fix: Fix `dispatch_queue_t` in Swift * fix: Disable FPs if not built * Revert "fix: Disable FPs if not built" This reverts commit 82cb560. * Remove Example Plugins from source code * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs (#2764) * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs * Split utils * fix: Refactor Android codebase into `core`, `react` and `frameprocessors` (#2765) * Add example pod helper again * feat: Also separate Android into `react`, `core` and `frameprocessor` * Log similar to iOS * Update build.gradle * fix: Fix weird refactor mistakes * fix: Refactor a bit * fix: `frameprocessor` -> `frameprocessors` * duplicate clang format with ln * chore: release 4.0.0-beta.17 * chore: Cleanup C++ * chore: Cleanup TS codebase and fix Skia Preview View style/layout issue (#2767) * chore: Cleanup TS codebase * chore: More refactors * fix: Put Skia Canvas into CameraView * Update Camera.tsx * fix: Fix JNI types * chore: Update to RN Skia 1.2.3 * fix: Simplify `cpp/` codebase into `android`/`ios` folders (#2766) * `Frame Processor` -> `FrameProcessors` * fix: Remove almost empty `cpp/` folder * fix: Add wrap ctor to `MutableRawBuffer` * fix: Add `.clang-format` file again * fix: Remove `cpp` folder references from everywhere * fix: Fix `already bound to a different lifecycle` error when flipping camera (#2768) * fix: Fix `already bound to a different lifecycle` error when flipping camera * Update CameraSession.kt * Delete .clang-format * feat: Rebuild FPS Graph as JS component (#2769) * feat: Rebuild FPS Graph as JS component * feat: Properly style graph * chore: Lint * fix: Fix FPS Graph not displaying on Android * chore: release 4.0.0-beta.18 * fix: Fix `ImageAnalysis is not supported when Extension is enabled` error (#2770) * feat: Upgrade to CameraX alpha5 (#2771) * fix: Separate `FpsSampleCollectorDelegate` * Downgrade to CameraX alpha04 because 05 fails to build * fix: Make `FpsSampleCollectorDelegate` weak * fix: Fix `delegate` guard * fix: Fix Android import path due to new subfolder * fix: Fix `Timer is already canceled` * chore: release 4.0.0-beta.19 --------- Co-authored-by: Kirill Zyusko <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Rodrigo Gomes <[email protected]> Co-authored-by: Versus <[email protected]>
fahadhaque007
pushed a commit
to equinoxventures/react-native-vision-science-camera
that referenced
this pull request
May 3, 2024
* feat: Give V4 🫴 * Update to compile SDK 34 * fix kotlin error * fix ui thread * Run all on Main Thread * Update lifecycle * Take Photo? * Type Camera Errors * Unbind before starting bind * Create Recording base? * Add `photoQualityBalance` to iOS * feat: Add `takeSnapshot()` to `PreviewView` * Add new API * Low Light Boost throttles * feat: Add CodeScanner pipeline (`ImageAnalysis`) * Format * Remove outputs * Update CameraSession.kt * Call back with Video result * Delete `RecordingSession` * Configure zoom and torch * Only update torch, zoom or exposure if needed * Create `takeSnapshot()` * Use `compressionQuality` * Format * Catch CameraErrors directly * Use new API for Snapshot * Throw `.fileError` * fix temp filename * fix: Implement `takeSnaphot` on iOS * Update Camera.tsx * Don't re-initialize outputs if only inputId changes * Update CameraSession.kt * Update CameraSession.kt * Use photo/video size selectors * fix: Use runOnUiThread only if needed * Code org * Update CameraSession.kt * feat: Use HDR or NIGHT extensions if available * fix: Check for ImageAnalysis support * feat: Shutter sound and stabilization * Update CameraSession.kt * restructure Threading a bit * Restructure Threading in CameraSession, configure is UI Thread * Update CameraSession.kt * Update CameraSession.kt * Better `mainExecutor` * Implement `focus()` * Use file type * Make `VideoPipeline` a CameraX use-case * Update CameraQueues.kt * Update VideoPipeline.kt * feat: Implement native HardwareBuffer rendering for `VideoPipeline` * Try use VideoPipeline * Try extending ImageAnalysis * Create `FrameProcessorEffect` as camera middleman * Use PREVIEW output * fix not running FP for PREVIEW * Update FrameProcessorEffect.kt * feat: Allow setting custom format in ImageWriter * Restructure code a bit * Make everything synchronized * Make `FrameProcessorEffect` closeable * chore: Clean up Codebase * Don't re-create Recorder if actively recording * docs: Update docs for `qualityBalance` * Update CameraProps.ts * Remove OpenGL VideoPipeline (now CameraX does that) * Properly disable CameraSession when `isActive=false` * Remove a few Errors * chore: release 4.0.0-beta.0 * chore: release 4.0.0-beta.1 * fix: Rename to `qualityBalance` * Update package.json * Small changes * Update CameraView.kt * Throw focus-canceled * Implement `onStarted` and `onStopped` for Android again * Always activate Camera lifecycle, ignore `isAttachedToWindow` * feat: Add `onShutter` event * chore: release 4.0.0-beta.2 * fix: Fix optional delegate * Update CameraError.kt * feat: Add `cancelRecording` * chore: release 4.0.0-beta.3 * Add `Preview` docs * feat: Add `androidPreviewViewType` prop * feat: Use CameraX for device details (mrousavy#2624) * feat: Use CameraX device details * Remove `maxZoom` from format * More * feat: Implement the rest (some Camera2 fallbacks) * Properly await ProcessCameraProvider and ExtensionsManager * Initialize earlier * Update CameraDeviceDetails.kt * feat: Properly get orientation and isMirrored in FP * fix: Properly close `ImageWriter` * fix: Fix transform matrix * fix: Use image transformation info from input surface * fix: Capture directly in lambda * fix: Fix focus not running on UI Thread * chore: release 4.0.0-beta.4 * fix: Fix calling `sendAvailableDevicesChangedEvent` before module is initialized * fix: Allow re-creating if session changed from outside * fix: Don't play shutter sound unless explicitly enabled * fix: Allow catching errors in `.await()` * feat: Remove `enableAutoStabilization` prop as this was deprecated in iOS 13 * fix: Export `Snapshot.ts` * feat: Add `type` to `onShutter` (`photo` or `snapshot`) * fix: Only unbind current outputs, not all * chore: Lint/Fix * chore: release 4.0.0-beta.5 * fix: Use dummy metering factory instead of display-oriented * fix: Remove unneeded context * fix: Fix rotation getting * Update Podfile.lock * fix: Check format requirements in outputs * fix: Remove `not-compatible-with-outputs` error on Android since CameraX supports StreamSharing * fix: Remove `didSessionChangeFromOutside` * fix: Catch Objective-C errors in iOS Frame Processor Plugins * feat: Catch Objective-C init errors on plugin init * fix: Fix `TimeoutException` by using `STARTED` lifecycle * chore: release 4.0.0-beta.6 * add todo about hdr extension bug * fix: Remove focus listeners after resetting * fix: Fix capture session configuration error by manually layouting PreviewView * feat: Write proper photo metadata (orientation & isMirrored) (mrousavy#2660) * feat: Write proper photo metadata (orientation & isMirrored) * Format * fix: Fix Camera not starting because PreviewView isn't laid out * fix: Use `CREATED` as default lifecycle state * fix: Fix 10-bit HDR in combination with HDR Extension * Update CameraSession.kt * fix: Throw correct errors * fix: Properly check SDR * fix: Update outputs if video stabilization or low-light-boost changes * fix: Also dont enable if other extension is enabled * chore: release 4.0.0-beta.7 * Update package.json * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) (mrousavy#2664) * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) * feat: Set target resolution * Update CameraSession.kt * Refactor * fix: Fix Frame Processor not being unbound * fix: Properly set `maxPhotoDimensions` on PhotoOutput (mrousavy#2659) * fix: Properly set `maxPhotoDimensions` on PhotoOutput * fix: Remove `maxPhotoDimensions` since it is the highest value by default anyways I think * fix: Remove `enableHighQualityPhotos` entirely * fix: Update RN Worklets * Revert "fix: Remove `enableHighQualityPhotos` entirely" This reverts commit 7110b76. * fix: Use high res capture / maxPhotoDimensions always. * Update CameraSession+Configuration.swift * chore: release 4.0.0-beta.8 * Update CameraSession+Photo.swift * fix: Fix focus on Android by using PX instead of DP (mrousavy#2666) * fix: focus on Android (v4) * format --------- Co-authored-by: Marc Rousavy <[email protected]> * Optimised images with calibre/image-actions * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) (mrousavy#2665) * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) * feat: Also add location tag to video * fix: Fix location tagging for videos * fix: Write proper location to video * Create VisionCamera branding * fix: Write branding * Use quicktime location instead of common location * Finish metadata track * Update Podfile.lock * Try writing metadata * Write video metadata * Throw error with cause * fix: Properly write metadata at start * Properly convert to ISO 6709 String * Use quicktime metadata location again * fix: Put branding into QuickTime information field * Format * Update MetadataProvider.swift * Only stream location if camera is active * fix: Set location * Format * fix: Fix changing object reference in `usePermission` * fix: Use proper location * Update MetadataProvider.kt * fix: Use last location as default * Add some logs * Only update location once every 5s * fix: Also allow coarse location * feat: Add location docs and `enableLocation` to expo config plugin * Optimised images with calibre/image-actions --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Optimised images with calibre/image-actions * fix: Remove HardwareBufferUtils * fix: Fix CodeScanner not working * chore: release 4.0.0-beta.9 * fix: Fix location permission not resolving on iOS (mrousavy#2672) * fix: Fix location permission not resolving on iOS * fix: Remove unused code * fix: Ignore first call * Use `GlobalReferenceHolder` * chore: Remove RN Fast Image (we dont have web images anyways) * Update package.json * chore: release 4.0.0-beta.10 * fix: Fix wrong orientation enum being send to ImageCapture (mrousavy#2683) * perf: Add `@FastNative` flags to critical Frame Processor methods * fix: Fix `RCTDeviceEventEmitter` import for RN 0.71.X (mrousavy#2692) fix: build for rn 0.71.X * chore: release 4.0.0-beta.11 * feat: Disable shutter sound if in silent * fix: Remove `enablePrecapture` * chore: release 4.0.0-beta.12 * fix: Remove `mustPlayShutterSound()` check * Update Podfile.lock * chore: release 4.0.0-beta.13 * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * fix: Use `ResolutionSelector`s with fallback values * fix: Fall-back to H.264 if custom codec is not supported * Update Camera.tsx * feat: Add `hardware-cost-too-high` error for iOS 16 * chore: release 4.0.0-beta.14 * fix: Use 1280x720 for QR scanner * feat: Skia for V4! 🥳 (mrousavy#2727) * feat: Add Skia support (`toSkImage()`) * feat: Add `frame.getPlaformBuffer()` * Try use getPlatformBuffer * Update Frame.ts * fix: Fix `getPlatformBuffer()` * fix: Fix buffer access * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * feat: Build Skia example renderer * fix: Render a bit better * fix: Use atomic updates now * fix: Remove SK flags * fix: Clear and copy Frame * fix: Use bigint * fix: Use `ImageProxy` and close it to avoid stalling * Only do copy on iOS * fix: Add throw method signature * feat: Box into `DrawableFrame` * feat: Create `SkiaCameraCanvas` * feat: Allow disabling `preview` use-case * feat: Catch some errors * Update FrameHostObject.cpp * fix: Use proper layouts * Update useSkiaFrameProcessor.ts * fix: Use main Coroutine Scope * fix: Fix wrong changes because we now properly abort changes * chore: Format * fix: Upgrade react-native-worklets-core Now includes shift() * fix: Add `focusRequiresPreview` error to iOS * chore: Use latest react-native-worklets-core * fix: Fix buffer orientation on iOS (it's currently always `.up` * fix: Only pass function into native, no object * fix: Format code * fix: Fix styling * fix: Fix pointer-events * Update Camera.tsx * Update useSkiaFrameProcessor.ts * Update Worklets * Draw radial face blur * circle that spins * fix: Proxify * fix: Also use worklets proxy lazily * fix: Try multiple error handlers * fix: Also use Skia Proxy * Add docs to the objects * fix: Use `createModuleProxy` * fix: Also re-create surface if height is different * perf: Remove unneeded mutex locks as Frame only locks refCount * Update FRAME_PROCESSORS_CREATE_OVERVIEW.mdx * fix: Remove unneeded CPU copy * fix: Catch PixelFormat error and fallback to default format * fix: Throw proper PixelFormat error with more info * Update CameraError.swift * Update Frame.java * fix: Use `jsi::Scope` In Hermes, this will probably not do anything. * chore: Update react-native-worklets-core * fix: Dispose previous Surface * Update useSkiaFrameProcessor.ts * fix: Fix surface re-creating each time * chore: Bump to latest RN Skia with the features we need * feat: Clean everything on unmount * Add CPU copy again. On iOS, we need it. * fix: Fix Android build * fix: Fix Android build again * Update ModuleProxy.ts * fix: Use type imports if possible * fix: Add RN Skia patch * Add react-native-fast-tflite * fix: Configure for tflite files as well * Revert "fix: Configure for tflite files as well" This reverts commit 37db0f5. * Revert "Add react-native-fast-tflite" This reverts commit 264dd4b. * Update @Shopify+react-native-skia+1.2.0.patch * Add `useSkiaFrameProcessor` to ESLint * Update yarn.lock * fix: Upgrade to latest Skia to use NativeBuffer API now * fix: Use `CVPixelBuffer` instead of `CMSampleBuffer` * fix: Fix package.json * chore: Update to latest react-native-worklets-core * fix: Use thread-local storage for SkSurface * Update Podfile.lock * fix: Migrate over to new Worklets API * Update FRAME_PROCESSORS.mdx * Update useSkiaFrameProcessor.ts * fix: Run destroy in interaction manager * fix: Safely remove surfaces * fix: Update RNWC to fix terminated threads using dead `thread_local` storage * fix: Install hierarchy fitter in `CameraView` * chore: Remove Skia example from example app * Update CameraPage.tsx * fix: Simplify default pixel format * fix: Rename `ReactLogger` to `VisionLogger` * chore: Format * fix: Fix ResolutionSelector `difference` abs (mrousavy#2746) * chore: release 4.0.0-beta.15 * Update CameraPage.tsx * chore: Fix `type` imports in TS * fix: Remove `enableGpuBuffers` * fix: Make dependencies (`ModuleProxy`) actually optional (mrousavy#2750) fix: Make dependencies actually optional * feat: Optionally compile out Location APIs (mrousavy#2751) * fix: Rename `disableFrameProcessors` to `enableFrameProcessors` * add podfile writer * Create vc_example_cocoapod_utils.rb * feat: Make Location compile optionally * fix: Only compile out location related permissio * Update Podfile * Use `$(inherited)` * fix: Fix `minFps` being larger in `Range` than `maxFps` (mrousavy#2755) * fix: Fix `minFps` being larger in `Range` than `maxFps` * import * fix: Also override `onProviderDisabled` to prevent abstract method crash * fix: Clean up Skia resources on Worklet context to avoid race condition (mrousavy#2754) * fix: Clean up Skia resources on Worklet context to avoid race condition * Update FrameProcessorPlugins.ts * Update useSkiaFrameProcessor.ts * docs: Update docs for V4 (mrousavy#2747) * fix: Update prism-react-renderer * fix: Fix docs * fix: Remove `enableGpuBuffers` * feat: Update Docusaurus to latest 3 * Update docusaurus.config.js * update languages * update languages * rename getting_started * Update some docs * Update CODE_SCANNING.mdx * update some docs * Update docusaurus.config.js * files * starting * fix: Properly display video * lots of updates * more docs * fix: Fix errors causing memory-leaks by not closing Frame * Add example * more * almost * fix broken links * update skia fp docs * Update FRAME_PROCESSORS_SKIA.mdx * Update FRAME_PROCESSORS_SKIA.mdx * fix broken link * chore: release 4.0.0-beta.16 * fix: Fix `Promise` resolve wrong name * fix: Fix build for Android because of Worklet usage * feat: Separate VisionCamera into two subspecs (mrousavy#2761) * feat: Separate VisionCamera into two subspecs * Swift * update podspec * fix: Refactor * some fixes * hmm * fix: Only export ObjC/Swift visible headers * fix: Also drop C++ sources * fix: Move to correct folder * fix: Fix any remaining C++ includes * chore: Format * fix: Fix missingg Swift include * fix: Add PluginRegistry Swift include * Update FrameProcessorPluginRegistry.h * fix: Disable location * fix: Use `public_header_files` instead of `preserve_paths` * Disable FP if needed * fix Podfile * fix: Fix `dispatch_queue_t` in Swift * fix: Disable FPs if not built * Revert "fix: Disable FPs if not built" This reverts commit 82cb560. * Remove Example Plugins from source code * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs (mrousavy#2764) * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs * Split utils * fix: Refactor Android codebase into `core`, `react` and `frameprocessors` (mrousavy#2765) * Add example pod helper again * feat: Also separate Android into `react`, `core` and `frameprocessor` * Log similar to iOS * Update build.gradle * fix: Fix weird refactor mistakes * fix: Refactor a bit * fix: `frameprocessor` -> `frameprocessors` * duplicate clang format with ln * chore: release 4.0.0-beta.17 * chore: Cleanup C++ * chore: Cleanup TS codebase and fix Skia Preview View style/layout issue (mrousavy#2767) * chore: Cleanup TS codebase * chore: More refactors * fix: Put Skia Canvas into CameraView * Update Camera.tsx * fix: Fix JNI types * chore: Update to RN Skia 1.2.3 * fix: Simplify `cpp/` codebase into `android`/`ios` folders (mrousavy#2766) * `Frame Processor` -> `FrameProcessors` * fix: Remove almost empty `cpp/` folder * fix: Add wrap ctor to `MutableRawBuffer` * fix: Add `.clang-format` file again * fix: Remove `cpp` folder references from everywhere * fix: Fix `already bound to a different lifecycle` error when flipping camera (mrousavy#2768) * fix: Fix `already bound to a different lifecycle` error when flipping camera * Update CameraSession.kt * Delete .clang-format * feat: Rebuild FPS Graph as JS component (mrousavy#2769) * feat: Rebuild FPS Graph as JS component * feat: Properly style graph * chore: Lint * fix: Fix FPS Graph not displaying on Android * chore: release 4.0.0-beta.18 * fix: Fix `ImageAnalysis is not supported when Extension is enabled` error (mrousavy#2770) * feat: Upgrade to CameraX alpha5 (mrousavy#2771) * fix: Separate `FpsSampleCollectorDelegate` * Downgrade to CameraX alpha04 because 05 fails to build * fix: Make `FpsSampleCollectorDelegate` weak * fix: Fix `delegate` guard * fix: Fix Android import path due to new subfolder * fix: Fix `Timer is already canceled` * chore: release 4.0.0-beta.19 --------- Co-authored-by: Kirill Zyusko <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Rodrigo Gomes <[email protected]> Co-authored-by: Versus <[email protected]>
fahadhaque007
pushed a commit
to equinoxventures/react-native-vision-science-camera
that referenced
this pull request
May 6, 2024
* feat: Give V4 🫴 * Update to compile SDK 34 * fix kotlin error * fix ui thread * Run all on Main Thread * Update lifecycle * Take Photo? * Type Camera Errors * Unbind before starting bind * Create Recording base? * Add `photoQualityBalance` to iOS * feat: Add `takeSnapshot()` to `PreviewView` * Add new API * Low Light Boost throttles * feat: Add CodeScanner pipeline (`ImageAnalysis`) * Format * Remove outputs * Update CameraSession.kt * Call back with Video result * Delete `RecordingSession` * Configure zoom and torch * Only update torch, zoom or exposure if needed * Create `takeSnapshot()` * Use `compressionQuality` * Format * Catch CameraErrors directly * Use new API for Snapshot * Throw `.fileError` * fix temp filename * fix: Implement `takeSnaphot` on iOS * Update Camera.tsx * Don't re-initialize outputs if only inputId changes * Update CameraSession.kt * Update CameraSession.kt * Use photo/video size selectors * fix: Use runOnUiThread only if needed * Code org * Update CameraSession.kt * feat: Use HDR or NIGHT extensions if available * fix: Check for ImageAnalysis support * feat: Shutter sound and stabilization * Update CameraSession.kt * restructure Threading a bit * Restructure Threading in CameraSession, configure is UI Thread * Update CameraSession.kt * Update CameraSession.kt * Better `mainExecutor` * Implement `focus()` * Use file type * Make `VideoPipeline` a CameraX use-case * Update CameraQueues.kt * Update VideoPipeline.kt * feat: Implement native HardwareBuffer rendering for `VideoPipeline` * Try use VideoPipeline * Try extending ImageAnalysis * Create `FrameProcessorEffect` as camera middleman * Use PREVIEW output * fix not running FP for PREVIEW * Update FrameProcessorEffect.kt * feat: Allow setting custom format in ImageWriter * Restructure code a bit * Make everything synchronized * Make `FrameProcessorEffect` closeable * chore: Clean up Codebase * Don't re-create Recorder if actively recording * docs: Update docs for `qualityBalance` * Update CameraProps.ts * Remove OpenGL VideoPipeline (now CameraX does that) * Properly disable CameraSession when `isActive=false` * Remove a few Errors * chore: release 4.0.0-beta.0 * chore: release 4.0.0-beta.1 * fix: Rename to `qualityBalance` * Update package.json * Small changes * Update CameraView.kt * Throw focus-canceled * Implement `onStarted` and `onStopped` for Android again * Always activate Camera lifecycle, ignore `isAttachedToWindow` * feat: Add `onShutter` event * chore: release 4.0.0-beta.2 * fix: Fix optional delegate * Update CameraError.kt * feat: Add `cancelRecording` * chore: release 4.0.0-beta.3 * Add `Preview` docs * feat: Add `androidPreviewViewType` prop * feat: Use CameraX for device details (mrousavy#2624) * feat: Use CameraX device details * Remove `maxZoom` from format * More * feat: Implement the rest (some Camera2 fallbacks) * Properly await ProcessCameraProvider and ExtensionsManager * Initialize earlier * Update CameraDeviceDetails.kt * feat: Properly get orientation and isMirrored in FP * fix: Properly close `ImageWriter` * fix: Fix transform matrix * fix: Use image transformation info from input surface * fix: Capture directly in lambda * fix: Fix focus not running on UI Thread * chore: release 4.0.0-beta.4 * fix: Fix calling `sendAvailableDevicesChangedEvent` before module is initialized * fix: Allow re-creating if session changed from outside * fix: Don't play shutter sound unless explicitly enabled * fix: Allow catching errors in `.await()` * feat: Remove `enableAutoStabilization` prop as this was deprecated in iOS 13 * fix: Export `Snapshot.ts` * feat: Add `type` to `onShutter` (`photo` or `snapshot`) * fix: Only unbind current outputs, not all * chore: Lint/Fix * chore: release 4.0.0-beta.5 * fix: Use dummy metering factory instead of display-oriented * fix: Remove unneeded context * fix: Fix rotation getting * Update Podfile.lock * fix: Check format requirements in outputs * fix: Remove `not-compatible-with-outputs` error on Android since CameraX supports StreamSharing * fix: Remove `didSessionChangeFromOutside` * fix: Catch Objective-C errors in iOS Frame Processor Plugins * feat: Catch Objective-C init errors on plugin init * fix: Fix `TimeoutException` by using `STARTED` lifecycle * chore: release 4.0.0-beta.6 * add todo about hdr extension bug * fix: Remove focus listeners after resetting * fix: Fix capture session configuration error by manually layouting PreviewView * feat: Write proper photo metadata (orientation & isMirrored) (mrousavy#2660) * feat: Write proper photo metadata (orientation & isMirrored) * Format * fix: Fix Camera not starting because PreviewView isn't laid out * fix: Use `CREATED` as default lifecycle state * fix: Fix 10-bit HDR in combination with HDR Extension * Update CameraSession.kt * fix: Throw correct errors * fix: Properly check SDR * fix: Update outputs if video stabilization or low-light-boost changes * fix: Also dont enable if other extension is enabled * chore: release 4.0.0-beta.7 * Update package.json * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) (mrousavy#2664) * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) * feat: Set target resolution * Update CameraSession.kt * Refactor * fix: Fix Frame Processor not being unbound * fix: Properly set `maxPhotoDimensions` on PhotoOutput (mrousavy#2659) * fix: Properly set `maxPhotoDimensions` on PhotoOutput * fix: Remove `maxPhotoDimensions` since it is the highest value by default anyways I think * fix: Remove `enableHighQualityPhotos` entirely * fix: Update RN Worklets * Revert "fix: Remove `enableHighQualityPhotos` entirely" This reverts commit 7110b76. * fix: Use high res capture / maxPhotoDimensions always. * Update CameraSession+Configuration.swift * chore: release 4.0.0-beta.8 * Update CameraSession+Photo.swift * fix: Fix focus on Android by using PX instead of DP (mrousavy#2666) * fix: focus on Android (v4) * format --------- Co-authored-by: Marc Rousavy <[email protected]> * Optimised images with calibre/image-actions * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) (mrousavy#2665) * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) * feat: Also add location tag to video * fix: Fix location tagging for videos * fix: Write proper location to video * Create VisionCamera branding * fix: Write branding * Use quicktime location instead of common location * Finish metadata track * Update Podfile.lock * Try writing metadata * Write video metadata * Throw error with cause * fix: Properly write metadata at start * Properly convert to ISO 6709 String * Use quicktime metadata location again * fix: Put branding into QuickTime information field * Format * Update MetadataProvider.swift * Only stream location if camera is active * fix: Set location * Format * fix: Fix changing object reference in `usePermission` * fix: Use proper location * Update MetadataProvider.kt * fix: Use last location as default * Add some logs * Only update location once every 5s * fix: Also allow coarse location * feat: Add location docs and `enableLocation` to expo config plugin * Optimised images with calibre/image-actions --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Optimised images with calibre/image-actions * fix: Remove HardwareBufferUtils * fix: Fix CodeScanner not working * chore: release 4.0.0-beta.9 * fix: Fix location permission not resolving on iOS (mrousavy#2672) * fix: Fix location permission not resolving on iOS * fix: Remove unused code * fix: Ignore first call * Use `GlobalReferenceHolder` * chore: Remove RN Fast Image (we dont have web images anyways) * Update package.json * chore: release 4.0.0-beta.10 * fix: Fix wrong orientation enum being send to ImageCapture (mrousavy#2683) * perf: Add `@FastNative` flags to critical Frame Processor methods * fix: Fix `RCTDeviceEventEmitter` import for RN 0.71.X (mrousavy#2692) fix: build for rn 0.71.X * chore: release 4.0.0-beta.11 * feat: Disable shutter sound if in silent * fix: Remove `enablePrecapture` * chore: release 4.0.0-beta.12 * fix: Remove `mustPlayShutterSound()` check * Update Podfile.lock * chore: release 4.0.0-beta.13 * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * fix: Use `ResolutionSelector`s with fallback values * fix: Fall-back to H.264 if custom codec is not supported * Update Camera.tsx * feat: Add `hardware-cost-too-high` error for iOS 16 * chore: release 4.0.0-beta.14 * fix: Use 1280x720 for QR scanner * feat: Skia for V4! 🥳 (mrousavy#2727) * feat: Add Skia support (`toSkImage()`) * feat: Add `frame.getPlaformBuffer()` * Try use getPlatformBuffer * Update Frame.ts * fix: Fix `getPlatformBuffer()` * fix: Fix buffer access * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * feat: Build Skia example renderer * fix: Render a bit better * fix: Use atomic updates now * fix: Remove SK flags * fix: Clear and copy Frame * fix: Use bigint * fix: Use `ImageProxy` and close it to avoid stalling * Only do copy on iOS * fix: Add throw method signature * feat: Box into `DrawableFrame` * feat: Create `SkiaCameraCanvas` * feat: Allow disabling `preview` use-case * feat: Catch some errors * Update FrameHostObject.cpp * fix: Use proper layouts * Update useSkiaFrameProcessor.ts * fix: Use main Coroutine Scope * fix: Fix wrong changes because we now properly abort changes * chore: Format * fix: Upgrade react-native-worklets-core Now includes shift() * fix: Add `focusRequiresPreview` error to iOS * chore: Use latest react-native-worklets-core * fix: Fix buffer orientation on iOS (it's currently always `.up` * fix: Only pass function into native, no object * fix: Format code * fix: Fix styling * fix: Fix pointer-events * Update Camera.tsx * Update useSkiaFrameProcessor.ts * Update Worklets * Draw radial face blur * circle that spins * fix: Proxify * fix: Also use worklets proxy lazily * fix: Try multiple error handlers * fix: Also use Skia Proxy * Add docs to the objects * fix: Use `createModuleProxy` * fix: Also re-create surface if height is different * perf: Remove unneeded mutex locks as Frame only locks refCount * Update FRAME_PROCESSORS_CREATE_OVERVIEW.mdx * fix: Remove unneeded CPU copy * fix: Catch PixelFormat error and fallback to default format * fix: Throw proper PixelFormat error with more info * Update CameraError.swift * Update Frame.java * fix: Use `jsi::Scope` In Hermes, this will probably not do anything. * chore: Update react-native-worklets-core * fix: Dispose previous Surface * Update useSkiaFrameProcessor.ts * fix: Fix surface re-creating each time * chore: Bump to latest RN Skia with the features we need * feat: Clean everything on unmount * Add CPU copy again. On iOS, we need it. * fix: Fix Android build * fix: Fix Android build again * Update ModuleProxy.ts * fix: Use type imports if possible * fix: Add RN Skia patch * Add react-native-fast-tflite * fix: Configure for tflite files as well * Revert "fix: Configure for tflite files as well" This reverts commit 37db0f5. * Revert "Add react-native-fast-tflite" This reverts commit 264dd4b. * Update @Shopify+react-native-skia+1.2.0.patch * Add `useSkiaFrameProcessor` to ESLint * Update yarn.lock * fix: Upgrade to latest Skia to use NativeBuffer API now * fix: Use `CVPixelBuffer` instead of `CMSampleBuffer` * fix: Fix package.json * chore: Update to latest react-native-worklets-core * fix: Use thread-local storage for SkSurface * Update Podfile.lock * fix: Migrate over to new Worklets API * Update FRAME_PROCESSORS.mdx * Update useSkiaFrameProcessor.ts * fix: Run destroy in interaction manager * fix: Safely remove surfaces * fix: Update RNWC to fix terminated threads using dead `thread_local` storage * fix: Install hierarchy fitter in `CameraView` * chore: Remove Skia example from example app * Update CameraPage.tsx * fix: Simplify default pixel format * fix: Rename `ReactLogger` to `VisionLogger` * chore: Format * fix: Fix ResolutionSelector `difference` abs (mrousavy#2746) * chore: release 4.0.0-beta.15 * Update CameraPage.tsx * chore: Fix `type` imports in TS * fix: Remove `enableGpuBuffers` * fix: Make dependencies (`ModuleProxy`) actually optional (mrousavy#2750) fix: Make dependencies actually optional * feat: Optionally compile out Location APIs (mrousavy#2751) * fix: Rename `disableFrameProcessors` to `enableFrameProcessors` * add podfile writer * Create vc_example_cocoapod_utils.rb * feat: Make Location compile optionally * fix: Only compile out location related permissio * Update Podfile * Use `$(inherited)` * fix: Fix `minFps` being larger in `Range` than `maxFps` (mrousavy#2755) * fix: Fix `minFps` being larger in `Range` than `maxFps` * import * fix: Also override `onProviderDisabled` to prevent abstract method crash * fix: Clean up Skia resources on Worklet context to avoid race condition (mrousavy#2754) * fix: Clean up Skia resources on Worklet context to avoid race condition * Update FrameProcessorPlugins.ts * Update useSkiaFrameProcessor.ts * docs: Update docs for V4 (mrousavy#2747) * fix: Update prism-react-renderer * fix: Fix docs * fix: Remove `enableGpuBuffers` * feat: Update Docusaurus to latest 3 * Update docusaurus.config.js * update languages * update languages * rename getting_started * Update some docs * Update CODE_SCANNING.mdx * update some docs * Update docusaurus.config.js * files * starting * fix: Properly display video * lots of updates * more docs * fix: Fix errors causing memory-leaks by not closing Frame * Add example * more * almost * fix broken links * update skia fp docs * Update FRAME_PROCESSORS_SKIA.mdx * Update FRAME_PROCESSORS_SKIA.mdx * fix broken link * chore: release 4.0.0-beta.16 * fix: Fix `Promise` resolve wrong name * fix: Fix build for Android because of Worklet usage * feat: Separate VisionCamera into two subspecs (mrousavy#2761) * feat: Separate VisionCamera into two subspecs * Swift * update podspec * fix: Refactor * some fixes * hmm * fix: Only export ObjC/Swift visible headers * fix: Also drop C++ sources * fix: Move to correct folder * fix: Fix any remaining C++ includes * chore: Format * fix: Fix missingg Swift include * fix: Add PluginRegistry Swift include * Update FrameProcessorPluginRegistry.h * fix: Disable location * fix: Use `public_header_files` instead of `preserve_paths` * Disable FP if needed * fix Podfile * fix: Fix `dispatch_queue_t` in Swift * fix: Disable FPs if not built * Revert "fix: Disable FPs if not built" This reverts commit 82cb560. * Remove Example Plugins from source code * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs (mrousavy#2764) * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs * Split utils * fix: Refactor Android codebase into `core`, `react` and `frameprocessors` (mrousavy#2765) * Add example pod helper again * feat: Also separate Android into `react`, `core` and `frameprocessor` * Log similar to iOS * Update build.gradle * fix: Fix weird refactor mistakes * fix: Refactor a bit * fix: `frameprocessor` -> `frameprocessors` * duplicate clang format with ln * chore: release 4.0.0-beta.17 * chore: Cleanup C++ * chore: Cleanup TS codebase and fix Skia Preview View style/layout issue (mrousavy#2767) * chore: Cleanup TS codebase * chore: More refactors * fix: Put Skia Canvas into CameraView * Update Camera.tsx * fix: Fix JNI types * chore: Update to RN Skia 1.2.3 * fix: Simplify `cpp/` codebase into `android`/`ios` folders (mrousavy#2766) * `Frame Processor` -> `FrameProcessors` * fix: Remove almost empty `cpp/` folder * fix: Add wrap ctor to `MutableRawBuffer` * fix: Add `.clang-format` file again * fix: Remove `cpp` folder references from everywhere * fix: Fix `already bound to a different lifecycle` error when flipping camera (mrousavy#2768) * fix: Fix `already bound to a different lifecycle` error when flipping camera * Update CameraSession.kt * Delete .clang-format * feat: Rebuild FPS Graph as JS component (mrousavy#2769) * feat: Rebuild FPS Graph as JS component * feat: Properly style graph * chore: Lint * fix: Fix FPS Graph not displaying on Android * chore: release 4.0.0-beta.18 * fix: Fix `ImageAnalysis is not supported when Extension is enabled` error (mrousavy#2770) * feat: Upgrade to CameraX alpha5 (mrousavy#2771) * fix: Separate `FpsSampleCollectorDelegate` * Downgrade to CameraX alpha04 because 05 fails to build * fix: Make `FpsSampleCollectorDelegate` weak * fix: Fix `delegate` guard * fix: Fix Android import path due to new subfolder * fix: Fix `Timer is already canceled` * chore: release 4.0.0-beta.19 --------- Co-authored-by: Kirill Zyusko <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Rodrigo Gomes <[email protected]> Co-authored-by: Versus <[email protected]>
isaaccolson
pushed a commit
to isaaccolson/deliveries-mobile
that referenced
this pull request
Oct 30, 2024
* feat: Give V4 🫴 * Update to compile SDK 34 * fix kotlin error * fix ui thread * Run all on Main Thread * Update lifecycle * Take Photo? * Type Camera Errors * Unbind before starting bind * Create Recording base? * Add `photoQualityBalance` to iOS * feat: Add `takeSnapshot()` to `PreviewView` * Add new API * Low Light Boost throttles * feat: Add CodeScanner pipeline (`ImageAnalysis`) * Format * Remove outputs * Update CameraSession.kt * Call back with Video result * Delete `RecordingSession` * Configure zoom and torch * Only update torch, zoom or exposure if needed * Create `takeSnapshot()` * Use `compressionQuality` * Format * Catch CameraErrors directly * Use new API for Snapshot * Throw `.fileError` * fix temp filename * fix: Implement `takeSnaphot` on iOS * Update Camera.tsx * Don't re-initialize outputs if only inputId changes * Update CameraSession.kt * Update CameraSession.kt * Use photo/video size selectors * fix: Use runOnUiThread only if needed * Code org * Update CameraSession.kt * feat: Use HDR or NIGHT extensions if available * fix: Check for ImageAnalysis support * feat: Shutter sound and stabilization * Update CameraSession.kt * restructure Threading a bit * Restructure Threading in CameraSession, configure is UI Thread * Update CameraSession.kt * Update CameraSession.kt * Better `mainExecutor` * Implement `focus()` * Use file type * Make `VideoPipeline` a CameraX use-case * Update CameraQueues.kt * Update VideoPipeline.kt * feat: Implement native HardwareBuffer rendering for `VideoPipeline` * Try use VideoPipeline * Try extending ImageAnalysis * Create `FrameProcessorEffect` as camera middleman * Use PREVIEW output * fix not running FP for PREVIEW * Update FrameProcessorEffect.kt * feat: Allow setting custom format in ImageWriter * Restructure code a bit * Make everything synchronized * Make `FrameProcessorEffect` closeable * chore: Clean up Codebase * Don't re-create Recorder if actively recording * docs: Update docs for `qualityBalance` * Update CameraProps.ts * Remove OpenGL VideoPipeline (now CameraX does that) * Properly disable CameraSession when `isActive=false` * Remove a few Errors * chore: release 4.0.0-beta.0 * chore: release 4.0.0-beta.1 * fix: Rename to `qualityBalance` * Update package.json * Small changes * Update CameraView.kt * Throw focus-canceled * Implement `onStarted` and `onStopped` for Android again * Always activate Camera lifecycle, ignore `isAttachedToWindow` * feat: Add `onShutter` event * chore: release 4.0.0-beta.2 * fix: Fix optional delegate * Update CameraError.kt * feat: Add `cancelRecording` * chore: release 4.0.0-beta.3 * Add `Preview` docs * feat: Add `androidPreviewViewType` prop * feat: Use CameraX for device details (mrousavy#2624) * feat: Use CameraX device details * Remove `maxZoom` from format * More * feat: Implement the rest (some Camera2 fallbacks) * Properly await ProcessCameraProvider and ExtensionsManager * Initialize earlier * Update CameraDeviceDetails.kt * feat: Properly get orientation and isMirrored in FP * fix: Properly close `ImageWriter` * fix: Fix transform matrix * fix: Use image transformation info from input surface * fix: Capture directly in lambda * fix: Fix focus not running on UI Thread * chore: release 4.0.0-beta.4 * fix: Fix calling `sendAvailableDevicesChangedEvent` before module is initialized * fix: Allow re-creating if session changed from outside * fix: Don't play shutter sound unless explicitly enabled * fix: Allow catching errors in `.await()` * feat: Remove `enableAutoStabilization` prop as this was deprecated in iOS 13 * fix: Export `Snapshot.ts` * feat: Add `type` to `onShutter` (`photo` or `snapshot`) * fix: Only unbind current outputs, not all * chore: Lint/Fix * chore: release 4.0.0-beta.5 * fix: Use dummy metering factory instead of display-oriented * fix: Remove unneeded context * fix: Fix rotation getting * Update Podfile.lock * fix: Check format requirements in outputs * fix: Remove `not-compatible-with-outputs` error on Android since CameraX supports StreamSharing * fix: Remove `didSessionChangeFromOutside` * fix: Catch Objective-C errors in iOS Frame Processor Plugins * feat: Catch Objective-C init errors on plugin init * fix: Fix `TimeoutException` by using `STARTED` lifecycle * chore: release 4.0.0-beta.6 * add todo about hdr extension bug * fix: Remove focus listeners after resetting * fix: Fix capture session configuration error by manually layouting PreviewView * feat: Write proper photo metadata (orientation & isMirrored) (mrousavy#2660) * feat: Write proper photo metadata (orientation & isMirrored) * Format * fix: Fix Camera not starting because PreviewView isn't laid out * fix: Use `CREATED` as default lifecycle state * fix: Fix 10-bit HDR in combination with HDR Extension * Update CameraSession.kt * fix: Throw correct errors * fix: Properly check SDR * fix: Update outputs if video stabilization or low-light-boost changes * fix: Also dont enable if other extension is enabled * chore: release 4.0.0-beta.7 * Update package.json * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) (mrousavy#2664) * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) * feat: Set target resolution * Update CameraSession.kt * Refactor * fix: Fix Frame Processor not being unbound * fix: Properly set `maxPhotoDimensions` on PhotoOutput (mrousavy#2659) * fix: Properly set `maxPhotoDimensions` on PhotoOutput * fix: Remove `maxPhotoDimensions` since it is the highest value by default anyways I think * fix: Remove `enableHighQualityPhotos` entirely * fix: Update RN Worklets * Revert "fix: Remove `enableHighQualityPhotos` entirely" This reverts commit 7110b76. * fix: Use high res capture / maxPhotoDimensions always. * Update CameraSession+Configuration.swift * chore: release 4.0.0-beta.8 * Update CameraSession+Photo.swift * fix: Fix focus on Android by using PX instead of DP (mrousavy#2666) * fix: focus on Android (v4) * format --------- Co-authored-by: Marc Rousavy <[email protected]> * Optimised images with calibre/image-actions * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) (mrousavy#2665) * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) * feat: Also add location tag to video * fix: Fix location tagging for videos * fix: Write proper location to video * Create VisionCamera branding * fix: Write branding * Use quicktime location instead of common location * Finish metadata track * Update Podfile.lock * Try writing metadata * Write video metadata * Throw error with cause * fix: Properly write metadata at start * Properly convert to ISO 6709 String * Use quicktime metadata location again * fix: Put branding into QuickTime information field * Format * Update MetadataProvider.swift * Only stream location if camera is active * fix: Set location * Format * fix: Fix changing object reference in `usePermission` * fix: Use proper location * Update MetadataProvider.kt * fix: Use last location as default * Add some logs * Only update location once every 5s * fix: Also allow coarse location * feat: Add location docs and `enableLocation` to expo config plugin * Optimised images with calibre/image-actions --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Optimised images with calibre/image-actions * fix: Remove HardwareBufferUtils * fix: Fix CodeScanner not working * chore: release 4.0.0-beta.9 * fix: Fix location permission not resolving on iOS (mrousavy#2672) * fix: Fix location permission not resolving on iOS * fix: Remove unused code * fix: Ignore first call * Use `GlobalReferenceHolder` * chore: Remove RN Fast Image (we dont have web images anyways) * Update package.json * chore: release 4.0.0-beta.10 * fix: Fix wrong orientation enum being send to ImageCapture (mrousavy#2683) * perf: Add `@FastNative` flags to critical Frame Processor methods * fix: Fix `RCTDeviceEventEmitter` import for RN 0.71.X (mrousavy#2692) fix: build for rn 0.71.X * chore: release 4.0.0-beta.11 * feat: Disable shutter sound if in silent * fix: Remove `enablePrecapture` * chore: release 4.0.0-beta.12 * fix: Remove `mustPlayShutterSound()` check * Update Podfile.lock * chore: release 4.0.0-beta.13 * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * fix: Use `ResolutionSelector`s with fallback values * fix: Fall-back to H.264 if custom codec is not supported * Update Camera.tsx * feat: Add `hardware-cost-too-high` error for iOS 16 * chore: release 4.0.0-beta.14 * fix: Use 1280x720 for QR scanner * feat: Skia for V4! 🥳 (mrousavy#2727) * feat: Add Skia support (`toSkImage()`) * feat: Add `frame.getPlaformBuffer()` * Try use getPlatformBuffer * Update Frame.ts * fix: Fix `getPlatformBuffer()` * fix: Fix buffer access * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * feat: Build Skia example renderer * fix: Render a bit better * fix: Use atomic updates now * fix: Remove SK flags * fix: Clear and copy Frame * fix: Use bigint * fix: Use `ImageProxy` and close it to avoid stalling * Only do copy on iOS * fix: Add throw method signature * feat: Box into `DrawableFrame` * feat: Create `SkiaCameraCanvas` * feat: Allow disabling `preview` use-case * feat: Catch some errors * Update FrameHostObject.cpp * fix: Use proper layouts * Update useSkiaFrameProcessor.ts * fix: Use main Coroutine Scope * fix: Fix wrong changes because we now properly abort changes * chore: Format * fix: Upgrade react-native-worklets-core Now includes shift() * fix: Add `focusRequiresPreview` error to iOS * chore: Use latest react-native-worklets-core * fix: Fix buffer orientation on iOS (it's currently always `.up` * fix: Only pass function into native, no object * fix: Format code * fix: Fix styling * fix: Fix pointer-events * Update Camera.tsx * Update useSkiaFrameProcessor.ts * Update Worklets * Draw radial face blur * circle that spins * fix: Proxify * fix: Also use worklets proxy lazily * fix: Try multiple error handlers * fix: Also use Skia Proxy * Add docs to the objects * fix: Use `createModuleProxy` * fix: Also re-create surface if height is different * perf: Remove unneeded mutex locks as Frame only locks refCount * Update FRAME_PROCESSORS_CREATE_OVERVIEW.mdx * fix: Remove unneeded CPU copy * fix: Catch PixelFormat error and fallback to default format * fix: Throw proper PixelFormat error with more info * Update CameraError.swift * Update Frame.java * fix: Use `jsi::Scope` In Hermes, this will probably not do anything. * chore: Update react-native-worklets-core * fix: Dispose previous Surface * Update useSkiaFrameProcessor.ts * fix: Fix surface re-creating each time * chore: Bump to latest RN Skia with the features we need * feat: Clean everything on unmount * Add CPU copy again. On iOS, we need it. * fix: Fix Android build * fix: Fix Android build again * Update ModuleProxy.ts * fix: Use type imports if possible * fix: Add RN Skia patch * Add react-native-fast-tflite * fix: Configure for tflite files as well * Revert "fix: Configure for tflite files as well" This reverts commit 37db0f5. * Revert "Add react-native-fast-tflite" This reverts commit 264dd4b. * Update @Shopify+react-native-skia+1.2.0.patch * Add `useSkiaFrameProcessor` to ESLint * Update yarn.lock * fix: Upgrade to latest Skia to use NativeBuffer API now * fix: Use `CVPixelBuffer` instead of `CMSampleBuffer` * fix: Fix package.json * chore: Update to latest react-native-worklets-core * fix: Use thread-local storage for SkSurface * Update Podfile.lock * fix: Migrate over to new Worklets API * Update FRAME_PROCESSORS.mdx * Update useSkiaFrameProcessor.ts * fix: Run destroy in interaction manager * fix: Safely remove surfaces * fix: Update RNWC to fix terminated threads using dead `thread_local` storage * fix: Install hierarchy fitter in `CameraView` * chore: Remove Skia example from example app * Update CameraPage.tsx * fix: Simplify default pixel format * fix: Rename `ReactLogger` to `VisionLogger` * chore: Format * fix: Fix ResolutionSelector `difference` abs (mrousavy#2746) * chore: release 4.0.0-beta.15 * Update CameraPage.tsx * chore: Fix `type` imports in TS * fix: Remove `enableGpuBuffers` * fix: Make dependencies (`ModuleProxy`) actually optional (mrousavy#2750) fix: Make dependencies actually optional * feat: Optionally compile out Location APIs (mrousavy#2751) * fix: Rename `disableFrameProcessors` to `enableFrameProcessors` * add podfile writer * Create vc_example_cocoapod_utils.rb * feat: Make Location compile optionally * fix: Only compile out location related permissio * Update Podfile * Use `$(inherited)` * fix: Fix `minFps` being larger in `Range` than `maxFps` (mrousavy#2755) * fix: Fix `minFps` being larger in `Range` than `maxFps` * import * fix: Also override `onProviderDisabled` to prevent abstract method crash * fix: Clean up Skia resources on Worklet context to avoid race condition (mrousavy#2754) * fix: Clean up Skia resources on Worklet context to avoid race condition * Update FrameProcessorPlugins.ts * Update useSkiaFrameProcessor.ts * docs: Update docs for V4 (mrousavy#2747) * fix: Update prism-react-renderer * fix: Fix docs * fix: Remove `enableGpuBuffers` * feat: Update Docusaurus to latest 3 * Update docusaurus.config.js * update languages * update languages * rename getting_started * Update some docs * Update CODE_SCANNING.mdx * update some docs * Update docusaurus.config.js * files * starting * fix: Properly display video * lots of updates * more docs * fix: Fix errors causing memory-leaks by not closing Frame * Add example * more * almost * fix broken links * update skia fp docs * Update FRAME_PROCESSORS_SKIA.mdx * Update FRAME_PROCESSORS_SKIA.mdx * fix broken link * chore: release 4.0.0-beta.16 * fix: Fix `Promise` resolve wrong name * fix: Fix build for Android because of Worklet usage * feat: Separate VisionCamera into two subspecs (mrousavy#2761) * feat: Separate VisionCamera into two subspecs * Swift * update podspec * fix: Refactor * some fixes * hmm * fix: Only export ObjC/Swift visible headers * fix: Also drop C++ sources * fix: Move to correct folder * fix: Fix any remaining C++ includes * chore: Format * fix: Fix missingg Swift include * fix: Add PluginRegistry Swift include * Update FrameProcessorPluginRegistry.h * fix: Disable location * fix: Use `public_header_files` instead of `preserve_paths` * Disable FP if needed * fix Podfile * fix: Fix `dispatch_queue_t` in Swift * fix: Disable FPs if not built * Revert "fix: Disable FPs if not built" This reverts commit 82cb560. * Remove Example Plugins from source code * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs (mrousavy#2764) * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs * Split utils * fix: Refactor Android codebase into `core`, `react` and `frameprocessors` (mrousavy#2765) * Add example pod helper again * feat: Also separate Android into `react`, `core` and `frameprocessor` * Log similar to iOS * Update build.gradle * fix: Fix weird refactor mistakes * fix: Refactor a bit * fix: `frameprocessor` -> `frameprocessors` * duplicate clang format with ln * chore: release 4.0.0-beta.17 * chore: Cleanup C++ * chore: Cleanup TS codebase and fix Skia Preview View style/layout issue (mrousavy#2767) * chore: Cleanup TS codebase * chore: More refactors * fix: Put Skia Canvas into CameraView * Update Camera.tsx * fix: Fix JNI types * chore: Update to RN Skia 1.2.3 * fix: Simplify `cpp/` codebase into `android`/`ios` folders (mrousavy#2766) * `Frame Processor` -> `FrameProcessors` * fix: Remove almost empty `cpp/` folder * fix: Add wrap ctor to `MutableRawBuffer` * fix: Add `.clang-format` file again * fix: Remove `cpp` folder references from everywhere * fix: Fix `already bound to a different lifecycle` error when flipping camera (mrousavy#2768) * fix: Fix `already bound to a different lifecycle` error when flipping camera * Update CameraSession.kt * Delete .clang-format * feat: Rebuild FPS Graph as JS component (mrousavy#2769) * feat: Rebuild FPS Graph as JS component * feat: Properly style graph * chore: Lint * fix: Fix FPS Graph not displaying on Android * chore: release 4.0.0-beta.18 * fix: Fix `ImageAnalysis is not supported when Extension is enabled` error (mrousavy#2770) * feat: Upgrade to CameraX alpha5 (mrousavy#2771) * fix: Separate `FpsSampleCollectorDelegate` * Downgrade to CameraX alpha04 because 05 fails to build * fix: Make `FpsSampleCollectorDelegate` weak * fix: Fix `delegate` guard * fix: Fix Android import path due to new subfolder * fix: Fix `Timer is already canceled` * chore: release 4.0.0-beta.19 --------- Co-authored-by: Kirill Zyusko <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Rodrigo Gomes <[email protected]> Co-authored-by: Versus <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Adds support for basic read-only Frame Processors using the
ImageAnalysisuse-case.By design, those Frame Processors are async, and Skia couldn't really be implemented ontop of this. But we'll figure it out!
Changes
Tested on
Related issues