Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: ✨ VisionCamera V4 ✨ (mrousavy#2623)
* feat: Give V4 🫴 * Update to compile SDK 34 * fix kotlin error * fix ui thread * Run all on Main Thread * Update lifecycle * Take Photo? * Type Camera Errors * Unbind before starting bind * Create Recording base? * Add `photoQualityBalance` to iOS * feat: Add `takeSnapshot()` to `PreviewView` * Add new API * Low Light Boost throttles * feat: Add CodeScanner pipeline (`ImageAnalysis`) * Format * Remove outputs * Update CameraSession.kt * Call back with Video result * Delete `RecordingSession` * Configure zoom and torch * Only update torch, zoom or exposure if needed * Create `takeSnapshot()` * Use `compressionQuality` * Format * Catch CameraErrors directly * Use new API for Snapshot * Throw `.fileError` * fix temp filename * fix: Implement `takeSnaphot` on iOS * Update Camera.tsx * Don't re-initialize outputs if only inputId changes * Update CameraSession.kt * Update CameraSession.kt * Use photo/video size selectors * fix: Use runOnUiThread only if needed * Code org * Update CameraSession.kt * feat: Use HDR or NIGHT extensions if available * fix: Check for ImageAnalysis support * feat: Shutter sound and stabilization * Update CameraSession.kt * restructure Threading a bit * Restructure Threading in CameraSession, configure is UI Thread * Update CameraSession.kt * Update CameraSession.kt * Better `mainExecutor` * Implement `focus()` * Use file type * Make `VideoPipeline` a CameraX use-case * Update CameraQueues.kt * Update VideoPipeline.kt * feat: Implement native HardwareBuffer rendering for `VideoPipeline` * Try use VideoPipeline * Try extending ImageAnalysis * Create `FrameProcessorEffect` as camera middleman * Use PREVIEW output * fix not running FP for PREVIEW * Update FrameProcessorEffect.kt * feat: Allow setting custom format in ImageWriter * Restructure code a bit * Make everything synchronized * Make `FrameProcessorEffect` closeable * chore: Clean up Codebase * Don't re-create Recorder if actively recording * docs: Update docs for `qualityBalance` * Update CameraProps.ts * Remove OpenGL VideoPipeline (now CameraX does that) * Properly disable CameraSession when `isActive=false` * Remove a few Errors * chore: release 4.0.0-beta.0 * chore: release 4.0.0-beta.1 * fix: Rename to `qualityBalance` * Update package.json * Small changes * Update CameraView.kt * Throw focus-canceled * Implement `onStarted` and `onStopped` for Android again * Always activate Camera lifecycle, ignore `isAttachedToWindow` * feat: Add `onShutter` event * chore: release 4.0.0-beta.2 * fix: Fix optional delegate * Update CameraError.kt * feat: Add `cancelRecording` * chore: release 4.0.0-beta.3 * Add `Preview` docs * feat: Add `androidPreviewViewType` prop * feat: Use CameraX for device details (mrousavy#2624) * feat: Use CameraX device details * Remove `maxZoom` from format * More * feat: Implement the rest (some Camera2 fallbacks) * Properly await ProcessCameraProvider and ExtensionsManager * Initialize earlier * Update CameraDeviceDetails.kt * feat: Properly get orientation and isMirrored in FP * fix: Properly close `ImageWriter` * fix: Fix transform matrix * fix: Use image transformation info from input surface * fix: Capture directly in lambda * fix: Fix focus not running on UI Thread * chore: release 4.0.0-beta.4 * fix: Fix calling `sendAvailableDevicesChangedEvent` before module is initialized * fix: Allow re-creating if session changed from outside * fix: Don't play shutter sound unless explicitly enabled * fix: Allow catching errors in `.await()` * feat: Remove `enableAutoStabilization` prop as this was deprecated in iOS 13 * fix: Export `Snapshot.ts` * feat: Add `type` to `onShutter` (`photo` or `snapshot`) * fix: Only unbind current outputs, not all * chore: Lint/Fix * chore: release 4.0.0-beta.5 * fix: Use dummy metering factory instead of display-oriented * fix: Remove unneeded context * fix: Fix rotation getting * Update Podfile.lock * fix: Check format requirements in outputs * fix: Remove `not-compatible-with-outputs` error on Android since CameraX supports StreamSharing * fix: Remove `didSessionChangeFromOutside` * fix: Catch Objective-C errors in iOS Frame Processor Plugins * feat: Catch Objective-C init errors on plugin init * fix: Fix `TimeoutException` by using `STARTED` lifecycle * chore: release 4.0.0-beta.6 * add todo about hdr extension bug * fix: Remove focus listeners after resetting * fix: Fix capture session configuration error by manually layouting PreviewView * feat: Write proper photo metadata (orientation & isMirrored) (mrousavy#2660) * feat: Write proper photo metadata (orientation & isMirrored) * Format * fix: Fix Camera not starting because PreviewView isn't laid out * fix: Use `CREATED` as default lifecycle state * fix: Fix 10-bit HDR in combination with HDR Extension * Update CameraSession.kt * fix: Throw correct errors * fix: Properly check SDR * fix: Update outputs if video stabilization or low-light-boost changes * fix: Also dont enable if other extension is enabled * chore: release 4.0.0-beta.7 * Update package.json * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) (mrousavy#2664) * feat: Basic read-only Frame Processors (`ImageAnalysis` Use-Case) * feat: Set target resolution * Update CameraSession.kt * Refactor * fix: Fix Frame Processor not being unbound * fix: Properly set `maxPhotoDimensions` on PhotoOutput (mrousavy#2659) * fix: Properly set `maxPhotoDimensions` on PhotoOutput * fix: Remove `maxPhotoDimensions` since it is the highest value by default anyways I think * fix: Remove `enableHighQualityPhotos` entirely * fix: Update RN Worklets * Revert "fix: Remove `enableHighQualityPhotos` entirely" This reverts commit 7110b76. * fix: Use high res capture / maxPhotoDimensions always. * Update CameraSession+Configuration.swift * chore: release 4.0.0-beta.8 * Update CameraSession+Photo.swift * fix: Fix focus on Android by using PX instead of DP (mrousavy#2666) * fix: focus on Android (v4) * format --------- Co-authored-by: Marc Rousavy <[email protected]> * Optimised images with calibre/image-actions * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) (mrousavy#2665) * feat: Add GPS Location EXIF tagging for photos and videos (`enableLocation={true}`) * feat: Also add location tag to video * fix: Fix location tagging for videos * fix: Write proper location to video * Create VisionCamera branding * fix: Write branding * Use quicktime location instead of common location * Finish metadata track * Update Podfile.lock * Try writing metadata * Write video metadata * Throw error with cause * fix: Properly write metadata at start * Properly convert to ISO 6709 String * Use quicktime metadata location again * fix: Put branding into QuickTime information field * Format * Update MetadataProvider.swift * Only stream location if camera is active * fix: Set location * Format * fix: Fix changing object reference in `usePermission` * fix: Use proper location * Update MetadataProvider.kt * fix: Use last location as default * Add some logs * Only update location once every 5s * fix: Also allow coarse location * feat: Add location docs and `enableLocation` to expo config plugin * Optimised images with calibre/image-actions --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Optimised images with calibre/image-actions * fix: Remove HardwareBufferUtils * fix: Fix CodeScanner not working * chore: release 4.0.0-beta.9 * fix: Fix location permission not resolving on iOS (mrousavy#2672) * fix: Fix location permission not resolving on iOS * fix: Remove unused code * fix: Ignore first call * Use `GlobalReferenceHolder` * chore: Remove RN Fast Image (we dont have web images anyways) * Update package.json * chore: release 4.0.0-beta.10 * fix: Fix wrong orientation enum being send to ImageCapture (mrousavy#2683) * perf: Add `@FastNative` flags to critical Frame Processor methods * fix: Fix `RCTDeviceEventEmitter` import for RN 0.71.X (mrousavy#2692) fix: build for rn 0.71.X * chore: release 4.0.0-beta.11 * feat: Disable shutter sound if in silent * fix: Remove `enablePrecapture` * chore: release 4.0.0-beta.12 * fix: Remove `mustPlayShutterSound()` check * Update Podfile.lock * chore: release 4.0.0-beta.13 * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * fix: Use `ResolutionSelector`s with fallback values * fix: Fall-back to H.264 if custom codec is not supported * Update Camera.tsx * feat: Add `hardware-cost-too-high` error for iOS 16 * chore: release 4.0.0-beta.14 * fix: Use 1280x720 for QR scanner * feat: Skia for V4! 🥳 (mrousavy#2727) * feat: Add Skia support (`toSkImage()`) * feat: Add `frame.getPlaformBuffer()` * Try use getPlatformBuffer * Update Frame.ts * fix: Fix `getPlatformBuffer()` * fix: Fix buffer access * fix: Always support `rgb` and `yuv`, remove `native` All Pixel Formats (`rgb` and `yuv`) are supported now, always. On Android this sometimes uses a conversion step. * feat: Build Skia example renderer * fix: Render a bit better * fix: Use atomic updates now * fix: Remove SK flags * fix: Clear and copy Frame * fix: Use bigint * fix: Use `ImageProxy` and close it to avoid stalling * Only do copy on iOS * fix: Add throw method signature * feat: Box into `DrawableFrame` * feat: Create `SkiaCameraCanvas` * feat: Allow disabling `preview` use-case * feat: Catch some errors * Update FrameHostObject.cpp * fix: Use proper layouts * Update useSkiaFrameProcessor.ts * fix: Use main Coroutine Scope * fix: Fix wrong changes because we now properly abort changes * chore: Format * fix: Upgrade react-native-worklets-core Now includes shift() * fix: Add `focusRequiresPreview` error to iOS * chore: Use latest react-native-worklets-core * fix: Fix buffer orientation on iOS (it's currently always `.up` * fix: Only pass function into native, no object * fix: Format code * fix: Fix styling * fix: Fix pointer-events * Update Camera.tsx * Update useSkiaFrameProcessor.ts * Update Worklets * Draw radial face blur * circle that spins * fix: Proxify * fix: Also use worklets proxy lazily * fix: Try multiple error handlers * fix: Also use Skia Proxy * Add docs to the objects * fix: Use `createModuleProxy` * fix: Also re-create surface if height is different * perf: Remove unneeded mutex locks as Frame only locks refCount * Update FRAME_PROCESSORS_CREATE_OVERVIEW.mdx * fix: Remove unneeded CPU copy * fix: Catch PixelFormat error and fallback to default format * fix: Throw proper PixelFormat error with more info * Update CameraError.swift * Update Frame.java * fix: Use `jsi::Scope` In Hermes, this will probably not do anything. * chore: Update react-native-worklets-core * fix: Dispose previous Surface * Update useSkiaFrameProcessor.ts * fix: Fix surface re-creating each time * chore: Bump to latest RN Skia with the features we need * feat: Clean everything on unmount * Add CPU copy again. On iOS, we need it. * fix: Fix Android build * fix: Fix Android build again * Update ModuleProxy.ts * fix: Use type imports if possible * fix: Add RN Skia patch * Add react-native-fast-tflite * fix: Configure for tflite files as well * Revert "fix: Configure for tflite files as well" This reverts commit 37db0f5. * Revert "Add react-native-fast-tflite" This reverts commit 264dd4b. * Update @Shopify+react-native-skia+1.2.0.patch * Add `useSkiaFrameProcessor` to ESLint * Update yarn.lock * fix: Upgrade to latest Skia to use NativeBuffer API now * fix: Use `CVPixelBuffer` instead of `CMSampleBuffer` * fix: Fix package.json * chore: Update to latest react-native-worklets-core * fix: Use thread-local storage for SkSurface * Update Podfile.lock * fix: Migrate over to new Worklets API * Update FRAME_PROCESSORS.mdx * Update useSkiaFrameProcessor.ts * fix: Run destroy in interaction manager * fix: Safely remove surfaces * fix: Update RNWC to fix terminated threads using dead `thread_local` storage * fix: Install hierarchy fitter in `CameraView` * chore: Remove Skia example from example app * Update CameraPage.tsx * fix: Simplify default pixel format * fix: Rename `ReactLogger` to `VisionLogger` * chore: Format * fix: Fix ResolutionSelector `difference` abs (mrousavy#2746) * chore: release 4.0.0-beta.15 * Update CameraPage.tsx * chore: Fix `type` imports in TS * fix: Remove `enableGpuBuffers` * fix: Make dependencies (`ModuleProxy`) actually optional (mrousavy#2750) fix: Make dependencies actually optional * feat: Optionally compile out Location APIs (mrousavy#2751) * fix: Rename `disableFrameProcessors` to `enableFrameProcessors` * add podfile writer * Create vc_example_cocoapod_utils.rb * feat: Make Location compile optionally * fix: Only compile out location related permissio * Update Podfile * Use `$(inherited)` * fix: Fix `minFps` being larger in `Range` than `maxFps` (mrousavy#2755) * fix: Fix `minFps` being larger in `Range` than `maxFps` * import * fix: Also override `onProviderDisabled` to prevent abstract method crash * fix: Clean up Skia resources on Worklet context to avoid race condition (mrousavy#2754) * fix: Clean up Skia resources on Worklet context to avoid race condition * Update FrameProcessorPlugins.ts * Update useSkiaFrameProcessor.ts * docs: Update docs for V4 (mrousavy#2747) * fix: Update prism-react-renderer * fix: Fix docs * fix: Remove `enableGpuBuffers` * feat: Update Docusaurus to latest 3 * Update docusaurus.config.js * update languages * update languages * rename getting_started * Update some docs * Update CODE_SCANNING.mdx * update some docs * Update docusaurus.config.js * files * starting * fix: Properly display video * lots of updates * more docs * fix: Fix errors causing memory-leaks by not closing Frame * Add example * more * almost * fix broken links * update skia fp docs * Update FRAME_PROCESSORS_SKIA.mdx * Update FRAME_PROCESSORS_SKIA.mdx * fix broken link * chore: release 4.0.0-beta.16 * fix: Fix `Promise` resolve wrong name * fix: Fix build for Android because of Worklet usage * feat: Separate VisionCamera into two subspecs (mrousavy#2761) * feat: Separate VisionCamera into two subspecs * Swift * update podspec * fix: Refactor * some fixes * hmm * fix: Only export ObjC/Swift visible headers * fix: Also drop C++ sources * fix: Move to correct folder * fix: Fix any remaining C++ includes * chore: Format * fix: Fix missingg Swift include * fix: Add PluginRegistry Swift include * Update FrameProcessorPluginRegistry.h * fix: Disable location * fix: Use `public_header_files` instead of `preserve_paths` * Disable FP if needed * fix Podfile * fix: Fix `dispatch_queue_t` in Swift * fix: Disable FPs if not built * Revert "fix: Disable FPs if not built" This reverts commit 82cb560. * Remove Example Plugins from source code * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs (mrousavy#2764) * feat: Separate VisionCamera into `React`, `Core` and `FrameProcessors` subspecs * Split utils * fix: Refactor Android codebase into `core`, `react` and `frameprocessors` (mrousavy#2765) * Add example pod helper again * feat: Also separate Android into `react`, `core` and `frameprocessor` * Log similar to iOS * Update build.gradle * fix: Fix weird refactor mistakes * fix: Refactor a bit * fix: `frameprocessor` -> `frameprocessors` * duplicate clang format with ln * chore: release 4.0.0-beta.17 * chore: Cleanup C++ * chore: Cleanup TS codebase and fix Skia Preview View style/layout issue (mrousavy#2767) * chore: Cleanup TS codebase * chore: More refactors * fix: Put Skia Canvas into CameraView * Update Camera.tsx * fix: Fix JNI types * chore: Update to RN Skia 1.2.3 * fix: Simplify `cpp/` codebase into `android`/`ios` folders (mrousavy#2766) * `Frame Processor` -> `FrameProcessors` * fix: Remove almost empty `cpp/` folder * fix: Add wrap ctor to `MutableRawBuffer` * fix: Add `.clang-format` file again * fix: Remove `cpp` folder references from everywhere * fix: Fix `already bound to a different lifecycle` error when flipping camera (mrousavy#2768) * fix: Fix `already bound to a different lifecycle` error when flipping camera * Update CameraSession.kt * Delete .clang-format * feat: Rebuild FPS Graph as JS component (mrousavy#2769) * feat: Rebuild FPS Graph as JS component * feat: Properly style graph * chore: Lint * fix: Fix FPS Graph not displaying on Android * chore: release 4.0.0-beta.18 * fix: Fix `ImageAnalysis is not supported when Extension is enabled` error (mrousavy#2770) * feat: Upgrade to CameraX alpha5 (mrousavy#2771) * fix: Separate `FpsSampleCollectorDelegate` * Downgrade to CameraX alpha04 because 05 fails to build * fix: Make `FpsSampleCollectorDelegate` weak * fix: Fix `delegate` guard * fix: Fix Android import path due to new subfolder * fix: Fix `Timer is already canceled` * chore: release 4.0.0-beta.19 --------- Co-authored-by: Kirill Zyusko <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Rodrigo Gomes <[email protected]> Co-authored-by: Versus <[email protected]>
- Loading branch information
5 people
authored
Apr 22, 2024
1 parent
124cda7
commit c7ad133