Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Galaxy S25 MediaPipe Hand Detection Camera Error. #5867

Open
hongsoonill opened this issue Feb 22, 2025 · 9 comments
Open

Galaxy S25 MediaPipe Hand Detection Camera Error. #5867

hongsoonill opened this issue Feb 22, 2025 · 9 comments
Assignees
Labels
platform::android Android Solutions platform:javascript MediaPipe Javascript issues task:hand landmarker Issues related to hand landmarker: Identify and track hands and fingers type:bug Bug in the Source Code of MediaPipe Solution

Comments

@hongsoonill
Copy link

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Android 15 / One UI 7.0

Mobile device if the issue happens on mobile device

Galaxy S25

Browser and version if the issue happens on browser

Google Chrome, Samsung Internet

Programming Language and version

Javascript

MediaPipe version

No response

Bazel version

No response

Solution

Hand Landmark detection

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

On the Galaxy S25, the camera only detects distant hands located in the bottom right corner of the screen. When a hand is placed in the center, MediaPipe fails to recognize it.

Describe the expected behaviour

In the demo version provided by MediaPipe, the camera index cannot be selected, and even when uploading a hand image, it is not recognized. The Galaxy S25 seems to misidentify the camera or only detect specific positions (a small palm in the bottom right corner).

Standalone code/steps you may have used to try to get what you need

https://mediapipe-studio.webapps.google.com/demo/hand_landmarker?hl=ko

Other info / Complete Logs

https://mediapipe-studio.webapps.google.com/demo/hand_landmarker?hl=ko
@hongsoonill hongsoonill added the type:bug Bug in the Source Code of MediaPipe Solution label Feb 22, 2025
@hongsoonill
Copy link
Author

hongsoonill commented Feb 22, 2025

This is the only area where hand landmarks are recognized on the Galaxy S25

[Test Result]
Image

[Original]

Image

@kuaashish kuaashish added platform::android Android Solutions platform:javascript MediaPipe Javascript issues task:hand landmarker Issues related to hand landmarker: Identify and track hands and fingers labels Feb 24, 2025
@kuaashish
Copy link
Collaborator

Hi @hongsoonill,

To further investigate, could you share screenshots where hands are positioned in the middle of the frame and are not being recognized? We also request testing with live camera input and a variety of static images to assess the consistency of the detection issue.

Thank you!!

@kuaashish kuaashish added the stat:awaiting response Waiting for user response label Feb 24, 2025
@mackinleysmith
Copy link

Not sure if it helps, but we are experiencing similar issues with object detection in JS on the S25. The S25 seems to prefer objects to be really far away before it detects them.

Here's a similar scene between my iPhone and my S25.

iPhone:

Image

S25:

Image

You can see in the S25 image that the objects that are further away on my desk are detected, but the PS5 controller is barely getting a box on it (and is wrongly categorized). The iPhone has no such trouble.

Sorry if this isn't quite the same problem as the original issue, but it feels somewhat related to me.

@google-ml-butler google-ml-butler bot removed the stat:awaiting response Waiting for user response label Feb 25, 2025
@kuaashish
Copy link
Collaborator

Hi @mackinleysmith,

This is useful and we will bring it to the team's attention. @hongsoonill, please check the comment above and let us know if you have any further information.

Thank you!!

@kuaashish kuaashish added the stat:awaiting response Waiting for user response label Feb 27, 2025
@mackinleysmith
Copy link

Great! My company bought me an S25 just for this purpose, so please don't hesitate to reach out to me if you need any help with testing things. We are keen on getting this resolved for our customer base.

@google-ml-butler google-ml-butler bot removed the stat:awaiting response Waiting for user response label Feb 27, 2025
@kuaashish
Copy link
Collaborator

@hongsoonill, @mackinleysmith,

We tested the S24 Ultra today, and there appear to be no issues. We will check if we can reproduce the behavior on the S25, although we currently do not have it.

@juniverse76
Copy link

juniverse76 commented Mar 13, 2025

Hi, @kuaashish
I would like to give you update on behalf of @hongsoonill .
We found out that there was some kind of error when the MediaPipe is using GPU in S25.
So explicitly setting to CPU mode, the MediaPipe seems to work.
However, this issue should still be worth investigation.
@hongsoonill , please elaborate if needed.

@hongsoonill
Copy link
Author

hongsoonill commented Mar 13, 2025

@kuaashish

We have confirmed that MediaPipe runs in CPU mode instead of GPU mode on the Samsung Galaxy S25, and the Landmark detection appears to be working properly. However, further investigation is needed to identify any specific issues that may arise due to this mode switch.

We are providing additional device details, including the Android version and Chrome version of the Galaxy S25, along with other relevant information. This will help in analyzing whether there are differences in MediaPipe’s behavior across different device environments and how it compares to other setups.

------------------------- About Galaxy S25 GPU Information --------------------
Graphics Feature Status

  • Canvas: Hardware accelerated
  • Canvas out-of-process rasterization: Enabled
  • Direct Rendering Display Compositor: Enabled
  • Compositing: Hardware accelerated
  • Multiple Raster Threads: Disabled
  • OpenGL: Enabled
  • Rasterization: Hardware accelerated
  • Raw Draw: Disabled
  • Skia Graphite: Disabled
  • Surface Control: Enabled
  • Video Decode: Hardware accelerated
  • Video Encode: Hardware accelerated
  • Vulkan: Disabled
  • WebGL: Hardware accelerated
  • WebGL2: Hardware accelerated
  • WebGPU: Hardware accelerated
  • WebNN: Disabled

Version Information

Data exported : 2025-03-10T08:47:28.925Z
Chrome version : Chrome/133.0.6943.121
Operating system : Android 15

Software rendering list URL: https://chromium.googlesource.com/chromium/src/+/20baa002cf6672d58cfd567492d58d8bd37b8dda/gpu/config/software_rendering_list.json
Driver bug list URL : https://chromium.googlesource.com/chromium/src/+/20baa002cf6672d58cfd567492d58d8bd37b8dda/gpu/config/gpu_driver_bug_list.json
ANGLE commit id : 8304737811a7
2D graphics backend : Skia/133 ecebe831881cdf52c65df518777210071f7970dd
Command Line : --use-mobile-user-agent --use-mobile-user-agent --enable-viewport --validate-input-event-stream --enable-longpress-drag-selection --touch-selection-strategy=direction --disable-composited-antialiasing --enable-dom-distiller --flag-switches-begin --flag-switches-end --top-controls-show-threshold=0.5 --top-controls-hide-threshold=0.5

Driver Information

Initialization time : 65
In-process GPU : false
Passthrough Command Decoder : false
Sandboxed : false
GPU0 : VENDOR= 0x0000 [Qualcomm], DEVICE=0x0000 [Adreno (TM) 830], DRIVER_VERSION=0800.17.11 ACTIVE
Optimus : false
AMD switchable : false
Pixel shader version : 3.20
Vertex shader version : 3.20
Max. MSAA samples : 4
Machine model name : SM-S938N
Machine model version :
GL implementation parts : (gl=egl-gles2,angle=none)
Display type : DEFAULT
GL_VENDOR : Qualcomm
GL_RENDERER : Adreno (TM) 830
GL_VERSION : OpenGL ES 3.2 [email protected] (GIT@7a7d1616fb, I1b2d45fd3f, 1734524510) (Date:12/18/24)
GL_EXTENSIONS : GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_sync GL_OES_vertex_half_float GL_OES_framebuffer_object GL_OES_rgb8_rgba8 GL_OES_compressed_ETC1_RGB8_texture GL_AMD_compressed_ATC_texture GL_KHR_texture_compression_astc_sliced_3d GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_OES_texture_compression_astc GL_EXT_texture_compression_s3tc GL_EXT_texture_compression_s3tc_srgb GL_EXT_texture_compression_rgtc GL_EXT_texture_compression_bptc GL_OES_texture_npot GL_EXT_texture_filter_anisotropic GL_EXT_texture_format_BGRA8888 GL_EXT_read_format_bgra GL_OES_texture_3D GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_float_blend GL_QCOM_alpha_test GL_OES_depth24 GL_OES_packed_depth_stencil GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_EXT_sRGB GL_OES_texture_float GL_OES_texture_float_linear GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_EXT_texture_type_2_10_10_10_REV GL_EXT_texture_sRGB_decode GL_EXT_texture_compression_astc_decode_mode GL_EXT_texture_mirror_clamp_to_edge GL_EXT_texture_format_sRGB_override GL_OES_element_index_uint GL_EXT_copy_image GL_EXT_geometry_shader GL_EXT_tessellation_shader GL_OES_texture_stencil8 GL_EXT_shader_io_blocks GL_OES_shader_image_atomic GL_OES_sample_variables GL_EXT_texture_border_clamp GL_EXT_EGL_image_external_wrap_modes GL_EXT_multisampled_render_to_texture GL_EXT_multisampled_render_to_texture2 GL_OES_shader_multisample_interpolation GL_EXT_texture_cube_map_array GL_EXT_draw_buffers_indexed GL_EXT_gpu_shader5 GL_EXT_robustness GL_EXT_texture_buffer GL_EXT_shader_framebuffer_fetch GL_ARM_shader_framebuffer_fetch_depth_stencil GL_OES_texture_storage_multisample_2d_array GL_OES_sample_shading GL_OES_get_program_binary GL_EXT_debug_label GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_QCOM_tiled_rendering GL_ANDROID_extension_pack_es31a GL_EXT_primitive_bounding_box GL_OES_standard_derivatives GL_OES_vertex_array_object GL_EXT_disjoint_timer_query GL_KHR_debug GL_EXT_YUV_target GL_EXT_sRGB_write_control GL_EXT_texture_norm16 GL_EXT_discard_framebuffer GL_OES_surfaceless_context GL_OVR_multiview GL_OVR_multiview2 GL_EXT_texture_sRGB_R8 GL_KHR_no_error GL_EXT_debug_marker GL_OES_EGL_image_external_essl3 GL_OVR_multiview_multisampled_render_to_texture GL_EXT_buffer_storage GL_EXT_external_buffer GL_EXT_blit_framebuffer_params GL_EXT_clip_cull_distance GL_EXT_protected_textures GL_EXT_shader_non_constant_global_initializers GL_QCOM_texture_foveated GL_QCOM_texture_foveated2 GL_QCOM_texture_foveated_subsampled_layout GL_QCOM_shader_framebuffer_fetch_noncoherent GL_QCOM_shader_framebuffer_fetch_rate GL_EXT_memory_object GL_EXT_memory_object_fd GL_EXT_EGL_image_array GL_NV_shader_noperspective_interpolation GL_KHR_robust_buffer_access_behavior GL_EXT_EGL_image_storage GL_EXT_blend_func_extended GL_EXT_clip_control GL_OES_texture_view GL_EXT_fragment_invocation_density GL_QCOM_motion_estimation GL_QCOM_validate_shader_binary GL_QCOM_YUV_texture_gather GL_QCOM_shading_rate GL_QCOM_frame_extrapolation GL_QCOM_texture_lod_bias GL_IMG_texture_filter_cubic GL_QCOM_render_shared_exponent GL_EXT_polygon_offset_clamp GL_EXT_texture_sRGB_RG8 GL_EXT_depth_clamp GL_EXT_fragment_shading_rate GL_EXT_fragment_shading_rate_primitive GL_EXT_fragment_shading_rate_attachment GL_EXT_shader_implicit_conversions GL_EXT_texture_storage_compression GL_EXT_EGL_image_storage_compression GL_EXT_render_snorm GL_QCOM_render_sRGB_R8_RG8 GL_QCOM_ycbcr_degamma
Disabled Extensions : GL_EXT_shader_framebuffer_fetch GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent
Disabled WebGL Extensions :
Window system binding vendor :
Window system binding version :
Window system binding extensions:
Direct rendering version : unknown
Reset notification strategy : 0x8252
GPU process crash count : 0
gfx::BufferFormats supported for allocation and texturing: R_8: not supported, R_16: not supported, RG_88: not supported, RG_1616: not supported, BGR_565: not supported, RGBA_4444: not supported, RGBX_8888: not supported, RGBA_8888: not supported, BGRX_8888: not supported, BGRA_1010102: not supported, RGBA_1010102: not supported, BGRA_8888: not supported, RGBA_F16: not supported, YVU_420: not supported, YUV_420_BIPLANAR: not supported, YUVA_420_TRIPLANAR: not supported, P010: not supported

Driver Bug Workarounds

  • disable_program_caching_for_transform_feedback
  • disable_timestamp_queries
  • dont_delete_source_texture_for_egl_image
  • exit_on_context_lost
  • force_cube_map_positive_x_allocation
  • force_enable_color_buffer_float_except_rgb32f
  • max_copy_texture_chromium_size_1048576
  • max_msaa_sample_count_4
  • remove_dynamic_indexing_of_swizzled_vector
  • round_down_uniform_bind_buffer_range_size
  • unbind_egl_context_to_flush_driver_caches
  • use_copyteximage2d_instead_of_readpixels_on_multisampled_textures
  • use_virtualized_gl_contexts
  • wake_up_gpu_before_drawing
  • webgl_or_caps_max_texture_size_limit_4096
  • disabled_extension_GL_EXT_shader_framebuffer_fetch
  • disabled_extension_GL_KHR_blend_equation_advanced
  • disabled_extension_GL_KHR_blend_equation_advanced_coherent

Problems Detected

  • Non-virtual contexts on Qualcomm sometimes cause out-of-order frames:
    (http://crbug.com/163464), (http://crbug.com/233612), (http://crbug.com/289461)
    Applied Workarounds: use_virtualized_gl_contexts

  • The first draw operation from an idle state is slow:
    (http://crbug.com/309734)
    Applied Workarounds: wake_up_gpu_before_drawing

  • glFinish doesn't clear caches on Android:
    (http://crbug.com/509727)
    Applied Workarounds: unbind_egl_context_to_flush_driver_caches

  • Android Adreno crashes on binding incomplete cube map texture to FBO:
    (http://crbug.com/518889)
    Applied Workarounds: force_cube_map_positive_x_allocation

  • CHROMIUM_copy_texture with 1MB copy per flush to avoid unwanted cache growth on Adreno:
    (http://crbug.com/542478)
    Applied Workarounds: max_copy_texture_chromium_size_1048576

  • Limit max texure size to 4096 on all of Android
    Applied Workarounds: webgl_or_caps_max_texture_size_limit_4096

  • Disable KHR_blend_equation_advanced until cc shaders are updated:
    (http://crbug.com/661715)
    Applied Workarounds: disable(GL_KHR_blend_equation_advanced),
    disable(GL_KHR_blend_equation_advanced_coherent)

  • Program binaries don't contain transform feedback varyings on Qualcomm GPUs:
    (http://crbug.com/658074)
    Applied Workarounds: disable_program_caching_for_transform_feedback

  • Limit MSAA to 4x on Android devices:
    (http://crbug.com/797243)
    Applied Workarounds: max_msaa_sample_count_4

  • A deleted query id for GL_TIMESTAMP cannot be reused for other query targets:
    (http://crbug.com/858879)
    Applied Workarounds: disable_timestamp_queries

  • ReadPixels is broken with EXT_multisampled_render_to_texture on recent Adreno drivers.:
    (http://crbug.com/890002)
    Applied Workarounds: use_copyteximage2d_instead_of_readpixels_on_multisampled_textures

  • Round down glBindBufferRange size to a multiple of 4 on Qualcomm Adreno GPUs for uniform buffers:
    (http://crbug.com/906743)
    Applied Workarounds: round_down_uniform_bind_buffer_range_size

  • Crash on Adreno when using GL_EXT_shader_framebuffer_fetch:
    (http://crbug.com/1010338)
    Applied Workarounds: disable(GL_EXT_shader_framebuffer_fetch)

  • Remove dynamic indexing for swizzled vectors on Android:
    (http://crbug.com/709351)
    Applied Workarounds: remove_dynamic_indexing_of_swizzled_vector

  • Some drivers seem to require as to use original texture whenever possible:
    (http://crbug.com/1052114), (http://crbug.com/1117370)
    Applied Workarounds: dont_delete_source_texture_for_egl_image

  • Always assume float formats are renderable on Android except RGB32F:
    (http://crbug.com/1127387)
    Applied Workarounds: force_enable_color_buffer_float_except_rgb32f

  • Context lost recovery often fails on Adreno with SkiaRenderer GL.:
    (http://crbug.com/1079161)
    Applied Workarounds: exit_on_context_lost

Dawn Info

OpenGLES backend - Adreno (TM) 830 (Compatibility Mode)

[WebGPU Status]

  • Blocklisted - crbug.com/333858788: OpenGLES not fully supported on Android.

[Adapter Supported Features]

  • texture-compression-bc
  • texture-compression-etc2
  • texture-compression-astc
  • float32blendable
  • dawn-internal-usages
  • dawn-native
  • implicit-device-synchronization
  • dual-source-blending
  • unorm16texture-formats
  • snorm16texture-formats
  • format-capabilities
  • norm16texture-formats
  • shared-texture-memory-a-hardware-buffer
  • shared-fence-sync-fd

[Enabled Toggle Names]

  • lazy_clear_resource_on_first_use:
    (https://crbug.com/dawn/145):
    Clears resource to zero on first usage. This initializes the resource so
    that no dirty bits from recycled memory is present in the new resource.

  • flush_before_client_wait_sync:
    (https://crbug.com/dawn/633):
    Call glFlush before glClientWaitSync to work around bugs in the latter

  • disallow_spirv:
    (https://crbug.com/1214923):
    Disallow usage of SPIR-V completely so that only WGSL is used for shader
    modules. This is useful to prevent a Chromium renderer process from
    successfully sending SPIR-V code to be compiled in the GPU process.

  • use_placeholder_fragment_in_vertex_only_pipeline:
    (https://crbug.com/dawn/136):
    Use a placeholder empty fragment shader in vertex only render pipeline.
    This toggle must be enabled for OpenGL ES backend, the Vulkan Backend, and
    serves as a workaround by default enabled on some Metal devices with Intel
    GPU to ensure the depth result is correct.

  • timestamp_quantization:
    (https://crbug.com/dawn/1800):
    Enable timestamp queries quantization to reduce the precision of timers
    that can be created with timestamp queries.

  • use_blit_for_buffer_to_stencil_texture_copy:
    (https://crbug.com/dawn/1389):
    Use a blit instead of a copy command to copy buffer data to the stencil
    aspect of a texture. Works around an issue where stencil writes by copy
    commands are not visible to a render or compute pass.

  • use_blit_for_depth16unorm_texture_to_buffer_copy:
    (https://crbug.com/dawn/1782):
    Use a blit instead of a copy command to copy depth aspect of a texture to
    a buffer.Workaround for OpenGL and OpenGLES.

  • use_blit_for_depth32float_texture_to_buffer_copy:
    (https://crbug.com/dawn/1782):
    Use a blit instead of a copy command to copy depth aspect of a texture to
    a buffer.Workaround for OpenGLES.

  • use_blit_for_stencil_texture_to_buffer_copy:
    (https://crbug.com/dawn/1782):
    Use a blit instead of a copy command to copy stencil aspect of a texture
    to a buffer.Workaround for OpenGLES.

  • use_blit_for_snorm_texture_to_buffer_copy:
    (https://crbug.com/dawn/1781):
    Use a blit instead of a copy command to copy snorm texture to a
    buffer.Workaround for OpenGLES.

  • use_blit_for_rgb9e5ufloat_texture_copy:
    (https://crbug.com/dawn/2079):
    Use a blit instead of a copy command to copy rgb9e5ufloat texture to a
    texture or a buffer.Workaround for OpenGLES.

  • use_t2b2t_for_srgb_texture_copy:
    (https://crbug.com/dawn/2362):
    Use T2B and B2T copies to emulate a T2T copy between sRGB and non-sRGB
    textures.Workaround for OpenGLES.

[WebGPU Required Toggles - enabled]

  • disallow_spirv:
    (https://crbug.com/1214923):
    Disallow usage of SPIR-V completely so that only WGSL is used for shader
    modules. This is useful to prevent a Chromium renderer process from
    successfully sending SPIR-V code to be compiled in the GPU process.

  • timestamp_quantization:
    (https://crbug.com/dawn/1800):
    Enable timestamp queries quantization to reduce the precision of timers
    that can be created with timestamp queries.

Vulkan backend - Adreno (TM) 830

[WebGPU Status]

  • Available

[Adapter Supported Features]

  • depth-clip-control
  • depth32float-stencil8
  • timestamp-query
  • texture-compression-bc
  • texture-compression-etc2
  • texture-compression-astc
  • indirect-first-instance
  • rg11b10ufloat-renderable
  • bgra8unorm-storage
  • float32filterable
  • float32blendable
  • subgroups
  • dawn-internal-usages
  • dawn-multi-planar-formats
  • dawn-native
  • implicit-device-synchronization
  • transient-attachments
  • dual-source-blending
  • unorm16texture-formats
  • snorm16texture-formats
  • adapter-properties-memory-heaps
  • adapter-properties-vk
  • format-capabilities
  • norm16texture-formats
  • shared-texture-memory-a-hardware-buffer
  • shared-texture-memory-opaque-fd
  • shared-fence-sync-fd
  • dawn-load-resolve-texture
  • clip-distances
  • flexible-texture-views

@guilherme-bakaukas
Copy link

Also not working for MediaPipe Face Mesh on Galaxy S25 Ultra

The detection works only for the first few frames after execution, but then it stops detecting faces entirely.

This problem also occurs when using the official MediaPipe Web Demo (https://codepen.io/mediapipe-preview/pen/OJBVQJm), where the face mesh initially detects properly but quickly stops working.

Has anyone experienced this issue or found a potential fix? Any insights or updates would be greatly appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform::android Android Solutions platform:javascript MediaPipe Javascript issues task:hand landmarker Issues related to hand landmarker: Identify and track hands and fingers type:bug Bug in the Source Code of MediaPipe Solution
Projects
None yet
Development

No branches or pull requests

5 participants