Question: Running MediaPipe Pose Landmark Detection on Videos with Covered Faces

[Mena-SA-Kamel]

Hello,
First, congratulations on developing such an excellent tool! I have a question regarding running the MediaPipe landmark detection model on videos where faces are blurred or hidden.

From the BlazePose paper, it’s noted that the face plays a crucial role in defining the torso position and contributes significantly to person detection. In my experiments, I’ve observed a noticeable drop in performance when processing videos where the face is not visible.

Is there a way to manually define the person's location in the video—perhaps using a dedicated person detection model—to mitigate this dependency on the face for accurate pose estimation as well as enable MediaPipe to work on multi-person videos? I’d appreciate any insights on how to improve performance in such scenarios.

Looking forward to your response.
Mena

[kuaashish]

Hi @Mena-SA-Kamel,

We believe this issue falls outside the scope of the MediaPipe Pose Landmarker. The model is not capable of detecting poses when the head is not visible, hidden, or the video is blurred. Additionally, we have not identified a method to manually define person detection in such cases. Detecting multiple people in videos is also currently beyond the scope of the MediaPipe Pose Landmarker, and this information is outlined in the model cards.

As a suggestion, if you feel this should be considered as a feature request, we encourage you to submit a feature request with detailed information on how it would benefit the community. Depending on future demand and our priorities, we may consider implementing it in the future.

Thank you!!