Skip to content

Commit

Permalink
New face detection example based on MediaPipe (#2360)
Browse files Browse the repository at this point in the history
### What

Closes #2338

New example based on
[MediaPipe](https://developers.google.com/mediapipe)'s Face Detector and
Landmarks solutions.

<img alt="image"
src="https://static.rerun.io/6efefd1de7c63166c4d99956711b6367206d9835_mp_face.png">

In the future, this example could benefit from a custom blueprint when
the API is stabilised enough:
- split face detect and face landmark into separate views (both with the
image in the background)
  - proper default orientation for the face 3D model
  - general layout order (cam views -> 3D view -> plots)

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)

<!-- This line will get updated when the PR build summary job finishes.
-->
PR Build Summary: https://build.rerun.io/pr/2360

<!-- pr-link-docs:start -->
Docs preview: https://rerun.io/preview/275e36b/docs
Examples preview: https://rerun.io/preview/275e36b/examples
<!-- pr-link-docs:end -->
  • Loading branch information
abey79 authored and emilk committed Jun 15, 2023
1 parent 865dcb8 commit 122832e
Show file tree
Hide file tree
Showing 7 changed files with 497 additions and 2 deletions.
3 changes: 3 additions & 0 deletions examples/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -132,3 +132,6 @@ root:
- name: tracking-hf-opencv
python: python/tracking_hf_opencv

- name: face-tracking
python: python/face_tracking

2 changes: 2 additions & 0 deletions examples/python/face_tracking/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
dataset/
model/
56 changes: 56 additions & 0 deletions examples/python/face_tracking/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
title: Face Tracking
python: https://github.com/rerun-io/rerun/tree/latest/examples/python/face_tracking/main.py
tags: [media-pipe, face-tracking, live, camera, 2d, 3d]
---

<picture>
<source media="(max-width: 480px)" srcset="https://static.rerun.io/9f4ecc9d8447375cbad0af17fe2faf8ad2761025_mp_face_480w.png">
<source media="(max-width: 768px)" srcset="https://static.rerun.io/16020f7f1cb4e07c0b481b4887e713e6dd827298_mp_face_768w.png">
<source media="(max-width: 1024px)" srcset="https://static.rerun.io/9e90eff729dee3252659a8ea528fe01ecb44bd92_mp_face_1024w.png">
<source media="(max-width: 1200px)" srcset="https://static.rerun.io/4897ba71ff5da36c5fed3d6bd8f4f134520ef90f_mp_face_1200w.png">
<img src="https://static.rerun.io/8b951a755f57a210d48c37d032156c872fd7cc41_mp_face_full.png" alt="screenshot of the Rerun visualization of the MediaPipe Face Detector and Landmarker">
</picture>


Use the [MediaPipe](https://google.github.io/mediapipe/) Face Detector and Landmarker solutions to detect and track a human face in image, videos, and camera stream.

```bash
pip install -r examples/python/face_tracking/requirements.txt
python examples/python/face_tracking/main.py
```

## Usage

CLI usage help is available using the `--help` option:

```
$ python examples/python/face_tracking/main.py --help
usage: main.py [-h] [--demo-image] [--image IMAGE] [--video VIDEO] [--camera CAMERA] [--max-frame MAX_FRAME] [--max-dim MAX_DIM] [--num-faces NUM_FACES] [--headless] [--connect] [--serve] [--addr ADDR] [--save SAVE]
Uses the MediaPipe Face Detection to track a human pose in video.
options:
-h, --help show this help message and exit
--demo-image Run on a demo image automatically downloaded
--image IMAGE Run on the provided image
--video VIDEO Run on the provided video file.
--camera CAMERA Run from the camera stream (parameter is the camera ID, usually 0
--max-frame MAX_FRAME
Stop after processing this many frames. If not specified, will run until interrupted.
--max-dim MAX_DIM Resize the image such as its maximum dimension is not larger than this value.
--num-faces NUM_FACES
Max number of faces detected by the landmark model (temporal smoothing is applied only for a value of 1).
--headless Don't show GUI
--connect Connect to an external viewer
--serve Serve a web viewer (WARNING: experimental feature)
--addr ADDR Connect to this ip:port
--save SAVE Save data to a .rrd file at this path
```

Here is an overview of the options specific to this example:

- *Running modes*: By default, this example streams images from the default webcam. Another webcam can be used by providing a camera index with the `--camera` option. Alternatively, images can be read from a video file (using `--video PATH`) or a single image file (using `--image PATH`). Also, a demo image with two faces can be automatically downloaded and used with `--demo-image`.
- *Max face count*: The maximum face detected by MediaPipe Face Landmarker can be set using `--num-faces NUM`. It defaults to 1, in which case the Landmarker applies temporal smoothing. This parameter doesn't affect MediaPipe Face Detector, which always attempts to detect all faces present in the input images.
- *Image downscaling*: By default, this example logs and runs on the native resolution of the provided images. Input images can be downscaled to a given maximum dimension using `--max-dim DIM`.
- *Limiting frame count*: When running from a webcam or a video file, this example can be set to stop after a given number of frames using `--max-frame MAX_FRAME`.
Loading

0 comments on commit 122832e

Please sign in to comment.