Skip to content

Commit e9fbe86

Browse files
MediaPipe Teamjqtang
MediaPipe Team
authored and
jqtang
committed
Project import generated by Copybara.
GitOrigin-RevId: e3a43e4e5e519cd14df7095749059e2613bdcf76
1 parent 67bd8a2 commit e9fbe86

File tree

96 files changed

+2547
-1176
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

96 files changed

+2547
-1176
lines changed

Diff for: BUILD

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright 2019-2020 The MediaPipe Authors.
1+
# Copyright 2019 The MediaPipe Authors.
22
#
33
# Licensed under the Apache License, Version 2.0 (the "License");
44
# you may not use this file except in compliance with the License.

Diff for: README.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ run code search using
101101

102102
## Videos
103103

104-
* [YouTube Channel](https://www.youtube.com/channel/UCObqmpuSMx-usADtL_qdMAw)
104+
* [YouTube Channel](https://www.youtube.com/c/MediaPipe)
105105

106106
## Events
107107

@@ -123,7 +123,7 @@ run code search using
123123

124124
* [Awesome MediaPipe](https://mediapipe.org) - A curated list of awesome
125125
MediaPipe related frameworks, libraries and software
126-
* [Slack community](https://mediapipe.slack.com) for MediaPipe users
126+
* [Slack community](https://https://mediapipe.page.link/joinslack) for MediaPipe users
127127
* [Discuss](https://groups.google.com/forum/#!forum/mediapipe) - General
128128
community discussion around MediaPipe
129129

Diff for: WORKSPACE

+12-3
Original file line numberDiff line numberDiff line change
@@ -37,10 +37,19 @@ http_archive(
3737
)
3838

3939
# GoogleTest/GoogleMock framework. Used by most unit-tests.
40+
# Last updated 2020-06-30.
4041
http_archive(
41-
name = "com_google_googletest",
42-
urls = ["https://github.com/google/googletest/archive/master.zip"],
43-
strip_prefix = "googletest-master",
42+
name = "com_google_googletest",
43+
urls = ["https://github.com/google/googletest/archive/aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e.zip"],
44+
patches = [
45+
# fix for https://github.com/google/googletest/issues/2817
46+
"@//third_party:com_google_googletest_9d580ea80592189e6d44fa35bcf9cdea8bf620d6.diff"
47+
],
48+
patch_args = [
49+
"-p1",
50+
],
51+
strip_prefix = "googletest-aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e",
52+
sha256 = "04a1751f94244307cebe695a69cc945f9387a80b0ef1af21394a490697c5c895",
4453
)
4554

4655
# Google Benchmark library.

Diff for: build_ios_examples.sh

+74
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
#!/bin/bash
2+
# Copyright 2020 The MediaPipe Authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
# =========================================================================
16+
#
17+
# Script to build all MediaPipe iOS example apps.
18+
#
19+
# To build all apps and store them in out_dir:
20+
# $ ./build_ios_examples.sh -d out_dir
21+
# Omitting -d and the associated directory saves all generated IPAs in the
22+
# current directory.
23+
# $ ./build_ios_examples.sh -d out_dir --nostrip
24+
# Same as above except that the symnbols are not stripped.
25+
26+
set -e
27+
28+
out_dir="."
29+
strip=true
30+
app_dir="mediapipe/examples/ios"
31+
bin_dir="bazel-bin"
32+
declare -a default_bazel_flags=(build -c opt --config=ios_arm64)
33+
34+
while [[ -n $1 ]]; do
35+
case $1 in
36+
-d)
37+
shift
38+
out_dir=$1
39+
;;
40+
--nostrip)
41+
strip=false
42+
;;
43+
*)
44+
echo "Unsupported input argument $1."
45+
exit 1
46+
;;
47+
esac
48+
shift
49+
done
50+
51+
echo "app_dir: $app_dir"
52+
echo "out_dir: $out_dir"
53+
echo "strip: $strip"
54+
55+
declare -a bazel_flags
56+
57+
apps="${app_dir}/*"
58+
for app in ${apps}; do
59+
if [[ -d "${app}" ]]; then
60+
target_name=${app##*/}
61+
target="${app}:${target_name}"
62+
63+
echo "=== Target: ${target}"
64+
65+
bazel_flags=("${default_bazel_flags[@]}")
66+
bazel_flags+=(${target})
67+
if [[ $strip == true ]]; then
68+
bazel_flags+=(--linkopt=-s)
69+
fi
70+
71+
bazel "${bazel_flags[@]}"
72+
cp -f "${bin_dir}/${app}/"*".ipa" "${out_dir}"
73+
fi
74+
done

Diff for: docs/framework_concepts/gpu.md

+12-12
Original file line numberDiff line numberDiff line change
@@ -149,15 +149,15 @@ When possible, these calculators use platform-specific functionality to share da
149149
150150
The below diagram shows the data flow in a mobile application that captures video from the camera, runs it through a MediaPipe graph, and renders the output on the screen in real time. The dashed line indicates which parts are inside the MediaPipe graph proper. This application runs a Canny edge-detection filter on the CPU using OpenCV, and overlays it on top of the original video using the GPU.
151151
152-
| ![How GPU calculators interact](../images/gpu_example_graph.png) |
153-
| :--------------------------------------------------------------------------: |
154-
| *Video frames from the camera are fed into the graph as `GpuBuffer` packets. |
155-
: The input stream is accessed by two calculators in parallel. :
156-
: `GpuBufferToImageFrameCalculator` converts the buffer into an `ImageFrame`, :
157-
: which is then sent through a grayscale converter and a canny filter (both :
158-
: based on OpenCV and running on the CPU), whose output is then converted into :
159-
: a `GpuBuffer` again. A multi-input GPU calculator, GlOverlayCalculator, :
160-
: takes as input both the original `GpuBuffer` and the one coming out of the :
161-
: edge detector, and overlays them using a shader. The output is then sent :
162-
: back to the application using a callback calculator, and the application :
163-
: renders the image to the screen using OpenGL.* :
152+
![How GPU calculators interact](../images/gpu_example_graph.png)
153+
154+
Video frames from the camera are fed into the graph as `GpuBuffer` packets. The
155+
input stream is accessed by two calculators in parallel.
156+
`GpuBufferToImageFrameCalculator` converts the buffer into an `ImageFrame`,
157+
which is then sent through a grayscale converter and a canny filter (both based
158+
on OpenCV and running on the CPU), whose output is then converted into a
159+
`GpuBuffer` again. A multi-input GPU calculator, GlOverlayCalculator, takes as
160+
input both the original `GpuBuffer` and the one coming out of the edge detector,
161+
and overlays them using a shader. The output is then sent back to the
162+
application using a callback calculator, and the application renders the image
163+
to the screen using OpenGL.

Diff for: docs/getting_started/building_examples.md

+20-15
Original file line numberDiff line numberDiff line change
@@ -184,12 +184,8 @@ app:
184184

185185
### Prerequisite
186186

187-
1. Install [Xcode](https://developer.apple.com/xcode/) and the Command Line
188-
Tools.
189-
190-
Follow Apple's instructions to obtain the required development certificates
191-
and provisioning profiles for your iOS device. Install the Command Line
192-
Tools by
187+
1. Install [Xcode](https://developer.apple.com/xcode/), and additionally
188+
install the Command Line Tools by:
193189

194190
```bash
195191
xcode-select --install
@@ -209,26 +205,31 @@ app:
209205
pip3 install --user six
210206
```
211207

212-
4. Clone the MediaPipe repository.
208+
4. Follow
209+
[Apple's instructions](https://developer.apple.com/support/certificates/) to
210+
obtain the required development certificates and provisioning profiles for
211+
your iOS device.
212+
213+
Tip: You can the following command to see the provisioning profiles you have
214+
previously downloaded using Xcode: `open
215+
~/Library/MobileDevice/"Provisioning Profiles"`. If there are none, generate
216+
and download a profile on
217+
[Apple's developer site](https://developer.apple.com/account/resources/).
218+
219+
5. Clone the MediaPipe repository.
213220

214221
```bash
215222
git clone https://github.com/google/mediapipe.git
216223
```
217224

218-
5. Symlink or copy your provisioning profile to
219-
`mediapipe/mediapipe/provisioning_profile.mobileprovision`.
225+
6. In the cloned MediaPipe repository, symlink or copy your provisioning profile
226+
to `mediapipe/provisioning_profile.mobileprovision`, e.g.,
220227

221228
```bash
222229
cd mediapipe
223230
ln -s ~/Downloads/MyProvisioningProfile.mobileprovision mediapipe/provisioning_profile.mobileprovision
224231
```
225232

226-
Tip: You can use this command to see the provisioning profiles you have
227-
previously downloaded using Xcode: `open
228-
~/Library/MobileDevice/"Provisioning Profiles"`. If there are none, generate
229-
and download a profile on
230-
[Apple's developer site](https://developer.apple.com/account/resources/).
231-
232233
### Option 1: Build with Bazel in Command Line
233234

234235
1. Modify the `bundle_id` field of the app's `ios_application` build target to
@@ -246,6 +247,10 @@ app:
246247
247248
You may see a permission request from `codesign` in order to sign the app.
248249
250+
Tip: You can run this
251+
[script](https://github.com/google/mediapipe/blob/master/build_ios_examples.sh)
252+
to build all MediaPipe iOS example apps.
253+
249254
3. In Xcode, open the `Devices and Simulators` window (command-shift-2).
250255
251256
4. Make sure your device is connected. You will see a list of installed apps.

Diff for: docs/getting_started/install.md

+28-8
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,18 @@ apps, see these [instructions](./building_examples.md#ios).
4444
[Bazel documentation](https://docs.bazel.build/versions/master/install-ubuntu.html)
4545
to install Bazel 2.0 or higher.
4646

47+
For Nvidia Jetson and Raspberry Pi devices with ARM Ubuntu, Bazel needs to
48+
be built from source.
49+
50+
```bash
51+
# For Bazel 3.0.0
52+
wget https://github.com/bazelbuild/bazel/releases/download/3.0.0/bazel-3.0.0-dist.zip
53+
sudo apt-get install build-essential openjdk-8-jdk python zip unzip
54+
unzip bazel-3.0.0-dist.zip
55+
env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh
56+
sudo cp output/bazel /usr/local/bin/
57+
```
58+
4759
3. Install OpenCV and FFmpeg.
4860

4961
Option 1. Use package manager tool to install the pre-compiled OpenCV
@@ -58,6 +70,14 @@ apps, see these [instructions](./building_examples.md#ios).
5870
libopencv-imgproc-dev libopencv-video-dev
5971
```
6072

73+
[`opencv_linux.BUILD`] is configured for x86_64 by default. For Nvidia
74+
Jetson and Raspberry Pi devices with ARM Ubuntu, the lib paths need to be
75+
modified.
76+
77+
```bash
78+
sed -i "s/x86_64-linux-gnu/aarch64-linux-gnu/g" third_party/opencv_linux.BUILD
79+
```
80+
6181
Option 2. Run [`setup_opencv.sh`] to automatically build OpenCV from source
6282
and modify MediaPipe's OpenCV config.
6383
@@ -493,14 +513,14 @@ cameras. Alternatively, you use a video file as input.
493513
494514
```bash
495515
username@DESKTOP-TMVLBJ1:~$ curl -sLO --retry 5 --retry-max-time 10 \
496-
https://storage.googleapis.com/bazel/2.0.0/release/bazel-2.0.0-installer-linux-x86_64.sh && \
497-
sudo mkdir -p /usr/local/bazel/2.0.0 && \
498-
chmod 755 bazel-2.0.0-installer-linux-x86_64.sh && \
499-
sudo ./bazel-2.0.0-installer-linux-x86_64.sh --prefix=/usr/local/bazel/2.0.0 && \
500-
source /usr/local/bazel/2.0.0/lib/bazel/bin/bazel-complete.bash
501-
502-
username@DESKTOP-TMVLBJ1:~$ /usr/local/bazel/2.0.0/lib/bazel/bin/bazel version && \
503-
alias bazel='/usr/local/bazel/2.0.0/lib/bazel/bin/bazel'
516+
https://storage.googleapis.com/bazel/3.0.0/release/bazel-3.0.0-installer-linux-x86_64.sh && \
517+
sudo mkdir -p /usr/local/bazel/3.0.0 && \
518+
chmod 755 bazel-3.0.0-installer-linux-x86_64.sh && \
519+
sudo ./bazel-3.0.0-installer-linux-x86_64.sh --prefix=/usr/local/bazel/3.0.0 && \
520+
source /usr/local/bazel/3.0.0/lib/bazel/bin/bazel-complete.bash
521+
522+
username@DESKTOP-TMVLBJ1:~$ /usr/local/bazel/3.0.0/lib/bazel/bin/bazel version && \
523+
alias bazel='/usr/local/bazel/3.0.0/lib/bazel/bin/bazel'
504524
```
505525
506526
6. Checkout MediaPipe repository.

Diff for: docs/index.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ run code search using
101101

102102
## Videos
103103

104-
* [YouTube Channel](https://www.youtube.com/channel/UCObqmpuSMx-usADtL_qdMAw)
104+
* [YouTube Channel](https://www.youtube.com/c/MediaPipe)
105105

106106
## Events
107107

@@ -123,7 +123,7 @@ run code search using
123123

124124
* [Awesome MediaPipe](https://mediapipe.org) - A curated list of awesome
125125
MediaPipe related frameworks, libraries and software
126-
* [Slack community](https://mediapipe.slack.com) for MediaPipe users
126+
* [Slack community](https://https://mediapipe.page.link/joinslack) for MediaPipe users
127127
* [Discuss](https://groups.google.com/forum/#!forum/mediapipe) - General
128128
community discussion around MediaPipe
129129

Diff for: docs/solutions/hands.md

+8-4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
layout: default
3-
title: Hand
3+
title: Hands
44
parent: Solutions
55
nav_order: 3
66
---
@@ -219,9 +219,13 @@ Please refer to [these instructions](../index.md#mediapipe-on-the-web).
219219

220220
## Resources
221221

222-
* Google AI Blog: [On-Device, Real-Time Hand Tracking with MediaPipe](https://ai.googleblog.com/2019/08/on-device-real-time-hand-tracking-with.html)
223-
* TensorFlow Blog: [Face and hand tracking in the browser with MediaPipe and
224-
TensorFlow.js](https://blog.tensorflow.org/2020/03/face-and-hand-tracking-in-browser-with-mediapipe-and-tensorflowjs.html)
222+
* Google AI Blog:
223+
[On-Device, Real-Time Hand Tracking with MediaPipe](https://ai.googleblog.com/2019/08/on-device-real-time-hand-tracking-with.html)
224+
* TensorFlow Blog:
225+
[Face and hand tracking in the browser with MediaPipe and TensorFlow.js](https://blog.tensorflow.org/2020/03/face-and-hand-tracking-in-browser-with-mediapipe-and-tensorflowjs.html)
226+
* Paper:
227+
[MediaPipe Hands: On-device Real-time Hand Tracking](https://arxiv.org/abs/2006.10214)
228+
([presentation](https://www.youtube.com/watch?v=I-UOrvxxXEk))
225229
* Palm detection model:
226230
[TFLite model](https://github.com/google/mediapipe/tree/master/mediapipe/models/palm_detection.tflite),
227231
[TF.js model](https://tfhub.dev/mediapipe/handdetector/1)

Diff for: docs/solutions/objectron.md

+3
Original file line numberDiff line numberDiff line change
@@ -188,5 +188,8 @@ to visualize its associated subgraphs, please see
188188
[Real-Time 3D Object Detection on Mobile Devices with MediaPipe](https://ai.googleblog.com/2020/03/real-time-3d-object-detection-on-mobile.html)
189189
* Paper: [MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak
190190
Shape Supervision](https://arxiv.org/abs/2003.03522)
191+
* Paper:
192+
[Instant 3D Object Tracking with Applications in Augmented Reality](https://drive.google.com/open?id=1O_zHmlgXIzAdKljp20U_JUkEHOGG52R8)
193+
([presentation](https://www.youtube.com/watch?v=9ndF1AIo7h0))
191194
* [TFLite model for shoes](https://github.com/google/mediapipe/tree/master/mediapipe/models/object_detection_3d_sneakers.tflite)
192195
* [TFLite model for chairs](https://github.com/google/mediapipe/tree/master/mediapipe/models/object_detection_3d_chair.tflite)

Diff for: docs/tools/tracing_and_profiling.md

+10-1
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,16 @@ available on Linux, Android, or iOS.
2121

2222
## Enabling tracing and profiling
2323

24-
To enable tracing/profiling of a mediapipe graph, the `CalculatorGraphConfig` (in
24+
To enable tracing and profiling of a mediapipe graph:
25+
26+
1. The profiling library must be linked to the framework.
27+
2. Tracing and profiling must be enabled in the graph configuration.
28+
29+
The profiling library is linked to the framework by default. If needed,
30+
the profiling library can be omitted from the framework using the bazel
31+
command line option: `--define MEDIAPIPE_PROFILING=0`.
32+
33+
To enable tracing and profiling, the `CalculatorGraphConfig` (in
2534
[calculator.proto](https://github.com/google/mediapipe/tree/master/mediapipe/framework/calculator.proto))
2635
representing the graph must have a `profiler_config` message at its root. Here
2736
is a simple setup that turns on a few extra options:

Diff for: mediapipe/calculators/image/bilateral_filter_calculator.cc

+1-1
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,7 @@ class BilateralFilterCalculator : public CalculatorBase {
107107
GLuint program_ = 0;
108108
GLuint vao_;
109109
GLuint vbo_[2]; // vertex storage
110-
#endif // !MEDIAPIPE_DISABLE_GPU
110+
#endif // !MEDIAPIPE_DISABLE_GPU
111111
};
112112
REGISTER_CALCULATOR(BilateralFilterCalculator);
113113

0 commit comments

Comments
 (0)