Skip to content

Fix tutorial for Qualcomm AI Engine Direct Backend #2956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kirklandsign wants to merge 2 commits into from
Closed

Fix tutorial for Qualcomm AI Engine Direct Backend #2956

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
687425d
Fix tutorial for Qualcomm AI Engine Direct Backend
kirklandsign Apr 9, 2024
b152bbc
Make minimal changes
kirklandsign Apr 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion backends/qualcomm/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ target_link_libraries(qnn_executorch_backend
qnn_executorch_header
qnn_schema
qnn_manager
executorch
executorch_no_prim_ops
Copy link
Contributor

@cccclai cccclai Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do we link it with prim ops lib?

Copy link
Contributor Author

@kirklandsign kirklandsign Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In

executorch/examples/qualcomm/CMakeLists.txt

Lines 102 to 108 in 687425d

target_link_libraries(qnn_executor_runner
qnn_executorch_backend
full_portable_ops_lib
etdump
${FLATCCRT_LIB}
gflags
)
, maybe full_portable_ops_lib has it… Otherwise we need to add executorch explicitly

For other users, add executorch explicitly to binary helps

Copy link
Contributor Author

@kirklandsign kirklandsign Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TLDR is for users who use qnn_executorch_backend as a dependency, it should also have executorch as a dependency now.

qcir_utils
)
target_link_libraries(utils
Expand Down
21 changes: 13 additions & 8 deletions docs/source/build-run-qualcomm-ai-engine-direct-backend.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,12 @@ Python APIs on x64 are required to compile models to Qualcomm AI Engine Direct b

```bash
cd $EXECUTORCH_ROOT
mkdir build_x86_64
cd build_x86_64
# Workaround for fbs files in exir/_serialize
cp schema/program.fbs exir/_serialize/program.fbs
cp schema/scalar_type.fbs exir/_serialize/scalar_type.fbs

mkdir cmake_out
cd cmake_out
cmake .. -DEXECUTORCH_BUILD_QNN=ON -DQNN_SDK_ROOT=${QNN_SDK_ROOT}
cmake --build . -t "PyQnnManagerAdaptor" "PyQnnWrapperAdaptor" -j8

Expand All @@ -134,12 +138,12 @@ Commands to build `qnn_executor_runner` for Android:

```bash
cd $EXECUTORCH_ROOT
mkdir build_android
cd build_android
mkdir cmake_android_out
cd cmake_android_out
Copy link
Contributor

@cccclai cccclai Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change is needed?

Copy link
Contributor Author

@kirklandsign kirklandsign Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can use old one. simply it's already in gitignore.

Copy link
Contributor

@cccclai cccclai Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use the old one probably? Try to make the change minimum

Copy link
Contributor Author

@kirklandsign kirklandsign Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

# build executorch & qnn_executorch_backend
cmake .. \
-DBUCK2=buck2 \
-DCMAKE_INSTALL_PREFIX=$PWD \
-DEXECUTORCH_BUILD_SDK=ON \
-DEXECUTORCH_BUILD_QNN=ON \
Copy link
Contributor

@chiwwang chiwwang Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly need something like CMAKE_INSTALL_PREFIX if we don't want to require root priviledge.
mmmm... would it be a good idea to leave a reminder here?

Copy link
Contributor

@chiwwang chiwwang Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

never mind! I saw -DCMAKE_INSTALL_PREFIX=$PWD
So this might be caused that the CMAKE_INSTALL_PREFIX is not propagated.

Copy link
Contributor Author

@kirklandsign kirklandsign Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this issue good now?

Copy link
Contributor

@chuntl chuntl Apr 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this issue good now?

Yes, it's good now.

-DQNN_SDK_ROOT=$QNN_SDK_ROOT \
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
Expand All @@ -162,7 +166,7 @@ cmake --build examples/qualcomm -j16

**Note:** If you want to build for release, add `-DCMAKE_BUILD_TYPE=Release` to the `cmake` command options.

You can find `qnn_executor_runner` under `build_android/examples/qualcomm/`.
You can find `qnn_executor_runner` under `cmake_android_out/examples/qualcomm/`.


## Deploying and running on device
Expand All @@ -174,7 +178,7 @@ We use deeplab-v3-resnet101 as an example in this tutorial. Run below commands t

```
cd $EXECUTORCH_ROOT
python -m examples.qualcomm.scripts.deeplab_v3 -b build_android -m SM8550 --compile_only --download
python -m examples.qualcomm.scripts.deeplab_v3 -b cmake_android_out -m SM8550 --compile_only --download
```

You might see something like below:
Expand Down Expand Up @@ -219,7 +223,8 @@ So, we can run `qnn_executor_runner` like

```bash
adb push ./deeplab_v3/dlv3_qnn.pte ${DEVICE_DIR}
adb push ${EXECUTORCH_ROOT}/build_android/examples/qualcomm/qnn_executor_runner ${DEVICE_DIR}
adb push ${EXECUTORCH_ROOT}/cmake_android_out/examples/qualcomm/qnn_executor_runner ${DEVICE_DIR}
adb push ${EXECUTORCH_ROOT}/cmake_android_out/lib/libqnn_executorch_backend.so ${DEVICE_DIR}
adb shell "cd ${DEVICE_DIR} \
&& export LD_LIBRARY_PATH=${DEVICE_DIR} \
&& export ADSP_LIBRARY_PATH=${DEVICE_DIR} \
Expand Down
1 change: 1 addition & 0 deletions examples/qualcomm/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ endif()
# portable_ops_lib, etdump, bundled_program.
find_package(executorch CONFIG REQUIRED)
target_compile_options(executorch INTERFACE -DET_EVENT_TRACER_ENABLED)
set(gflags_DIR ${CMAKE_CURRENT_BINARY_DIR}/../../third-party/gflags)
find_package(gflags REQUIRED)

set(_common_compile_options -Wno-deprecated-declarations -fPIC)
Expand Down
4 changes: 3 additions & 1 deletion examples/qualcomm/executor_runner/qnn_executor_runner.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,8 +202,10 @@ int main(int argc, char** argv) {
// be used by a single thread at at time, but it can be reused.
//
torch::executor::ETDumpGen etdump_gen = torch::executor::ETDumpGen();
// TODO: So far we have issues with etdump_gen during load_method. Enable it
// after the issues are fixed.
Result<Method> method =
program->load_method(method_name, &memory_manager, &etdump_gen);
program->load_method(method_name, &memory_manager, nullptr);
ET_CHECK_MSG(
method.ok(),
"Loading of method %s failed with status 0x%" PRIx32,
Expand Down