Qualcomm AI Engine Direct - Refactor QNN Manager

[shewu-quic]

Summary:

• Include the contextmanager QnnManagerContext and QnnManagerRegistry to handle the life cycle management of QnnManager in AOT.
• Introduce a singleton QnnBackendUnifiedRegistry to manage shared QNN components.
• Separate the QnnManager Init function into InitBackend and InitContext to prevent creating the same backend and device for each context in AOT and runtime.
  • InitBackend should initialize shared QNN components such as QnnBackend, QnnDevice, QnnLogger, and the QNN library.
  • InitContext will be responsible for setting up QnnContext and QnnGraph.
• Resolved the outdated link in the documentation

Background

• It takes some time to re-initialize the QNN components for both qnn_partitioner and qnn_preprocess in AOT.
  • We currently create and initialize a QnnManager in the qnn_partitioner, and then repeat this process for each partition in the qnn_preprocess. This could be improved by using a single QnnManager for both qnn_partitioner and qnn_preprocess to enhance lowering time.
• It takes some fixed time to do QNN component initialization during QNN ExecuTorch runtime, especially to QNN device API.
  • We can use static initialization of the QNN device to improve model loading times when the model is initialized frequently.

Goals and Objectives

• In AOT, make sure to load the QNN library and configure the QNN Backend and Device, ensuring these are shared between both qnn_partitioner and qnn_preprocess.
• During runtime, we can create one backend and device once in QnnExecutorchBackend and sharing for all partitions and multi-methods

cc: @haowhsu-quic

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15903

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 11 New Failures, 2 Unrelated Failures

As of commit 31dbcca with merge base aff5086 ():

NEW FAILURES - The following jobs have failed:

• pull / android / build-llm-demo / linux-job (gh)
  RuntimeError: Command docker exec -t fc32aae357ed6cfcca7a84cb4d3152c719112d189df6201cb289439ac38ac403 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 10f29e3b092c8bb557f46f3d481b44bf69f9382d04eaaedccec81fd0f0482e56 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 28e3c40ba129c5269bcf41343b60ee48c38173a46ee9ac08c1cdac5dc8ff949f /exec failed with exit code 92
• pull / test-models-linux (linear, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 17ac2a6313caec11704481cc7af6c47450d686999ca59075ea88790d62973a6b /exec failed with exit code 1
• pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t ef315dac96e9706eb55f78c8b9befdd1b18f0206d4f3fdd48ecda54b16932854 /exec failed with exit code 1
• pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
  RuntimeError: Command docker exec -t 21a8279f948447b478dd713cd37106e2192f66a4452d7beee097bce8549541e7 /exec failed with exit code 1
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
  RuntimeError: Command docker exec -t 1eaaa6857668738f4fca1b856f40ba9bd5fd6a7e0fdabd3405e0c47985bf5cb2 /exec failed with exit code 1
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t dcf69f8f2b72ef5b0624161d26a7a962358db376ffd76a5858988a8b048b3a4a /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 22db8244a1eeb15127af9f74acb0e6c28fca34db90d3588ac08f69448831bc37 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t 5175805a1033e9cbc903ddf0f4fc2483d858f9c41359eb15eeb015849165da97 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t c94e468f524fd9226e36c2c35427362e65f5cd581f72cd1f589c3d96fa0650ee /exec failed with exit code 1

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / unittest / linux / linux-job (gh) (similar failure)
  exir/backend/test/test_lowered_backend_module.py::TestBackendAPI::test_emit_nested_lowered_backend_module

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest-editable / linux / linux-job (gh) (trunk failure)
  exir/backend/test/test_lowered_backend_module.py::TestBackendAPI::test_emit_nested_lowered_backend_module

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[shewu-quic]

@pytorchbot label "release notes: qualcomm"