Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions .github/scripts/export-qnn/Dockerfile-2.40
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
FROM ubuntu:22.04

ARG QNN_SDK_VERSION=2.40.0.251030
ARG QNN_TOOLKIT_URL=https://huggingface.co/csukuangfj/qnn-toolkit/resolve/main/v${QNN_SDK_VERSION}.zip

ENV DEBIAN_FRONTEND=noninteractive

# Use bash as default shell (envsetup.sh requires bash syntax)
SHELL ["/bin/bash", "-c"]

# Install basic system dependencies (Ubuntu 22.04 ships Python 3.10 natively)
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
curl \
unzip \
software-properties-common \
lsb-release \
adb \
clang \
libc++-dev \
libc++abi-dev \
libclang-dev \
libsndfile1 \
python3 \
python3-pip \
python3-venv \
python3-dev \
&& rm -rf /var/lib/apt/lists/*

# Install Android NDK r29 (placed early so failures surface quickly)
ARG ANDROID_NDK_VERSION=29.0.14206865
ARG ANDROID_NDK_REVISION=r29
RUN curl -SL -o /tmp/ndk.zip https://dl.google.com/android/repository/android-ndk-${ANDROID_NDK_REVISION}-linux.zip \
&& unzip -q /tmp/ndk.zip -d /opt \
&& rm -f /tmp/ndk.zip
ENV ANDROID_NDK_LATEST_HOME=/opt/android-ndk-${ANDROID_NDK_REVISION}
ENV PATH=${ANDROID_NDK_LATEST_HOME}:${PATH}

# Verify NDK is set up correctly
RUN ndk-build --help

# Download, install QNN SDK toolkit, then remove the zip in the same layer
WORKDIR /opt
RUN curl -SL -O ${QNN_TOOLKIT_URL} \
&& unzip v${QNN_SDK_VERSION}.zip \
&& rm -f v${QNN_SDK_VERSION}.zip

# Set QNN SDK environment variables
ENV QNN_SDK_ROOT=/opt/qairt/${QNN_SDK_VERSION}
ENV PATH=${QNN_SDK_ROOT}/bin/x86_64-linux-clang:${QNN_SDK_ROOT}/bin:${PATH}
ENV LD_LIBRARY_PATH=${QNN_SDK_ROOT}/lib/x86_64-linux-clang:${LD_LIBRARY_PATH}

# Install linux dependencies required by QNN SDK (source envsetup.sh first, matching workflow)
RUN cd ${QNN_SDK_ROOT}/bin && . envsetup.sh \
&& yes | ${QNN_SDK_ROOT}/bin/check-linux-dependency.sh || true \
&& rm -rf /var/lib/apt/lists/*
Comment on lines +54 to +56

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Bash Operator Precedence & Missing Package Lists\n\nThere are two issues in this step:\n1. Operator Precedence Pitfall: In Bash, && and || have equal precedence and are left-associative. The expression A && B && C || true && D is evaluated as (((A && B && C) || true) && D). If cd or source envsetup.sh fails, the chain A && B && C fails, but the || true evaluates to true, and the shell proceeds to run D (rm -rf ...). This silently ignores failures in sourcing envsetup.sh.\n2. Missing Package Lists: The check-linux-dependency.sh script runs apt-get install under the hood. However, since /var/lib/apt/lists/* was deleted in the previous layer, apt-get will fail to locate packages. Running apt-get update before the script ensures dependencies can be successfully installed.\n\nTo fix both, wrap the dependency check in { ... || true; } and run apt-get update beforehand.

RUN apt-get update && cd ${QNN_SDK_ROOT}/bin && . envsetup.sh \\n    && { yes | ${QNN_SDK_ROOT}/bin/check-linux-dependency.sh || true; } \\n    && rm -rf /var/lib/apt/lists/*


# Create and activate a Python 3.10 virtual environment
RUN python3 -m venv /opt/py310
ENV VIRTUAL_ENV=/opt/py310
ENV PATH=/opt/py310/bin:${PATH}

# Install all Python dependencies in a single layer to minimize image size
# The virtual environment is already active via ENV PATH above
RUN . /opt/py310/bin/activate && \
pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir \
mock \
numpy \
opencv-python \
optuna \
packaging \
pandas \
paramiko \
pathlib2 \
pillow \
plotly \
protobuf \
psutil \
pydantic \
pytest \
pyyaml \
rich \
scikit-optimize \
scipy \
six \
tabulate \
typing-extensions \
xlsxwriter \
&& python3 "${QNN_SDK_ROOT}/bin/check-python-dependency" || true \
&& pip install --no-cache-dir \
torch==2.0.0+cpu -f https://download.pytorch.org/whl/torch \
kaldi_native_fbank \
"numpy<2" \
onnx==1.17.0 \
onnxruntime \
soundfile \
librosa \
onnxsim \
sentencepiece \
pyyaml \
&& pip cache purge 2>/dev/null || true
Comment on lines +65 to +102

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Silent Installation Failures & Duplicate Dependencies\n\nThere are two main issues in this block:\n1. Silent Failures due to Operator Precedence: In Bash, && and || have equal precedence and are left-associative. The expression A && B || true && C && D || true is evaluated as ((((A && B) || true) && C) && D) || true. If the first pip install fails, the || true evaluates to true, and the shell proceeds to the second pip install. More critically, if the second pip install (which installs major packages like PyTorch and ONNX) fails, the trailing || true causes the entire RUN instruction to succeed, silently producing a broken Docker image.\n2. Duplicate Dependencies: pyyaml is listed twice (lines 82 and 101). numpy is also listed twice (once as numpy on line 69 and once as \"numpy<2\" on line 94), which causes pip to install the latest numpy first and then downgrade it later.\n\nGrouping the commands with { ... || true; } and cleaning up the duplicates resolves these issues.

RUN . /opt/py310/bin/activate && \\n    pip install --no-cache-dir --upgrade pip && \\n    pip install --no-cache-dir \\n    mock \\n    \"numpy<2\" \\n    opencv-python \\n    optuna \\n    packaging \\n    pandas \\n    paramiko \\n    pathlib2 \\n    pillow \\n    plotly \\n    protobuf \\n    psutil \\n    pydantic \\n    pytest \\n    pyyaml \\n    rich \\n    scikit-optimize \\n    scipy \\n    six \\n    tabulate \\n    typing-extensions \\n    xlsxwriter && \\n    { python3 \"${QNN_SDK_ROOT}/bin/check-python-dependency\" || true; } && \\n    pip install --no-cache-dir \\n    torch==2.0.0+cpu -f https://download.pytorch.org/whl/torch \\n    kaldi_native_fbank \\n    onnx==1.17.0 \\n    onnxruntime \\n    soundfile \\n    librosa \\n    onnxsim \\n    sentencepiece && \\n    { pip cache purge 2>/dev/null || true; }


# Verify QNN tools are accessible (source envsetup.sh first, matching workflow)
RUN cd ${QNN_SDK_ROOT}/bin && . envsetup.sh \
&& qnn-onnx-converter --help \
&& qnn-model-lib-generator --help \
&& qnn-net-run --help

WORKDIR /workspace
46 changes: 46 additions & 0 deletions .github/workflows/qnn-sdk-docker-build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Build docker image for qnn SDK

on:
push:
branches:
- docker-qnn-2
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false

permissions:
contents: read
packages: write

steps:
- name: Checkout repository
uses: actions/checkout@v4

# Buildx is required for cache + multi-platform correctness
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
uses: docker/build-push-action@v5
with:
context: ./.github/scripts/export-qnn
file: ./.github/scripts/export-qnn/Dockerfile-2.40
platforms: linux/amd64
push: true
tags: |
ghcr.io/${{ github.repository_owner }}/qnn-sdk:2.40
cache-from: type=gha
cache-to: type=gha,mode=max
70 changes: 70 additions & 0 deletions .github/workflows/test-qnn-sdk-docker.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
name: test-qnn-sdk-docker

on:
push:
branches:
- docker-qnn
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
container:
image: ghcr.io/${{ github.repository_owner }}/qnn-sdk:2.40

steps:
- uses: actions/checkout@v4

- name: Show system information
shell: bash
run: |
df -h

pwd
id
cat /etc/*version
ls -lh

- name: Show Python version
shell: bash
run: |
python3 --version
which python3

- name: Show environment variables
shell: bash
run: |
echo "ANDROID_NDK_LATEST_HOME: $ANDROID_NDK_LATEST_HOME"
echo "PATH: $PATH"
echo "LD_LIBRARY_PATH: $LD_LIBRARY_PATH"
echo "QNN_SDK_ROOT: $QNN_SDK_ROOT"

- name: qnn-onnx-converter show help
shell: bash
run: |
pushd $QNN_SDK_ROOT/bin
source envsetup.sh
popd

which qnn-onnx-converter
qnn-onnx-converter --help

- name: qnn-model-lib-generator help
shell: bash
run: |
pushd $QNN_SDK_ROOT/bin
source envsetup.sh
popd

which qnn-model-lib-generator
echo "qnn-model-lib-generator --help"

- name: qnn-net-run help
shell: bash
run: |
pushd $QNN_SDK_ROOT/bin
source envsetup.sh
popd

which qnn-net-run
qnn-net-run --help