Skip to content

Conversation

@chraac
Copy link
Owner

@chraac chraac commented Feb 20, 2025

Closes #22

Here we will add windows platform support to qnn backend

Build with msvc 2022

  1. Download Qualcomm AI Engine Direct SDK, from here, and then extract it into a folder

  2. Install the latest Visual Studio, make sure clang toolchain and 'cmake' are installed
    Image
    Image

  3. Launch vs2022, tap Continue without code, and then in File menu, select Open -> CMake, in file open dialog, navigate to llama.cpp root directory, select CMakeLists.txt

  4. Edit llama.cpp\CMakePresets.json, add following line to config arm64-windows-llvm

    {
        "name": "arm64-windows-llvm", "hidden": true,
        "architecture": { "value": "arm64",    "strategy": "external" },
        "toolset":      { "value": "host=x64", "strategy": "external" },
        "cacheVariables": {
    -        "CMAKE_TOOLCHAIN_FILE": "${sourceDir}/cmake/arm64-windows-llvm.cmake"
    +        "CMAKE_TOOLCHAIN_FILE": "${sourceDir}/cmake/arm64-windows-llvm.cmake",
    +        "GGML_QNN": "ON",
    +        "GGML_QNN_SDK_PATH": "<path to your Qualcomm AI Engine Direct SDK, like x:/ml/qnn_sdk/qairt/2.31.0.250130/>",
    +        "BUILD_SHARED_LIBS": "OFF"
        }
    },
  5. Select config arm64-windows-llvm-debug
    Image

  6. In Build menu, tap Build All, output file are located at build-arm64-windows-llvm-debug\bin\

  7. Please copy those files from sdk lib directory (at qairt\2.31.0.250130\lib\aarch64-windows-msvc\) to the output directory before run:

QnnSystem.dll
QnnCpu.dll
QnnGpu.dll
QnnHtp.dll
QnnHtp*.dll

Tests

image

#22 (comment)

For detailed build instructions, please refer to #22 (comment)

@chraac chraac added enhancement New feature or request windows labels Feb 20, 2025
@chraac chraac self-assigned this Feb 20, 2025
@chraac chraac linked an issue Feb 20, 2025 that may be closed by this pull request
4 tasks
@chraac chraac marked this pull request as draft February 20, 2025 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request windows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat] Port to windows

1 participant