vcpkg_find_acquire_program does not respect VCPKG_FORCE_SYSTEM_BINARIES for shared dependencies like Ninja

[chao-sun01io]

Host Environment

• OS: Ubuntu on Jetson Nx (arm-linux)
• Compiler: g++7

To Reproduce
I tried to install qt5-base with manifest file vcpkg.json on arm-linux platform (Nvidia Jetson Nx),but failed to install harfbuzz (I think it is a dep of qt5?)

Steps to reproduce the behavior:
./vcpkg install harfbuzz

Failure logs

[cmake] Building package harfbuzz[core]:arm64-linux...
[cmake] -- Using community triplet arm64-linux. This triplet configuration is not guaranteed to succeed.
[cmake] -- [COMMUNITY] Loading triplet configuration from: /home/user1/vcpkg/triplets/community/arm64-linux.cmake
[cmake] -- Using cached harfbuzz-harfbuzz-3.0.0.tar.gz.
[cmake] -- Cleaning sources at /home/user1/vcpkg/buildtrees/harfbuzz/src/3.0.0-685ae48a4e.clean. Use --editable to skip cleaning for the packages you specify.
[cmake] -- Extracting source /home/user1/vcpkg/downloads/harfbuzz-harfbuzz-3.0.0.tar.gz
[cmake] -- Applying patch 0001-circumvent-samefile-error.patch
[cmake] -- Applying patch 0002-fix-uwp-build.patch
[cmake] -- Using source at /home/user1/vcpkg/buildtrees/harfbuzz/src/3.0.0-685ae48a4e.clean
[cmake] -- Getting CMake variables for arm64-linux-dbg
[cmake] -- Getting CMake variables for arm64-linux-rel
[cmake] -- Configuring arm64-linux-dbg
[cmake] CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:127 (message):
[cmake]     Command failed: /usr/bin/python3 /home/user1/vcpkg/downloads/tools/meson/meson-aeda7f249c4a5dbbecc52e44f382246a2377b5b0/meson.py -Dicu=disabled -Dgraphite=disabled -Dcoretext=disabled -Dglib=disabled -Dgobject=disabled -Dfreetype=enabled -Dcairo=disabled -Dintrospection=disabled -Ddocs=disabled -Dtests=disabled -Dbenchmark=disabled --buildtype plain --backend ninja --wrap-mode nodownload --native /home/user1/vcpkg/buildtrees/harfbuzz/meson-native-arm64-linux.log --default-library static --libdir lib --native /home/user1/vcpkg/buildtrees/harfbuzz/meson-native-arm64-linux-debug.log -Ddebug=true --prefix /home/user1/vcpkg/packages/harfbuzz_arm64-linux/debug --includedir ../include -Dcmake_prefix_path=['/home/user1/aidi_vision_v2/out/build/Nx-arm-gcc-Debug/vcpkg_installed/arm64-linux/debug','/home/user1/aidi_vision_v2/out/build/Nx-arm-gcc-Debug/vcpkg_installed/arm64-linux'] /home/user1/vcpkg/buildtrees/harfbuzz/src/3.0.0-685ae48a4e.clean
[cmake]     Working Directory: /home/user1/vcpkg/buildtrees/harfbuzz/arm64-linux-dbg
[cmake]     Error code: 2
[cmake]     See logs for more information:
[cmake]       /home/user1/vcpkg/buildtrees/harfbuzz/config-arm64-linux-dbg-out.log
[cmake]       /home/user1/vcpkg/buildtrees/harfbuzz/config-arm64-linux-dbg-err.log
[cmake] 
[cmake] Call Stack (most recent call first):
[cmake]   scripts/cmake/vcpkg_configure_meson.cmake:512 (vcpkg_execute_required_process)
[cmake]   ports/harfbuzz/portfile.cmake:48 (vcpkg_configure_meson)
[cmake]   scripts/ports.cmake:142 (include)

[cmake] Error: Building package harfbuzz:arm64-linux failed with: BUILD_FAILED
[cmake] Please ensure you're using the latest portfiles with `git pull` and `./vcpkg update`, then
[cmake] submit an issue at https://github.com/Microsoft/vcpkg/issues including:
[cmake]   package: harfbuzz[core]:arm64-linux -> 3.0.0
[cmake]   vcpkg version: 2021-11-02-unknownhash
[cmake]   vcpkg-tool version: a4b5cde7f 2021-11-16 (42 minutes ago)

/home/user1/vcpkg/buildtrees/harfbuzz/config-arm64-linux-dbg-err.log

Traceback (most recent call last):
  File "/home/user1/vcpkg/downloads/tools/meson/meson-aeda7f249c4a5dbbecc52e44f382246a2377b5b0/mesonbuild/mesonmain.py", line 134, in run
    return options.run_func(options)
  File "/home/user1/vcpkg/downloads/tools/meson/meson-aeda7f249c4a5dbbecc52e44f382246a2377b5b0/mesonbuild/msetup.py", line 281, in run
    app.generate()
  File "/home/user1/vcpkg/downloads/tools/meson/meson-aeda7f249c4a5dbbecc52e44f382246a2377b5b0/mesonbuild/msetup.py", line 184, in generate
    self._generate(env)
  File "/home/user1/vcpkg/downloads/tools/meson/meson-aeda7f249c4a5dbbecc52e44f382246a2377b5b0/mesonbuild/msetup.py", line 246, in _generate
    intr.backend.generate()
  File "/home/user1/vcpkg/downloads/tools/meson/meson-aeda7f249c4a5dbbecc52e44f382246a2377b5b0/mesonbuild/backend/ninjabackend.py", line 509, in generate
    ninja = environment.detect_ninja_command_and_version(log=True)
  File "/home/user1/vcpkg/downloads/tools/meson/meson-aeda7f249c4a5dbbecc52e44f382246a2377b5b0/mesonbuild/environment.py", line 220, in detect_ninja_command_and_version
    p, found = Popen_safe(prog.command + ['--version'])[0:2]
  File "/home/user1/vcpkg/downloads/tools/meson/meson-aeda7f249c4a5dbbecc52e44f382246a2377b5b0/mesonbuild/mesonlib/universal.py", line 1330, in Popen_safe
    stdout=stdout, stderr=stderr, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/home/user1/vcpkg/downloads/tools/ninja/1.10.2-linux/ninja'

[JackBoosY]

Please provide the out log /home/user1/vcpkg/buildtrees/harfbuzz/config-arm64-linux-dbg-out.log.

[chao-sun01io]

@JackBoosY /home/user1/vcpkg/buildtrees/harfbuzz/config-arm64-linux-dbg-out.log

DEPRECATION: c_args in the [properties] section of the machine file is deprecated, use the [built-in options] section.
DEPRECATION: cpp_args in the [properties] section of the machine file is deprecated, use the [built-in options] section.
DEPRECATION: c_link_args in the [properties] section of the machine file is deprecated, use the [built-in options] section.
DEPRECATION: cpp_link_args in the [properties] section of the machine file is deprecated, use the [built-in options] section.
WARNING: Recommend using either -Dbuildtype or -Doptimization + -Ddebug. Using both is redundant since they override each other. See: https://mesonbuild.com/Builtin-options.html#build-type-options
The Meson build system
Version: 0.58.1
Source dir: /home/user1/vcpkg/buildtrees/harfbuzz/src/3.0.0-685ae48a4e.clean
Build dir: /home/user1/vcpkg/buildtrees/harfbuzz/arm64-linux-dbg
Build type: native build
Project name: harfbuzz
Project version: 3.0.0
C compiler for the host machine: /usr/bin/cc (gcc 7.5.0 "cc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0")
C linker for the host machine: /usr/bin/cc ld.bfd 2.30
C++ compiler for the host machine: /usr/bin/c++ (gcc 7.5.0 "c++ (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0")
C++ linker for the host machine: /usr/bin/c++ ld.bfd 2.30
Host machine cpu family: aarch64
Host machine cpu: aarch64
Compiler for C++ supports link arguments -Bsymbolic-functions: YES 
Compiler for C++ supports arguments -fno-exceptions: YES 
Compiler for C++ supports arguments -fno-rtti: YES 
Compiler for C++ supports arguments -fno-threadsafe-statics: YES 
Compiler for C++ supports arguments -fvisibility-inlines-hidden: YES 
Library m found: YES
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Run-time dependency freetype2 found: YES 24.0.18
Dependency glib-2.0 skipped: feature glib disabled
Dependency gobject-2.0 skipped: feature gobject disabled
Dependency graphite2 skipped: feature graphite disabled
Found CMake: /usr/local/bin/cmake (3.21.4)
Run-time dependency chafa found: NO (tried pkgconfig and cmake)
Compiler for C++ supports arguments -Wno-non-virtual-dtor: YES 
Run-time dependency threads found: YES
Has header "unistd.h" : YES 
Has header "sys/mman.h" : YES 
Has header "stdbool.h" : YES 
Checking for function "atexit" : YES 
Checking for function "mprotect" : YES 
Checking for function "sysconf" : YES 
Checking for function "getpagesize" : YES 
Checking for function "mmap" : YES 
Checking for function "isatty" : YES 
Checking for function "FT_Get_Var_Blend_Coordinates" with dependency freetype2: YES 
Checking for function "FT_Set_Var_Blend_Coordinates" with dependency freetype2: YES 
Checking for function "FT_Done_MM_Var" with dependency freetype2: YES 
Program gen-hb-version.py found: YES (/home/user1/vcpkg/buildtrees/harfbuzz/src/3.0.0-685ae48a4e.clean/src/gen-hb-version.py)
Configuring hb-version.h with command
Program ragel found: NO
../src/3.0.0-685ae48a4e.clean/src/meson.build:300: WARNING: You have to install ragel if you are going to develop HarfBuzz itself
Program gen-harfbuzzcc.py found: YES (/home/user1/vcpkg/buildtrees/harfbuzz/src/3.0.0-685ae48a4e.clean/src/gen-harfbuzzcc.py)
Program gen-def.py found: YES (/home/user1/vcpkg/buildtrees/harfbuzz/src/3.0.0-685ae48a4e.clean/src/gen-def.py)
Configuring harfbuzz-config.cmake using configuration
Configuring config.h using configuration
Build targets in project: 5

harfbuzz 3.0.0

  Additional shapers
    Graphite2        : NO

  Dependencies used for command-line utilities
    Cairo            : NO
    Chafa            : NO

  Directories
    prefix           : /home/user1/vcpkg/packages/harfbuzz_arm64-linux/debug
    bindir           : bin
    libdir           : lib
    includedir       : ../include
    datadir          : share

  Font callbacks (the more the merrier)
    FreeType         : YES

  Other features
    Documentation    : NO
    GObject bindings : NO
    Introspection    : NO
    Experimental APIs: NO

  Platform shapers (not normally needed)
    CoreText         : NO
    DirectWrite      : NO
    GDI/Uniscribe    : NO

  Testing
    Tests            : NO
    Benchmark        : NO

  Unicode callbacks (you want at least one)
    Builtin          : YES
    Glib             : NO
    ICU              : NO

Option wrap_mode is: nodownload [default: nofallback]

[chao-sun01io]

I think ninja is not a arm-build version in /home/user1/vcpkg/downloads/tools/ninja/1.10.2-linux:

$ file ninja
ninja: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 2.6.32, BuildID[sha1]=3c1def4659e39fa9a2870be5e78feea4a96e8bc3, stripped

But I don't know how to repair.

[chao-sun01io]

I think I found the reason: my default ninja in the system(/usr/bin/ninja) installed by apt is 1.8.x while this package requires>1.10, vcpkg choose to use its tools installed in $VCPKG_ROOT/download/tools/ninja , but it is a x86-64 built executable file while I am in arm-Linux.
I can work around it by building an arm-version of ninja from the source then replacing the older version in my OS(/usr/bin/ninja) and deleting the wrong architecture in $VCPKG_ROOT/download/tools/ninja.

Maybe we need to be careful if vcpkg sometimes downloads software with the wrong platform architecture。Maybe it is a bug of vcpkg.
@JackBoosY

[JackBoosY]

vcpkg uses ninja release version which provided by https://github.com/ninja-build/ninja.
This issue should be reported earlier if it's a bug.
Can you please manually run command /home/user1/vcpkg/downloads/tools/ninja/1.10.2-linux/ninja and provide the result?

[chao-sun01io]

vcpkg uses ninja release version which provided by the upstream. This issue should be reported earlier if it's a bug. Can you please manually run command /home/user1/vcpkg/downloads/tools/ninja/1.10.2-linux/ninja and provide the result?

$ ./ninja
/ninja: cannot execute binary file: Exec format error
$ file ninja
ninja: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 2.6.32, BuildID[sha1]=3c1def4659e39fa9a2870be5e78feea4a96e8bc3, stripped

It is a x86-64 executable while on an arm-linux
@JackBoosY

[JackBoosY]

cc @BillyONeal, should we handle this issue and how can we handle it?

[BillyONeal]

Maybe we need to be careful if vcpkg sometimes downloads software with the wrong platform architecture。Maybe it is a bug of vcpkg.

It's less a bug and more that vcpkg never had an architecture selection feature for dependencies like this at all. First class support for arm64-linux is a thing we want to do but do not have right now.

[AlexLeungZ]

I have the same issue, here is the work around for that.

Host Environment

• OS: Ubuntu 20.04 on Jetson Xavier NX aarch64
• Compiler: gcc11 g++11

1. Build ninja from source (https://github.com/ninja-build/ninja/releases)

wget https://github.com/ninja-build/ninja/archive/refs/tags/v1.10.2.tar.gz
unpigz v1.10.2.tar.gz
tar -xvf v1.10.2.tar
cd ninja-1.10.2
./configure.py --bootstrap

2. Copy the local built ninja to bin

sudo cp ninja /usr/bin/ninja
sudo cp /usr/bin/ninja /usr/local/bin/
sudo update-alternatives --install /usr/bin/ninja ninja /usr/local/bin/ninja 1 --force

3. Check version

ninja --version

4. Replace ninja in vcpkg/dl/tool with the one in bin

 cp /usr/local/bin/ninja .../vcpkg/downloads/tools/ninja/1.10.2-linux/ninja

Result

[build] Installing package harfbuzz[core]:arm64-linux...
[build] Elapsed time for package harfbuzz:arm64-linux: 5.359 min

[JackBoosY]

The ninja upstream should provide arm-linux pre-build binary first.

[vserdyuk]

I ran into the same issue when building pkgconf on armhf Ubuntu 20.04.
I think the actual issue here is that vcpkg_find_acquire_program function does not respect VCPKG_FORCE_SYSTEM_BINARIES environment variable. Why does vcpkg download wrong architecture Ninja binary when I have perfectly fine binary from my distro repository?

[BillyONeal]

vcpkg_find_acquire_program function does not respect VCPKG_FORCE_SYSTEM_BINARIES environment variable

I agree this is ultimately the problem. It won't fix the case where there's 1.8 available and the port needs 1.10, but we should at least get it to the point for that to happen.

[JackBoosY]

If we want to use the system library in these cmake scripts, we should first unfilter the environment variable PATH when VCPKG_FORCE_SYSTEM_BINARIES is defined, which also affects the entire build system.

[BillyONeal]

Rather than passing through the environment variable, the right thing is to power vcpkg_find_acquire_program with vcpkg fetch to unify the tree's access to this stuff rather than implementing it 3 separate and incompatible times.

[kambala-decapitator]

faced the same issue on 32-bit windows (prebuilt ninja is available only for x64), applying the same fix as provided by @AlexLeungZ helped

[rcoup]

there's a CI job for linux/arm64 upstream now, so I presume new upstreams releases will have binaries available.

[KUGA2]

I have the same issue as stated in headline. However discussion here drifted very much into Ninja, so tell me if I should create a new Issue.

My example ist the package/port nanopb that has vcpkg_find_acquire_program(PYTHON3).

On the build agent we have python installed (but an older version) so vcpkg decides to download it:
-- Downloading https://www.python.org/ftp/python/3.10.7/python-3.10.7-embed-amd64.zip -> python-3.10.7-embed-amd64.zip...

In my opinion this should not happen because I set VCPKG_FORCE_SYSTEM_BINARIES=1! It should stop with an error (Incorrect python version) or try the old python version.

A quick look at the code (vcpkg_find_acquire_program.cmake) shows, that it calls vcpkg_download_distfile() without checking the environment variable. vcpkg_download_distfile() again does not check it (but it should not as this function is also used for downloading sources).

[github-actions]

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.

[KUGA2]

keep open.

[github-actions]

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.