Fix building where posix_spawn_file_actions_addchdir_np is unavailable - #104
Conversation
posix_spawn_file_actions_addchdir_np arrived in glibc 2.29, so subprocess_create_ex fails to compile and link on older systems such as manylinux2014. Add a SUBPROCESS_HAVE_CWD probe, report a requested cwd as ENOSYS where the call is unavailable, and skip the cwd test there.
musl gained posix_spawn_file_actions_addchdir_np in 1.1.24 but exposes no version macro, so the detection cannot cover older releases. Skip it when the macro is already defined, letting those users set it themselves.
Refresh patch 0009 to match sheredom/subprocess.h#104 at 620ce44, which added the overridable SUBPROCESS_HAVE_CWD detection. Verified to apply cleanly to the vendored header and to reproduce that head byte for byte, so the applier will skip it once the pin is bumped. Add the upstream-facing write-up for patch 0001, including a reproducer that makes llama.cpp's own test-arg-parser fail on unmodified master. Assisted-by: Claude Opus 5
|
@bernardladenthin We also need this for macOS: #108 |
Drops patch 0009 (subprocess.h old-glibc guard): upstream merged the exact fix it submitted (sheredom/subprocess.h#104) via #26606, so the patch is now redundant and its context no longer matches after a neighboring Windows argv-quoting rewrite (confirmed fail-loud "does not apply cleanly" before removing it). Also fixes a real break carried over unbuilt from the b10270->b10275 step (that step was verified configure-only, so a full build never caught it until this checkpoint): eval_llama_cmpl_schema (tools/server/server-schema.h, included directly by jllama.cpp) dropped its n_ctx_slot parameter at b10275, along with the -1=ctx-size sentinel for penalty_last_n/dry_penalty_last_n (hard lower limit moved -1 -> 0). jllama.cpp's populate_completion_task drops its now-unused n_ctx_slot parameter; test_server.cpp's parse_params helper drops its n_ctx parameter and the two *_ExpandsToNCtxSlot tests become *_MinusOne_Throws (a request-supplied -1 is now out-of-range instead of expanding). Added tools/server/server-schema.h to CLAUDE.md's priority-ordered API-compat review list so a directly-included tools/server/*.h header isn't missed again by future bump steps. Full local verification (checkpoint): fresh configure + full cmake --build (libjllama + jllama_test link cleanly) + ctest 482/482 passing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QyzMNtQf878jjtLc6KCxVv
posix_spawn_file_actions_addchdir_np arrived in macOS 10.15, so older SDKs neither declare nor export it and subprocess_create_ex fails to compile, as reported for MacPorts on PowerPC in sheredom#108. A newer SDK paired with an older deployment target does compile, but warns and leaves the binary referencing a symbol the target system does not have. Extend the SUBPROCESS_HAVE_CWD probe with a Darwin arm. It deliberately omits a defined() guard on MAC_OS_X_VERSION_MIN_REQUIRED: an absent macro folds to 0, so a toolchain that does not supply it answers "no cwd" rather than calling a function that may not be there. That also gives the right answer on iOS, tvOS and watchOS, where the SDKs mark the call unavailable outright.
done 😃 |
|
@barracuda156 Thanks for the pointer and #108 is now handled in this PR. The probe gained a Darwin arm: #elif defined(__APPLE__) && MAC_OS_X_VERSION_MIN_REQUIRED < 101500
#define SUBPROCESS_HAVE_CWD 0Apple's SDKs put the boundary at exactly 10.15, and mark the call unavailable on iOS, int posix_spawn_file_actions_addchdir_np(posix_spawn_file_actions_t *,
const char * __restrict) __API_AVAILABLE(macos(10.15)) __API_UNAVAILABLE(ios, tvos, watchos);Where the capability is missing, a requested There is deliberately no That last part is the one thing I could not verify. I tested against real 10.14, 10.15 Two questions, if you can spare the time:
Either way |
| #endif | ||
| if (0 != posix_error) { | ||
| saved_errno = posix_error; | ||
| result = subprocess_error_from_errno(posix_error); |
There was a problem hiding this comment.
Don't we need to make subprocess_error_from_errno handle ENOSYS too?
|
Your LLM should be made to be succinct, that is a wall of text with about 1/100 useful words. |
| mark it unavailable on iOS, tvOS and watchOS, where the undefined version | ||
| macro folds to 0 and so answers correctly. Define this yourself to override | ||
| the detection, for instance on musl older than 1.1.24. */ | ||
| #if !defined(SUBPROCESS_HAVE_CWD) |
There was a problem hiding this comment.
We don't need this new define - just add it inline where you set ENOSYS.
There was a problem hiding this comment.
Changed my mind actually - you are right to add the define for the testing too. I generally dislike defines s'all!
|
Is there anyway we can test this in CI? I'd love not to regress y'all in future! |
|
On length: I do prefer recording what was tested, why, and what wasn't - it saves the On the override: it isn't there because of how I tested, it's there because musl exposes On CI: happy to set it up if you want it. Two things are straightforward:
What can't be done on hosted runners is the pre-10.15 SDK case, since that needs SDKs I kept this patch as small as possible on purpose; CI would be a separate PR. |
Merging main after sheredom#104 dropped the SUBPROCESS_SPAWN_REPORTS_EXEC_ERRORS block while keeping its use site, so the macro was undefined and folded to 0. That turned the access() pre-check on unconditionally, on every platform, which is the opposite of what this change intends.
|
Would happily accept a CI patch to test more things for sure. I just don't want to inadvertently break anyone if I can help it! |
* Fix building against glibc older than 2.29 posix_spawn_file_actions_addchdir_np arrived in glibc 2.29, so subprocess_create_ex fails to compile and link on older systems such as manylinux2014. Add a SUBPROCESS_HAVE_CWD probe, report a requested cwd as ENOSYS where the call is unavailable, and skip the cwd test there. * Allow overriding SUBPROCESS_HAVE_CWD and document it musl gained posix_spawn_file_actions_addchdir_np in 1.1.24 but exposes no version macro, so the detection cannot cover older releases. Skip it when the macro is already defined, letting those users set it themselves. * Report a missing executable on glibc older than 2.24 * Restore the exec-error probe lost in the merge Merging main after #104 dropped the SUBPROCESS_SPAWN_REPORTS_EXEC_ERRORS block while keeping its use site, so the macro was undefined and folded to 0. That turned the access() pre-check on unconditionally, on every platform, which is the opposite of what this change intends.
* Add a portability workflow beside the existing matrix
cmake.yml covers x86-64 Linux, Windows and macOS across four build types and
four compilers, and it covers them well. What it cannot reach is anything that
is not selected by the runner it happens to be on: the capability probes only
ever resolve one way per machine, every runner is 64-bit, and no BSD or non-x86
architecture is present at all.
The new workflow adds one job per dimension:
capability-paths force each combination of SUBPROCESS_HAVE_CWD,
SUBPROCESS_SPAWN_REPORTS_EXEC_ERRORS and
SUBPROCESS_SPAWN_VIA_FORK on an ordinary runner
linux-32-bit, windows-x86 ILP32, where uint64_t becomes unsigned long long
musl a second libc
foreign-arch ppc64le, riscv64, s390x, armv7 under qemu-user
bsd real FreeBSD, NetBSD 9.4 and 10.1, OpenBSD VMs
macos-deployment-target 10.14, 10.15 and 11.0, either side of the boundary
that SUBPROCESS_HAVE_CWD turns on
capability-paths is the cheapest and catches the most. The library selects its
implementation at compile time, so on any given runner the other paths are never
executed — including the ones added by #104 and #106, which as far as I can tell
had never actually run anywhere before, only compiled.
NetBSD is listed at both 9.4 and 10.1 on purpose: posix_spawn_file_actions_addchdir
arrived in 10.0 and is spelled without the _np suffix, so the two take different
paths through the header.
foreign-arch runs under qemu-user, which is faithful for instruction-set
semantics but not for posix_spawn's exec-error reporting — it does not propagate
the errno, and a create_ex failure there would be an artifact rather than a
platform property. That is noted in the file so nobody reads it the wrong way.
* Narrow the emulated-architecture jobs to what emulation reproduces
The first run of this workflow failed all four foreign-arch jobs, and for a
reason that has nothing to do with the architectures: qemu-user does not
propagate the errno that glibc's posix_spawn uses to report a failed exec, so
create_ex_subprocess_create_failure_preserves_error fails there regardless of
the target. s390x failed identically to ppc64le despite having no architectural
difference in play, which is what gives it away.
Running the whole suite under emulation therefore cannot go green, and a job
that is permanently red is worse than no job. These now build the whole suite —
which still covers compilation for each architecture — and run only the tests
whose outcome actually depends on the instruction set:
./subprocess_test '--filter=*divzero'
That is the reason the jobs exist: integer division by zero traps on x86, Arm
and s390x but not on PowerPC or RISC-V. The comment explains the narrowing so it
does not read as suppressing inconvenient failures.
Also noted in the file that the SUBPROCESS_SPAWN_VIA_FORK combination in
capability-paths is a no-op until #112 lands, so its green tick is not mistaken
for evidence today.
* Trim the workflow comments to what the YAML does not already say
Reviewer feedback on #113: the file carried explanatory prose that repeated
what the job and matrix names state. The paragraphs above capability-paths,
linux-32-bit, musl and macos-deployment-target are gone entirely — "no chdir
file action, no exec error reporting" and "macOS deployment target 10.14"
already say it.
Two survive as single lines, both where deleting the comment would invite a
wrong edit rather than merely lose background:
foreign-arch why it runs only *divzero. Without it the narrowing looks
like an oversight and the obvious "fix" is a permanently
red job.
bsd why NetBSD is listed twice, so the duplicate does not get
tidied away and take the 9.x path with it.
The note that SUBPROCESS_SPAWN_VIA_FORK is a no-op until #112 lands is
dropped as well. It was accurate when written but describes a moment rather
than the workflow, and it goes stale the day #112 merges.
No job, step or matrix entry is touched: 7 jobs, 17 runs, before and after.
The reasoning that left the file is preserved in the two commits that added
it and in the PR description.
* Refresh the vendored utest.h to sheredom/utest.h@b6230bc
The old copy is a pristine snapshot of utest.h@f4610ee. Since then upstream
fixed the ILP32 constants and the NetBSD PRIu64 problem (#188), the test
filter dropping tests it should run (#190) and added AIX support (#189).
* Keep the PRI macros visible on NetBSD in C++98
<sys/param.h>, included here since #112 for __NetBSD_Version__, pulls in
<sys/inttypes.h>. That header gates <machine/int_fmtio.h> on
__STDC_FORMAT_MACROS for C++ before C++11 and sits behind an include guard,
so anything defining the macro afterwards cannot get PRId64 and PRIu64 back.
Consumers including subprocess.h first lost them silently; the test suite's
C++98 translation unit is one.
* Fall back to the compiler default where -std=c++2a is unknown too
#105 added -std=c++2a for GCC 8 and clang 9, which do not know -std=c++20.
GCC 7 and older know neither, so the fallback passed a flag those compilers
reject and the build stopped before running anything. NetBSD 9.4 ships GCC
7.5.0 and hit exactly this.
test20.cpp now drops the standard flag when neither spelling is available,
which is what already happens for test11.c without c_std_11 and for MSVC,
where no -std flag is passed at all. The suite then runs that translation
unit at the compiler default rather than not at all.
Measured in GCC 7.5.0: -std=c++20 and -std=c++2a are both rejected, and the
file compiles clean at the default gnu++14 under the flags CMakeLists sets.
GCC 13 still selects -std=c++20 and the suite stays at 444/444.
Fix building where posix_spawn_file_actions_addchdir_np is unavailable
Fixes #108.
Problem
subprocess.hdoes not compile oncesubprocess_create_exis used on platforms whereposix_spawn_file_actions_addchdir_npdoes not exist. That function (added by #83, macOSbranch added by #99) is called unconditionally on the POSIX path, but it only exists from
glibc 2.29 and macOS 10.15 onwards, and Apple marks it unavailable on iOS, tvOS and
watchOS entirely.
glibc
On manylinux2014 (CentOS 7, glibc 2.17, devtoolset-10 / GCC 10.2) the failure appears in
three shapes depending on the language mode:
The third one matters: in
gnu89the implicit declaration is only a warning, so thecompiler is happy and the linker fails instead. The symbol is genuinely absent from
libc.so. This is therefore not the_GNU_SOURCEvisibility issue thattest/CMakeLists.txtalready works around — no feature-test macro can conjure a symbolthat does not exist.
Affected in practice: manylinux2014 (glibc 2.17) and manylinux_2_28 (glibc 2.28), i.e. the
standard build environments for redistributable Linux binaries. llama.cpp vendors this
header, so its manylinux jobs hit this too.
macOS
#108 reports the same breakage under MacPorts on PowerPC,
-arch ppc -mmacosx-version-min=10.6, again via llama.cpp's vendored copy:There are two distinct macOS failure modes, and only the first is a compile error:
error in C++, implicit-declaration in C, link error in
gnu89. This is Regression from introduction of posix_spawn_file_actions_addchdir_np:subprocess.h: error: 'posix_spawn_file_actions_addchdir_np' was not declared in this scope; did you mean 'posix_spawn_file_actions_addopen'?#108.symbol, so the build succeeds. clang emits
-Wunguarded-availability-newby default,which is fatal under
-Werror=unguarded-availability-new; otherwise the warning iseasy to miss and the binary ships with a reference to a symbol that does not exist on
the target OS.
Fix
Add a
SUBPROCESS_HAVE_CWDcompile-time probe next to the POSIX includes, and report arequested
process_cwdasENOSYSwhere the platform cannot honour it. Thecwdtest iscompiled only where the capability exists.
Deliberate choices:
pass a
cwd, not how it is implemented — and the answer now comes from two unrelatedversion boundaries.
#if !defined(_WIN32). An undefined macroevaluates to 0 inside
#if, which would have silently dropped the test on Windows.#iffor the glibc arm, not#if defined(__GLIBC__) && !__GLIBC_PREREQ(2, 29).The preprocessor replaces unknown identifiers with
0before evaluating, so the flatform becomes
0 (2, 29)— a syntax error on macOS, musl and Windows.&&does not help,because this fails at parse time, not evaluation time.
#iffor the Darwin arm, and deliberately nodefined(MAC_OS_X_VERSION_MIN_REQUIRED)guard.
MAC_OS_X_VERSION_MIN_REQUIREDis object-like, so it folds to0rather thanproducing a syntax error, and
0 < 101500is true. That makes the arm fail-closed:any Apple toolchain that does not supply the macro answers "no cwd" instead of
optimistically claiming the capability.
AvailabilityMacros.his already included forApple targets at the top of the POSIX block (added by Fix strict builds with current toolchains #99), so on real macOS the macro is
present and correctly valued. This also produces the right answer on iOS, tvOS and
watchOS, where the macro is undefined and the function does not exist at all.
it cannot recognise are not stuck with the wrong answer. See the limitations below.
Behaviour on glibc >= 2.29 and on macOS >= 10.15 is byte-for-byte unchanged, as is every
non-glibc, non-Apple platform. The macro is documented in
README.mdnext to theprocess_cwdargument.Primary-source evidence for the 10.15 boundary
Quoted verbatim from
usr/include/spawn.h, identical inMacOSX10.15.sdkandMacOSX15.5.sdk:Two things follow. The macOS boundary is exactly 10.15, and the function is macOS-only —
it is not available on iOS 13, contrary to what the naming convention might suggest. The
fail-closed Darwin arm therefore answers correctly on those platforms for free.
The boundary is a real symbol boundary, not merely a declaration one. From
usr/lib/libSystem.B.tbd:_npinspawn.h_posix_spawn_file_actions_addchdir_npTesting
Linux and Windows
Full test suite in Docker, one container per row, sanitizer probes pinned off for
reproducibility. Windows was built natively with MSVC.
SUBPROCESS_HAVE_CWDBefore this change, the first two rows do not build at all.
The 2.29 boundary is hit exactly: 2.28 → 0, 2.29 → 1, 2.31 → 1. The 9
cwdtest cases areexcluded only where the function is missing — all 432 are still compiled and green
everywhere else, including musl, where the
#elsebranch applies. On Windows all 9 arepresent and pass; the total of 387 there is 432 minus the 45 cases (5 tests x 9 language
modes) that
test_shared.halready excludes under#if !defined(_MSC_VER).The Darwin arm cannot affect these rows — it is inside
#if defined(__APPLE__).macOS, cross-compiled against real Apple SDKs
Cross-compiled on Linux in Docker with clang against unmodified Apple SDK headers, this
branch versus
main. Compiled and linked, not run: Mach-O binaries cannot execute on Linux.SUBPROCESS_HAVE_CWDmainRows 1 and 2 reproduce #108 exactly and show the fix. Both C++ and C were exercised; the C
runs used
-Werror=implicit-function-declaration, since otherwise the C compiler acceptsthe implicit declaration and the failure only surfaces at link time.
Row 5 is failure mode 2. On
main,-Wunguarded-availability-newfires by default:Under
-Werror=unguarded-availability-newthat build fails onmainand succeeds on thisbranch.
Linking the same sources with osxcross (cctools/ld64, SDK 15.5) and inspecting the result
with
nm -mshows what actually ships:main(undefined) weak external _posix_spawn_file_actions_addchdir_np (from libSystem)The weak import does not stop the binary from loading on an older system — dyld binds a
missing weak symbol to null — so on
mainthe failure surfaces as a crash at the call siterather than at load time. Building with a deployment target of 10.15 or newer instead
produces a strong reference, which is what turns into the
dyld: Symbol not found: _posix_spawn_file_actions_addchdir_npreports seen in the wild. This branch emits noreference at all below 10.15, so neither shape can occur.
The 10.15 boundary is hit exactly, and
MAC_OS_X_VERSION_MIN_REQUIREDresolves to theexpected numeric values at the probe point (1090, 101300, 101400, 150000), confirming that
the
AvailabilityMacros.hinclude from #99 is in scope there.A separate preprocessor matrix covers the probe in isolation across 12 scenarios (glibc
2.17/2.29, macOS 10.6/10.14/10.15/11/26, iOS-like, musl-like, and both override
directions). Negative controls with deliberately wrong expectations were included in
both matrices and do fail, so the passing rows are not a silent no-op.
Limitations
musl older than 1.1.24. musl gained
posix_spawn_file_actions_addchdir_npin 1.1.24,so the
#elsebranch is wrong for releases before that: Alpine 3.10 (musl 1.1.22) stillfails to link. musl deliberately exposes no version macro, so this cannot be detected.
That is what the override is for —
-DSUBPROCESS_HAVE_CWD=0builds cleanly there, verified.The exact #108 toolchain was not reproduced. All macOS verification above used clang
against x86_64 SDKs. PowerPC Darwin with MacPorts GCC could not be reproduced: there is no
usable PowerPC-Darwin cross toolchain, and old
AvailabilityMacros.hgatesMAC_OS_X_VERSION_MIN_REQUIREDon__APPLE_CC__, which MacPorts GCC does not define.This is precisely why the Darwin arm is fail-closed: if the macro is absent, the probe
answers 0 and the ENOSYS path is taken, rather than calling a function that is not there.
Confirmation on the reporter's actual setup would be welcome.
GCC does not implement Apple's availability attributes. Under GCC,
__API_AVAILABLEexpands to nothing, so there is no weak import and no
-Wunguarded-availabilitydiagnostic — the version check is the only line of defence. Another reason the arm must
fail closed rather than open.
Pre-existing failures on glibc 2.17, unrelated to this change
The 18 remaining failures on manylinux2014 are
create_ex_subprocess_create_failure_preserves_errorand
create_ex_subprocess_create_failure_does_not_leak_resourcesacross the 9 languagemodes. They expect
subprocess_error_not_foundand get0.The cause is in glibc, not in this library. A standalone probe on the same image:
Old
posix_spawndoes not report a failedexecback to the caller; glibc reworked thisin 2.24 — the same probe returns
rc=0on 2.17 and 2.23, andrc=2on 2.24 and 2.28.AlmaLinux 8 (glibc 2.28) shows the same
SUBPROCESS_HAVE_CWD=0code path withzero failures, which isolates it cleanly. These failures were simply never observable
before, because the library did not build there at all. Not addressed here.
Also noticed
test/CMakeLists.txtpasses-std=c++20, which GCC 8 (the RHEL 8 system compiler) rejectsin favour of
-std=c++2a— the suite cannot be built with it, though the library itselfcompiles fine. Not touched here; #105 fixes it separately.
The macOS 26 branch added by #99 calls the non-
_npposix_spawn_file_actions_addchdirunder
MAC_OS_X_VERSION_MIN_REQUIRED >= 260000. That spelling is POSIX.1-2024 and existson Solaris, FreeBSD, glibc and musl, but it is absent from both
spawn.handlibSystem.B.tbdin every SDK up to 15.5, and no SDK 26 was available to check whetherApple actually ships it. If it does not, that branch is a latent link error on macOS 26.
Pre-existing and out of scope here, but worth a look.