Skip to content

Make the macOS bootstrap tests pass on a SIP-enabled Mac without npm (tests only; supersedes #178) - #228

Merged
seathatflowsinourveins merged 3 commits into
mainfrom
claude/macos-test-env-20260924
Sep 25, 2026
Merged

seathatflowsinourveins merged 3 commits into
mainfrom
claude/macos-test-env-20260924

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Owner

Summary

On a developer Mac with System Integrity Protection (SIP) enabled and no npm on PATH, tests/test_adoption_bootstrap_macos.py has one failure and one error on unmodified main. Both are test-harness problems, so this PR changes tests only. adoption/bootstrap-macos.sh is unchanged.

Reproduced at d478fd5 on macOS 26.5.1 (25F80) arm64. csrutil status reports enabled, and ls -lO /bin/sh shows restricted. The interpreter is uv-managed CPython 3.13.15. Temp paths are shortened to <tmp>:

FAIL:  LlamaWrapperQuotingTests.test_install_path_with_shell_syntax_is_data_in_the_wrapper
       AssertionError: Lists differ: ['<tmp>/…/llama-server', ''] != ['<tmp>/…/llama-server', '<tmp>/…/llama-cpp-b1']
ERROR: PlatformDependencyInstallTests.test_places_the_verified_tarball_correctly_when_the_prefix_is_reached_through_a_symlink
       AttributeError: 'PlatformDependencyInstallTests' object has no attribute 'verified_sha256'
Ran 116 tests: FAILED (failures=1, errors=1, skipped=37)

1. Llama wrapper quoting test under SIP (commit carried over from #178)

2. Missing npm skip

Checks on this Mac

Run of tests.test_adoption_bootstrap_macos Result
Unmodified main, npm absent 116 tests: failures=1, errors=1, skipped=37
This branch, npm absent 116 tests OK, skipped=38. The llama test ran and passed; the symlink test skipped with native npm unavailable
This branch, npm present (mise-managed node 24.21.0 / npm 11.19.0 prepended to PATH) 116 tests OK, skipped=16. Both target tests ran and passed

The remaining skips are environment gaps: PyYAML not installed (14), shellcheck not installed (1), the non-macOS guard test (1), and the npm tests (22, only when npm is absent).

The branch was rebased onto 636751a (#224), which touches none of these files. After the rebase:

  • Validation: python3 scripts/validate.py (system Python 3.9.6) passed with 69 components, 5338 hashed files, 4 profiles and 145 receipts.
  • Module: tests.test_adoption_bootstrap_macos was re-run: OK, skipped=38.
  • Full suite (python3.13 -m unittest, npm absent, at this PR's head): ran 4714 tests with failures=1 and skipped=647.
    • The one failure is tests.test_new_host_grand_list.RecordHostToGrandListTests.test_recorded_host_gets_non_null_measured_in_the_grand_list. It fails the same way on unmodified main (d478fd5, re-run in a clean checkout) on this host, so it is unrelated to this PR.
    • Cause: a host-specific false positive in the local-user-name guard makes hardware_profile.py --record-host refuse the test's fixture id (exit 2).
    • A run before the rebase, on d478fd5 plus these two commits, gave the same single failure (4713 tests).
  • Evidence: manifests/evidence.json re-hashes only tests/test_adoption_bootstrap_macos.py, through host_receipts.register_file.

Relation to #178

This PR contains #178's commit plus the npm fix, re-hashed once. If this merges, #178 is redundant and can be closed. It has been left open for you to decide.

Evidence class: local integration on one SIP-enabled Mac. No product change, no receipt added, and no launchd, embedding or model run.

🤖 Generated with Claude Code

LlamaWrapperQuotingTests failed on a SIP-enabled Mac (macOS 26.5.1) and
passed on GitHub's hosted macos-15 runner. Its stand-in llama-server was a
#!/bin/sh script: with System Integrity Protection enabled, macOS purges
DYLD_* variables when it launches a SIP-protected binary such as /bin/sh,
so the stand-in printed an empty DYLD_LIBRARY_PATH even though the wrapper
had exported it.

The product wrapper is not affected. It sets DYLD_LIBRARY_PATH after its
own /bin/sh has started and execs the real llama-server, an unprotected,
ad-hoc signed Mach-O, so the value arrives. On the same SIP-enabled Mac the
pinned b11057 asset installed through install_llama_cpp into a
shell-syntax path runs through the generated wrapper, and with its dylibs
moved away the wrapper's DYLD_LIBRARY_PATH alone still loads them.

The stand-in is now a script for the test's own Python, unprotected like
the real binary. A control run first checks that this interpreter sees a
DYLD_* value handed to it directly; if not, the test skips only the
DYLD_LIBRARY_PATH assertion, after the data-safety assertions have run.
The wrapper now receives DYLD_LIBRARY_PATH unset rather than empty: an
inherited variable stays exported, so a missing export line had gone
unnoticed on every non-SIP host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 3fd39cd)
test_places_the_verified_tarball_correctly_when_the_prefix_is_reached_through_a_symlink
uses the npm-packed fixtures that PlatformDependencyInstallTests.setUpClass
builds only when npm is on PATH: self.verified_tarball directly and
self.verified_sha256 through _platform_dependency_pin. Unlike its
npm-dependent siblings it had no @unittest.skipUnless(NPM, ...) since it
was added in #94, so on a host without npm it errored with AttributeError
instead of skipping. Hosted runners have npm, so CI never hit it.

The class's other three undecorated tests use no npm fixtures and still
run without npm.

Re-hash tests/test_adoption_bootstrap_macos.py in manifests/evidence.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

# Conflicts:
#	manifests/evidence.json
@seathatflowsinourveins
seathatflowsinourveins enabled auto-merge (squash) September 25, 2026 13:53
@seathatflowsinourveins
seathatflowsinourveins merged commit e8a843c into main Sep 25, 2026
24 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/macos-test-env-20260924 branch September 25, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant