Skip to content

fix(deps): resolve Pillow from piwheels on ARM32 - #72134

Open
arasovic wants to merge 1 commit into
NousResearch:mainfrom
arasovic:fix/arm32-pillow-core
Open

fix(deps): resolve Pillow from piwheels on ARM32#72134
arasovic wants to merge 1 commit into
NousResearch:mainfrom
arasovic:fix/arm32-pillow-core

Conversation

@arasovic

@arasovic arasovic commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Prevents Pillow source builds during Hermes install and update on supported
32-bit Raspberry Pi platforms.

PyPI does not publish ARMv6/ARMv7 wheels for the locked Pillow==12.3.0
release. piwheels publishes compatible CPython 3.11 and 3.13 wheels, but uv
does not read Raspberry Pi OS's pip-specific /etc/pip.conf. Without a
project source declaration, uv therefore selects the PyPI sdist and Pillow
requires native JPEG/zlib development headers.

This change declares piwheels as an explicit, package-scoped uv source:

  • only pillow can resolve from piwheels;
  • only armv6l/armv7l on Python 3.11 or 3.13 use that source;
  • aarch64 and all other platforms continue to use PyPI; and
  • both PyPI and piwheels artifacts remain SHA256-pinned in uv.lock.

Using explicit = true avoids the unrelated-package resolution problem of a
global extra index: uv cannot select any other dependency from piwheels.

Related Issue

Fixes #72132

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • Added a marker-scoped, explicit piwheels source for Pillow in
    pyproject.toml.
  • Regenerated uv.lock with separate PyPI and piwheels Pillow entries.
  • Added metadata tests for source isolation, platform/Python routing, PyPI
    fallback, and SHA256-locked ARM32 wheels.
  • Added an executable regression test that runs the real
    scripts/install.sh --stage python-deps route, forces locked sync to fail,
    and verifies the fallback under the installer's UV_NO_CONFIG=1 isolation.
  • Added the contributor attribution mapping required by repository policy.

Compatibility and Risk

  • ARM32 Python 3.12 still resolves Pillow from the PyPI sdist and therefore
    needs the JPEG/zlib build headers. Raspberry Pi OS Bookworm uses Python 3.11
    and Trixie uses Python 3.13, so the practical unsupported gap is narrow.
  • On matching ARM32 Python 3.11/3.13 systems, piwheels is an availability
    dependency. If it is unreachable or the selected wheel disappears, uv fails
    resolution instead of falling back to the PyPI sdist.
  • Trust moves from the host's /etc/pip.conf to a repository-scoped package
    source. The source is restricted to Pillow with explicit = true; locked
    sync verifies each selected wheel against its SHA256 in uv.lock.
  • piwheels does not expose upload timestamps required by uv's age gate, so
    exclude-newer-package.pillow = false disables that gate package-wide for
    Pillow. The exact version pin, package-scoped explicit source, and lock
    hashes bound the locked-sync exposure. The existing unlocked installer
    fallback does not enforce lock hashes.
  • The identical armv6l/armv7l hashes are intentional: piwheels publishes the
    same artifact under both platform tags.
  • The uv2nix/Nix path was not tested because this repository has no Nix CI job
    and Nix was unavailable locally. The marker is false on the common
    x86_64/aarch64 Nix hosts, but lock parsing remains an unverified risk.

How to Test

  1. Run
    scripts/run_tests.sh tests/test_install_sh_uv_sources.py tests/test_project_metadata.py -q.
  2. Run uv lock --check --offline.
  3. On ARMv7 Raspberry Pi OS Bookworm with Python 3.11, install the locked
    Pillow version from piwheels and confirm that
    PIL/_imaging.cpython-311-arm-linux-gnueabihf.so imports successfully.

Verification performed

Raspberry Pi 3 Model B, Raspberry Pi OS 12, armv7l, Python 3.11.2:
  actual installer fallback regression test with uv 0.9.28: 1 passed
  actual installer fallback regression test with uv 0.12.3: 1 passed
  Pillow 12.3.0 piwheels artifact installed successfully
  native PIL/_imaging ARM extension imported successfully
  extension identified as ELF 32-bit ARM EABI5

Installer fallback controls:
  UV_NO_SOURCES=1 negative control: failed as expected
  installer-exported UV_NO_CONFIG=1: project-scoped source remained active

Lock source-marker matrix:
  66 platform/Python combinations checked
  0 routing holes
  0 routing overlaps

uv lock --check --offline with CI uv 0.9.28:
  resolved 251 packages; passed

git diff --check:
  passed

GitHub CI for 2d6724d5caf46f317fa6eeb4146592a971fc0c0c:
  all 12 Python test slices passed
  Python e2e, Ruff, ty diff, and Windows-footgun checks passed
  macOS-only and Windows-only tests passed
  uv lock, attribution, OSV, and supply-chain checks passed
  Docker amd64 and arm64 builds and integration tests passed
  All required checks pass gate passed

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Raspberry Pi OS 12, 32-bit ARMv7, Python 3.11.2

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — N/A; the non-obvious source invariants are documented next to the configuration
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — all non-ARM32 platforms retain PyPI
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

Original report (Pillow==12.2.0):

DEBUG Selecting: pillow==12.2.0 [compatible] (pillow-12.2.0.tar.gz)
RequiredDependencyException: The headers or library files could not be found for jpeg

Current branch (Pillow==12.3.0):

Pillow 12.3.0 installed from the piwheels ARMv7 wheel
PIL/_imaging.cpython-311-arm-linux-gnueabihf.so: ELF 32-bit LSB shared object, ARM, EABI5

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard area/install-update Installer, updater, packaging, wheels, doctor sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Jul 26, 2026
@arasovic
arasovic marked this pull request as draft July 26, 2026 18:56
@arasovic
arasovic marked this pull request as ready for review July 26, 2026 19:33
@arasovic
arasovic marked this pull request as draft July 26, 2026 19:37
@arasovic
arasovic force-pushed the fix/arm32-pillow-core branch from 3502719 to 42253dd Compare July 26, 2026 21:48
@arasovic arasovic changed the title fix(install): use ARM32 Pillow wheel on Raspbian fix(deps): resolve Pillow from piwheels on ARM32 Jul 26, 2026
@arasovic
arasovic marked this pull request as ready for review July 26, 2026 21:59
@arasovic
arasovic requested a review from a team July 26, 2026 21:59

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused ARM32 investigation. The current lock still lacks an ARMv6/ARMv7 Pillow wheel, so the reported failure is real.

Problems

  • pyproject.toml:320 adds the piwheels source only through [tool.uv.sources], but the shell installer exports UV_NO_CONFIG=1 at scripts/install.sh:31-33. If locked sync fails, its fallback invokes uv pip install -e at scripts/install.sh:1632-1645; that path does not discover the project-level uv configuration, so it can still resolve Pillow from PyPI and attempt the ARM32 source build.

Suggested changes

  • Cover the installer fallback/recovery route as well as locked sync, while preserving the existing isolation from user/home uv configuration.
  • The PR is currently conflicting with main; current main regenerated uv.lock with CI's uv 0.11.33 in 3d2cc39158, so the lock should be regenerated during salvage.

Automated hermes-sweeper review.

Comment thread pyproject.toml
# On matching versions, a missing piwheels wheel is a hard resolution failure;
# uv does not fall back to the sdist. See #72132.
[tool.uv.sources]
pillow = [

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this source table is bypassed by the shell installer's UV_NO_CONFIG=1 (scripts/install.sh:31-33). When locked sync fails, its uv pip install -e fallback (scripts/install.sh:1632-1645) will not discover this configuration, leaving ARM32 able to fall back to the PyPI Pillow sdist. Please cover that fallback/recovery route too.

@arasovic arasovic Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 2d6724d5c.

The branch is rebased onto main at 338bca796, and uv.lock is regenerated for Pillow==12.3.0.

tests/test_install_sh_uv_sources.py now executes the actual scripts/install.sh --stage python-deps path. The test forces locked sync to fail, verifies a negative control with UV_NO_SOURCES=1, and then verifies that the real fallback succeeds while the installer exports UV_NO_CONFIG=1. This covers the recovery route without weakening isolation from user/home uv configuration.

The same installer regression test passed on the reported ARMv7 / Python 3.11.2 host with both CI uv 0.9.28 and current uv 0.12.3. Separately, the piwheels Pillow==12.3.0 artifact installed on that host and its native _imaging.cpython-311-arm-linux-gnueabihf.so extension imported successfully; file identifies it as ELF 32-bit ARM EABI5.

The current PR CI is fully green, including all 12 Python slices, uv lock --check, supply-chain checks, and amd64/arm64 Docker builds.

No runtime installer change was required: UV_NO_CONFIG=1 isolates ambient configuration but does not disable project tool.uv.sources; UV_NO_SOURCES=1 is the control that disables those sources.

@teknium1 teknium1 added the sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform label Jul 30, 2026
@arasovic
arasovic force-pushed the fix/arm32-pillow-core branch from 42253dd to 9e2b9cc Compare July 30, 2026 15:31
@GottZ

GottZ commented Aug 3, 2026

Copy link
Copy Markdown

This was generated by AI during triage.

Summary

One PR addresses issue #72132. #72134 adds package-scoped piwheels routing and hash-locked ARM32 Pillow wheels for Python 3.11 and 3.13, directly targeting the reported PyPI source-build failure, but its diff does not add coverage for the installer's fallback path.

Related pull requests

  • fix(deps): resolve Pillow from piwheels on ARM32 #72134 best fix — (+159/-8) — best partial fix: The diff routes only Pillow on armv6l/armv7l with Python 3.11 or 3.13 through an explicit piwheels index, preserves PyPI routing elsewhere, locks the ARM32 wheels by SHA256, and adds metadata tests. The keep_open review on fix(deps): resolve Pillow from piwheels on ARM32 #72134 remains unresolved: although the author reports a successful ARMv7 installation with UV_NO_CONFIG=1, the diff contains no executable regression test for the scripts/install.sh:1632-1645 fallback identified by the blocking contributor review, and the lock must also be regenerated after rebasing onto current main.

Suggested consolidation

Keep #72134 open with a salvage path: retain its scoped piwheels routing and locked ARM32 artifacts, rebase onto main and regenerate uv.lock consistently with the current main lock from 3d2cc39158, then add an executable regression test covering the actual installer fallback under UV_NO_CONFIG=1. This addresses the blocking contributor review while converting the reported ARMv7 result into repository-verifiable evidence; there are no duplicate PRs to close.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I72132(["issue #72132 (open)"])
    P72134["PR #72134 (open)"]
    P72134 -->|best fix| I72132
    class I72132 open
    class P72134 open
    class P72134 best
    class P72134 target
    click I72132 "https://github.com/NousResearch/hermes-agent/issues/72132"
    click P72134 "https://github.com/NousResearch/hermes-agent/pull/72134"
Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 1 pull request and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 12 kB of PR diffs, 11 kB of issue/PR text, 3 kB of discussion (3 comments), 3 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

@arasovic
arasovic force-pushed the fix/arm32-pillow-core branch from 9e2b9cc to 2d6724d Compare August 10, 2026 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/install-update Installer, updater, packaging, wheels, doctor comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: ARM32 update builds Pillow from source because uv ignores Raspbian piwheels config

4 participants