Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,13 @@ jobs:
- name: Validate LWE and Garage page contracts
run: pnpm run pages:check

# Only the DECLARATION tier gates here. It reads shell source and needs no
# binary present, so it runs anywhere. The presence tier prints what is
# absent and passes, because a hosted runner carries none of the photo
# toolchain and never runs the pipeline that wants it.
- name: Check system tool declaration
run: pnpm run tools:check

- name: Test coffee Worker
run: pnpm --filter cal-aadhar-sh test

Expand Down
35 changes: 35 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -835,6 +835,41 @@ Two encoders + one transform tool, all built from source:
this pipeline. The 64-bin RGB/luminance bake moved into `zenc histogram` on
2026-08-14, so nothing in add-photos.sh or extract-photo-metadata.sh needs it.

The four below serve the STUDY pages rather than the photo pipeline, and every
one of them was undocumented until `tools:check` went looking (2026-08-14):

- **webp** (`brew install webp`) — `cwebp` draws the WebP point on both
`/garage/encoding` grids.
- **ffmpeg** (`brew install ffmpeg`) — one PNG-to-PPM conversion in
`gen-encoding-grids.sh`, because `cjpeg` will not read PNG and sips' BMP
output confuses it.
- **ssimulacra2** and **butteraugli_main** — the two perceptual metrics
`export-for-instagram.sh` searches quality against. These are libjxl TOOLS,
built with `-DJPEGXL_ENABLE_TOOLS=ON`, and **`brew install jpeg-xl` does not
ship them**, which is why the script falls back to `/opt/zerobrew/prefix/bin`.
The error message in that script names the formula rather than the tools and
is misleading; the binaries are what to go find.

> **The whole list is DECLARED in [`config/tools.json`](config/tools.json) and
> `pnpm run tools:check` fails on drift.** It sits outside `infra.json` on
> purpose: that file declares Cloudflare and GitHub state and diffs it against
> those APIs, while nothing here is remote and nothing here has an API.
>
> The check is tiered the same way `infra:check` is. Its DECLARATION tier reads
> source text, needs no binary, and runs on every PR: every `for cmd in …` guard,
> every literal `command -v`, and every `brew install` hint in the shell scripts
> must have an entry, and both this file and MAINTENANCE.md must name it. Its
> PRESENCE tier probes the machine and is ADVISORY in CI, because a hosted runner
> has none of these and is not meant to.
>
> The guard scanner is the load-bearing part, and it is why four prerequisites
> could stay undocumented. Most of these preconditions are written `for cmd in
> sips exiftool`, so the binary name exists only as a loop word and a grep for the
> name finds nothing. That is gotcha 29's blind spot in different clothes: a
> command assembled from list elements is invisible to a search for the assembled
> form. Each scanner also carries a FLOOR and fails if its match count collapses,
> since a scanner that matches nothing otherwise reports a pass.

### `<picture>` + content-addressed thumbnails

Photo thumbnails are dual-encoded AVIF + JPG, served via `<picture>` from
Expand Down
140 changes: 140 additions & 0 deletions config/tools.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
{
"$comment": [
"The external binaries the photo and encoding pipelines shell out to, declared",
"once so `pnpm run tools:check` can verify them. NOT in infra.json on purpose:",
"that file declares Cloudflare and GitHub state and `infra:check` diffs it",
"against those APIs, while nothing here is remote and nothing here has an API.",
"Mixing them would give one file two unrelated contracts.",
"",
"This exists because the list used to live in three places that disagreed:",
"CLAUDE.md's encoder section, MAINTENANCE.md's brew line, and inline guards in",
"the scripts. ffmpeg, cwebp, ssimulacra2 and butteraugli_main were required and",
"documented nowhere, so a clean machine met them as a raw shell error.",
"",
"`optional` means a script degrades rather than exits: avifenc falls back to",
"sips in add-photos.sh and reencode-thumbnails.sh, and is REQUIRED by the two",
"encoding-grid scripts, so it is listed optional and its hard callers are named",
"in `required_by` anyway. A tool that is optional everywhere is still declared,",
"because the point is a complete list rather than a blocking one.",
"",
"`path` is for a binary a script resolves by absolute location rather than",
"PATH. jpegtran is the only one: mozjpeg is keg-only, so `command -v jpegtran`",
"can find libjpeg-turbo's copy while the scripts use mozjpeg's, and probing",
"PATH would report a pass on the wrong binary."
],

"tools": [
{
"bin": "exiftool",
"install": "brew install exiftool",
"why": "EXIF, orientation, and the Fujifilm recipe card",
"required_by": [
"www/scripts/extract-photo-metadata.sh",
"www/scripts/add-photos.sh",
"www/scripts/download-remote-photos.sh",
"www/scripts/export-for-instagram.sh",
"www/scripts/gen-encoding-samples.sh",
"www/scripts/reencode-thumbnails.sh"
]
},
{
"bin": "jq",
"install": "brew install jq",
"why": "JSON assembly in the metadata shell scripts",
"required_by": [
"www/scripts/extract-photo-metadata.sh",
"www/scripts/download-remote-photos.sh"
]
},
{
"bin": "sips",
"install": "ships with macOS",
"why": "resize, HEIF decode, and the AVIF fallback encoder",
"required_by": [
"www/scripts/add-photos.sh",
"www/scripts/export-for-instagram.sh",
"www/scripts/gen-encoding-grids.sh",
"www/scripts/gen-encoding-samples.sh",
"www/scripts/reencode-thumbnails.sh"
]
},
{
"bin": "curl",
"install": "ships with macOS",
"why": "pulls the R2 originals back down",
"required_by": ["www/scripts/download-remote-photos.sh"]
},
{
"bin": "cargo",
"install": "https://rustup.rs",
"why": "builds zenc, the shipped JPEG encoder, on first run",
"required_by": [
"www/scripts/add-car-photo.sh",
"www/scripts/add-photos.sh",
"www/scripts/export-for-instagram.sh",
"www/scripts/gen-encoding-grids.sh",
"www/scripts/gen-encoding-samples.sh",
"www/scripts/reencode-thumbnails.sh"
]
},
{
"bin": "jpegtran",
"path": "/opt/homebrew/opt/mozjpeg/bin/jpegtran",
"install": "brew install mozjpeg",
"why": "lossless EXIF-orientation rotation before recompression",
"required_by": [
"www/scripts/add-photos.sh",
"www/scripts/reencode-thumbnails.sh"
]
},
{
"bin": "cjpeg",
"install": "brew install mozjpeg",
"why": "the chroma-subsampling row of the /garage/encoding grids",
"required_by": ["www/scripts/gen-encoding-grids.sh"]
},
{
"bin": "avifenc",
"install": "brew install libavif",
"optional": true,
"why": "primary 10-bit AVIF tier; add-photos and reencode-thumbnails fall back to sips, the grid scripts do not",
"required_by": [
"www/scripts/gen-encoding-grids.sh",
"www/scripts/gen-encoding-samples.sh"
]
},
{
"bin": "cwebp",
"install": "brew install webp",
"why": "the WebP point on the /garage/encoding format grid",
"required_by": [
"www/scripts/gen-encoding-grids.sh",
"www/scripts/gen-encoding-samples.sh"
]
},
{
"bin": "ffmpeg",
"install": "brew install ffmpeg",
"why": "PNG to PPM, because cjpeg will not read PNG and sips BMP confuses it",
"required_by": ["www/scripts/gen-encoding-grids.sh"]
},
{
"bin": "ssimulacra2",
"install": "build libjxl with -DJPEGXL_ENABLE_TOOLS=ON; brew's jpeg-xl does not ship it",
"why": "the perceptual metric the Instagram export searches quality against",
"required_by": ["www/scripts/export-for-instagram.sh"]
},
{
"bin": "butteraugli_main",
"install": "build libjxl with -DJPEGXL_ENABLE_TOOLS=ON; brew's jpeg-xl does not ship it",
"why": "the second metric bounding that same search",
"required_by": ["www/scripts/export-for-instagram.sh"]
},
{
"bin": "python3",
"install": "brew install python",
"why": "alt text in add-photos.sh, plus build-recipes.py and gen-pixel-peeper.py; only the last of those wants Pillow, from www/scripts/requirements.txt",
"required_by": ["www/scripts/add-photos.sh"]
}
]
}
21 changes: 20 additions & 1 deletion docs/MAINTENANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1115,15 +1115,34 @@ workflow that hand-tracked the from-source jpegli commit. Only Homebrew formulas
## Local fallback setup

```bash
brew install exiftool jq mozjpeg libavif # mozjpeg = jpegtran; libavif = avifenc (optional, sips falls back)
# the photo pipeline
brew install exiftool jq mozjpeg libavif # mozjpeg = jpegtran + cjpeg; libavif = avifenc (optional, sips falls back)
python3 -m pip install -r www/scripts/requirements.txt # Pillow, for gen-pixel-peeper.py only
# the JPEG encoder (zenc) builds itself on first pipeline run; needs rust (rustup.rs)
cargo build --release --manifest-path www/scripts/zenc/Cargo.toml
wrangler login # Cloudflare auth (deploys + KV + R2 all use it)

# the study pages, which are NOT needed to add a photo
brew install webp ffmpeg # cwebp for the encoding grids; ffmpeg for their PNG -> PPM step
```
This is an emergency fallback only. sips is macOS-native (no install), and the
normal path is the remote workflow above.

`export-for-instagram.sh` additionally wants **ssimulacra2** and
**butteraugli_main**, the two perceptual metrics it searches quality against.
Those are libjxl tools built with `-DJPEGXL_ENABLE_TOOLS=ON`, and Homebrew's
`jpeg-xl` formula does not ship them, so there is no brew line for this row. The
script falls back to `/opt/zerobrew/prefix/bin`, which is where this workstation's
source build put them.

**`pnpm run tools:check` is the check on all of it**, declared in
[`config/tools.json`](../config/tools.json). Its declaration tier runs on every
PR and needs no binary; its presence tier probes this machine and is advisory in
CI. Run it before a pipeline session on a fresh machine and it names what is
missing and how to get it, rather than letting a script exit on a raw shell
error four steps in. Four of the tools above were required and documented nowhere
until it was written.

---

## Add photos (local fallback only)
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
"infra:check": "node scripts/check-infra.mjs",
"infra:apply": "node scripts/apply-infra.mjs",
"pages:check": "node scripts/check-page-contracts.mjs",
"tools:check": "node scripts/check-tools.mjs",
"test": "node --test scripts/contract-tests.mjs",
"lint": "oxlint --type-aware",
"typecheck": "tsc -p config/tsconfig.json && tsc -p config/tsconfig.browser.json",
Expand Down
Loading
Loading