Apple GPU C-ABI signature registry + off-device drift guard + bind_or_skip

[gstoner]

What & why

The graph audit of this repo surfaced that the whole Apple-GPU test suite fans
into one bind_symbol() symbol-resolver, and on inspection 87 distinct C-ABI
symbols had their ctypes signatures hand-written at each of ~19 test files + 8
production modules — signature knowledge with no single source of truth, and
validated only on a real Apple GPU (every such test skips off-device). A
renamed/removed C export or a drifted signature was invisible on ordinary CI.

This PR centralizes that surface and adds the CI-runnable guard that was missing.

Fix 1 — APPLE_ABI registry (single source of truth)

python/tessera/_apple_gpu_dispatch.py now holds the canonical
{symbol: (argtypes, restype)} for all 87 exports, plus:

• bind_registered(symbol) — bind by name, signature pulled from the registry;
• expected_symbols() — the full declared ABI set.

The registry was mechanically extracted by AST-parsing every bind_symbol()
call site and proven conflict-free (every symbol resolves to exactly one
evaluated ctypes signature). Per-file aliases were resolved — e.g. runtime.py's
cf = POINTER(c_float) and ci = POINTER(c_int64) (note ci is int64, easy
to get wrong by eye).

Fix 2 — tests/unit/test_apple_gpu_abi_registry.py, two guards

• Off-device drift net (runs on ordinary CI, no GPU): AST-scans the tree and
  asserts every literal bind_symbol("sym", …) agrees with APPLE_ABI (symbol
  registered + identical evaluated signature, resolving per-file ctypes aliases).
  A drift or an unregistered symbol now fails immediately instead of silently
  skipping. It already earned its keep — rebasing onto main pulled in 4 new
  symbols (optimizer/scatter/sddmm/spmm_csr f32) and the guard flagged
  every one.
• On-device net (skips without the dylib, like the NVIDIA/ROCm runtime-symbol
  tests): when the runtime loads, every registry symbol must resolve in it — one
  place covering the whole ABI, catching a renamed/removed C export.

Fix 3 — bind_or_skip(symbol) test helper

tests/unit/_apple_gpu_abi_testutil.py — bind by name via the registry, skip
off-device. Migrated the two clean exemplars (commit_and_wait_timeout,
row_major_strides) off inline signatures. The rest can adopt incrementally; the
drift guard keeps their inline signatures honest meanwhile.

Verification done here (Linux/WSL, no Apple GPU)

• 3 new off-device guard tests pass; on-device net skips cleanly.
• ruff + mypy clean on the changed module.
• Full tests/unit/test_apple*.py suite: no new failures — identical
  pre-existing count with and without this change (those are stale-tessera-opt
  build failures on this box, unrelated to ctypes; confirmed by stashing).

⚠️ Continue on Apple silicon (what this box could NOT verify)

This branch was authored on a Linux/WSL box with no Apple GPU, so everything
gated on the runtime dylib is unproven. On an Apple-silicon Mac:

# 1. Environment (Homebrew toolchain, off-venv — see CLAUDE.md)
cd tessera && git fetch origin && git checkout apple/abi-signature-registry

# 2. THE key check — on-device ABI net must PASS (not skip) with the runtime built:
python3 -m pytest tests/unit/test_apple_gpu_abi_registry.py -v
#   Expect: test_dylib_exports_resolve_every_registry_symbol PASSES (all 87
#   registry symbols resolve in the compiled runtime dylib). If it fails, the
#   registry lists a symbol the .mm runtime doesn't export (or vice-versa) —
#   reconcile APPLE_ABI with apple_gpu_runtime.mm.

# 3. The two MIGRATED tests must stay green on-device (they were only collected,
#    not executed, off-device):
python3 -m pytest tests/unit/test_apple_commit_and_wait_timeout.py \
                  tests/unit/test_apple_row_major_strides.py -v

# 4. Full Apple suite regression — no NEW failures vs. main:
python3 -m pytest tests/unit/test_apple*.py -q

Then (optional follow-on, same pattern)

Finish Fix 3 by migrating the remaining exercise-and-skip sites to
bind_or_skip("sym") / bind_registered("sym"), one file at a time, running
each on-device after. Do NOT migrate the presence-assertion tests (e.g.
test_apple_mlpkg_pk2.py, which does assert fn is not None) or negative tests —
bind_or_skip turns a missing symbol into a skip, which would weaken them. The
drift guard already covers their inline signatures against the registry, so
leaving them is safe.

If the runtime adds/renames a C export, update APPLE_ABI in
python/tessera/_apple_gpu_dispatch.py — the off-device drift guard will point
at any call site that disagrees.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b5d6b26152

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Exclude the registry from the unused-symbol scan

When a symbol is added only to APPLE_ABI and never bound anywhere else, this test still marks it as referenced because _SCAN_ROOTS includes python/tessera, so _apple_gpu_dispatch.py itself is scanned and the registry definition contains every key as a string. That makes orphaned effectively unable to catch stale or mistyped registry entries; exclude the registry file or count references outside the APPLE_ABI definition.

Useful? React with 👍 / 👎.