fix(x86): stop leaking AVX-512/AMX flags into the Target IR dialect

[gstoner]

Fixes the Validate / lit failure in run 31893492411 — 14 x86 fixtures crashing inside TesseraX86Dialect::initialize().

Root cause: a build-flag leak, not an IR defect

The dialect and its TileType registration were always correct.

add_compile_options(-mavx512f …) is directory scoped, and add_subdirectory(lib/IR) sat below that call — so the hardware-free Target IR dialect was compiled with all 11 AVX-512/AMX flags. The compiler was then entitled to emit an AVX-512-only encoding into dialect registration itself. Confirmed by disassembling the TU: vpbroadcastq %rgpr, %xmm, a GPR-source broadcast that exists only under AVX-512.

The tell was the signal, not the backtrace. All 14 fixtures failed with SIGILL (signal 4) at one identical address, not SIGSEGV. An illegal instruction is a build-configuration fact; Dialect::addType<TileType>() was merely the first code from that object file to execute.

This is also why the P0 was closed in good faith on Zen 5 and reopened by CI: Zen 5 has AVX-512, so the instruction is legal there. The GitHub runner does not.

Fix

• Flags collected into TESSERA_X86_ARCH_FLAGS, applied with target_compile_options to this directory's kernel targets, enumerated via BUILDSYSTEM_TARGETS. A future kernel target is covered automatically; subdirectories are structurally excluded. No ordering left for a reader to maintain by hand — the previous arrangement was correct only until someone added a subdirectory below it.
• tessera_x86_base stays excluded so its baseline pin isn't silently re-enabled.
• lib/IR/CMakeLists.txt fails configure if any host-specific ISA flag is in scope (Decision #21a, fail closed), so the next occurrence is a build error naming the cause rather than a runtime SIGILL that reads as a corrupt MLIR install.

Verification

Check	Result
Dialect TU compile flags, before	11 arch flags (-mavx512f … -mamx-bf16)
Dialect TU compile flags, after	none
Kernel TU (avx512_gemm_bf16.cpp)	unchanged, all 11
tessera_x86_base	unchanged, baseline pin intact
ninja tessera-opt	clean; tessera_x86 registers
x86_dialect_load / x86_target_ir / x86_target_ir_invalid	pass
Guard negative test (leak reintroduced)	configure fails with the naming message

Acceptance evidence is the green lit lane on the non-AVX-512 runner, not a local rebuild — hence the lit-smoke label.

Separate defect documented, not fixed

TileToX86Pass.cpp:1045 calls getOrLoadDialect("tessera_x86") from inside runOnOperation(). MLIR forbids loading a dialect during pass execution: it's a hard LLVM ERROR on assertions-enabled LLVM (fails 12 of the 15 x86 fixtures) and silent UB on the NDEBUG build CI uses — CI structurally cannot catch it.

Left unfixed deliberately: the canonical remedy is getDependentDialects(), which needs the C++ type and therefore couples TesseraPasses to the optional TesseraX86IR. TesseraPasses links no backend Target IR library today, so that's a new dependency direction under Decisions #19/#31 — an owner call, not a bug edit. The todo records the recommended TESSERA_HAS_X86_TARGET_IR-guarded shape, and notes that Pass::initialize() is not an escape hatch (its contract explicitly forbids loading dialects).

Docs

• docs/audit/backend/x86/todo.md — P0 reopened and root-caused; new P0 for the dialect-load defect.
• CLAUDE.md Decision Apple GPU Tier-2/3: reductions, native GQA, fused batched attention #19 — replaces the stale SUSPENDED banner, and records the standing lesson: a host that has the ISA cannot falsify a host-portability claim.

🤖 Generated with Claude Code