[ZenCPU] Add zencpu Platform Runtime Logging and Docs

docs/getting_started/installation/cpu.md

@@ -142,20 +142,51 @@ VLLM_USE_PRECOMPILED=1 VLLM_PRECOMPILED_WHEEL_VARIANT=cpu VLLM_TARGET_DEVICE=cpu
 === "IBM Z (S390X)"
     --8<-- "docs/getting_started/installation/cpu.s390x.inc.md:build-image-from-source"
 
+## AMD Zen optimizations {#amd-zen-optimizations}
+
+--8<-- "docs/getting_started/installation/cpu.x86.inc.md:amd-zen-optimizations"
+
 ## Related runtime environment variables
 
 - `VLLM_CPU_KVCACHE_SPACE`: specify the KV Cache size (e.g, `VLLM_CPU_KVCACHE_SPACE=40` means 40 GiB space for KV cache), larger setting will allow vLLM to run more requests in parallel. This parameter should be set based on the hardware configuration and memory management pattern of users. Default value is `0`.
 - `VLLM_CPU_OMP_THREADS_BIND`: specify the CPU cores dedicated to the OpenMP threads, can be set as CPU id lists, `auto` (by default), or `nobind` (to disable binding to individual CPU cores and to inherit user-defined OpenMP variables). For example, `VLLM_CPU_OMP_THREADS_BIND=0-31` means there will be 32 OpenMP threads bound on 0-31 CPU cores. `VLLM_CPU_OMP_THREADS_BIND=0-31|32-63` means there will be 2 tensor parallel processes, 32 OpenMP threads of rank0 are bound on 0-31 CPU cores, and the OpenMP threads of rank1 are bound on 32-63 CPU cores. By setting to `auto`, the OpenMP threads of each rank are bound to the CPU cores in each NUMA node respectively. If set to `nobind`, the number of OpenMP threads is determined by the standard `OMP_NUM_THREADS` environment variable.
 - `VLLM_CPU_NUM_OF_RESERVED_CPU`: specify the number of CPU cores which are not dedicated to the OpenMP threads for each rank. The variable only takes effect when VLLM_CPU_OMP_THREADS_BIND is set to `auto`. Default value is `None`. If the value is not set and use `auto` thread binding, no CPU will be reserved for `world_size == 1`, 1 CPU per rank will be reserved for `world_size > 1`.
 - `CPU_VISIBLE_MEMORY_NODES`: specify visible NUMA memory nodes for vLLM CPU workers, similar to ```CUDA_VISIBLE_DEVICES```. The variable only takes effect when VLLM_CPU_OMP_THREADS_BIND is set to `auto`. The variable provides more control for the auto thread-binding feature, such as masking nodes and changing nodes binding sequence.
 - `VLLM_CPU_SGL_KERNEL` (x86 only, Experimental): whether to use small-batch optimized kernels for linear layer and MoE layer, especially for low-latency requirements like online serving. The kernels require AMX instruction set, BFloat16 weight type and weight shapes divisible by 32. Default is `0` (False).
+- `VLLM_ZENTORCH_WEIGHT_PREPACK` (AMD Zen only): when `ZenCpuPlatform` is active, eagerly prepack linear weights into ZenDNN's blocked layout at model load time, eliminating per-inference layout conversion overhead. Default is `1` (enabled). See [AMD Zen optimizations](#amd-zen-optimizations).
 
 ## FAQ
 
 ### Which `dtype` should be used?
 
 - Currently, vLLM CPU uses model default settings as `dtype`. However, due to unstable float16 support in torch CPU, it is recommended to explicitly set `dtype=bfloat16` if there are any performance or accuracy problem.  
 
+### How do I enable AMD Zen optimizations?
+
+On an AMD Zen 4 / Zen 5 CPU, install the CPU wheel with the `zen` extra so
+vLLM pulls the tested `zentorch` version for that release:
+
+```bash
+export VLLM_VERSION=0.20.2
+pip install "vllm[zen] @ https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}%2Bcpu-cp38-abi3-manylinux_2_35_x86_64.whl" \
+    --extra-index-url https://download.pytorch.org/whl/cpu
+```
+
+This is safer than installing `zentorch` separately, because the `zen` extra
+keeps the vLLM wheel and ZenDNN plugin on the tested version combination.
+
+vLLM auto-detects the platform and routes linear layers through
+ZenDNN-optimized kernels - no flag needed. To verify it is engaged, run with
+INFO-level logs and look for the activation banner:
+
+```bash
+VLLM_LOGGING_LEVEL=INFO vllm serve facebook/opt-125m --dtype bfloat16 \
+    2>&1 | grep -E "ZenCpuPlatform activated|CPU unquantized GEMM dispatch"
+```
+
+See [AMD Zen optimizations](#amd-zen-optimizations) for details on detection
+rules, supported dtypes, and the `VLLM_ZENTORCH_WEIGHT_PREPACK` knob.
+
 ### How to launch a vLLM service on CPU?
 
 - When using the online serving, it is recommended to reserve 1-2 CPU cores for the serving framework to avoid CPU oversubscription. For example, on a platform with 32 physical CPU cores, reserving CPU 31 for the framework and using CPU 0-30 for inference threads:

docs/getting_started/installation/cpu.x86.inc.md

@@ -1,7 +1,15 @@
 <!-- markdownlint-disable MD041 -->
 --8<-- [start:installation]
 
-vLLM supports basic model inferencing and serving on x86 CPU platform, with data types FP32, FP16 and BF16.
+vLLM supports basic model inferencing and serving on x86 CPU platform, with
+data types FP32 and BF16.
+
+On AMD Zen 4 / Zen 5 (Genoa, Bergamo, Turin) CPUs, vLLM auto-activates a
+`ZenCpuPlatform` when a compatible
+[`zentorch`](https://github.com/amd/ZenDNN-pytorch-plugin) package is
+available. For release wheels, install the CPU wheel with the `zen` extra so
+vLLM pulls the tested `zentorch` version for that release. See
+[AMD Zen optimizations](#amd-zen-optimizations) below.
 
 --8<-- [end:installation]
 --8<-- [start:requirements]
@@ -216,5 +224,69 @@ docker run --rm \
 ```
 
 --8<-- [end:build-image-from-source]
+--8<-- [start:amd-zen-optimizations]
+
+On AMD Zen CPUs, vLLM auto-selects `ZenCpuPlatform` (a subclass of the default
+`CpuPlatform`) which dispatches linear layers through
+[`zentorch`](https://github.com/amd/ZenDNN-pytorch-plugin)'s ZenDNN-optimized
+kernels.
+
+#### Install
+
+```bash
+export VLLM_VERSION=0.20.2
+pip install "vllm[zen] @ https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}%2Bcpu-cp38-abi3-manylinux_2_35_x86_64.whl" \
+    --extra-index-url https://download.pytorch.org/whl/cpu
+```
+
+This keeps the vLLM CPU wheel and `zentorch` on the tested version combination,
+instead of installing `zentorch` separately and risking dependency skew.
+That's it - no flag or env var is required. vLLM's platform discovery picks up
+`ZenCpuPlatform` automatically the next time `vllm` is imported.
+
+#### Detection rules
+
+`ZenCpuPlatform` is selected when **all** of the following hold:
+
+- vLLM is built for CPU
+- `/proc/cpuinfo` reports `AuthenticAMD` and `avx512`.
+- `import zentorch` succeeds.
+
+Otherwise, vLLM falls back to the default `CpuPlatform` (oneDNN/sgl-kernel
+paths).
+
+#### Supported dtypes
+
+`ZenCpuPlatform.supported_dtypes` advertises only `bfloat16` and `float32`.
+Models declared with `torch_dtype=float16` are auto-downcast to `bfloat16` at
+load time with the standard
+`"Your device 'cpu' doesn't support torch.float16. Falling back to torch.bfloat16 for compatibility."`
+warning emitted from `vllm/config/model.py`.
+
+#### Verify it is engaged
+
+Run any model with INFO-level logs and grep for the activation banner:
+
+```bash
+VLLM_LOGGING_LEVEL=INFO vllm serve facebook/opt-125m --dtype bfloat16 \
+    2>&1 | grep -E "ZenCpuPlatform activated|CPU unquantized GEMM dispatch"
+```
+
+You should see one line confirming `ZenCpuPlatform activated` (with the
+`zentorch` version, `VLLM_ZENTORCH_WEIGHT_PREPACK` value, and AVX-512 /
+AVX-512_BF16 capability flags) and one line confirming each unquantized linear
+layer is routed through `zentorch_linear_unary`.
+
+#### Environment variables
+
+- `VLLM_ZENTORCH_WEIGHT_PREPACK` (default `1`): when set, eagerly prepacks
+  linear weights into ZenDNN's blocked layout at model load time, eliminating
+  per-inference layout conversion overhead. Set to `0` to disable.
+
+#### Reference
+
+For the design rationale, see [RFC #35089: In-Tree AMD Zen CPU Backend via zentorch](https://github.com/vllm-project/vllm/issues/35089).
+
+--8<-- [end:amd-zen-optimizations]
 --8<-- [start:extra-information]
 --8<-- [end:extra-information]

docs/models/hardware_supported_models/cpu.md

@@ -1,5 +1,8 @@
 # CPU - Intel® Xeon®
 
+!!! note "AMD Zen CPUs"
+    On AMD Zen 4 / Zen 5 CPUs, AMD Zen optimizations are auto-enabled when the [`zentorch`](https://github.com/amd/ZenDNN-pytorch-plugin) package is installed. All models supported by vLLM on CPU are supported on AMD Zen as well; model compatibility does not change. This page reflects the current CPU reference validation matrix on Intel systems. See [AMD Zen optimizations](../../getting_started/installation/cpu.md#amd-zen-optimizations) for details.
+
 ## Validated Hardware
 
 | Hardware |

tests/model_executor/test_cpu_unquantized_gemm_dispatch.py

@@ -2,11 +2,15 @@
 # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
 """Tests for CPU unquantized GEMM dispatch behavior."""
 
+import sys
+import types
+
 import pytest
 import torch
 
 from vllm.model_executor.layers import utils
 from vllm.platforms import current_platform
+from vllm.platforms import zen_cpu
 
 
 @pytest.fixture(scope="module")
@@ -66,3 +70,67 @@ def test_dispatch_cpu_unquantized_gemm_zen_remove_weight(monkeypatch):
     utils.dispatch_cpu_unquantized_gemm(layer, remove_weight=True)
 
     assert layer.weight.numel() == 0
+
+
+@pytest.mark.usefixtures("_mock_zentorch_linear_unary")
+def test_dispatch_cpu_unquantized_gemm_logs_zentorch_dispatch(monkeypatch):
+    monkeypatch.setattr(current_platform, "is_zen_cpu", lambda: True)
+    expected_prepacked = bool(utils.envs.VLLM_ZENTORCH_WEIGHT_PREPACK) and hasattr(
+        torch.ops.zentorch, "zentorch_weight_prepack_for_linear"
+    )
+
+    log_calls = []
+    monkeypatch.setattr(utils.logger, "info_once", lambda *args: log_calls.append(args))
+
+    layer = torch.nn.Linear(16, 8, bias=True)
+    utils.dispatch_cpu_unquantized_gemm(layer, remove_weight=False)
+
+    assert log_calls == [
+        (
+            "CPU unquantized GEMM dispatch: using zentorch_linear_unary "
+            "(prepacked=%s)",
+            expected_prepacked,
+        )
+    ]
+
+
+def test_zen_cpu_platform_logs_activation(monkeypatch):
+    fake_zentorch = types.SimpleNamespace(
+        __version__="5.2.1",
+        __config__="zentorch build config",
+    )
+    monkeypatch.setitem(sys.modules, "zentorch", fake_zentorch)
+    monkeypatch.setattr(zen_cpu.envs, "VLLM_ZENTORCH_WEIGHT_PREPACK", 1)
+    monkeypatch.setattr(zen_cpu.torch.cpu, "_is_avx512_supported", lambda: True)
+    monkeypatch.setattr(
+        zen_cpu.torch.cpu, "_is_avx512_bf16_supported", lambda: False
+    )
+
+    log_calls = []
+    monkeypatch.setattr(
+        zen_cpu.logger, "info_once", lambda *args: log_calls.append(args)
+    )
+
+    def fake_check_and_update_config(cls, vllm_config):
+        pass
+
+    monkeypatch.setattr(
+        zen_cpu.CpuPlatform,
+        "check_and_update_config",
+        classmethod(fake_check_and_update_config),
+    )
+
+    zen_cpu.ZenCpuPlatform.check_and_update_config(object())
+
+    assert log_calls == [
+        (
+            "ZenCpuPlatform activated | zentorch=%s | "
+            "VLLM_ZENTORCH_WEIGHT_PREPACK=%d | "
+            "AVX-512=%s | AVX-512_BF16=%s",
+            "5.2.1",
+            1,
+            True,
+            False,
+        ),
+        ("zentorch build config: %s", "zentorch build config"),
+    ]

vllm/model_executor/layers/utils.py

@@ -253,6 +253,11 @@ def dispatch_cpu_unquantized_gemm(
         )
         if remove_weight:
             layer.weight = torch.nn.Parameter(torch.empty(0), requires_grad=False)
+        logger.info_once(
+            "CPU unquantized GEMM dispatch: using zentorch_linear_unary "
+            "(prepacked=%s)",
+            is_prepacked,
+        )
         return
 
     if envs.VLLM_CPU_SGL_KERNEL and check_cpu_sgl_kernel(N, K, dtype):
@@ -266,6 +271,10 @@ def dispatch_cpu_unquantized_gemm(
         )
         if remove_weight:
             layer.weight = torch.nn.Parameter(torch.empty(0), requires_grad=False)
+        logger.info_once(
+            "CPU unquantized GEMM dispatch: using sgl-kernel (AMX) "
+            "weight_packed_linear"
+        )
         return
     elif (
         ops._supports_onednn
@@ -277,6 +286,9 @@ def dispatch_cpu_unquantized_gemm(
             layer.cpu_linear = lambda x, weight, bias: ops.onednn_mm(handler, x, bias)
             if remove_weight:
                 layer.weight = torch.nn.Parameter(torch.empty(0), requires_grad=False)
+            logger.info_once(
+                "CPU unquantized GEMM dispatch: using oneDNN onednn_mm"
+            )
             return
         except RuntimeError as e:
             logger.warning_once(
@@ -288,6 +300,10 @@ def dispatch_cpu_unquantized_gemm(
     layer.cpu_linear = lambda x, weight, bias: torch.nn.functional.linear(
         x, weight, bias
     )
+    logger.info_once(
+        "CPU unquantized GEMM dispatch: using torch.nn.functional.linear "
+        "(fallback)"
+    )
 
 
 def cpu_unquantized_gemm(

vllm/platforms/zen_cpu.py

@@ -1,13 +1,21 @@
 # SPDX-License-Identifier: Apache-2.0
 # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
 
+from typing import TYPE_CHECKING
+
 import torch
 
+from vllm import envs
 from vllm.logger import init_logger
 from vllm.platforms.cpu import CpuPlatform
 
 logger = init_logger(__name__)
 
+if TYPE_CHECKING:
+    from vllm.config import VllmConfig
+else:
+    VllmConfig = None
+
 
 class ZenCpuPlatform(CpuPlatform):
     """CPU platform with AMD Zen (ZenDNN/zentorch) optimizations.
@@ -30,3 +38,27 @@ def is_zen_cpu(self) -> bool:
     @property
     def supported_dtypes(self) -> list[torch.dtype]:
         return [torch.bfloat16, torch.float32]
+
+    @classmethod
+    def check_and_update_config(cls, vllm_config: VllmConfig) -> None:
+        super().check_and_update_config(vllm_config)
+
+        import zentorch
+
+        zentorch_version = getattr(zentorch, "__version__", "unknown")
+        avx512 = torch.cpu._is_avx512_supported()
+        avx512_bf16 = torch.cpu._is_avx512_bf16_supported()
+
+        logger.info_once(
+            "ZenCpuPlatform activated | zentorch=%s | "
+            "VLLM_ZENTORCH_WEIGHT_PREPACK=%d | "
+            "AVX-512=%s | AVX-512_BF16=%s",
+            zentorch_version,
+            int(envs.VLLM_ZENTORCH_WEIGHT_PREPACK),
+            avx512,
+            avx512_bf16,
+        )
+
+        zentorch_config = getattr(zentorch, "__config__", None)
+        if zentorch_config:
+            logger.info_once("zentorch build config: %s", zentorch_config)