Emit GGUF zero points explicitly (#459)

## Summary

This is the clean-main rebuild of the urgent correctness fix from #458.
It contains only the GGUF zero-point fix, based on current
`origin/main`.

GGUF Q4_0 and Q8_0 are symmetric formats, but their dequantization
formulas still require non-zero zero points:

- Q4_0: `(q - 8) * scale`
- Q8_0: `(q - 128) * scale`

Mobius previously marked those GGUF imports as symmetric, so
`QuantizedLinear` and `QuantizedEmbedding` omitted the optional
`zero_points` input for `MatMulNBits` and `GatherBlockQuantized`. That
is not portable: `GatherBlockQuantized` defaults diverge between ORT CPU
and ORT CUDA when `zero_points` is omitted, corrupting CUDA embeddings
before the first decoder layer runs.

This PR makes GGUF Q4_0/Q8_0 emit explicit zero-point initializers, so
both `GatherBlockQuantized` and `MatMulNBits` receive the intended
values instead of relying on EP defaults.

## Regression test

Added a single-node `GatherBlockQuantized` runtime regression:

- qweight nibbles are all `10`
- scales are known constants
- explicit zero point is `8`
- output is asserted against hand-computed `(q - 8) * scale`
- rerunning with zero point `0` is asserted not to match

I also temporarily flipped the test's zero point to `0`; it failed with
a 100% output mismatch, so the test checks the value and not just the
presence of the input.

Synthetic GGUF tests now also assert:

- Q4_0 `MatMulNBits` nodes have the explicit fourth input
- Q4_0 `GatherBlockQuantized` has the explicit fourth input
- Q4_0 embedding zero points are packed as `0x88`

## Quantization types checked

Direct GGUF repackers already produce explicit zero-points for supported
direct formats:

- Q4_0: zp=8
- Q4_1: per-block affine zp
- Q4_K: requantized per-block affine zp
- Q8_0: zp=128
- Q1_0: zp=1; Tencent Q1_0 uses its custom zp path

Q5_0/Q5_1/Q5_K/Q6_K are not direct `MatMulNBits` repack targets here;
when they appear in mixed presets they go through
dequantize+requantize/native-block fallback paths, which produce
explicit zero-points where needed.

## Validation

- `python -m pytest src\mobius\integrations\gguf\_builder_test.py -q` ->
`33 passed`
- `python -m pytest src\mobius\integrations\gguf\ src\mobius\_configs\
src\mobius\_model_package_test.py` -> `304 passed`
- `python -m ruff check src\mobius\integrations\gguf\_builder.py
src\mobius\integrations\gguf\_builder_test.py` -> passed
- `python -m ruff format --check
src\mobius\integrations\gguf\_builder.py
src\mobius\integrations\gguf\_builder_test.py` -> passed

Fresh conversion validated with
`C:\Users\justinchu\dev\models-gguf\qwen2.5-0.5b-instruct-q4_0.gguf`:

```powershell
python -m mobius build-gguf C:\Users\justinchu\dev\models-gguf\qwen2.5-0.5b-instruct-q4_0.gguf --output C:\Users\justinchu\dev\models\qwen2.5-0.5b-q4_0-mobius --keep-quantized --dtype f16 --ep cuda
```

Converted graph verification:

- `GatherBlockQuantized` input count: 4
- all `MatMulNBits` input counts: 4
- zero-point initializers: 170

Generation checks on the freshly converted model, no post-hoc graph
patching:

- ORT CPU via onnx-genai CLI: `The capital of France is Paris. It is the
largest city in`
- ORT CUDA via onnx-genai CLI (`ONNX_GENAI_ORT_LIB_DIR` pointed at the
installed onnxruntime-gpu package, CUDA/cuDNN DLL dirs on PATH): `The
capital of France is Paris. It is the largest city in`
- native CUDA via onnx-genai CLI (`--features native-cuda`, CUDA/cuDNN
DLL dirs on PATH): `The capital of France is Paris. It is the largest
city in`

## Upstream ORT issue

Reported the provider default divergence here:
microsoft/onnxruntime#31692

## Relationship to #458

#458 was accidentally based on another in-flight feature branch and
includes unrelated commits. This PR is the clean-main replacement for
the urgent zero-point correctness fix only.

---------

Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: justinchuby <11205048+justinchuby@users.noreply.github.com>

Author: 3 people

src/mobius/integrations/gguf/_builder.py

@@ -528,19 +528,25 @@ def _detect_quant_params(gguf_model, gguf_arch: str) -> tuple[int, int, bool]:
         map_gguf_to_hf_names,
     )
 
-    # Symmetry of each supported GGUF quantization type.
+    # Whether the graph can omit zero_points for each supported GGUF type.
     #
     # Mainline Q1_0 (1-bit binary) is repacked into 2-bit MatMulNBits
     # with zp=1 — see _repack_q1_0. Tencent's custom Q1_0 (2-bit SEQ,
     # 512-elt blocks) is inflated to 4-bit MatMulNBits with zp=3 — see
     # parse_tencent_q1_0_tensor — because the ORT CPU unpacked-float-zp
     # path is currently only implemented for bits=4, and the half-integer
     # SEQ offset 1.5 cannot be expressed with integer zp at bits=2.
-    type_symmetry: dict = {
-        GGMLQuantizationType.Q4_0: True,
+    #
+    # Q4_0 and Q8_0 are symmetric formats, but their GGUF dequantization
+    # formulas are still ``(q - 8) * scale`` and ``(q - 128) * scale``.
+    # Emit those zero_points explicitly: GatherBlockQuantized has diverging
+    # CPU/CUDA defaults when the input is omitted, which corrupts embeddings
+    # on CUDA before the first decoder layer runs.
+    type_can_omit_zero_points: dict = {
+        GGMLQuantizationType.Q4_0: False,
         GGMLQuantizationType.Q4_1: False,
         GGMLQuantizationType.Q4_K: False,
-        GGMLQuantizationType.Q8_0: True,
+        GGMLQuantizationType.Q8_0: False,
         GGMLQuantizationType.Q1_0: False,
     }
 
@@ -561,17 +567,26 @@ def _detect_quant_params(gguf_model, gguf_arch: str) -> tuple[int, int, bool]:
         {qtype: count for qtype, count in counts.items() if _native_block_format(qtype)}
     )
     if native_counts:
-        asymmetric_types = {"Q2_K", "Q4_1", "Q4_K", "Q5_1", "Q5_K"}
-        is_sym = not any(
-            getattr(qtype, "name", None) in asymmetric_types
+        explicit_zero_point_types = {
+            "Q1_0",
+            "Q2_K",
+            "Q4_0",
+            "Q4_1",
+            "Q4_K",
+            "Q5_1",
+            "Q5_K",
+            "Q8_0",
+        }
+        can_omit_zero_points = not any(
+            getattr(qtype, "name", None) in explicit_zero_point_types
             for qtype in counts
             if qtype not in native_counts
         )
         logger.info(
             "Native GGUF quant types present; using 4-bit/block-32 module "
             "scaffolding for non-native quantized tensors",
         )
-        return 4, 32, is_sym
+        return 4, 32, can_omit_zero_points
 
     # Q4_K_M is deliberately a mixed preset. Depending on tensor dimensions
     # and importance it may contain mostly Q5_0 plus Q4_K, Q6_K, and Q8_0.
@@ -598,7 +613,7 @@ def _detect_quant_params(gguf_model, gguf_arch: str) -> tuple[int, int, bool]:
     params = repack_quant_params(dominant_value)
     assert params is not None
     bits, block_size = params
-    is_sym = type_symmetry[dominant]
+    is_sym = type_can_omit_zero_points[dominant]
 
     # Tencent Q1_0 files reuse the Q1_0 type id but ship a different
     # on-disk layout (2-bit SEQ, 512-element blocks, fp16 scale per block).

src/mobius/integrations/gguf/_builder_test.py

@@ -8,9 +8,72 @@
 from pathlib import Path
 
 import numpy as np
+import onnx_ir as ir
+import onnxruntime as ort
 import pytest
 
 
+def _run_gather_block_quantized(
+    tmp_path: Path,
+    *,
+    zero_point: int,
+) -> np.ndarray:
+    """Run a tiny GatherBlockQuantized graph with a controlled zero point."""
+    qweight = np.full((2, 16), 0xAA, dtype=np.uint8)
+    scales = np.array([[0.5], [0.25]], dtype=np.float16)
+    zero_points = np.full((2, 1), zero_point, dtype=np.uint8)
+
+    def _const(name: str, arr: np.ndarray) -> ir.Value:
+        value = ir.Value(name=name)
+        tensor = ir.tensor(arr)
+        value.const_value = tensor
+        value.shape = ir.Shape(arr.shape)
+        value.dtype = tensor.dtype
+        return value
+
+    input_ids = ir.Value(
+        name="input_ids",
+        shape=ir.Shape([2]),
+        type=ir.TensorType(ir.DataType.INT64),
+    )
+    output = ir.Value(
+        name="output",
+        shape=ir.Shape([2, 32]),
+        type=ir.TensorType(ir.DataType.FLOAT16),
+    )
+    qweight_init = _const("qweight", qweight)
+    scales_init = _const("scales", scales)
+    zero_points_init = _const("zero_points", zero_points)
+    node = ir.Node(
+        "com.microsoft",
+        "GatherBlockQuantized",
+        inputs=[
+            qweight_init,
+            input_ids,
+            scales_init,
+            zero_points_init,
+        ],
+        outputs=[output],
+        attributes=ir.convenience.convert_attributes(
+            {"bits": 4, "block_size": 32, "gather_axis": 0, "quantize_axis": 1}
+        ),
+    )
+    graph = ir.Graph(
+        inputs=[input_ids],
+        outputs=[output],
+        nodes=[node],
+        initializers=[qweight_init, scales_init, zero_points_init],
+        opset_imports={"": 18, "com.microsoft": 1},
+        name="gbq_zero_point",
+    )
+    model = ir.Model(graph, ir_version=10)
+    path = tmp_path / f"gbq_zp_{zero_point}.onnx"
+    ir.save(model, path)
+    session = ort.InferenceSession(str(path), providers=["CPUExecutionProvider"])
+    (result,) = session.run(None, {"input_ids": np.array([0, 1], dtype=np.int64)})
+    return result
+
+
 def _write_quantized_gguf(
     path: Path,
     *,
@@ -311,6 +374,20 @@ def test_model_has_matmulnbits_ops(self, q4_0_gguf: Path):
             f"Expected MatMulNBits in ops, got: {sorted(op_types)}"
         )
 
+    def test_q4_0_matmulnbits_has_explicit_zero_points(self, q4_0_gguf: Path):
+        """GGUF Q4_0 projections explicitly encode zp=8 instead of EP defaults."""
+        from mobius.integrations.gguf import build_from_gguf
+
+        model = build_from_gguf(q4_0_gguf, keep_quantized=True)["model"]
+        nodes = [node for node in model.graph if node.op_type == "MatMulNBits"]
+        assert nodes
+        for node in nodes:
+            assert len(node.inputs) == 4
+            zero_point_name = node.inputs[3].name
+            assert zero_point_name.endswith(".zero_points")
+            zero_points = model.graph.initializers[zero_point_name]
+            np.testing.assert_array_equal(zero_points.const_value.numpy(), 0x88)
+
     def test_native_blocks_emit_block_quantized_matmul_and_preserve_bytes(
         self,
         native_block_gguf: tuple[Path, str, int, int],
@@ -380,6 +457,7 @@ def test_quantized_embedding_uses_gatherblockquantized(self, q4_0_embedding_gguf
         gather_nodes = [node for node in model.graph if node.op_type == "GatherBlockQuantized"]
         assert len(gather_nodes) == 1
         assert gather_nodes[0].domain == "com.microsoft"
+        assert len(gather_nodes[0].inputs) == 4
 
         qweight = model.graph.initializers["model.embed_tokens.qweight"]
         assert qweight.dtype == ir.DataType.UINT8
@@ -388,8 +466,26 @@ def test_quantized_embedding_uses_gatherblockquantized(self, q4_0_embedding_gguf
             256,
             2,
         ]
+        zero_points = model.graph.initializers["model.embed_tokens.zero_points"]
+        assert zero_points.dtype == ir.DataType.UINT8
+        assert list(zero_points.shape) == [256, 1]
+        np.testing.assert_array_equal(zero_points.const_value.numpy(), 0x88)
         assert "model.embed_tokens.weight" not in model.graph.initializers
 
+    def test_gatherblockquantized_zero_point_dequantizes_q4_0(self, tmp_path: Path):
+        """GatherBlockQuantized output must match GGUF Q4_0's ``(q - 8) * scale``."""
+        actual = _run_gather_block_quantized(tmp_path, zero_point=0x08).astype(np.float32)
+        expected = np.stack(
+            [
+                np.full(32, (10 - 8) * 0.5, dtype=np.float32),
+                np.full(32, (10 - 8) * 0.25, dtype=np.float32),
+            ]
+        )
+        np.testing.assert_allclose(actual, expected)
+
+        wrong = _run_gather_block_quantized(tmp_path, zero_point=0x00).astype(np.float32)
+        assert not np.allclose(wrong, expected)
+
     def test_tied_quantized_embedding_drives_matmulnbits_head(
         self, q4_0_tied_embedding_gguf: Path
     ):
@@ -401,6 +497,7 @@ def test_tied_quantized_embedding_drives_matmulnbits_head(
         assert op_types.count("GatherBlockQuantized") == 1
         assert "MatMulNBits" in op_types
         assert "model.embed_tokens.qweight" in model.graph.initializers
+        assert "model.embed_tokens.zero_points" in model.graph.initializers
         assert not any(name.startswith("lm_head.") for name in model.graph.initializers)
 
     def test_untied_quantized_head_uses_q4_matmulnbits(
@@ -479,7 +576,7 @@ def test_detect_quant_params(self, q4_0_gguf: Path):
         bits, block_size, is_sym = _detect_quant_params(gguf_model, gguf_model.architecture)
         assert bits == 4
         assert block_size == 32
-        assert is_sym is True
+        assert is_sym is False
 
     def test_embedding_quantization_check_is_metadata_only(self, monkeypatch):
         """Embedding compatibility does not read or repack tensor data."""