QVAC-24253 feat[api]: expose split-mode 'tensor' in llm-llamacpp

packages/embed-llamacpp/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## [0.37.0] - 2026-08-29
+
+### Changed
+
+- `qvac-fabric` dependency bumped `10297.0.0` -> `10297.1.1` (MTP drafter, pipeline-parallel ACCEL fix, Metal optimisations, Qwen4-Next support and fit host-memory budgeting, plus the Qwen4-Next perf follow-ups and the Vulkan top-k radix-select shader; no API change for this package).
+
 ## [0.36.0] - 2026-08-24
 
 ### Changed

packages/embed-llamacpp/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qvac/embed-llamacpp",
-  "version": "0.36.0",
+  "version": "0.37.0",
   "description": "bert addon for qvac",
   "addon": true,
   "engines": {

packages/embed-llamacpp/vcpkg.json

@@ -8,7 +8,7 @@
     },
     {
       "name": "qvac-fabric",
-      "version>=": "10297.0.0",
+      "version>=": "10297.1.1",
       "features": [
         "vector-index"
       ]

packages/fabric/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## [0.10.0] - 2026-08-29
+
+### Changed
+
+- `qvac-fabric` dependency bumped `10297.0.0` -> `10297.1.1` (MTP drafter, pipeline-parallel ACCEL fix, Metal optimisations, Qwen4-Next support and fit host-memory budgeting, plus the Qwen4-Next perf follow-ups and the Vulkan top-k radix-select shader; no API change for this package).
+
 ## [0.9.0] - 2026-08-25
 
 ### Added

packages/fabric/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qvac/fabric",
-  "version": "0.9.0",
+  "version": "0.10.0",
   "description": "Shared bare addon hosting the qvac-fabric (forked llama.cpp + ggml) runtime for QVAC inference addons",
   "addon": true,
   "engines": {

packages/fabric/vcpkg.json

@@ -6,7 +6,7 @@
     },
     {
       "name": "qvac-fabric",
-      "version>=": "10297.0.0",
+      "version>=": "10297.1.1",
       "features": [
         "hip-backend"
       ]

packages/llm-llamacpp/CHANGELOG.md

@@ -1,5 +1,47 @@
 # Changelog
 
+## [0.49.0] - 2026-08-31
+
+### Added
+
+- `split-mode: 'tensor'` enables qvac-fabric's meta-device tensor parallelism, splitting weights
+  and KV cache across all visible GPUs. **EXPERIMENTAL and desktop-only** (still rejected on
+  Android/iOS with the other multi-GPU parameters). Three constraints, all enforced up front with
+  `InvalidArgument` rather than surfacing as an opaque native failure:
+  - Requires flash attention — a falsey `flash-attn` is rejected. qvac-fabric treats `off`,
+    `disabled`, `false` and `0` as equivalent, and all four are rejected under both the
+    `flash-attn` and `flash_attn` spellings. Leaving it unset is fine; it already defaults to `on`.
+  - Disables auto-fit, which qvac-fabric does not implement for this mode: `gpu_layers` then
+    defaults to every layer and `ctx_size` to the model's trained context, so **set `ctx_size`
+    explicitly for large models** or the load can OOM. The override is applied after argument
+    parsing, so an explicit `fit: 'on'` cannot silently re-enable it. Logged at WARNING.
+  - Unavailable for some architectures (Mamba/Jamba-family, BitNet, Grok, T5, DeepSeek-V2/3.2,
+    MiniMax, Qwen3-Next and others as of qvac-fabric v10297.1.1); rejected before loading with
+    the architecture named. `deepseek4`, `qwen35` and `qwen35moe` were unsupported at v10297.0.0
+    and are supported from v10297.1.0.
+
+  Tensor mode pins its own `--device` list. qvac-fabric selects devices for this mode with no
+  device-type filter and no deduplication, so left alone it splits weights and KV cache onto the
+  integrated GPU of any discrete + integrated host and shards a dual-registered GPU twice. The
+  addon enumerates devices itself — discrete when present, otherwise integrated, deduplicated by
+  the backend-reported `device_id` (PCI bus id), not by description: two identical cards report
+  identical descriptions. `layer` and `row` are unchanged and still let qvac-fabric choose.
+
+  **Not selectable through the SDK yet:** `@qvac/inference`'s config schema still enumerates
+  `none`/`layer`/`row`, so `'tensor'` is reachable only via direct addon `loadModel`.
+
+  Unrelated to `split-mode: 'row'`, which needs split buffers no shipped backend provides and is
+  still degraded to `'layer'`. `examples/multiGpuBenchmark.js` now benchmarks the new mode
+  alongside the existing three. See `docs/multi-gpu.md`.
+
+- `flash-attn` (and the `flash_attn` alias) is now a narrowed field on `LlamaConfig` rather than
+  reaching callers only through the `[key: string]` escape hatch, making the tensor-mode
+  requirement visible at compile time. Propagating it to the SDK schema is separate SDK-pod work.
+
+### Changed
+
+- `qvac-fabric` dependency bumped `10297.0.0` -> `10297.1.1` (MTP drafter, pipeline-parallel ACCEL fix, Metal optimisations, Qwen4-Next support and fit host-memory budgeting, plus the Qwen4-Next perf follow-ups and the Vulkan top-k radix-select shader).
+
 ## [0.48.0] - 2026-08-31
 
 ### Removed

packages/llm-llamacpp/README.md

@@ -176,8 +176,8 @@ const config = {
 | tools             | `"true"` or `"false"`                       | `"false"`                    | Enable tool calling with jinja templating             |
 | verbosity         | 0 – 3 (0=ERROR, 1=WARNING, 2=INFO, 3=DEBUG) | 0                            | Logging verbosity level                               |
 | main-gpu          | integer, `"integrated"`, or `"dedicated"`   | —                            | GPU selection for multi-GPU systems                   |
-| split-mode        | `"none"`, `"layer"`, or `"row"`             | `"none"`                     | How to split the model across GPUs ([details](./docs/multi-gpu.md)) |
-| tensor-split      | comma-separated proportions (e.g. `"1,1"`)  | —                            | GPU split ratios for layer/row parallelism ([details](./docs/multi-gpu.md)) |
+| split-mode        | `"none"`, `"layer"`, `"row"`, or `"tensor"` | `"none"`                     | How to split the model across GPUs. `"tensor"` is EXPERIMENTAL and desktop-only ([details](./docs/multi-gpu.md)) |
+| tensor-split      | comma-separated proportions (e.g. `"1,1"`)  | —                            | GPU split ratios for the multi-GPU split modes ([details](./docs/multi-gpu.md)) |
 | parallel          | integer                                     | 1                            | Concurrent sequence slots for continuous batching. Values `>= 2` enable batch `run()` and split the KV cache uniformly across slots ([details](./docs/continuous-batching.md)) |
 | cache-type-k      | `f16`, `f32`, `bf16`, `q8_0`, `q4_0`, …      | auto (see below)             | KV-cache **key** quantization type. Unset = auto-default (see KV-cache type below) |
 | cache-type-v      | `f16`, `f32`, `bf16`, `q8_0`, `q4_0`, …      | auto (see below)             | KV-cache **value** quantization type. Quantizing V requires `flash-attn` on |