llama : add backends of the other model to the context

[wOvAN]

For models that share tensors with another model through ctx_other (e.g. Gemma4Assistant, Eagle3, DFlash, DSpark), add the other model's device backends to the context scheduler so that pre-allocated shared tensors (tok_embd, output) can be scheduled. Fixes an abort when the speculative draft model uses a device list that does not cover the GPU holding the target model's output projection.

Assisted-by: opencode

Overview

Bug (observed behavior)

• Abort during llama_context construction: "pre-allocated tensor (output.weight) in a buffer (CUDA10) that cannot run the operation (NONE)"
• Location: ggml/src/ggml-backend.cpp (line ~898/930 depending on version), called from llama_context::sched_reserve -> graph_reserve -> resolve_fused_ops
• Trigger: speculative decoding with ctx_other (draft-dspark / draft-dflash / draft-mtp), draft model on its own device via --device-draft/--spec-draft-device, target model on other GPU(s)
• Root cause : the draft context scheduler only had backends for the draft's own devices; shared pre-allocated tensors (tok_embd/output.weight, taken from the target model via ctx_other in src/models/dflash.cpp) live in buffers on the target's devices; scheduler found no backend for those buffers and aborted

Reproduction (your setup)

• llama-server in router mode, model preset:
  • device = CUDA3,CUDA4,CUDA5,CUDA7,CUDA8,CUDA9,CUDA10 (--split-mode layer)
  • spec-type = draft-dspark, spec-draft-n-max = 8
  • spec-draft-device = CUDA0
  • model-draft = dspark/dspark-DeepSeek-V4-Flash-0731-Q8_0.gguf
• Before fix: GGML_ABORT, spec model failed to load
• After fix: draft context logs "adding backend for device CUDA3..CUDA10: shared tensors with the other model (e.g. tok_embd, output)", speculative implementation 'draft-dspark' initializes (n_max=8, block_size=5), server healthy
• Verified with CI-built image version 10282 (34ec9c9bb) on 15-GPU host (RTX 3080/4080/4090/3090), CUDA 12, 16x 3090-class workload

Additional information

Related issues

• Eval bug: using -devd CUDA0 on draft-dspark causes model to crash #26475 - "using -devd CUDA0 on draft-dspark causes model to crash": same abort message (output.weight, CUDA8), same stack, draft-dspark + tensor split. Directly matches. (open)
• Eval bug: Gemma 4 31B MTP (draft-mtp) crashes on Vulkan backend, pre-allocated tensor cannot run operation NONE #24492 - Gemma 4 31B MTP, Vulkan: "pre-allocated tensor (cache_k_l58) ... cannot run the operation (NONE)" - same abort class, ctx_other/KV sharing; mechanism may differ (Vulkan backend support), unknown whether fixed
• Eval bug: Gemma MTP: Tensor in buffer cannot run (NONE) #24366 - Gemma MTP, tensor split, buffer "Meta()": same abort class; unknown whether fixed
• Dspark model fails to load with -sm tensor on MI50 #26339 - "Dspark model fails to load with -sm tensor on MI50": possible same class; unknown

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - opencode + ds4

[wOvAN]

in short: now you can set spec-draft-device = CUDA0 on a separate device(s) from the main model device = CUDA3,CUDA4,CUDA5,CUDA7,CUDA8,CUDA9,CUDA10

[wOvAN]

also the err mentioned in
https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF/blob/main/dspark/README.md#usage

"Do not pass -devd / --spec-draft-device"