feat: evaluate DSpark alongside DFlash 2, add DSpark llama-server presets - #4595
Merged
Conversation
…sets DSpark's `--spec-type draft-dspark` merged into llama.cpp on 2026-07-28 (ggml-org/llama.cpp#25173), while DFlash 2's engine modules are still an open PR (#27342) needing a from-source build. Since PortOS already manages llama-server and passes `--spec-type` straight through, DSpark is reachable today at zero backend cost — and is the more available of the two families. - Settings -> Local LLMs: DFLASH_PRESETS becomes SPEC_DECODE_PRESETS, led by two `draft-dspark` presets that run on a stock `brew install llama.cpp`. The DFlash 2 presets stay, relabelled with the build they require, because llama.cpp falls back silently on a spec-type/drafter mismatch rather than erroring. - Hugging Face model search: `dspark` joins the MLX drafter name filter. A real mlx-community repo declares `mlx` + `dspark` + `speculative-decoding` but no `draft-model` tag, so PortOS was offering a 30B target's drafter sidecar as a one-click install — a model that cannot chat. - Research note comparing the two families, covering the Ollama question: Ollama's vendored engine has carried DSpark since 0.32.6 but surfaces no control for it (its server hardcodes `--spec-type draft-mtp`), so the Ollama backend is not a usable path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Investigates DSpark (via ARahim3/mlx-dspark) against the DFlash 2 evaluation from earlier today, and answers the open question about Ollama support.
Verdict: worth adding — as a preset, not a backend. The prior DFlash 2 note concluded "not adoptable" partly because PortOS orchestrated no llama.cpp server. That changed: PortOS now manages
llama-server, and its Spec Type field passes straight through to--spec-type. That flips the ranking between the two drafter families:brew install llama.cppPortOS shipped presets exclusively for DFlash 2 — the one family needing an unmerged branch. DSpark needs zero new backend, zero new dependency, and no server change.
mlx-dsparkitself is not worth adopting — it would be a fourth local inference backend (pip, Apple-Silicon-only, MLX-safetensors-only) bought for a speedup PortOS already reaches, and a user who wants it needs nothing from PortOS anyway:mlx-dspark servelistens on127.0.0.1:8080with an OpenAI-compatible API, exactly where theopencode-llama-tuiprovider preset already points.Ollama support for DSpark: partial, and not usable. Ollama's vendored engine has carried DSpark since 0.32.6 (#17545), but Ollama surfaces no control — its server hardcodes
--spec-type draft-mtp. Selecting DSpark requires overridingLLAMA_ARG_SPEC_TYPEon the Ollama process and pointing at a raw blob-store sha256 path; the one reported measurement through that path was ~13%. ollama#17016 is still open.Changes
client/src/components/settings/LocalLlmTab.jsx—DFLASH_PRESETS→SPEC_DECODE_PRESETS, led by twodraft-dsparkpresets that run on a stock llama.cpp. The DFlash 2 presets stay, relabelled with the build they require, because llama.cpp falls back silently on a spec-type/drafter mismatch rather than erroring.aliasstaysdflashso the provider's default model alias keeps resolving.server/services/huggingFaceCatalog.js—dsparkjoins the MLX drafter name filter.mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-DSpark-bf16declaresmlx+dspark+speculative-decodingbut nodraft-modeltag, so the tag predicate alone offered a 30B target's drafter sidecar as a one-click install — a model that cannot chat. Same hazard class the DFlash 2 change fixed, new family.docs/research/2026-08-19-dspark-vs-dflash2.md— the full comparison, including the quantified reason the GGUF branch is deliberately left alone (name-matchingdsparkthere would hide mainstream complete models, the worse failure).docs/features/dflash2.md— generalized to cover both families, with the availability guidance and an Ollama section.Test plan
cd server && npm test— 1486 files, 30670 passedcd client && npm test— 681 files, 8408 passedhuggingFaceCatalog.test.js:727verified failing before the fix (expected true to be false) and passing after.