Skip to content

model: add DSpark support for Nemotron3.5 - #27804

Merged
ruixiang63 merged 3 commits into
ggml-org:masterfrom
ruixiang63:nemotron-dspark
Aug 27, 2026
Merged

model: add DSpark support for Nemotron3.5#27804
ruixiang63 merged 3 commits into
ggml-org:masterfrom
ruixiang63:nemotron-dspark

Conversation

@ruixiang63

Copy link
Copy Markdown
Member

Overview

Support DSpark model for Nemotron3.5-lightning NVFP4: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark

Performance results: (Tested with NVIDIA-Nemotron-3.5-Lightning-30B-A3B-UD-Q4_K_M.gguf on NVIDIA L40S)

Comparison: baseline=speed-bench-qualitative-nemotron.json speculative=speed-bench-qualitative-nemotron-dspark.json
category       base_avg_pred_t/s  spec_avg_pred_t/s  decode_speedup  base_avg_latency  spec_avg_latency  latency_speedup  accept_rate
-------------  -----------------  -----------------  --------------  ----------------  ----------------  ---------------  -----------
coding         165.48             214.77             1.30x           3.337s            2.925s            1.14x            0.4431     
humanities     165.16             202.66             1.23x           4.943s            4.286s            1.15x            0.3731     
math           164.42             221.93             1.35x           3.220s            2.403s            1.34x            0.4621     
qa             164.70             188.28             1.14x           3.244s            2.892s            1.12x            0.3553     
rag            165.12             244.60             1.48x           5.019s            3.847s            1.30x            0.5454     
reasoning      164.99             222.26             1.35x           3.228s            2.443s            1.32x            0.4621     
stem           164.82             220.01             1.33x           3.150s            2.375s            1.33x            0.4621     
writing        163.94             186.67             1.14x           3.400s            3.111s            1.09x            0.3620     
multilingual   164.76             235.47             1.43x           3.285s            2.424s            1.35x            0.4849     
summarization  165.49             221.08             1.34x           3.182s            2.527s            1.26x            0.4626     
roleplay       165.25             224.46             1.36x           8.268s            6.387s            1.29x            0.4742     
overall        164.92             216.56             1.31x           4.025s            3.238s            1.24x            0.4432  

Additional information

Requirements

@ruixiang63
ruixiang63 requested review from a team and CISC as code owners August 27, 2026 14:23
@github-actions github-actions Bot added model Model specific conversion labels Aug 27, 2026
@ruixiang63
ruixiang63 requested a review from ggerganov August 27, 2026 14:34
Comment thread src/models/dflash.cpp Outdated
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
@ruixiang63

Copy link
Copy Markdown
Member Author

@ggerganov needs one more approval.

@ruixiang63

Copy link
Copy Markdown
Member Author

Thanks @ngxson for resolving the conflicts. Merging this PR.

@ruixiang63
ruixiang63 merged commit ca3d5a3 into ggml-org:master Aug 27, 2026
27 of 30 checks passed
@ruixiang63
ruixiang63 deleted the nemotron-dspark branch August 28, 2026 22:12
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request Sep 5, 2026
* model: add DSpark support for Nemotron3.5

* Update src/models/dflash.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
thecodacus pushed a commit to thecodacus/llama.cpp that referenced this pull request Sep 7, 2026
* model: add DSpark support for Nemotron3.5

* Update src/models/dflash.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conversion model Model specific

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants