server : add stuck-loop escape for ngram-mod (WIP)

[yychyo]

❗ This is WiP and a mitigation PR, not an actual fix.

Overview

When ngram-mod speculative decoding fails verification, spec_draft is set to the accepted tokens (including the correction token) and the checkpoint is restored. On the next iteration the draft is reused instead of regenerated and sometimes fails again, creating a loop with non-deterministic end condition.

This PRs adds logs to diagnose the problem, and a "band-aid" mitigation - it detects loop, and breaks out of it. However, the root cause of the loop is not yet clear.

Logs

Here are the logs that shows the issue:

❯ grep -B 60 -r "^8.36.225.102" debug-logs/ | grep 'issue#23268'
debug-logs/llama-20260717-144519.log-8.34.175.998 D slot   operator(): id  7 | task 0 | [issue#23268] PRE_DRAFT: spec_draft.empty() = no, n_past = 47206
debug-logs/llama-20260717-144519.log-8.34.690.806 D [issue#23268] VERIFY_FAIL: idx=20 draft=63 (logit=24.2188) sampled=7561 (logit=24.2500)
debug-logs/llama-20260717-144519.log-8.34.690.816 D slot   operator(): id  7 | task 0 | [issue#23268] ROLLBACK: streak = 2, spec_draft = 21 tokens
debug-logs/llama-20260717-144519.log-8.34.696.216 D slot   operator(): id  7 | task 0 | [issue#23268] PRE_DRAFT: spec_draft.empty() = no, n_past = 47206
debug-logs/llama-20260717-144519.log-8.35.205.159 D [issue#23268] VERIFY_FAIL: idx=20 draft=7561 (logit=24.2188) sampled=63 (logit=24.3125)
debug-logs/llama-20260717-144519.log-8.35.205.170 D slot   operator(): id  7 | task 0 | [issue#23268] ROLLBACK: streak = 3, spec_draft = 21 tokens
debug-logs/llama-20260717-144519.log-8.35.210.298 D slot   operator(): id  7 | task 0 | [issue#23268] PRE_DRAFT: spec_draft.empty() = no, n_past = 47206
debug-logs/llama-20260717-144519.log-8.35.717.474 D [issue#23268] VERIFY_FAIL: idx=20 draft=63 (logit=24.2188) sampled=7561 (logit=24.2500)
debug-logs/llama-20260717-144519.log-8.35.717.489 W slot   operator(): id  7 | task 0 | [issue#23268] STUCK speculative loop: 4 consecutive checkpoint restores with no progress (matched 20/21 draft tokens), diverging at draft index 20: draft 63 ('`', logit 24.2188) vs sampled 7561 ('`,', logit 24.2500). Applying mitigation to force progress - please report with server logs at https://github.com/ggml-org/llama.cpp/issues/23268
debug-logs/llama-20260717-144519.log-8.35.717.491 D slot   operator(): id  7 | task 0 | [issue#23268] ROLLBACK: streak = 4, spec_draft = 20 tokens
debug-logs/llama-20260717-144519.log-8.35.721.472 D slot   operator(): id  7 | task 0 | [issue#23268] PRE_DRAFT: spec_draft.empty() = no, n_past = 47206
debug-logs/llama-20260717-144519.log:8.36.225.102 I slot   operator(): id  7 | task 0 | [issue#23268] recovered from stuck speculative loop after 4 iterations

Additional information

• Eval bug: Speculative Decoding intermittent timeout #23268

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES, llama.cpp + qwen3.6 35b-a3b + pi

[Janno]

This PR fixes reproducable (endless?) loops I get on Nemotron-3.5-Lightning-30B-A3B.

EDIT: The PR title implies this is related to ngram-mod but even with only MTP I get loops that are fixed by the changes here.

[leinad-sch]

Same for Qwen3.x models, using ngram-mod.
Funnily I did not face the issue until I applied the fixes for the slot and checkpoint save/restore.

Would be great to have a fix merged.