feat(agent): nudge model on empty response after tool calls

[g4bwy]

Weaker models sometimes return an empty response after executing tool calls (no text, no tool uses, just end_turn), silently abandoning incomplete multi-step tasks.

When this happens, inject a synthetic user message asking the model to process the tool results and continue. Guarded by a per-tool-round flag and the global turn limit to prevent infinite loops.

• Add AgentEvent::Nudge variant and UI handling
• Add History::has_recent_tool_results() helper
• Make Message::first_text_content() public
• Add nudge detection in turn() with synthetic message injection

Ported from: NousResearch/hermes-agent#10065

Addresses issue #299 (among others)

[g4bwy]

draft until issue reproduced

[sdroege]

You can get into this situation quite easily with Mistral Small 4 in case you can run that. Needs some "go ahead" poking to actually continue until the end :)

[g4bwy]

You can get into this situation quite easily with Mistral Small 4 in case you can run that. Needs some "go ahead" poking to actually continue until the end :)

"small" is kind of ironic in that case, but I may be able to load a Q2 quant of it 🤣 otherwise, I used to get this issue quite a lot with Q4 quants of qwen3.6 before I got more VRAM, maybe I'll try that first

[g4bwy]

You can get into this situation quite easily with Mistral Small 4 in case you can run that. Needs some "go ahead" poking to actually continue until the end :)

I managed to load the IQ2_XXS quant on my rig, but I probably have a wrong template or sampling parameters, can't get anything useful out of it without having it do infinite loops on the same failing tool calls... do you know how to run it with llama.cpp ?

[sdroege]

[unsloth/Mistral-Small-4-119B-2603-GGUF:Q4_K_M]
mmap = true
fit = on
fit-target = 0
no-mmproj = true
spec-type = ngram-mod

hf = unsloth/Mistral-Small-4-119B-2603-GGUF:Q4_K_M
c = 131072
temp = 0.7
chat-template-kwargs = {"reasoning_effort": "high"}

is my configuration. FWIW, just today I had the problematic behaviour with qwen3.6 27B but I didn't have this PR applied...

[g4bwy]

[unsloth/Mistral-Small-4-119B-2603-GGUF:Q4_K_M]
mmap = true
fit = on
fit-target = 0
no-mmproj = true
spec-type = ngram-mod

hf = unsloth/Mistral-Small-4-119B-2603-GGUF:Q4_K_M
c = 131072
temp = 0.7
chat-template-kwargs = {"reasoning_effort": "high"}

is my configuration. FWIW, just today I had the problematic behaviour with qwen3.6 27B but I didn't have this PR applied...

can't get much out of it even with these params... the damn thing just returns directly with "Task completed" after doing nothing when I ask it to chain random tool calls in a loop 😢

which quant/params + prompt do you use with qwen3.6 27b to reproduce ? I haven't seen a stall for several days in a row (sphaela/Qwen3.6-27B-AutoRound-GGUF:Q8_0 here with 262144 context + MTP + temp 0.6 + top-p/top-k 0.95/20 + preserve_thinking). I tried with the default template and several different community templates to no avail...

[sdroege]

which quant/params + prompt do you use with qwen3.6 27b to reproduce ? I haven't seen a stall for several days in a row (sphaela/Qwen3.6-27B-AutoRound-GGUF:Q8_0 here with 262144 context + MTP + temp 0.6 + top-p/top-k 0.95/20 + preserve_thinking). I tried with the default template and several different community templates to no avail...

It happened exactly once yesterday in the middle of a long session in maki. It's definitely not easy to trigger.

FWIW:

[unsloth/Qwen3.6-27B-MTP-GGUF:Q4_K_M]
ngl = 65 # exactly 1 layer offloaded to CPU is fastest on my setup for whatever reason!
ctk = q8_0
ctv = q8_0
ctkd = q8_0
ctvd = q8_0
spec-type = ngram-mod,draft-mtp
spec-draft-n-max = 3
no-mmproj = true

c = 131072
temp = 0.6
top-p = 0.95
top-k = 20
min-p = 0.0
presence-penalty = 0.0
repeat-penalty = 1.0
chat-template-kwargs = {"preserve_thinking": true}
chat-template-file = froggeric_Qwen3.6_jinja.templ

chat template is the v19 template from https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates (v20 is completely broken, default and unsloth templates have all kinds of silly bugs). The chat template situation with qwen is a bit of a disaster.

---

I'm surprised you can run qwen 3.6 27B at Q8 but had to use IQ2_XSS for Mistral Small 4 :)

My setup here is "just" a 9070XT (16GB VRAM) and 128GB of RAM. ~120B MoE models (with CPU offloading) have about the same speed here as ~30B dense models (with something like 6-8GB swapped out from VRAM to GTT) at Q4_K_M.

[g4bwy]

I'm surprised you can run qwen 3.6 27B at Q8 but had to use IQ2_XSS for Mistral Small 4 :)

My setup here is "just" a 9070XT (16GB VRAM) and 128GB of RAM. ~120B MoE models (with CPU offloading) have about the same speed here as ~30B dense models (with something like 6-8GB swapped out from VRAM to GTT) at Q4_K_M.

I have a pair of RTX3090 (2x48Go) so I don't offload to CPU.
it happens this model has some cuda OOM issue with --split-mode tensor (but not with layer), and I reduced lots of stuff before realizing that. when it finally got loaded without OOM I didn't bother to fetch a larger quant given that HF's CDN doesn't have it in cache and it takes hours to download...

[g4bwy]

I started a long session with your model/config, I didn't get the issue yet, but hit another weird behavior: right after auto-compaction, the batch tool started spinning and the model decoded 124k tokens for several minutes (starting at an unlikely ~400 t/s!) then it ended with this message: invalid tool input: internal validator bug: missing field 'tool' and continued the session like nothing happened after showing this in a thinking block: thinking> Let me fix the batch call - I need to use "tool" not "tool_name".

[sdroege]

400t/s sounds like some ngram-related bug in llama.cpp, maybe try without ngram-mod to see if that helps

[g4bwy]

dammit, just reproduced it twice in a row while reworking the ollama branch, after I spent 3 days trying with the nudge branch 🤷‍♂️

[g4bwy]

for reference:

{"t":"msg","d":{"role":"user","content":[{"type":"text","text":"verify in all _clean.c files that no functions are missing. use sequential subtasks"}]
}}
{"t":"msg","d":{"role":"assistant","content":[{"type":"thinking","thinking":"The user wants me to verify that no functions are missing from the _clean
.c files compared to the original .c files. I need to do this sequentially using subtasks. Let me start by checking the first few files."}]}}
{"t":"meta","title":"deobfuscate all the functions found in decompiled.c.…","token_usage":{"input_tokens":8373254,"output_tokens":1076323,"cache_creat
ion_input_tokens":0,"cache_read_input_tokens":72857350},"updated_at":1782847182,"mode":"build","plan_path":null,"plan_written":false,"session_rules":[
{"tool":"bash","scope":"sort *","effect":"allow"},{"tool":"bash","scope":"uniq *","effect":"allow"},{"tool":"bash","scope":"echo *","effect":"allow"}]
,"context_size":174548,"thinking":{"kind":"off"}}

stuck here, then the manual nudge:

{"t":"msg","d":{"role":"user","content":[{"type":"text","text":"are you doing it ?"}]}}
{"t":"msg","d":{"role":"assistant","content":[{"type":"thinking","thinking":"The user is asking if I'm actually doing the verification. Let me start immediately with the first subtask.\n"},{"type":"text","text":"Starting now.\n\n"},{"type":"tool_use","id":"ZwGfSdoZJEehRajJ91ynOcCdggPm6Jzb","name":"task","input":{"description":"Verify 01-05 clean files","prompt":"Compare function counts between original and cleaned files for sections 01-05. For each pair, extract function names from both files and check none are missing.\n\nUse grep to find function definitions (lines matching pattern: type name(params) {). Compare the function name lists. Report any missing functions.","subagent_type":"general"}}]}}

[g4bwy]

finally managed to reproduce and test this in a real session... it works and the session continued

[smohr-sevenai]

Excited to see this, I run into it with GLM-5.2 and GPT-5.5 even, maybe a couple times a day, of course with smaller models too.

[sdroege]

@tontinton Let's get that in? Seems like a good addition :)

[g4bwy]

I've been very conservative about wanting to push this, but given how hard it is for me to reproduce on my setup, maybe it's a good idea to put this in more hand then revert/revise if anything bad happens...

[g4bwy]

Excited to see this, I run into it with GLM-5.2 and GPT-5.5 even, maybe a couple times a day, of course with smaller models too.

I had no idea this could happen to the big boys too, only using local models here (and some google + old anthropic ones locked with ridiculous quota in antigravity CLI with a free trial google AI pro promotion 😅 )

[tontinton]

Oh yeah no problem, I'll have time tomorrow to look

[tontinton]

made a small change to get this in faster: https://github.com/tontinton/maki/compare/faee2c7ca65d4dfcff1a64ed67a588f1a08cc872..0b80b3b1aba13c60a20201fdd6e7eb94a9340820

now we don't end up with two consecutive assistant messages in history (the empty original + the "(empty)" replacement)