inst : preview generation + improve prompt

[ggerganov]

cont #96

• Streaming response
• Better system prompt
• Continue instructions (<leader>llc)
• Rerun instructions (<leader>llr)
• Update default keymaps - use <leader>ll as common keymap

Next PRs:

• Multiple suggestions using parallel n_cmpl
• Auto-trigger instructions based on git diff?

llama.vim-inst-1-lq.mp4

[alopatindev]

let l:system_prompt .= "... Respond ONLY with the result of applying INSTRUCTION to SELECTION given the CONTEXT. .... Do not output any extra separators.\n"

let l:extra = s:ring_get_extra()

let l:system_prompt .= "\n"
let l:system_prompt .= "--- CONTEXT   --------------------------------------------------\n"
...

Looks kinda sad. Maybe at least serialize inputs as JSONs?

This for instance works okay with ollama most of the time:

curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-coder:7b",
    "prompt": "You are a code-editing assistant that receives JSONs with \"selection\" value as input. Apply \"instruction\" value to the input, answer ONLY with single VALID minified JSON object with the ONLY value \"replacement\" that contains ONLY the output code. No markdown wrapping. {\"selection\":\"// TODO\",\"instruction\":\"implement quicksort in C\"}",
    "stream": false
  }' | jq -rM '.response' | jq -rM '.replacement'

[ggerganov]

Yeah, I was planning to change to json. Btw, I'm using gpt-oss-120b and it's solid.

[ggerganov]

@alopatindev Should be improved now - I find this feature quite useful.

If you give it a try would appreciate feedback.

[alopatindev]

Works for me (nvim 0.11.5), thanks!

Tested with --fim-qwen-3b-default and --fim-qwen-7b-default. I don't have enough VRAM to test with larger models.