Fix zeta2 prompt format selection

[probably-neb]

Self-Review Checklist:

• I've reviewed my own diff for quality, security, and reliability
• Unsafe blocks (if any) have justifying comments
• The content is consistent with the UI/UX checklist
• Tests cover the new/changed behavior
• Performance impact has been considered and is acceptable

Closes #52585

Release Notes:

• Fixed local zeta2 edit predictions using the wrong prompt format.

[kaytwo]

This isn't going to be very helpful but - I built this fork and tested it against an unquantized zeta 2 running on VLLM and was still getting nonsense completions just as I was before this fix.

[probably-neb]

Thanks @kaytwo! Very helpful I haven't merged this because I was waiting to be at home where I could test it properly. Will investigate!
By any chance do you have the model set to zeta in your settings instead of zeta2?

[kaytwo]

I've got the model set to zeta2 (trying zeta made for different gibberish edits). I'm not sure at this point whether it's an issue with Zed itself vs. the version of the model posted to huggingface. When I switch to "Zed AI" in the same buffer, I get edit suggestions that I would expect.

Here are some captures of the completion requests that are being made by zed, which look like I'd expect from the description of zeta 2. By far the most often completion suggestion I get is the "delete the whole file" version that just sends back "\n>>>>>>> UPDATED\n" as its completion, but sometimes I get one that essentially regurgitates the entire file. I can't recall having seen anything meaningful come out ever.

empty completion

POST /v1/completions HTTP/1.1
content-type: application/json
accept: */*
user-agent: Zed/1.2.0+dev.29a55cf31fd009823e6e21c62abb18eb02f66eb4 (linux; x86_64)
host: localhost:8000
content-length: 3083

{"model":"zed-industries/zeta-2","prompt":"<|file_sep|>edit history\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -5,5 +5,6 @@\n     print(\"--- Hello, World! in different languages ---\")\n     \n     \n+    \n \n \n// User accepted prediction:\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -4,8 +4,7 @@\n def print_hello_world():\n     print(\"--- Hello, World! in different languages ---\")\n     \n-    \n-    \n+\n \n \n def fizzbuzz():\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -3,6 +3,7 @@\n \n def print_hello_world():\n     print(\"--- Hello, World! in different languages ---\")\n+    print(\"English: \")\n     \n \n \n// User accepted prediction:\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -1,27 +1,1 @@\n-# hello_world.py\n \n-\n-def print_hello_world():\n-    print(\"--- Hello, World! in different languages ---\")\n-    print(\"English: \")\n-    \n-\n-\n-\n-def fizzbuzz():\n-    print(\"\\n--- FizzBuzz (1-100) ---\")\n-\n-    for number in range(1, 101):\n-        if number % 5 == 0 and number % 3 == 0:\n-            print(\"FizzBuzz\")\n-        elif number % 5 == 0:\n-            print(\"Buzz\")\n-        elif number % 3 == 0:\n-            print(\"Fizz\")\n-        else:\n-            print(number)\n-    return\n-\n-if __name__ == \"__main__\":\n-    print_hello_world()\n-    fizzbuzz()\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -1,1 +1,27 @@\n+# hello_world.py\n \n+\n+def print_hello_world():\n+    print(\"--- Hello, World! in different languages ---\")\n+    print(\"English: \n+    \n+\n+\n+\n+def fizzbuzz():\n+    print(\"\\n--- FizzBuzz (1-100) ---\")\n+\n+    for number in range(1, 101):\n+        if number % 5 == 0 and number % 3 == 0:\n+            print(\"FizzBuzz\")\n+        elif number % 5 == 0:\n+            print(\"Buzz\")\n+        elif number % 3 == 0:\n+            print(\"Fizz\")\n+        else:\n+            print(number)\n+    return\n+\n+if __name__ == \"__main__\":\n+    print_hello_world()\n+    fizzbuzz()\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -3,7 +3,8 @@\n \n def print_hello_world():\n     print(\"--- Hello, World! in different languages ---\")\n-    print(\"English: \n+    # English\n+    \n     \n \n \n<|file_sep|>leo/chapter1/hello_world.py\n<|fim_prefix|><<<<<<< CURRENT\n# hello_world.py\n\n\ndef print_hello_world():\n    print(\"--- Hello, World! in different languages ---\")\n    # English\n    <|user_cursor|>\n    \n\n\n\ndef fizzbuzz():\n    print(\"\\n--- FizzBuzz (1-100) ---\")\n\n    for number in range(1, 101):\n        if number % 5 == 0 and number % 3 == 0:\n            print(\"FizzBuzz\")\n        elif number % 5 == 0:\n            print(\"Buzz\")\n        elif number % 3 == 0:\n            print(\"Fizz\")\n        else:\n            print(number)\n    return\n\nif __name__ == \"__main__\":\n    print_hello_world()\n    fizzbuzz()\n=======\n<|fim_suffix|>\n<|fim_middle|>","max_tokens":256,"stop":[]}

HTTP/1.1 200 OK
date: Tue, 05 May 2026 17:18:49 GMT
server: uvicorn
content-length: 459
content-type: application/json

{"id":"cmpl-98a73fee1a440b54","object":"text_completion","created":1778001529,"model":"zed-industries/zeta-2","choices":[{"index":0,"text":"\n>>>>>>> UPDATED\n","logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null,"prompt_logprobs":null,"prompt_token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":1027,"total_tokens":1034,"completion_tokens":7,"prompt_tokens_details":null},"kv_transfer_params":null}

nonsense completion

POST /v1/completions HTTP/1.1
content-type: application/json
accept: */*
user-agent: Zed/1.2.0+dev.29a55cf31fd009823e6e21c62abb18eb02f66eb4 (linux; x86_64)
host: localhost:8000
content-length: 2896

{"model":"zed-industries/zeta-2","prompt":"<|file_sep|>edit history\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -5,5 +5,6 @@\n     print(\"--- Hello, World! in different languages ---\")\n     \n     \n+    \n \n \n// User accepted prediction:\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -4,8 +4,7 @@\n def print_hello_world():\n     print(\"--- Hello, World! in different languages ---\")\n     \n-    \n-    \n+\n \n \n def fizzbuzz():\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -3,6 +3,7 @@\n \n def print_hello_world():\n     print(\"--- Hello, World! in different languages ---\")\n+    print(\"English: \")\n     \n \n \n// User accepted prediction:\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -1,27 +1,1 @@\n-# hello_world.py\n \n-\n-def print_hello_world():\n-    print(\"--- Hello, World! in different languages ---\")\n-    print(\"English: \")\n-    \n-\n-\n-\n-def fizzbuzz():\n-    print(\"\\n--- FizzBuzz (1-100) ---\")\n-\n-    for number in range(1, 101):\n-        if number % 5 == 0 and number % 3 == 0:\n-            print(\"FizzBuzz\")\n-        elif number % 5 == 0:\n-            print(\"Buzz\")\n-        elif number % 3 == 0:\n-            print(\"Fizz\")\n-        else:\n-            print(number)\n-    return\n-\n-if __name__ == \"__main__\":\n-    print_hello_world()\n-    fizzbuzz()\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -1,1 +1,28 @@\n+# hello_world.py\n \n+\n+def print_hello_world():\n+    print()\n+    \n+    \n+    \n+\n+\n+\n+def fizzbuzz():\n+    print(\"\\n--- FizzBuzz (1-100) ---\")\n+\n+    for number in range(1, 101):\n+        if number % 5 == 0 and number % 3 == 0:\n+            print(\"FizzBuzz\")\n+        elif number % 5 == 0:\n+            print(\"Buzz\")\n+        elif number % 3 == 0:\n+            print(\"Fizz\")\n+        else:\n+            print(number)\n+    return\n+\n+if __name__ == \"__main__\":\n+    print_hello_world()\n+    fizzbuzz()\n--- a/leo/chapter1/hello_world.py\n+++ b/leo/chapter1/hello_world.py\n@@ -2,7 +2,7 @@\n \n \n def print_hello_world():\n-    print()\n+    print(\n     \n     \n     \n<|file_sep|>leo/chapter1/hello_world.py\n<|fim_prefix|><<<<<<< CURRENT\n# hello_world.py\n\n\ndef print_hello_world():\n    print(<|user_cursor|>\n    \n    \n    \n\n\n\ndef fizzbuzz():\n    print(\"\\n--- FizzBuzz (1-100) ---\")\n\n    for number in range(1, 101):\n        if number % 5 == 0 and number % 3 == 0:\n            print(\"FizzBuzz\")\n        elif number % 5 == 0:\n            print(\"Buzz\")\n        elif number % 3 == 0:\n            print(\"Fizz\")\n        else:\n            print(number)\n    return\n\nif __name__ == \"__main__\":\n    print_hello_world()\n    fizzbuzz()\n=======\n<|fim_suffix|>\n<|fim_middle|>","max_tokens":256,"stop":[]}

HTTP/1.1 200 OK
date: Tue, 05 May 2026 17:20:13 GMT
server: uvicorn
content-length: 969
content-type: application/json

{"id":"cmpl-96bb8b8ea5ced69a","object":"text_completion","created":1778001613,"model":"zed-industries/zeta-2","choices":[{"index":0,"text":"\n# hello_world.py\n\n\ndef print_hello_world():\n    print(\"\")\n    \n    \n\n\n\ndef fizzbuzz():\n    print(\"\\n--- FizzBuzz (1-100) ---\")\n\n    for number in range(1, 101):\n        if number % 5 == 0 and number % 3 == 0:\n            print(\"FizzBuzz\")\n        elif number % 5 == 0:\n            print(\"Buzz\")\n        elif number % 3 == 0:\n            print(\"Fizz\")\n        else:\n            print(number)\n    return\n\nif __name__ == \"__main__\":\n    print_hello_world()\n    fizzbuzz()\n>>>>>>> UPDATED\n","logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null,"prompt_logprobs":null,"prompt_token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":981,"total_tokens":1150,"completion_tokens":169,"prompt_tokens_details":null},"kv_transfer_params":null}

[probably-neb]

Hmm, finally had a chance to test, and I'm not seeing any issues. @kaytwo do you happen to have ZED_ZETA_FORMAT set in env? Do you mind sharing your settings (omitting URLS if you choose)?

[konradmb]

Works for me. Thank you!

$ env | grep ZED | wc
      0       0       0        
$ ./target/release-fast/zed

Video_2026-05-08_01-13-45.mp4

PS Do I smell Zeta 2.1 incoming with diagnostics support? 👀

[probably-neb]

Glad it works @konradmb!

@kaytwo if it's still not working, I'd appreciate if you opened an issue (and @-mentioned me for visibility). I'm going to merge this as is, but I do want to get things working for you

[probably-neb]

/cherry-pick preview
/cherry-pick stable

[zed-zippy]

🍒💥 Cherry-pick did not succeed
https://github.com/zed-industries/zed/actions/runs/25558703709

[zed-zippy]

🍒💥 Cherry-pick did not succeed
https://github.com/zed-industries/zed/actions/runs/25558705704

[probably-neb]

/cherry-pick preview
/cherry-pick stable

[kaytwo]

@kaytwo if it's still not working, I'd appreciate if you opened an issue (and @-mentioned me for visibility). I'm going to merge this as is, but I do want to get things working for you

Local zeta2.1 on vllm is giving me roughly the same suggestions I am getting through Zed AI - Zeta2 was still not really working, but not sure how useful it is to chase that particular issue down. Thanks for your help @probably-neb!

[probably-neb]

Glad to hear it @kaytwo!

Out of curiousity what GPU are you running it on, what's your vllm command, and what is your latency like? Trying to get a better feel for how it's running out in the wild

[kaytwo]

@probably-neb:
Running on a 4090FE, using the raw BF16 so it uses quite a bit of my available VRAM. The completions come in with 0.3–1.5 seconds of latency, so totally usable. They all seemed correctly formed too, but haven't seen whether a quant will maintain the same accuracy.
uv run --with vllm vllm serve zed-industries/zeta-2.1 --gpu-memory-utilization=.85 --max-model-len=8182 --language-model-only --host 0.0.0.0