Asthenia0412 · Oct 13, 2026
@@ -0,0 +1,105 @@
+---
+title: Qwen3-Reranker
+metatags:
+    description: "Serve Qwen3-Reranker with SGLang - decoder-only yes/no logprob reranking with launch commands, request formats, and common pitfalls."
+---
+
+## 1. Model Introduction
+
+[Qwen3-Reranker](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B) is a series of decoder-only reranker models (0.6B and 4B/8B) from the Qwen team. Instead of a cross-encoder head, they score query–document pairs with yes/no logprob scoring, and achieve strong performance on multilingual retrieval benchmarks.
+
+## 2. SGLang Installation
+
+SGLang offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang installation guide](https://docs.sglang.ai/get_started/install.html) for installation instructions.
+
+## 3. Model Deployment
+
+### 3.1 Launch Command
+
+```shell
+python3 -m sglang.launch_server \
+  --model-path Qwen/Qwen3-Reranker-0.6B \
+  --trust-remote-code \
+  --disable-radix-cache \
+  --host 0.0.0.0 \
+  --port 8001 \
+  --chat-template examples/chat_template/qwen3_reranker.jinja
+```
+
+<Note>
+Qwen3-Reranker uses decoder-only logprob scoring (yes/no). Do NOT launch it with `--is-embedding`.
+</Note>
+
+### 3.2 Common Pitfalls
+
+- **`--chat-template` is required.** Without `--chat-template examples/chat_template/qwen3_reranker.jinja`, the server does not recognize the model as a decoder-only reranker and returns a 400 error: `"This model does not appear to be an embedding model by default. Please add --is-embedding..."`. The fix is to add the chat template flag, NOT `--is-embedding`.
+- If you launch Qwen3-Reranker with `--is-embedding`, `/v1/rerank` cannot compute yes/no logprob scores. Relaunch **without** `--is-embedding`.
+- If you see a validation error like "score should be a valid number" and the backend returned a list, upgrade to a version that coerces `embedding[0]` into `score` for rerank responses.
+
+## 4. Model Invocation
+
+### 4.1 Example Client Request
+
+Supports optional `instruct`, `top_n`, and `return_documents`:
+
+```shell
+curl -X POST http://127.0.0.1:8001/v1/rerank \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "Qwen3-Reranker-0.6B",
+    "query": "What is the capital of France?",
+    "documents": [
+      "The capital of France is Paris.",
+      "The capital of Germany is Berlin.",
+      "Bananas are yellow fruit."
+    ],
+    "instruct": "Given a web search query, retrieve relevant passages that answer the query.",
+    "top_n": 2,
+    "return_documents": true
+  }'
+```
+
+**Request Parameters:**
+
+- `query` (required): The query text to rank documents against
+- `documents` (required): List of documents to be ranked
+- `model` (required): Model to use for reranking
+- `instruct` (optional): Instruction text for the reranker
+- `top_n` (optional): Maximum number of documents to return. Defaults to returning all documents. If specified value is greater than the total number of documents, all documents will be returned.
+- `return_documents` (optional): Whether to return documents in the response. Defaults to `True`.
+
+### 4.2 Response Format
+
+`/v1/rerank` returns a list of objects (sorted by descending score):
+
+- `score`: float, higher means more relevant
+- `document`: the original document string (only included when `return_documents` is `true`)
+- `index`: the original index in the input `documents`
+
+The number of returned results is controlled by the `top_n` parameter. If `top_n` is not specified or is greater than the total number of documents, all documents are returned.
+
+Example (with `return_documents: true`):
+
+```json
+[
+  {"score": 0.99, "document": "The capital of France is Paris.", "index": 0},
+  {"score": 0.01, "document": "The capital of Germany is Berlin.", "index": 1},
+  {"score": 0.00, "document": "Bananas are yellow fruit.", "index": 2}
+]
+```
+
+Example (with `top_n: 2`):
+
+```json
+[
+  {"score": 0.99, "document": "The capital of France is Paris.", "index": 0},
+  {"score": 0.01, "document": "The capital of Germany is Berlin.", "index": 1}
+]
+```
+
+## 5. References
+
+- [Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B) on Hugging Face
+- [SGLang rerank models documentation](https://docs.sglang.ai/supported_models/retrieval_ranking/rerank_models.html)
@@ -0,0 +1,167 @@
+---
+title: Qwen3-VL-Reranker
+metatags:
+    description: "Serve Qwen3-VL-Reranker with SGLang - multimodal decoder-only reranking over text, images, and videos with yes/no logprob scoring."
+---
+
+## 1. Model Introduction
+
+[Qwen3-VL-Reranker](https://huggingface.co/Qwen/Qwen3-VL-Reranker-2B) extends Qwen3-Reranker to support multimodal content, allowing reranking of documents containing text, images, and videos. Like Qwen3-Reranker, it uses decoder-only yes/no logprob scoring.
+
+<Note>
+Currently only `Qwen3-VL-Reranker-2B` is tested and supported. The 8B model may have different behavior and is not guaranteed to work with this template.
+</Note>
+
+## 2. SGLang Installation
+
+SGLang offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang installation guide](https://docs.sglang.ai/get_started/install.html) for installation instructions.
+
+## 3. Model Deployment
+
+### 3.1 Launch Command
+
+```shell
+python3 -m sglang.launch_server \
+  --model-path Qwen/Qwen3-VL-Reranker-2B \
+  --trust-remote-code \
+  --disable-radix-cache \
+  --host 0.0.0.0 \
+  --port 30000 \
+  --chat-template examples/chat_template/qwen3_vl_reranker.jinja
+```
+
+### 3.2 Common Pitfalls
+
+- Always use `--chat-template examples/chat_template/qwen3_vl_reranker.jinja` for Qwen3-VL-Reranker.
+- Do NOT launch with `--is-embedding` (it uses decoder-only logprob scoring like Qwen3-Reranker).
+- For best results, use `--disable-radix-cache` to avoid caching issues with multimodal content.
+
+## 4. Model Invocation
+
+### 4.1 Text-Only Reranking (backward compatible)
+
+```python
+import requests
+
+url = "http://127.0.0.1:30000/v1/rerank"
+
+payload = {
+    "model": "Qwen3-VL-Reranker-2B",
+    "query": "What is machine learning?",
+    "documents": [
+        "Machine learning is a branch of artificial intelligence that enables computers to learn from data.",
+        "The weather in Paris is usually mild with occasional rain.",
+        "Deep learning is a subset of machine learning using neural networks with many layers.",
+    ],
+    "instruct": "Retrieve passages that answer the question.",
+    "return_documents": True
+}
+
+response = requests.post(url, json=payload)
+results = response.json()
+
+for item in results:
+    print(f"Score: {item['score']:.4f} - {item['document'][:60]}...")
+```
+
+### 4.2 Image Reranking (text query, image/mixed documents)
+
+```python
+import requests
+
+url = "http://127.0.0.1:30000/v1/rerank"
+
+payload = {
+    "query": "A woman playing with her dog on a beach at sunset.",
+    "documents": [
+        # Document 1: Text description
+        "A woman shares a joyful moment with her golden retriever on a sun-drenched beach at sunset.",
+        # Document 2: Image URL
+        [
+            {
+                "type": "image_url",
+                "image_url": {
+                    "url": "https://example.com/beach_dog.jpeg"
+                }
+            }
+        ],
+        # Document 3: Text + Image (mixed)
+        [
+            {"type": "text", "text": "A joyful scene at the beach:"},
+            {
+                "type": "image_url",
+                "image_url": {
+                    "url": "https://example.com/beach_dog.jpeg"
+                }
+            }
+        ]
+    ],
+    "instruct": "Retrieve images or text relevant to the user's query.",
+    "return_documents": False
+}
+
+response = requests.post(url, json=payload)
+results = response.json()
+
+for item in results:
+    print(f"Index: {item['index']}, Score: {item['score']:.4f}")
+```
+
+### 4.3 Multimodal Query Reranking (query with image)
+
+```python
+import requests
+
+url = "http://127.0.0.1:30000/v1/rerank"
+
+payload = {
+    # Query with text and image
+    "query": [
+        {"type": "text", "text": "Find similar images to this:"},
+        {
+            "type": "image_url",
+            "image_url": {
+                "url": "https://example.com/reference_image.jpeg"
+            }
+        }
+    ],
+    "documents": [
+        "A cat sleeping on a couch.",
+        "A woman and her dog enjoying the sunset at the beach.",
+        "A busy city street with cars and pedestrians.",
+        [
+            {
+                "type": "image_url",
+                "image_url": {
+                    "url": "https://example.com/similar_image.jpeg"
+                }
+            }
+        ]
+    ],
+    "instruct": "Find images or descriptions similar to the query image."
+}
+
+response = requests.post(url, json=payload)
+results = response.json()
+
+for item in results:
+    print(f"Index: {item['index']}, Score: {item['score']:.4f}")
+```
+
+### 4.4 Request Parameters (Multimodal)
+
+- `query` (required): Can be a string (text-only) or a list of content parts:
+  - `&#123;"type": "text", "text": "..."&#125;` for text
+  - `&#123;"type": "image_url", "image_url": &#123;"url": "..."&#125;&#125;` for images
+  - `&#123;"type": "video_url", "video_url": &#123;"url": "..."&#125;&#125;` for videos
+- `documents` (required): List where each document can be a string or list of content parts (same format as query)
+- `instruct` (optional): Instruction text for the reranker
+- `top_n` (optional): Maximum number of documents to return
+- `return_documents` (optional): Whether to return documents in the response (default: `false`)
+
+## 5. References
+
+- [Qwen3-VL-Reranker-2B](https://huggingface.co/Qwen/Qwen3-VL-Reranker-2B) on Hugging Face
+- [SGLang rerank models documentation](https://docs.sglang.ai/supported_models/retrieval_ranking/rerank_models.html)
@@ -1288,6 +1288,8 @@
                       "cookbook/autoregressive/Qwen/Qwen3-Coder",
                       "cookbook/autoregressive/Qwen/Qwen3-Coder-Next",
                       "cookbook/autoregressive/Qwen/Qwen3-VL",
+                      "cookbook/autoregressive/Qwen/Qwen3-Reranker",
+                      "cookbook/autoregressive/Qwen/Qwen3-VL-Reranker",
                       "cookbook/autoregressive/Qwen/Qwen2.5-VL"
                     ]
                   },

@@ -18,15 +18,12 @@ sglang serve \
   --is-embedding
 ```
 
-### EmbeddingGemma
-
-EmbeddingGemma uses bidirectional attention and is auto-detected, so its best default command is simply:
-
-```bash
-sglang serve --model-path google/embeddinggemma-300m
-```
-
-On CUDA, SGLang automatically uses breakable CUDA graph (BCG) for its full encoder prefill and disables incompatible radix-cache and chunked-prefill behavior. Do not add the deprecated piecewise CUDA graph knobs.
+<Note>
+Model-specific setup (prerequisites, performance defaults, launch flags) lives
+in the [SGLang Cookbook](/cookbook/autoregressive/Google/EmbeddingGemma) — for
+example, for EmbeddingGemma, which is auto-detected and served with breakable
+CUDA graph by default.
+</Note>
 
 ### Client Request
 
@@ -138,7 +135,7 @@ print("Embedding:", response["data"][0]["embedding"])
   </thead>
   <tbody>
     <tr>
-      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>EmbeddingGemma</td>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><a href="/cookbook/autoregressive/Google/EmbeddingGemma">EmbeddingGemma</a></td>
       <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`google/embeddinggemma-300m`</td>
       <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>N/A</td>
       <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Bidirectional Gemma3 text encoder; auto-detected and served with BCG by default</td>