[router] openai router: support grok model

[key4ng]

Motivation

Support xai openai compatible api in openai router. now it supports history management and mcp execution for grok model.

Modifications

1. Fix the spec to auto set default value for some fields. some of the fields is not supported by xai endpoint, like background, service_tier etc
2. Xai response api currently doesn't fully support all openai input type, so preprocessing the request to meet the spec for both openai and xai:
   • xai doesn't support item as input, so it doesn't support id and status field
   • xai doesn't support output_text as type of assistant content, so mark it to input_text

Accuracy Tests

Using xai backend

cargo run --bin sglang-router --  --worker-urls https://api.x.ai/ --backend openai --history-backend oracle --prometheus-port 30002

base call

ubuntu@moirai-a10-exp:~/sglang$ curl http://localhost:30000/v1/responses \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $XAI_API_KEY" \
-d '{
"model": "grok-4-fast",
"input": "could you introduce me two programming languages both in one line"
}' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2612  100  2508  100   104    452     18  0:00:05  0:00:05 --:--:--   577
{
  "created_at": 1760294550,
  "id": "fd51be12-06c2-dc86-5e4f-bd09c615e325_us-east-1",
  "max_output_tokens": null,
  "model": "grok-4-fast-reasoning",
  "object": "response",
  "output": [
    {
      "id": "rs_fd51be12-06c2-dc86-5e4f-bd09c615e325_us-east-1",
      "summary": [],
      "type": "reasoning",
      "status": "completed",
      "encrypted_content": "W2MQGT2aQzTRNITN/g5mTCWg3lq2NWhqu6GHz67xXssQcqut8StfBx6bSHb/GDVl+wBTjIVDuNHm/kCjK2ndAVl+6o/GmwMHOfCb5iGf76796jX+29QenJ27TThsQ10+TFXX/iSbiGORgjqWxKtGnT9jljFXXs9TNwzvy8WyH30GoNgLla0S1wBHYGoElpL76ghLfEYo0df3qvZCrlesvxqS1+t3JkZUPTqqEXUjV80TwPhQ87laoIq9FllA07EJXYgEdNqDiY4Hi3gwtBRoY8yy9ZN+horjtEbFhuaJ2mFPHR9VzU/igdUfKuUq1nEjKdbYCqkq6ItAenRHAl4XwSqQ5O07jTf6XP3iDVDL+9cmXgWbHWyoKUlDDtaAYX5Ei9YhM0eSjkkS+lrzKOfIjKKmzRE2p6c0AhjPrl8HXvcOY3QMPFMFNkVwqm+ur8XA78x4zpR9GfGMt5BIEIDpiEg+i5Ds+mV2HRFhklTVuKClvyOILUKNoh3QopBviavEUvExEE23aqvd9bD6i4PeiMauBUPwpzcPQLehLp7AgUyGRrHJL9CGfhuQ4uaBl2F1l97PFehMawn+pX2dWY5/Ob5LmvLQaABG0uAMcC6fOVZ7KBP8xWTTlGE+xSe/eo86p2khr3iQKQjqt2RS09CvwwvALq6XHovKDjHA/AXQb+ye7jZQcdx+coGJtaRE/TJBOvDQv99f38jTnu6IgdBbn20I3lgYTuT7eWNFZ+9dZJhPlIczYfBzKvnQZbjXJFZWse/smXAIsC/Rd4cNvvwMPMvp8QhW/xUjxkhn36Rp9uqG/g59LYUbKaRlQKlxBfGo38kyVSnSVUgIsfZOn+VDfvpUsK1wUleVNXFMBUZuuV2P81MJbCaqYk7znJyoT0d+3WecLuVv+O7tIC5HdKy4XjKa95GQmixYoXIEiSlhWDdTyn3QUGH2Tm5IkUVwsdqIRFcXW1PXwVnkg8DW2pq5+44o0WBTWwWnkWHplTsijeKfrWq7zkIIeKcWpL5DYRWF67M5hHZcgSB5OQ5sQXvZTgs2Ay8Vda29L19ouIVXbFeGVs/KOqXYB5A7I79y5qcYApPFrRZUTTYAWPEtLMZOVe4r5IiXFRzCbwX4mYy2sI40O9Ue34DcR/uwr5vnJgWGm3oT91ELFwZxUJdldLRfuY1vno8l/EYksqpAnmW40lqtAA27xAYlH3LjpFI7XsN9R7vZUCUrJ2t6ZY4"
    },
    {
      "content": [
        {
          "type": "output_text",
          "text": "Python: A versatile, high-level interpreted language renowned for its simplicity, readability, and wide use in web development, data science, automation, and AI.  \nJavaScript: A dynamic, prototype-based scripting language primarily used for client-side web development to create interactive user interfaces and applications.",
          "logprobs": null,
          "annotations": []
        }
      ],
      "id": "msg_fd51be12-06c2-dc86-5e4f-bd09c615e325_us-east-1",
      "role": "assistant",
      "type": "message",
      "status": "completed"
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": "medium",
    "summary": "detailed"
  },
  "temperature": null,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_p": null,
  "usage": {
    "input_tokens": 167,
    "input_tokens_details": {
      "cached_tokens": 152
    },
    "output_tokens": 234,
    "output_tokens_details": {
      "reasoning_tokens": 177
    },
    "total_tokens": 401
  },
  "user": null,
  "incomplete_details": null,
  "status": "completed",
  "store": false,
  "metadata": {}
}

append history by providing previous_response_id

 curl http://localhost:30000/v1/responses \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $XAI_API_KEY" \
-d '{
"model": "grok-4-fast",
"input": "could you summarize what you mentioned, and introduce two more, both in 1 sentence",
"previous_response_id": "fd51be12-06c2-dc86-5e4f-bd09c615e325_us-east-1"
}' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3239  100  3044  100   195    702     45  0:00:04  0:00:04 --:--:--   972
{
  "created_at": 1760294662,
  "id": "9e286cac-8410-b5f3-ddb0-68f589d2fec6_us-east-1",
  "max_output_tokens": null,
  "model": "grok-4-fast-reasoning",
  "object": "response",
  "output": [
    {
      "id": "rs_9e286cac-8410-b5f3-ddb0-68f589d2fec6_us-east-1",
      "summary": [],
      "type": "reasoning",
      "status": "completed",
      "encrypted_content": "PX/D9lNvn3NMu2fywvkxxgJz+MU728w+ESrL76pZcQRjuC6kp+X+2gMNET/VF2MBD49MAvC7X9qqYIli1QGrkEUt6RguP+mku8osyhBPv61YM9pamqPa34lTHHNoa6/Bidg3wwVatPF5E6qjh4DHZ39qYHo0/YVlIEx8FKME9GO07gkiwOfWyurBEOlXdnKM/38zh7bPTTEmIxcAp51/8ra5ffrNgONYBYPmNb+gWs+PuqKhFSRJ6X/w2deGIUcGPTVCbW6DWRz0SZpKhPlab5VAYAMOSROWAleLUbZpOv6iPQVbqfiR4We3g+fRSXjV8M8M4yiquFl8vBK99LRXXlUoUDhqWgOI2Q2NcDSQDDyAsjNvDfiwZBhahGT9a2PD/Wvc6sD7I82ZtQEqG7tcxasCmYbH/CB6yaG6lQTMpyZSDUXONLsdcNxXpL+aOkANzywZTan+rSJ3QM1Fp9USPQIcEf6QdjMaJPALmoGalBc88gxGeKlHwP0Gwdh5lFI5MMW8xdV5caxsAkuh8B959PlMs9OA1wpYfFW7y1Tj9Tl2T3Pg5mvsZb6gkbcBfnRzOdm5l7g1NKviidjFSuLYdFHUrm9Kopl9HDaoRGZwyVvM0lAfVMNdQRVHeXyLN+w5cN1PF3pBN5NayV8G4j4qAhU2o0+ws61qfnXbtq3H/JRL0XGnt1i9V+EEoQg+d/V5+xOcW2zOHpHRjSG3gG+yUKP7PtgZ0/2kSf40td9NpUs5TKEXfL0BIrcM0l5MlZzNYWzNlvLzFp2VUtvK9qPKMN6fla1jUE9BjeYxq6Th4ckzCKCngXxMZ0DhkKZulJ83s2fHrYmgFzI+kivNxnNTzBoZE+Pk9xKJeUY3hvAZwNvEgiSNqAN8UzAF30WDnvZBRQZu4qbaurhdKTFFUzCCAuDh7EVKxxlK2YbEPsJ5csDFIMLd6U4sw1D33341BKOA2030Em/b1BT3dUwHblxRb/LZ7wXdfF+DHQmHw3c+3YVbHGOEG16fZJo67tQqbN7toESqSDE4EeIG6fReQFsfuUgySS8pDdaqKsESwrpTiws7/3fs4XsQrf4/yEION8TbbhwtavIvob48dBSRKGsoMV0BwaoXJH3sxUliGd83dk4/CSYxEnrH0cHIuFML7R4ocLQVgUsV/E9g3TR8xAvNJa4WBJ3h929iSnRnumVqbR1m3aun4yQ3dbSdeEmIVa57dtMzkKXb0EOu4guiwUtqYxrH2JmQxpfC5bUPDuti0mPLhmzCCmwC1gbL9NXf4gGPPuRKekbmd+rMyH0z4PFNqj63J082vKOeH8cjlndt2ocfDFGW772vnD9G0LAu5KuYWa5HgOXMpxEc4WXJJAuWHlzh8QNAu8Yc+8sPP/TZDA9guiEWWlvelnWudXZM/TaGUUzDNWoJFrzOeux+XlZIZRneVMY5+qt2GKV90/v9kvuGD/Fk7rzwjffbvNs"
    },
    {
      "content": [
        {
          "type": "output_text",
          "text": "Python and JavaScript are versatile programming languages: Python excels in simplicity for data science, AI, and automation, while JavaScript powers interactive web development.  \nJava: A robust, object-oriented language widely used for enterprise applications, Android development, and large-scale systems due to its platform independence and strong security features.  \nC++: A high-performance, compiled language extending C with object-oriented capabilities, ideal for system software, game development, and real-time applications requiring fine-grained control over hardware.",
          "logprobs": null,
          "annotations": []
        }
      ],
      "id": "msg_9e286cac-8410-b5f3-ddb0-68f589d2fec6_us-east-1",
      "role": "assistant",
      "type": "message",
      "status": "completed"
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": "fd51be12-06c2-dc86-5e4f-bd09c615e325_us-east-1",
  "reasoning": {
    "effort": "medium",
    "summary": "detailed"
  },
  "temperature": null,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_p": null,
  "usage": {
    "input_tokens": 249,
    "input_tokens_details": {
      "cached_tokens": 151
    },
    "output_tokens": 335,
    "output_tokens_details": {
      "reasoning_tokens": 237
    },
    "total_tokens": 584
  },
  "user": null,
  "incomplete_details": null,
  "status": "completed",
  "store": false,
  "metadata": {}
}

MCP call

ubuntu@moirai-a10-exp:~/sglang$ curl http://localhost:30000/v1/responses \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $XAI_API_KEY" \
-d '{
  "model": "grok-4-fast",
  "tools": [
    {
      "type": "mcp",
      "server_label": "brave",
      "server_description": "A Tool to do web search",
      "server_url": "http://localhost:8001/sse",
      "require_approval": "never"
    }
  ],
  "input": "could you quickly recommend me some good restaurants opened recently in Seattle"
}' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12066  100 11724  100   342    296      8  0:00:42  0:00:39  0:00:03  3570
{
  "created_at": 1760295107,
  "id": "be0b1277-7763-82f3-8416-9ebfdd4c6c27_us-east-1",
  "max_output_tokens": null,
  "model": "grok-4-fast-reasoning",
  "object": "response",
  "output": [
    {
      "id": "mcpl_5ed0fd23caadc52c1fe32d8ff8388f144d425e95fc1373f220",
      "type": "mcp_list_tools",
      "server_label": "brave",
      "tools": [
        {
          "name": "brave_local_search",
          "description": "Searches for local businesses and places using Brave's Local Search API. Best for queries related to physical locations, businesses, restaurants, services, etc. Returns detailed information including:\n- Business names and addresses\n- Ratings and review counts\n- Phone numbers and opening hours\nUse this when the query implies 'near me' or mentions specific locations. Automatically falls back to web search if no local results are found.",
          "input_schema": {
            "type": "object",
            "properties": {
              "query": {
                "type": "string",
                "description": "Local search query (e.g. 'pizza near Central Park')"
              },
              "count": {
                "type": "number",
                "description": "Number of results (1-20, default 5)",
                "default": 5
              }
            },
            "required": [
              "query"
            ]
          },
          "annotations": {
            "read_only": false
          }
        },
        {
          "name": "brave_web_search",
          "description": "Performs a web search using the Brave Search API, ideal for general queries, news, articles, and online content. Use this for broad information gathering, recent events, or when you need diverse web sources. Supports pagination, content filtering, and freshness controls. Maximum 20 results per request, with offset for pagination. ",
          "input_schema": {
            "type": "object",
            "properties": {
              "query": {
                "type": "string",
                "description": "Search query (max 400 chars, 50 words)"
              },
              "count": {
                "type": "number",
                "description": "Number of results (1-20, default 10)",
                "default": 10
              },
              "offset": {
                "type": "number",
                "description": "Pagination offset (max 9, default 0)",
                "default": 0
              }
            },
            "required": [
              "query"
            ]
          },
          "annotations": {
            "read_only": false
          }
        }
      ]
    },
    {
      "id": "mcp_c8efd91cda586709fa39dda2a94925a4713d19b46bde48a4bc",
      "type": "mcp_call",
      "status": "completed",
      "approval_request_id": null,
      "arguments": "{\"query\":\"best new restaurants opened in Seattle 2024\",\"count\":10}",
      "error": null,
      "name": "brave_web_search",
      "output": "{\"content\":[{\"type\":\"text\",\"text\":\"Title: The Hit List: New Seattle Restaurants To Try Right Now - Seattle - The Infatuation\\nDescription: The Hit List is where you’ll find all of the <strong>best</strong> <strong>new</strong> <strong>restaurants</strong> <strong>in</strong> <strong>Seattle</strong>. As long as it <strong>opened</strong> within the past several months and we’re still talking about it, it’s on this guide.\\nURL: https://www.theinfatuation.com/seattle/guides/best-new-seattle-restaurants-hit-list\\n\\nTitle: Seattle’s Best New Restaurants of 2024 | Seattle Met\\nDescription: These rookies brought smiles with big murals, small cocktails, and creative cooking.\\nURL: https://www.seattlemet.com/eat-and-drink/best-new-restaurants-seattle\\n\\nTitle: The Best New Restaurants in the Seattle Area | Eater Seattle\\nDescription: Italian sandwiches, Roman pizza, and Thai street food join the list this month.\\nURL: https://seattle.eater.com/maps/best-new-restaurants-seattle-heatmap\\n\\nTitle: Seattle’s New Restaurant Openings - Seattle - The Infatuation\\nDescription: The <strong>new</strong> <strong>restaurant</strong> <strong>openings</strong> <strong>in</strong> <strong>Seattle</strong> you should know about.\\nURL: https://www.theinfatuation.com/seattle/guides/seattles-new-restaurant-openings\\n\\nTitle: Best New Restaurants Seattle, WA - Last Updated October 2025 - Yelp\\nDescription: Top 10 <strong>Best</strong> <strong>Best</strong> <strong>New</strong> <strong>Restaurants</strong> <strong>in</strong> <strong>Seattle</strong>, WA - Last Updated September 2025 - Yelp - Paju, La Loba, Haerfest, Kalabaw Bar &amp; Kitchen, Sophon, House Of Eve, <strong>Restaurant</strong> Christine, Tanoor, Tendon Kohaku, D&#x27; La Santa\\nURL: https://www.yelp.com/search?find_desc=Best+New+Restaurants&find_loc=Seattle%2C+WA\\n\\nTitle: Restaurants That Opened and Closed in 2024 | Seattle Met\\nDescription: Goodbye to Machiavelli, Coastal Kitchen, the Fat Hen, and more.\\nURL: https://www.seattlemet.com/eat-and-drink/2024/12/closed-restaurants-2024-opened\\n\\nTitle: The 15 Best Restaurants in the Seattle Area 2024 - The Stranger\\nDescription: Hope you&#x27;re hungry: Here are the <strong>best</strong> <strong>restaurants</strong> <strong>in</strong> <strong>the</strong> <strong>Seattle</strong> area.\\nURL: https://www.thestranger.com/art-and-performance-fall-2024/2024/09/18/79696774/the-15-best-restaurants-in-the-seattle-area-2024\\n\\nTitle: Seattle’s 2024 Eater Award Winners | Eater Seattle\\nDescription: The Eater Award for the <strong>Best</strong> <strong>New</strong> <strong>Restaurant</strong>, <strong>Best</strong> <strong>New</strong> Bar, and <strong>Best</strong> Burger, <strong>Best</strong> Counter, and <strong>Best</strong> <strong>Best</strong> Pop-Up Turned <strong>Restaurant</strong> <strong>in</strong> <strong>Seattle</strong>.\\nURL: https://seattle.eater.com/2024/12/3/24299775/eater-awards-winners-seattle-2024-best-new-restaurant\\n\\nTitle: Seattle's Best New Restaurants Of 2024 - Seattle - The Infatuation\\nDescription: This year marked an era of restoration for dining <strong>in</strong> <strong>Seattle</strong>. Menus are for the most part tangible and 3D again. While “two to three small plates per person” might feel as familiar as “stop freakin’, call Beacon,” entrees have returned with a filling vengeance. And pop-ups-turned-permanent proved that a stellar ... omakase. <strong>2024</strong>’<strong>s</strong> <strong>best</strong> <strong>new</strong> <strong>restaurants</strong> ...\\nURL: https://www.theinfatuation.com/seattle/guides/best-new-restaurants-seattle-2024\\n\\nTitle: r/Seattle on Reddit: Which restaurants are new(ish) and deserve more business?\\nDescription: 62 votes, 86 comments. Saw a post early asking the opposite and realized how out of touch I am. I’m also accepting walk up counters/delis/grocers!\\nURL: https://www.reddit.com/r/Seattle/comments/18b7u07/which_restaurants_are_newish_and_deserve_more/\"}],\"isError\":false}",
      "server_label": "brave"
    },
    {
      "id": "mcp_656aecb1c6caaeb2606235dc0206a425c7a01b36047e34ea4d",
      "type": "mcp_call",
      "status": "completed",
      "approval_request_id": null,
      "arguments": "{\"query\":\"recently opened restaurants Seattle\",\"count\":10}",
      "error": null,
      "name": "brave_local_search",
      "output": "{\"content\":[{\"type\":\"text\",\"text\":\"Error: Rate limit exceeded\"}],\"isError\":true}",
      "server_label": "brave"
    },
    {
      "id": "rs_be0b1277-7763-82f3-8416-9ebfdd4c6c27_us-east-1",
      "summary": [],
      "type": "reasoning",
      "status": "completed",
      "encrypted_content": "lVc5yssK/rwBLXlRVWmBQGpI1RgNz71P/YuN00RTtVGe8fEBzNQVHpZD7rBT5OAIXjdGBQ3bNmxF9XoTpMCFDxDHJ8XhSgp4lh33kh+iDvH6u+xIa0TbBxuKuJ9QUNB27ufRxtR9kvv6Of01183LK12kD6WthiSAxb5C60oqxD1i2Jx+g2vFJxY9GT5v7uNNN/HmDX/eiJ2xJXE4duWJkrit0bJh1rSHZbGMKnDpCB6fWIZEI8ZpDDDT/mgtohwc1Q7EdZfMEROwQv24EWw/K/G95Akjo7Jup57s/TL3SC1vTs75dDdNa3Y3DNC9X2CQbPX7/W2GgJXkQW15Bf/N2CjBrV48pbBdPeYNJPv4JyJvynwvNapRF11rnqSbqRnDvSuNhhJG9bijv9Y7kE20VMWQ4W74Jdf0iH7VBbwyBfLuLpT+Iwldn1LYT01mjg+f1uTsI7A/3F4ItDSHvWnsiB57YdrGpBwYq2+HlC1Pp1hnd5do5XZMjq7vABEm7LH8CgWuPRwRTvxeJQ5d321qUQGwl0JlvCd/T5UiUtDvYsmaNfd1gged2o9+uz+BWiIcq79Y2fuVTpEnc3w8ykt/tHBlJ4ExKO974nM310IEzQJVlVu7iNJWBUZAzmO0e3MtE+JMbEgRn03TEPOaYgPHsdwZvnVB/XdLFobytoWYlbnWRw0XzFNGxKtQwiDlaukIJ/8QrIqVMYuXulu2lW1Qhys8YtyPXWGKJ0o2wehlz2/au4I+Y4JxrQYkfy3mpFTue2VN1p7tXZxiDSAIkthRbNTjNnTTPGir2VH0T8aijUX4ZXsnGiEhdh/vyNoBYVMTW0XfdFS7RhwnPV9iDEwZ7VCc94gSrZfq5trz12kRMhg5K7AtwZZBlw3LTN9AcrSiamigHvSy7sgdTPRDYwlkTIdjAfpOZ6a0UiPMWhsbcYtrkJNGr06VN0EHEm36GOfpwa1GKNqthUqFydru32CBucYrvBsi+61HwPs6DQBYQPXuWnz3AML5vMJUDrdW1K5b+TzOgSKVg6bEqpvu3oXIbFNWsfwxl1W14lpOKzXw8qboFe5accSqOFK3cUMpcxP72Mv6kQ+RJzweUZx15SS4vLbPyRwU4G9CArRifEfVKncZuubYTuc7/BjwctXCCjM9xh2/iYaZBvLQUcEK/TXw4OpQoEN/1HcJgzLi1zY06XorGIVrk4u9sEFixPbgqTBrqjLx8Ynj3UA3NHp8arkognBE6J7+qU+inwyK4czXg75pHGy7yJotlXpzFnpIfSkAGoeTLJsFWQnL6SrpvYBeJ09DMzbSKIkG20zFvYoN0RSMFIwh+wCj2TKPWxYIq/bm6Qn1MhFfii68yH3fs1UaUTYdgdUvPl2R6pJMAU+MmvjQYdIrHlDWLgRC87IGu8VUCQBTE21xjFHQixU8Di7lqvySmJBd9r9G79LvTkGyTT862AFV5SNSOSv20JFrv/JxV/FGox3KLKgSWb5jSnH/CB3rInd5Db3YEZys9LhsAEesr3d8GrxSt12xYGNoJQw"
    },
    {
      "content": [
        {
          "type": "output_text",
          "text": "Based on recent 2024 openings in Seattle (pulled from sources like Eater Seattle, The Infatuation, Seattle Met, and Yelp), here are a few standout new restaurants that have been getting rave reviews for their fresh concepts, food quality, and vibes. These are all relatively recent (opened in the last year or so). I'd recommend checking their websites or apps for current hours and reservations, as things can change quickly.\n\n### 1. **Paju** (Capitol Hill)\n   - **Cuisine**: Modern Korean with a focus on small plates and inventive twists.\n   - **Why?**: Top-rated new spot on Yelp and Eater lists for its bold flavors and cozy atmosphere. Opened in early 2024; great for sharing dishes like kimchi fried rice or grilled meats.\n   - **Address**: Around 11th Ave (exact: 1101 Pike St, Seattle, WA 98101).\n   - **Rating**: 4.5+ stars, praised for value and creativity.\n\n### 2. **Sophon** (Capitol Hill)\n   - **Cuisine**: Thai street food with elevated, shareable plates.\n   - **Why?**: Featured in Seattle Met's best new restaurants for its vibrant, authentic flavors (think spicy salads and curries). Opened mid-2024; perfect for groups or date nights.\n   - **Address**: 15th Ave area (exact: 1401 15th Ave, Seattle, WA 98122).\n   - **Rating**: 4.5 stars, loved for freshness and spice levels.\n\n### 3. **La Loba** (Ballard)\n   - **Cuisine**: Mexican-inspired with wood-fired pizzas and tacos.\n   - **Why?**: A hit on The Infatuation's Hit List for its fun, casual energy and high-quality ingredients. Opened late 2023/early 2024; try the namesake \"wolf\" pizza.\n   - **Address**: Ballard neighborhood (exact: 5405 Leary Ave NW, Seattle, WA 98107).\n   - **Rating**: 4.5+ stars, noted for lively patio and cocktails.\n\n### 4. **Kalabaw Bar & Kitchen** (International District)\n   - **Cuisine**: Filipino fusion with bold, comforting dishes.\n   - **Why?**: Yelp darling for new spots, highlighting its adobo and lumpia. Opened 2024; affordable and flavorful, great for first-timers to Filipino food.\n   - **Address**: Around King St (exact: 659 S King St, Seattle, WA 98104).\n   - **Rating**: 4.5 stars, buzzing for authenticity.\n\n### 5. **Tanoor** (Fremont)\n   - **Cuisine**: Middle Eastern with a focus on grilled meats and flatbreads.\n   - **Why?**: On Eater's new openings map for its smoky kebabs and welcoming vibe. Opened spring 2024; ideal for casual dinners.\n   - **Address**: Fremont area (exact: 707 N 36th St, Seattle, WA 98103).\n   - **Rating**: 4.5 stars, commended for portions and warmth.\n\nIf you're in a specific neighborhood or craving a certain cuisine, let me know for more tailored picks! These are based on current buzz, but always verify openings.",
          "logprobs": null,
          "annotations": []
        }
      ],
      "id": "msg_be0b1277-7763-82f3-8416-9ebfdd4c6c27_us-east-1",
      "role": "assistant",
      "type": "message",
      "status": "completed"
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": "medium",
    "summary": "detailed"
  },
  "temperature": null,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [
    {
      "type": "mcp",
      "server_label": "brave",
      "server_url": "http://localhost:8001/sse",
      "server_description": "A Tool to do web search",
      "require_approval": "never"
    }
  ],
  "top_p": null,
  "usage": {
    "input_tokens": 2426,
    "input_tokens_details": {
      "cached_tokens": 2397
    },
    "output_tokens": 940,
    "output_tokens_details": {
      "reasoning_tokens": 257
    },
    "total_tokens": 3366
  },
  "user": null,
  "incomplete_details": null,
  "status": "completed",
  "store": false,
  "metadata": {}
}

streaming also tested

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Summary of Changes

Hello @key4ng, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates support for the Grok model into the existing router, leveraging an XAI OpenAI compatible API. The core objective is to enable seamless interaction with Grok, including conversational history and advanced tool-use capabilities via Multi-Chain Prompting. To achieve this, the PR refines the request and response structures, making certain parameters optional and implementing specific data transformations to bridge the compatibility gaps between the router's internal specification and XAI's API requirements.

Highlights

• Grok Model Support: The router now supports the Grok model through an XAI OpenAI compatible API, including history management and Multi-Chain Prompting (MCP) execution.
• XAI API Compatibility: Adjustments have been made to the request specification to align with XAI endpoints, making several fields optional (e.g., background, service_tier) that are not universally supported by XAI.
• Request Preprocessing for XAI: Requests are now preprocessed to ensure compatibility with XAI's API, which includes removing 'id' and 'status' fields from input items and normalizing 'output_text' to 'input_text' for assistant content.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds support for the Grok model in the OpenAI router by making several request fields optional to handle API differences. The changes are well-implemented across the codebase, including updates to data structures, default values, and tests. I've identified a few areas for improvement: some leftover debugging code should be removed, there's a potential typo in a field name, and a complex block of code could be refactored to enhance readability and maintainability.