fix(executors): DuckDuckGo AI Chat uses duckduckgo.com (fixes 400) (#4037)

CHANGELOG.md

@@ -22,6 +22,7 @@ _In development — bullets added per PR; finalized at release._
 
 ### 🐛 Fixed
 
+- **fix(executors): DuckDuckGo AI Chat uses duckduckgo.com (fixes 400)** — the DuckDuckGo AI Chat executor fetched status/chat and set `Origin`/`Referer` against `https://duck.ai` while still sending `Sec-Fetch-Site: same-origin`, so the request's same-origin triplet (host + Origin + Referer) was inconsistent and the backend rejected it with HTTP 400. All current DDG reverse-engineering references — and the provider registry's own `baseUrl` — use `https://duckduckgo.com`; the executor now uses it consistently for the status URL, chat URL, `Origin`, and `Referer` (the same-origin header is now coherent). The `x-fe-version` scrape regex also required a 40-hex tail but the real served token has a 20-hex tail (e.g. `serp_20250401_100419_ET-19d438eb199b2bf7c300`), so it silently fell back to a hardcoded default; the pattern is relaxed to a bounded `{20,40}` tail (still ReDoS-safe). This addresses the DuckDuckGo half of the report; the separate Chipotle/`chipotle` upstream breakage is tracked independently. ([#4037](https://github.com/diegosouzapw/OmniRoute/issues/4037) — thanks @daniij)
 - **fix(security): bound the prompt-injection scan to the first 16 KB (hot-path perf)** — the prompt-injection guard joined every message/system string into one buffer and ran several regexes over the **whole** thing on every chat request, with no size cap — so a 300 KB body (pasted code, RAG context) meant O(body) CPU scanning on the hot path, a self-inflicted latency/GC source under concurrency. Both detection call sites (`detectInjection` in `inputSanitizer.ts` and the custom-pattern scan in `promptInjection.ts`) now slice the joined text to the first **16 KB** (`MAX_INJECTION_SCAN_BYTES`) before the regex loop. Injection directives sit near the top of a prompt, so the generous cap preserves real detection while scanning only a bounded prefix; the existing 10 MB body-size cap (which protects ingestion) is unchanged. ([#3932](https://github.com/diegosouzapw/OmniRoute/issues/3932) — thanks @KooshaPari)
 - **fix(sse): retry direct-connection socket failures on a fresh socket (fewer `502` bursts)** — the default direct-connection undici dispatcher pools keep-alive sockets for up to 4 s, but some edges (e.g. `nvidia`, `opencode-zen`) silently close idle keep-alive sockets within that window, so the next request reusing a pooled socket fails with `UND_ERR_SOCKET` ("other side closed") — in bursts. `proxyFetch` already retried once on such transient errors, but the retry reused the **same** pooled dispatcher and could grab another stale socket, then fell through to native fetch (which also pools) → the job sat in the rate-limit queue until the 30 s timeout → `502` + circuit-breaker open. The retry now uses a dedicated **no-keep-alive / no-pipelining** dispatcher so it opens a brand-new socket that can't be a dead pooled one; the first attempt still uses the pooled dispatcher (healthy keep-alive reuse is preserved). Complements the v3.8.29 diagnostics (`describeFetchCause`, #4281). ([#4252](https://github.com/diegosouzapw/OmniRoute/issues/4252) — thanks @klimadev)
 - **fix(sse): combo now stops at the first body-specific 400 instead of trying every target** — the `#2101` guard that detects a body-specific 400 (context overflow / malformed / model-access-denied, e.g. "model is not supported when using Codex with a ChatGPT account") logged "stopping combo" but executed a bare `break`, which only exited the inner retry loop; `executeTarget` then returned `null` and the outer target loop treated that as "this target produced nothing" and advanced to the next model. A combo of N targets that all reject the same request body therefore marched through all N (the report shows a 143-model Codex combo iterating every target), wasting upstream calls and per-attempt work. The guard now surfaces the 400 via the `{ ok, response }` contract (mirroring the 499 client-disconnect path) so the combo resolves and stops immediately. ([#4279](https://github.com/diegosouzapw/OmniRoute/issues/4279))

config/quality/file-size-baseline.json

@@ -82,7 +82,7 @@
     "open-sse/executors/codex.ts": 1449,
     "open-sse/executors/cursor.ts": 1391,
     "open-sse/executors/deepseek-web.ts": 1117,
-    "open-sse/executors/duckduckgo-web.ts": 917,
+    "open-sse/executors/duckduckgo-web.ts": 925,
     "open-sse/executors/grok-web.ts": 1871,
     "open-sse/executors/muse-spark-web.ts": 1284,
     "open-sse/executors/perplexity-web.ts": 1013,

open-sse/executors/duckduckgo-web.ts

@@ -9,25 +9,33 @@ import { tryBackedChat } from "../services/browserBackedChat.ts";
 import { sanitizeErrorMessage } from "../utils/error.ts";
 
 export const DUCKDUCKGO_BASE = "https://duckduckgo.com";
-const DUCKAI_BASE = "https://duck.ai";
-const AUTH_TOKEN_URL = `${DUCKAI_BASE}/duckchat/v1/auth/token`;
-const COUNTRY_URL = `${DUCKAI_BASE}/country.json`;
-const STATUS_URL = `${DUCKAI_BASE}/duckchat/v1/status`;
-const CHAT_URL = `${DUCKAI_BASE}/duckchat/v1/chat`;
+// #4037: the live DuckDuckGo AI Chat backend is served from duckduckgo.com. The
+// status/chat fetches, Origin, and Referer must all use this host so the request's
+// same-origin triplet (host + Origin + Referer) stays consistent with
+// `Sec-Fetch-Site: same-origin`; pointing them at duck.ai produced an inconsistent
+// triplet the backend rejected with HTTP 400.
+const AUTH_TOKEN_URL = `${DUCKDUCKGO_BASE}/duckchat/v1/auth/token`;
+const COUNTRY_URL = `${DUCKDUCKGO_BASE}/country.json`;
+export const STATUS_URL = `${DUCKDUCKGO_BASE}/duckchat/v1/status`;
+export const CHAT_URL = `${DUCKDUCKGO_BASE}/duckchat/v1/chat`;
 const DEFAULT_FE_VERSION = "serp_20260424_180649_ET-0bdc33b2a02ebf8f235def65d887787f694720a1";
-const FE_VERSION_PATTERN = /serp_\d{8}_\d{6}_[A-Z]{2}-[0-9a-f]{40}/;
+// #4037: the real served x-fe-version token has a 20-hex tail (e.g.
+// `serp_20250401_100419_ET-19d438eb199b2bf7c300`); the previous `{40}` requirement
+// never matched the live token, so the scrape silently fell back to DEFAULT_FE_VERSION.
+// Bounded `{20,40}` keeps the pattern ReDoS-safe.
+export const FE_VERSION_PATTERN = /serp_\d{8}_\d{6}_[A-Z]{2}-[0-9a-f]{20,40}/;
 const DEFAULT_USER_AGENT =
   "Mozilla/5.0 (X11; Linux x86_64) " +
   "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36";
 
-const FAKE_HEADERS: Record<string, string> = {
+export const FAKE_HEADERS: Record<string, string> = {
   Accept: "*/*",
   "Accept-Encoding": "gzip, deflate, br, zstd",
   "Accept-Language": "en-US,en;q=0.9",
   "Cache-Control": "no-cache",
-  Origin: DUCKAI_BASE,
+  Origin: DUCKDUCKGO_BASE,
   Pragma: "no-cache",
-  Referer: `${DUCKAI_BASE}/`,
+  Referer: `${DUCKDUCKGO_BASE}/`,
   Priority: "u=1, i",
   "Sec-Ch-Ua": '"Chromium";v="146", "Not-A.Brand";v="24", "Google Chrome";v="146"',
   "Sec-Ch-Ua-Mobile": "?0",
@@ -740,8 +748,8 @@ export class DuckDuckGoWebExecutor extends BaseExecutor {
     if (this.warmed || signal.aborted) return;
     this.warmed = true;
     this.seedBrowserCookies();
-    const duckAiResponse = await this.warmFetch(
-      `${DUCKAI_BASE}/`,
+    const homepageResponse = await this.warmFetch(
+      `${DUCKDUCKGO_BASE}/`,
       this.buildRequestHeaders({
         Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
         "Sec-Fetch-Dest": "document",
@@ -751,10 +759,10 @@ export class DuckDuckGoWebExecutor extends BaseExecutor {
       }),
       signal
     );
-    if (duckAiResponse) {
+    if (homepageResponse) {
       try {
-        const duckAiHtml = await duckAiResponse.clone().text();
-        const feVersion = extractDuckDuckGoFeVersion(duckAiHtml);
+        const homepageHtml = await homepageResponse.clone().text();
+        const feVersion = extractDuckDuckGoFeVersion(homepageHtml);
         if (feVersion) this.feVersion = feVersion;
       } catch (error) {
         void error;

tests/unit/duckduckgo-domain-4037.test.ts

@@ -0,0 +1,88 @@
+import { describe, it } from "node:test";
+import assert from "node:assert/strict";
+import {
+  DUCKDUCKGO_BASE,
+  STATUS_URL,
+  CHAT_URL,
+  FAKE_HEADERS,
+  FE_VERSION_PATTERN,
+} from "../../open-sse/executors/duckduckgo-web.ts";
+
+// Regression for GitHub #4037 (DuckDuckGo half only): DuckDuckGo AI Chat returns HTTP 400.
+// Root cause 1 (primary): the executor's STATUS_URL/CHAT_URL/Origin/Referer pointed at
+// `https://duck.ai` while `Sec-Fetch-Site: same-origin` was sent and the request hit
+// duck.ai — an inconsistent same-origin triplet the backend rejects with 400. Every current
+// DDG reverse-engineering reference (and the registry baseUrl) uses `https://duckduckgo.com`.
+// Root cause 2 (secondary): FE_VERSION_PATTERN required a 40-hex tail, but the real served
+// x-fe-version token has a 20-hex tail, so the scrape silently fell back to a hardcoded
+// future-dated default.
+describe("DuckDuckGo AI Chat domain consistency (#4037)", () => {
+  describe("URL/header host is duckduckgo.com (not duck.ai)", () => {
+    it("STATUS_URL uses duckduckgo.com", () => {
+      assert.ok(
+        STATUS_URL.startsWith(`${DUCKDUCKGO_BASE}/`),
+        `STATUS_URL should start with ${DUCKDUCKGO_BASE}, got ${STATUS_URL}`
+      );
+      assert.ok(!STATUS_URL.includes("duck.ai"), `STATUS_URL must not reference duck.ai: ${STATUS_URL}`);
+    });
+
+    it("CHAT_URL uses duckduckgo.com", () => {
+      assert.ok(
+        CHAT_URL.startsWith(`${DUCKDUCKGO_BASE}/`),
+        `CHAT_URL should start with ${DUCKDUCKGO_BASE}, got ${CHAT_URL}`
+      );
+      assert.ok(!CHAT_URL.includes("duck.ai"), `CHAT_URL must not reference duck.ai: ${CHAT_URL}`);
+    });
+
+    it("Origin header points at duckduckgo.com", () => {
+      assert.equal(FAKE_HEADERS.Origin, "https://duckduckgo.com");
+      assert.ok(!FAKE_HEADERS.Origin.includes("duck.ai"), "Origin must not be duck.ai");
+    });
+
+    it("Referer header points at duckduckgo.com", () => {
+      assert.equal(FAKE_HEADERS.Referer, "https://duckduckgo.com/");
+      assert.ok(!FAKE_HEADERS.Referer.includes("duck.ai"), "Referer must not be duck.ai");
+    });
+
+    it("keeps Sec-Fetch-Site: same-origin consistent with duckduckgo.com Origin/Referer", () => {
+      // The same-origin triplet (request host + Origin + Referer) must all agree.
+      assert.equal(FAKE_HEADERS["Sec-Fetch-Site"], "same-origin");
+      const originHost = new URL(FAKE_HEADERS.Origin).host;
+      const refererHost = new URL(FAKE_HEADERS.Referer).host;
+      const statusHost = new URL(STATUS_URL).host;
+      const chatHost = new URL(CHAT_URL).host;
+      assert.equal(originHost, refererHost, "Origin and Referer hosts must match");
+      assert.equal(originHost, statusHost, "Origin host must match STATUS_URL host");
+      assert.equal(originHost, chatHost, "Origin host must match CHAT_URL host");
+      assert.equal(originHost, "duckduckgo.com");
+    });
+  });
+
+  describe("FE_VERSION_PATTERN matches the real served token", () => {
+    it("matches a real 20-hex-tail token", () => {
+      // Real served example from the DDG SERP HTML.
+      const realToken = "serp_20250401_100419_ET-19d438eb199b2bf7c300";
+      assert.equal(
+        FE_VERSION_PATTERN.test(realToken),
+        true,
+        `FE_VERSION_PATTERN should match the real 20-hex token: ${realToken}`
+      );
+    });
+
+    it("still matches a 40-hex-tail token (backward compatible)", () => {
+      const fortyHexToken =
+        "serp_20260424_180649_ET-0bdc33b2a02ebf8f235def65d887787f694720a1";
+      assert.equal(
+        FE_VERSION_PATTERN.test(fortyHexToken),
+        true,
+        "FE_VERSION_PATTERN should still match a 40-hex token"
+      );
+    });
+
+    it("extracts the token from surrounding HTML", () => {
+      const html = `<script>window.__fe="serp_20250401_100419_ET-19d438eb199b2bf7c300";</script>`;
+      const match = html.match(FE_VERSION_PATTERN)?.[0];
+      assert.equal(match, "serp_20250401_100419_ET-19d438eb199b2bf7c300");
+    });
+  });
+});