Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **fix(sse):** retire the Gemini Web image-generation bridge while provenance and licensing remain on HOLD, preserving Gemini Web chat/auth/tools and legitimate Gemini/Nano Banana image providers ([#11708](https://github.com/diegosouzapw/OmniRoute/pull/11708))
8 changes: 8 additions & 0 deletions config/quality/test-masking-allowlist.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,14 @@
"tests/unit/providers-page-utils.test.ts": "v3.8.47 #6675: remoção dos providers obsoletos glhf/kluster/cablyai/inclusionai — os 2 asserts que citavam providers removidos do catálogo foram removidos junto (net 276→274). Superfície aposentada, não mascaramento. Verificado legítimo. Prune após v3.8.47 mergear para main.",
"_deletedWithReplacement": {
"_comment": "Deleções de arquivo de teste com SUBSTITUTO verificado (o gate exige que o replacement exista no HEAD e seja arquivo de teste). Uso restrito ao caso 'reescrito em outro path sem rename detectável pelo -M do git'. Cada entrada precisa de reason com PR ref e passa por revisão humana no release PR. Prune após o release mergear para main.",
"tests/unit/gemini-web-image-account-fallback.test.ts": {
"replacement": "tests/unit/gemini-web-image-retirement.test.ts",
"reason": "v3.8.50 #11708: a capacidade Gemini Web Images foi aposentada em HOLD de procedencia/licenca, sem aposentar Gemini Web chat/auth/tools. O teste positivo de fallback da integracao removida foi substituido pelo guard negativo no catalogo/handler e pelo controle que preserva o executor/registry de chat e provedores legitimos de Nano Banana. Superficie extinta, nao mascaramento."
},
"tests/unit/gemini-web-image-generation-10466.test.ts": {
"replacement": "tests/unit/gemini-web-image-retirement.test.ts",
"reason": "v3.8.50 #11708: parser, image-mode, handler e modelo gemini-web/nano-banana-web foram aposentados em HOLD de procedencia/licenca. O replacement prova a ausencia no catalogo e request path, a remocao dos artefatos dedicados, a neutralizacao da antiga extensao image-mode e a preservacao de Gemini Web chat e imagens Gemini/Nano Banana em provedores legitimos. Superficie extinta, nao mascaramento."
},
"open-sse/services/combo/__tests__/targetExhaustion.test.ts": {
"replacement": "tests/unit/combo/combo-target-exhaustion.test.ts",
"reason": "v3.8.44 #5976: os testes de exaustão eram flake-prone (delays Math.random, timeouts 30s, >3min no CI) e foram REESCRITOS como unit determinístico com MAIS cobertura (21 casos/52 asserts vs 13 casos/37 asserts). Documentado no commit 5fe225850. Revisão humana: apresentado ao operador no STOP #1 do release v3.8.44."
Expand Down
25 changes: 4 additions & 21 deletions open-sse/config/imageRegistry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -248,26 +248,6 @@ export const IMAGE_PROVIDERS: Record<string, ImageProviderConfig> = {
supportedSizes: ["1024x1024", "1024x1536", "1536x1024"],
},

// #10466: Gemini Web session image generation (Nano Banana). Same
// web-cookie transport as the gemini-web chat provider — the handler
// drives the session executor in image mode and extracts the generated
// asset URLs from the StreamGenerate frames.
"gemini-web": {
id: "gemini-web",
alias: "gweb",
baseUrl: "https://gemini.google.com/app",
authType: "apikey",
authHeader: "cookie",
format: "gemini-web",
// `-web` suffix on purpose: the bare `nano-banana` id is owned by
// adobe-firefly (operator decision 2026-07-31, pinned by the
// cheaperinference-image-models guard). parseImageModel's bare-model scan
// walks providers in insertion order, so a bare `nano-banana` here would
// steal that resolution. Keep this id distinct.
models: [{ id: "nano-banana-web", name: "Nano Banana (Gemini Web Image)" }],
supportedSizes: ["1024x1024", "1024x1536", "1536x1024"],
},

// Cursor plan image generation via the Agent CLI native `generateImage` tool.
// Reuses the same OAuth/API-key connection as chat (`provider: "cursor"`).
// Requires the `agent` binary (CURSOR_AGENT_BIN) — see cursorAgentImage handler.
Expand Down Expand Up @@ -925,7 +905,10 @@ export function parseImageModel(modelStr) {

// No provider prefix — try to find the model in every provider, excluding cookie-auth (web) bridges
for (const [providerId, config] of Object.entries(IMAGE_PROVIDERS)) {
if (config.authHeader !== "cookie" && (config.routingAliases?.includes(modelStr) || config.models.some((m) => m.id === modelStr))) {
if (
config.authHeader !== "cookie" &&
(config.routingAliases?.includes(modelStr) || config.models.some((m) => m.id === modelStr))
) {
return { provider: providerId, model: modelStr };
}
}
Expand Down
141 changes: 12 additions & 129 deletions open-sse/executors/gemini-web.ts
Original file line number Diff line number Diff line change
Expand Up @@ -265,70 +265,6 @@ export function parseStreamResponse(raw: string): string {
return lastText;
}

/**
* Extract generated-image URLs from a Gemini StreamGenerate response (#10466).
*
* When the web UI generates images (Nano Banana), the model's answer frames
* carry the assets in the candidate's extension block, NOT in the text:
*
* inner[4][0][12][7][0] → array of generated-image entries
* entry[0][3][3] → the image URL — either a plain string or a
* list of strings (take the first http(s) one)
*
* This path is corroborated by the two maintained reverse-engineered clients
* (gpt4free's Gemini provider and HanaokaYuzu/Gemini-API's _parse_candidate).
* Deliberately NOT collected: `inner[4][0][12][1]` — those are web-search
* result thumbnails, not generated content; mixing them in would serve
* scraped images as "generated" (#10466 acceptance criteria).
*
* Frames are cumulative snapshots, so later frames repeat earlier images;
* we dedupe while preserving first-seen order. A `=s2048` size suffix is
* appended (gpt4free's proven heuristic) so callers get full-resolution
* assets instead of UI thumbnails.
*/
export function parseStreamResponseImages(raw: string): string[] {
const urls: string[] = [];
const seen = new Set<string>();
const lines = raw.split("\n");

for (const rawLine of lines) {
const line = rawLine.trim();
if (!line || line === ")]}'" || /^\d+$/.test(line)) continue;
if (!line.includes("wrb.fr")) continue;
try {
const arr = JSON.parse(line);
if (!Array.isArray(arr) || !Array.isArray(arr[0]) || arr[0][0] !== "wrb.fr") continue;
const payload = arr[0]?.[2];
if (typeof payload !== "string") continue;
const inner = JSON.parse(payload);
const imageEntries = inner?.[4]?.[0]?.[12]?.[7]?.[0];
if (!Array.isArray(imageEntries)) continue;
for (const entry of imageEntries) {
const urlField = entry?.[0]?.[3]?.[3];
let url = "";
if (typeof urlField === "string") {
url = urlField;
} else if (Array.isArray(urlField)) {
const firstHttp = urlField.find(
(u: unknown) => typeof u === "string" && /^https?:\/\//.test(u)
);
url = typeof firstHttp === "string" ? firstHttp : "";
}
if (!url || !/^https?:\/\//.test(url)) continue;
// Upgrade to full resolution unless a size directive is already present
// (googleusercontent size syntax: trailing `=s2048`, `=w1024-h512`, ...).
if (!/=[swh]\d+/.test(url)) url += "=s2048";
if (seen.has(url)) continue;
seen.add(url);
urls.push(url);
}
} catch {
// Skip unparseable lines
}
}
return urls;
}

function readCredentialString(value: unknown): string {
if (typeof value !== "string") return "";
const trimmed = value.trim();
Expand Down Expand Up @@ -567,52 +503,24 @@ export class GeminiWebExecutor extends BaseExecutor {

const page = await context.newPage();

// #10466: image mode — the /v1/images/generations handler sets
// x_gemini_web_image_mode. Generated images arrive in the candidate's
// extension block ([12][7][0]) of the StreamGenerate frames, sometimes
// only in a LATER frame of the stream (or a follow-up StreamGenerate
// call), so image mode captures every StreamGenerate response, merges
// image URLs across frames, and resolves as soon as one is found.
// Chat mode keeps the original first-response-only behavior.
const imageMode = (body as Record<string, unknown>)?.x_gemini_web_image_mode === true;

// Capture first StreamGenerate response
let responseText = "";
const responseImages: string[] = [];
let captured = false;
const responsePromise = new Promise<void>((resolve) => {
page.on("response", async (resp: any) => {
if (!resp.url().includes("StreamGenerate")) return;
if (!imageMode && captured) return;
if (imageMode) {
// Image mode: merge text + image URLs across every frame and
// resolve as soon as an image appears (images can land in a
// later frame than the text).
try {
const raw = await resp.text();
const text = parseStreamResponse(raw);
if (text) responseText = text;
for (const url of parseStreamResponseImages(raw)) {
if (!responseImages.includes(url)) responseImages.push(url);
}
} catch {
/* ignore unreadable frames */
}
if (responseImages.length > 0) resolve();
} else {
// Chat mode: byte-for-byte the original first-response capture —
// resolve even if reading the body throws, so the flow falls
// through to the "No response from Gemini" 502 instead of
// burning the full wait window.
captured = true;
try {
const raw = await resp.text();
responseText = parseStreamResponse(raw);
} catch {
/* ignore */
}
resolve();
if (captured) return;
// Resolve even if reading the body throws, so the flow falls through
// to the "No response from Gemini" 502 instead of burning the full
// wait window.
captured = true;
try {
const raw = await resp.text();
responseText = parseStreamResponse(raw);
} catch {
/* ignore */
}
resolve();
});
});

Expand All @@ -631,36 +539,11 @@ export class GeminiWebExecutor extends BaseExecutor {
await page.waitForTimeout(300);
await page.keyboard.press("Enter");

// Wait for response or timeout. Image generation (Nano Banana) is
// noticeably slower than text — the UI renders the asset only after
// the full generation completes — so image mode gets a wider window.
await Promise.race([responsePromise, page.waitForTimeout(imageMode ? 90000 : 30000)]);
await Promise.race([responsePromise, page.waitForTimeout(30000)]);
if (signal?.aborted) {
throw signal.reason instanceof Error ? signal.reason : new Error("Request aborted");
}

// #10466 image mode: return the captured image URLs to the image
// handler via a custom field (same precedent as chatgpt-web's
// x_image_resolution_failed). An image-only answer can carry little or
// no text, so the empty-text 502 below must not fire when images
// were captured.
if (imageMode) {
await this.persistRotatedCookies(context, cookie, credentials, onCredentialsRefreshed, log);
const modelId = model || "gemini-2.5-pro";
return {
response: new Response(
JSON.stringify({
...formatChatCompletion(responseText, modelId),
x_gemini_web_image_urls: responseImages,
}),
{ status: 200, headers: { "Content-Type": "application/json" } }
),
url: GEMINI_URL,
headers: {},
transformedBody: body,
};
}

if (!responseText) {
return {
response: new Response(JSON.stringify({ error: "No response from Gemini" }), {
Expand Down
16 changes: 1 addition & 15 deletions open-sse/handlers/imageGeneration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ import {
extractMarkdownImageUrls,
CHATGPT_WEB_IMAGE_ID_RE,
} from "./imageGeneration/providers/chatgptWeb.ts";
import { handleGeminiWebImageGeneration } from "./imageGeneration/providers/geminiWeb.ts";
import { handleNvidiaNimImageGeneration } from "./imageGeneration/providers/nvidiaNim.ts";
import { handleSegmindImageGeneration } from "./imageGeneration/providers/segmind.ts";
import { handleDesignerWebImageGeneration } from "./imageGeneration/providers/designerWeb.ts";
Expand Down Expand Up @@ -579,19 +578,6 @@ export async function handleImageGeneration({
});
}

// #10466: Gemini Web session image generation (Nano Banana)
if (providerConfig.format === "gemini-web") {
return handleGeminiWebImageGeneration({
model,
provider,
body,
credentials,
log,
signal,
clientHeaders,
});
}

if (providerConfig.format === "cursor-agent-image") {
return handleCursorAgentImageGeneration({
model,
Expand Down Expand Up @@ -2897,7 +2883,7 @@ export function saveImageErrorResult({
error,
requestBody = null,
path = "/v1/images/generations",
// #10494: opt-in signal for executeImageWithCredentialFallback — set by a
// #8307: opt-in signal for executeImageWithCredentialFallback — set by a
// provider handler when the failure is account/session-specific (expired
// or blocked credentials) rather than a generic request/provider error, so
// the retry loop tries the next eligible account even when the upstream
Expand Down
Loading
Loading