Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions open-sse/executors/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ import { QwenWebExecutor } from "./qwen-web.ts";
import { KimiExecutor } from "./kimi.ts"
import { TheOldLlmExecutor } from "./theoldllm.ts";
import { ChipotleExecutor } from "./chipotle.ts";
import { LMArenaExecutor } from "./lmarena.ts";

const executors = {
antigravity: new AntigravityExecutor(),
Expand Down Expand Up @@ -140,6 +141,8 @@ const executors = {
tllm: new TheOldLlmExecutor(), // Alias
chipotle: new ChipotleExecutor(),
pepper: new ChipotleExecutor(), // Alias
lmarena: new LMArenaExecutor(),
lma: new LMArenaExecutor(), // Alias
};

const defaultCache = new Map();
Expand Down Expand Up @@ -198,3 +201,4 @@ export { InnerAiExecutor } from "./inner-ai.ts";
export { QwenWebExecutor } from "./qwen-web.ts";
export { TheOldLlmExecutor } from "./theoldllm.ts";
export { ChipotleExecutor } from "./chipotle.ts";
export { LMArenaExecutor } from "./lmarena.ts";
100 changes: 100 additions & 0 deletions open-sse/executors/lmarena.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
/**
* LMArenaExecutor — LMArena Web Session Provider
*
* Routes requests through LMArena's web interface using session credentials.
* LMArena is a model comparison platform with 40+ models (GPT, Claude, Gemini, Llama).
*
* API Structure (to be completed based on actual LMArena API):
* Endpoint: https://lmarena.ai/api/chat (placeholder)
* Method: POST
* Content-Type: application/json
* Accept: text/event-stream
*
* Auth pipeline (per request):
* 1. Extract session cookie from credentials
* 2. Build request with model and messages
* 3. Make authenticated POST request to LMArena API
* 4. Handle SSE response stream
*
* NOTE: This is a skeleton implementation. The actual LMArena API integration
* needs to be completed based on reverse-engineering or official documentation.
* See issue #3368 for tracking.
*/
import { BaseExecutor, type ExecuteInput } from "./base.ts";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Import ProviderCredentials from ./base.ts to use it for the corrected method signatures of buildUrl, buildHeaders, and transformRequest.

Suggested change
import { BaseExecutor, type ExecuteInput } from "./base.ts";
import { BaseExecutor, type ExecuteInput, type ProviderCredentials } from "./base.ts";


const LMARENA_API_BASE = "https://lmarena.ai/api";

function readLMArenaCookie(credentials: unknown): string {
if (!credentials || typeof credentials !== "object") return "";
const c = credentials as Record<string, unknown>;
const direct = typeof c.cookie === "string" ? c.cookie : "";
if (direct.trim()) return direct;
const apiKey = typeof c.apiKey === "string" ? c.apiKey : "";
if (apiKey.trim()) return apiKey;
const psd = c.providerSpecificData;
if (psd && typeof psd === "object") {
const nested = (psd as Record<string, unknown>).cookie;
if (typeof nested === "string" && nested.trim()) return nested;
}
return "";
}

export class LMArenaExecutor extends BaseExecutor {
constructor(providerConfig = {}) {
super("lmarena", { format: "openai", ...providerConfig });
}

protected buildUrl(_model: string, _credentials: unknown): string {
// TODO: Implement actual LMArena API endpoint
return `${LMARENA_API_BASE}/chat`;
}
Comment on lines +91 to +93

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The signature of buildUrl does not match the parent class BaseExecutor.buildUrl. In BaseExecutor, it is defined as buildUrl(model: string, stream: boolean, urlIndex?: number, credentials?: ProviderCredentials | null). Correcting this signature ensures proper inheritance and avoids runtime issues if super.execute or other standard execution paths are used.

  buildUrl(
    model: string,
    stream: boolean,
    urlIndex = 0,
    credentials: ProviderCredentials | null = null
  ): string {
    // TODO: Implement actual LMArena API endpoint
    return LMARENA_API_BASE + "/chat";
  }


protected buildHeaders(
_model: string,
credentials: unknown,
_body: unknown
): Record<string, string> {
const cookie = readLMArenaCookie(credentials);
const headers: Record<string, string> = {
"Content-Type": "application/json",
Accept: "text/event-stream",
"User-Agent":
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
};

if (cookie) {
headers.Cookie = cookie;
}

return headers;
}
Comment on lines +95 to +114

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The signature of buildHeaders is mismatched with BaseExecutor.buildHeaders. In BaseExecutor, the first parameter is credentials: ProviderCredentials, but here it is _model: string. This causes BaseExecutor.execute to pass stream (a boolean) as the credentials argument, resulting in readLMArenaCookie failing to extract the cookie. Correcting the signature and making it public resolves this critical bug.

Suggested change
protected buildHeaders(
_model: string,
credentials: unknown,
_body: unknown
): Record<string, string> {
const cookie = readLMArenaCookie(credentials);
const headers: Record<string, string> = {
"Content-Type": "application/json",
Accept: "text/event-stream",
"User-Agent":
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
};
if (cookie) {
headers.Cookie = cookie;
}
return headers;
}
buildHeaders(
credentials: ProviderCredentials,
stream = true,
clientHeaders?: Record<string, string> | null,
model?: string
): Record<string, string> {
const cookie = readLMArenaCookie(credentials);
const headers: Record<string, string> = {
"Content-Type": "application/json",
Accept: stream ? "text/event-stream" : "application/json",
"User-Agent":
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
};
if (cookie) {
headers.Cookie = cookie;
}
return headers;
}


protected transformRequest(body: unknown, _model: string): unknown {
// TODO: Transform OpenAI format to LMArena format
// For now, pass through as-is
return body;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The signature of transformRequest is mismatched with BaseExecutor.transformRequest. Additionally, calling super.transformRequest is highly recommended to preserve the common request sanitizations (such as removing empty optional parameters and stripping prompt_cache_retention) implemented in the base class.

  transformRequest(
    model: string,
    body: unknown,
    stream: boolean,
    credentials: ProviderCredentials
  ): unknown {
    // TODO: Transform OpenAI format to LMArena format
    // For now, pass through as-is
    return super.transformRequest(model, body, stream, credentials);
  }


async execute(input: ExecuteInput): Promise<Response> {
const { model, body, stream, credentials, signal, log } = input;

log?.info?.("LMArenaExecutor", `Executing request for model: ${model}`);

// TODO: Implement actual LMArena API call
// This is a placeholder that returns an error indicating the API is not yet implemented
return new Response(
JSON.stringify({
error: {
message:
"LMArena executor is not yet fully implemented. API integration pending.",
type: "not_implemented",
code: "not_implemented",
},
}),
{
status: 501,
headers: { "Content-Type": "application/json" },
}
);
}
}
14 changes: 14 additions & 0 deletions src/shared/constants/providers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,20 @@ export const WEB_COOKIE_PROVIDERS = {
authHint:
"Paste your __client cookie value from .clerk.agent.adapta.one (DevTools → Application → Cookies)",
},
lmarena: {
id: "lmarena",
alias: "lma",
name: "LMArena (Free)",
icon: "auto_awesome",
color: "#FF6B6B",
textIcon: "LMA",
website: "https://lmarena.ai",
hasFree: true,
freeNote: "Free model comparison platform — 40+ models (GPT, Claude, Gemini, Llama). No subscription required.",
authHint:
"Paste your session cookie from lmarena.ai (DevTools → Application → Cookies). Optional — works with free tier for basic comparisons.",
riskNoticeVariant: "webCookie",
},
huggingchat: {
id: "huggingchat",
// "hc" belongs to the hackclub provider; huggingchat uses its own id as alias.
Expand Down
7 changes: 7 additions & 0 deletions src/shared/providers/webSessionCredentials.ts
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,13 @@ export const WEB_SESSION_CREDENTIAL_REQUIREMENTS = {
acceptsFullCookieHeader: true,
storageKeys: ["cookie", "manus_session"],
},
lmarena: {
kind: "cookie",
credentialName: "session",
placeholder: "session=... or full Cookie header from lmarena.ai",
acceptsFullCookieHeader: true,
storageKeys: ["cookie", "session"],
},
} satisfies Record<keyof typeof WEB_COOKIE_PROVIDERS, WebSessionCredentialRequirement>;

export function getWebSessionCredentialRequirement(
Expand Down
133 changes: 133 additions & 0 deletions tests/unit/lmarena-provider.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
/**
* LMArena Provider — Unit Tests (Phase 2A of issue #3368)
*
* Run: node --import tsx/esm --test tests/unit/lmarena-provider.test.ts
*/

import { describe, it } from "node:test";
import assert from "node:assert/strict";
import { WEB_COOKIE_PROVIDERS } from "../../src/shared/constants/providers.ts";
import {
getWebSessionCredentialRequirement,
requiresWebSessionCredential,
hasUsableWebSessionCredential,
} from "../../src/shared/providers/webSessionCredentials.ts";
import { LMArenaExecutor } from "../../open-sse/executors/lmarena.ts";

describe("LMArena Provider Definition", () => {
it("is registered in WEB_COOKIE_PROVIDERS", () => {
assert.ok(WEB_COOKIE_PROVIDERS.lmarena, "lmarena should be in WEB_COOKIE_PROVIDERS");
assert.equal(WEB_COOKIE_PROVIDERS.lmarena.id, "lmarena");
assert.equal(WEB_COOKIE_PROVIDERS.lmarena.alias, "lma");
assert.equal(WEB_COOKIE_PROVIDERS.lmarena.name, "LMArena (Free)");
assert.equal(WEB_COOKIE_PROVIDERS.lmarena.website, "https://lmarena.ai");
assert.equal(WEB_COOKIE_PROVIDERS.lmarena.hasFree, true);
assert.equal(WEB_COOKIE_PROVIDERS.lmarena.riskNoticeVariant, "webCookie");
});

it("has correct metadata", () => {
const provider = WEB_COOKIE_PROVIDERS.lmarena;
assert.ok(provider.freeNote, "Should have freeNote");
assert.ok(provider.authHint, "Should have authHint");
assert.ok(provider.icon, "Should have icon");
assert.ok(provider.color, "Should have color");
assert.ok(provider.textIcon, "Should have textIcon");
});
});

describe("LMArena Credential Requirements", () => {
it("requires web session credential", () => {
assert.equal(requiresWebSessionCredential("lmarena"), true);
});

it("has correct credential requirement", () => {
const req = getWebSessionCredentialRequirement("lmarena");
assert.ok(req, "Should have credential requirement");
assert.equal(req.kind, "cookie");
assert.equal(req.credentialName, "session");
assert.ok(req.placeholder.includes("lmarena.ai"));
assert.equal(req.acceptsFullCookieHeader, true);
assert.ok(req.storageKeys.includes("cookie"));
assert.ok(req.storageKeys.includes("session"));
});

it("validates usable credentials correctly", () => {
assert.equal(
hasUsableWebSessionCredential("lmarena", { cookie: "session=abc123" }),
true
);
assert.equal(
hasUsableWebSessionCredential("lmarena", { session: "abc123" }),
true
);
assert.equal(
hasUsableWebSessionCredential("lmarena", { cookie: "" }),
false
);
assert.equal(
hasUsableWebSessionCredential("lmarena", {}),
false
);
});
});

describe("LMArena Executor", () => {
it("can be instantiated", () => {
const executor = new LMArenaExecutor();
assert.ok(executor, "Executor should be instantiated");
});

it("has correct provider ID", () => {
const executor = new LMArenaExecutor();
assert.equal((executor as any).provider, "lmarena");
});
Comment on lines +80 to +83

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since provider is a public property of BaseExecutor, there is no need to cast executor to any here. Removing the cast improves type safety and resolves a lint warning.

Suggested change
it("has correct provider ID", () => {
const executor = new LMArenaExecutor();
assert.equal((executor as any).provider, "lmarena");
});
it("has correct provider ID", () => {
const executor = new LMArenaExecutor();
assert.equal(executor.provider, "lmarena");
});


it("builds correct URL", () => {
const executor = new LMArenaExecutor();
const url = (executor as any).buildUrl("gpt-4", {});
assert.ok(url.includes("lmarena.ai"), "URL should include lmarena.ai");
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the test to call buildUrl with the correct signature matching BaseExecutor.buildUrl. This also eliminates the need for the as any cast.

Suggested change
it("builds correct URL", () => {
const executor = new LMArenaExecutor();
const url = (executor as any).buildUrl("gpt-4", {});
assert.ok(url.includes("lmarena.ai"), "URL should include lmarena.ai");
});
it("builds correct URL", () => {
const executor = new LMArenaExecutor();
const url = executor.buildUrl("gpt-4", true, 0, {});
assert.ok(url.includes("lmarena.ai"), "URL should include lmarena.ai");
});


it("builds headers with cookie", () => {
const executor = new LMArenaExecutor();
const headers = (executor as any).buildHeaders("gpt-4", { cookie: "session=abc123" }, {});
assert.ok(headers.Cookie, "Should have Cookie header");
assert.equal(headers.Cookie, "session=abc123");
assert.equal(headers["Content-Type"], "application/json");
assert.equal(headers.Accept, "text/event-stream");
});
Comment on lines +92 to +99

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the test to call buildHeaders with the correct signature matching BaseExecutor.buildHeaders. This also eliminates the need for the as any cast.

Suggested change
it("builds headers with cookie", () => {
const executor = new LMArenaExecutor();
const headers = (executor as any).buildHeaders("gpt-4", { cookie: "session=abc123" }, {});
assert.ok(headers.Cookie, "Should have Cookie header");
assert.equal(headers.Cookie, "session=abc123");
assert.equal(headers["Content-Type"], "application/json");
assert.equal(headers.Accept, "text/event-stream");
});
it("builds headers with cookie", () => {
const executor = new LMArenaExecutor();
const headers = executor.buildHeaders({ cookie: "session=abc123" }, true, null, "gpt-4");
assert.ok(headers.Cookie, "Should have Cookie header");
assert.equal(headers.Cookie, "session=abc123");
assert.equal(headers["Content-Type"], "application/json");
assert.equal(headers.Accept, "text/event-stream");
});


it("builds headers without cookie when not provided", () => {
const executor = new LMArenaExecutor();
const headers = (executor as any).buildHeaders("gpt-4", {}, {});
assert.ok(!headers.Cookie, "Should not have Cookie header when no cookie provided");
});
Comment on lines +101 to +105

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the test to call buildHeaders with the correct signature matching BaseExecutor.buildHeaders. This also eliminates the need for the as any cast.

Suggested change
it("builds headers without cookie when not provided", () => {
const executor = new LMArenaExecutor();
const headers = (executor as any).buildHeaders("gpt-4", {}, {});
assert.ok(!headers.Cookie, "Should not have Cookie header when no cookie provided");
});
it("builds headers without cookie when not provided", () => {
const executor = new LMArenaExecutor();
const headers = executor.buildHeaders({}, true, null, "gpt-4");
assert.ok(!headers.Cookie, "Should not have Cookie header when no cookie provided");
});


it("reads cookie from credentials correctly", () => {
const executor = new LMArenaExecutor();

// Direct cookie field
let headers = (executor as any).buildHeaders("gpt-4", { cookie: "session=abc" }, {});
assert.equal(headers.Cookie, "session=abc");

// apiKey field (dashboard form)
headers = (executor as any).buildHeaders("gpt-4", { apiKey: "session=def" }, {});
assert.equal(headers.Cookie, "session=def");

// providerSpecificData.cookie
headers = (executor as any).buildHeaders(
"gpt-4",
{ providerSpecificData: { cookie: "session=ghi" } },
{}
);
assert.equal(headers.Cookie, "session=ghi");

// Priority: direct > apiKey > providerSpecificData
headers = (executor as any).buildHeaders(
"gpt-4",
{ cookie: "session=abc", apiKey: "session=def" },
{}
);
assert.equal(headers.Cookie, "session=abc");
});
Comment on lines +107 to +133

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the tests to call buildHeaders with the correct signature matching BaseExecutor.buildHeaders. This eliminates all remaining as any casts in the test file, resolving the 8 lint warnings mentioned in the PR description.

  it("reads cookie from credentials correctly", () => {
    const executor = new LMArenaExecutor();

    // Direct cookie field
    let headers = executor.buildHeaders({ cookie: "session=abc" }, true, null, "gpt-4");
    assert.equal(headers.Cookie, "session=abc");

    // apiKey field (dashboard form)
    headers = executor.buildHeaders({ apiKey: "session=def" }, true, null, "gpt-4");
    assert.equal(headers.Cookie, "session=def");

    // providerSpecificData.cookie
    headers = executor.buildHeaders(
      { providerSpecificData: { cookie: "session=ghi" } },
      true,
      null,
      "gpt-4"
    );
    assert.equal(headers.Cookie, "session=ghi");

    // Priority: direct > apiKey > providerSpecificData
    headers = executor.buildHeaders(
      { cookie: "session=abc", apiKey: "session=def" },
      true,
      null,
      "gpt-4"
    );
    assert.equal(headers.Cookie, "session=abc");
  });

});