Skip to content

fix(lmarena): modernize Arena web provider + static Direct-chat catalog - #6280

Merged
diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.47from
backryun:codex/fix-lmarena-provider
Jul 11, 2026
Merged

diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.47from
backryun:codex/fix-lmarena-provider

Conversation

@backryun

@backryun backryun commented Jul 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Modernize the Arena web-session provider (formerly LMArena; wire id stays lmarena) for arena.ai.

  • Chat via https://arena.ai/nextjs-api/stream/create-evaluation with Chrome TLS impersonation (tls-client-node) and optional browser-minted recaptchaV3Token.
  • Static Direct-chat catalog (2026-07-09 scrape, TypeScript only — no scrape JSON in-repo):
    • Text + Search (48) → chat registry
    • Image (27) → IMAGE_PROVIDERS only (not the chat provider page)
  • Disable live arena.ai HTML model discovery; resolve public names → Arena UUIDs from the static seed.
  • Soft-exclude models that 404/502; slow/stop bulk test-all probes for this provider.
  • Specialty catalog: do not fold image/video rows into chat specialty when a chat registry exists (lmarena / openai / xai).
  • UI rebrand: display name Arena (Free); theme-aware icons arena-light.svg / arena-dark.svg.
  • Keep split Supabase SSR cookie reconstruction for arena-auth-prod-v1.*.

Wire / back-compat

Field Value
Provider id lmarena (unchanged)
Alias lma
Display name Arena (Free)
Website https://arena.ai

Validation

  • Targeted unit tests (lmarena executor, split-cookie, specialty catalog, themed icons, web credentials)
  • Live smoke with real session cookie (operator-verified)
  • CI gates on this PR (lint / unit / quality)

Local examples:

node --import tsx/esm --test \
  tests/unit/lmarena-provider.test.ts \
  tests/unit/lmarena-split-cookie-4271.test.ts \
  tests/unit/provider-icon-arena-themed.test.ts \
  tests/unit/bailian-coding-plan-provider.test.ts

Tests added / updated

  • tests/unit/lmarena-provider.test.ts
  • tests/unit/lmarena-split-cookie-4271.test.ts
  • tests/unit/provider-icon-arena-themed.test.ts
  • tests/unit/bailian-coding-plan-provider.test.ts (specialty no image dump)
  • related web-cookie / unsupported-validation / catalog tests

Reviewer notes

  • Image models are listed for the image catalog only; full /v1/images/generations path for Arena is out of scope unless follow-up.
  • Cloudflare may still require a residential path and/or operator-supplied reCAPTCHA v3 token in hard environments.
  • Scrape JSON stays local/desktop only; allowlist ships as directModels.ts.

@backryun
backryun requested a review from diegosouzapw as a code owner July 5, 2026 12:57

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the LMArena provider integration to support the new direct-battle evaluation endpoint and handles the updated AI-SDK SSE stream prefixes. It introduces dynamic model discovery for LMArena by fetching and parsing the initial models directly from arena.ai, complete with caching and display-name-to-internal-ID mapping. Additionally, it refactors the UI validation flow to treat "unsupported" validation results as non-blocking warnings (rendered as a neutral "N/A" badge) rather than blocking errors, and updates LMArena credential hints to reflect Supabase SSR chunked cookies. The reviewer provided several constructive feedback items for open-sse/executors/lmarena.ts, including guarding against null/undefined request bodies in transformRequest, caching the active fetch promise in getLMArenaModels to prevent thundering herd issues, optimizing string trimming and redundant operations in SSE parsing, and adding a safety guard in parseLMArenaInitialModels to prevent slicing errors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread open-sse/executors/lmarena.ts Resolved
Comment thread open-sse/executors/lmarena.ts Resolved
Comment thread open-sse/executors/lmarena.ts Resolved
Comment thread open-sse/executors/lmarena.ts Resolved
Comment thread open-sse/executors/lmarena.ts Resolved
@backryun
backryun force-pushed the codex/fix-lmarena-provider branch 6 times, most recently from de4c682 to 35e92a6 Compare July 6, 2026 02:59
@diegosouzapw
diegosouzapw changed the base branch from release/v3.8.45 to release/v3.8.46 July 6, 2026 04:08
@diegosouzapw

Copy link
Copy Markdown
Collaborator

🔀 Cycle moved: base retargeted to release/v3.8.46 — release/v3.8.45 is frozen for the release in progress (#6313). No action needed.

@backryun
backryun force-pushed the codex/fix-lmarena-provider branch 8 times, most recently from 93f643d to 3dc07e6 Compare July 7, 2026 06:22
@diegosouzapw
diegosouzapw changed the base branch from release/v3.8.46 to release/v3.8.47 July 7, 2026 11:04
@backryun
backryun force-pushed the codex/fix-lmarena-provider branch 9 times, most recently from 8c7e63f to dba3357 Compare July 8, 2026 18:18
@backryun backryun changed the title fix(lmarena): update web provider integration fix(provider): update lmarena integration Jul 8, 2026
@backryun
backryun force-pushed the codex/fix-lmarena-provider branch from 4f0cf6f to 652c088 Compare July 8, 2026 19:27
@backryun
backryun force-pushed the codex/fix-lmarena-provider branch 15 times, most recently from 196f4bc to 065960d Compare July 10, 2026 02:38
@backryun

Copy link
Copy Markdown
Contributor Author

@diegosouzapw I’d like to request a quick merge. It’s been five days, and I’ve done my best to ensure there are no conflicts with recent commits.

@backryun
backryun force-pushed the codex/fix-lmarena-provider branch from 065960d to e19ac82 Compare July 10, 2026 03:03
@backryun
backryun force-pushed the codex/fix-lmarena-provider branch from d299663 to e0ddb8f Compare July 11, 2026 03:38
backryun and others added 3 commits July 11, 2026 13:12
Update the lmarena provider for arena.ai (product rebranded from LMArena):

- Route chat via arena.ai create-evaluation with Chrome TLS impersonation
  (tls-client-node) and optional browser-minted recaptchaV3Token.
- Seed Text+Search (48) into the chat registry; seed Image (27) only into
  IMAGE_PROVIDERS. Disable live HTML model discovery; resolve public names to
  Arena UUIDs from the static TypeScript allowlist (no scrape JSON in-repo).
- Soft-exclude 404/502 model ids; slow/stop bulk test-all probes for this provider.
- Do not fold IMAGE_PROVIDERS/video specialty into the chat provider catalog when
  a chat registry already exists (lmarena/openai/xai).
- Display name Arena (Free); keep wire id `lmarena` / alias `lma` for back-compat.
- Theme-aware provider icons: arena-light.svg / arena-dark.svg.
- Preserve split Supabase SSR cookie reconstruction for arena-auth-prod-v1.*.
…ider reference

Fold the topaz image-only catalog entry's apiFormat/supportedEndpoints
into the local-catalog test fixture (route now tags media-only
providers per the lmarena PR's staticModels.ts change), regenerate
PROVIDER_REFERENCE.md against the merged release providers.ts, and add
the changelog fragment for cheaper-inference#6280.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…try (probe path)

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@backryun
backryun force-pushed the codex/fix-lmarena-provider branch from 835b311 to 31db0e6 Compare July 11, 2026 04:13
@diegosouzapw
diegosouzapw merged commit c92bdd1 into cheaper-inference:release/v3.8.47 Jul 11, 2026
11 checks passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator

@backryun thanks for the Arena web provider modernization — merged into release/v3.8.47 (fixture + fallback-suite aligned to the new registry entry). 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants