feat(models): add RanoAI provider

[steebchen]

Adds RanoAI (ranoai) — an OpenAI-compatible inference provider serving Gemma 4 31B on Furiosa RNGD NPU hardware, via https://api.ranoai.com/v1/chat/completions.

Changes

• Provider registration (packages/models/src/providers.ts): ranoai, API key via LLM_RANOAI_API_KEY, streaming + cancellation, full metadata (website, terms, privacy, HQ US, data policy).
• Endpoint resolution (packages/actions/src/get-provider-endpoint.ts): default base URL https://api.ranoai.com, routed to /v1/chat/completions.
• Streaming transform (apps/gateway/src/chat/tools/transform-streaming-to-openai.ts): added to the OpenAI-compatible passthrough branch.
• Model mapping (packages/models/src/models/google.ts): new ranoai mapping on the existing gemma-4-31b-it model (externalId: gemma-4-31b).
• Brand logo (packages/shared/src/components/provider-icons.tsx, apps/ui/.../provider-logo.ts): RanoAIIcon registered in ProviderIcons + both providerLogoUrls maps.
• Env: .env.example, .env.unified.example, Helm values.yaml, and the e2e CI workflow.

Pricing

Provider list price, passed through as-is: $0.10/M input, $0.30/M output — currently the cheapest Gemma 4 31B mapping in the catalogue.

Capability flags — measured against the live deployment

Every flag was verified by probing the deployment directly rather than reading the docs. RanoAI shipped several fixes during review (image input, tool-call formatting including streaming deltas and tool-result round-trips, tool_choice: "none", and a window expansion from 8K to 20480), so this reflects the state as of 2026-07-28.

Capability	Measured	Flag
Context window	20480 tokens (prompt + max_tokens), rejected with a proper OpenAI-style error	contextSize: 20480, maxOutput: 20480
Vision	works — data URIs and remote HTTPS URLs both decode and are described accurately	vision: true
Tools	correct tool_calls non-streaming and streaming; parallel multi-tool works; tool-result round-trips clean	tools: true
JSON schema	strict json_schema conforms to nested enum schemas (3/3)	jsonOutputSchema: true
Multiple choices	n=3 returns 3 distinct indexed choices, streaming included	supportsN: true
tool_choice	auto and none honoured; required accepted but not enforced; named choices ignored (see below)	supportedToolChoices: ["auto", "none"]
Streaming	works, incl. stream_options.include_usage	streaming: true
JSON output	json_object and json_schema both return valid JSON	jsonOutput: true
Reasoning	reasoning_content across all 7 effort tiers, streamed as deltas; only emitted when reasoning_effort is passed	reasoning: true + full reasoningEfforts

Vision was confirmed with a synthetic image the model could not have guessed (a 64×64 half-red/half-blue PNG, correctly described as "red and blue", 285 prompt tokens confirming real tokenization), and separately with a remote URL using the exact message shape our e2e sends.

tool_choice coverage

auto and none are passed through — none correctly returns finish_reason: "stop" with no tool calls and a clean refusal (3/3). Two modes are imperfect upstream and therefore downgrade to auto:

• required is accepted but not enforced: with a tool-inviting prompt it returns a correct tool call, but with a neutral prompt the model simply answers instead of being forced to call one (3/3).
• A named function choice is ignored — requesting get_time returned a get_weather call (3/3).

Since the upstream disregards both constraints, downgrading them to auto is behaviourally identical and keeps the mapping from implying a guarantee it cannot make. Widening the list is a one-line change once either is enforced upstream.

An earlier /v1/models mismatch (stale context_length: 131072 / modality: "text->text") and a malformed non-streaming finish_reason under required were both reported upstream during review and have since been fixed.

Testing

• TEST_MODELS="ranoai/gemma-4-31b-it" FULL_MODE=true pnpm test:e2e — 27 files passed, 0 failed; 89 passed. All 11 RanoAI cases green: basic (exercises vision), streaming, tool calls, streaming tool calls, tool-call results, JSON output, JSON output streaming, Responses API single-turn/multi-turn/tool-calls, empty-response DONE handling.
• CONTEXT_SIZE_TEST=true context-size e2e — passed, driving 14,336 real input tokens through the gateway to confirm the 20480 window.
• pnpm test:unit — 3088 passed / 2 failed; both pre-existing and unrelated (fallback.spec.ts fails identically on a clean checkout of main; stealth-error-redaction.spec.ts passes in isolation and is order-flaky).
• pnpm build 17/17, pnpm lint 17/17, pnpm format clean.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features
  • Added RanoAI as a supported LLM provider, including default endpoint routing and streaming transformation support.
  • Updated the UI provider selector to display the RanoAI icon/logo.
  • Added Gemma 4 (RanoAI) model support with configured token limits and enabled capabilities (streaming, vision, tools, and JSON output).
  • Expanded RanoAI API key configuration across environment examples, Helm chart values, and e2e workflow runs.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

RanoAI is registered as an LLM provider with Gemma model support, OpenAI-compatible endpoint and streaming routing, API key configuration, e2e secret wiring, and provider icon mappings.

Changes

RanoAI provider integration

Layer / File(s)	Summary
Provider and model registration packages/models/src/providers.ts, packages/models/src/models/google.ts	Adds RanoAI metadata and configures gemma-4-31b-it with RanoAI capabilities, pricing, limits, and streaming support.
Endpoint and streaming routing packages/actions/src/get-provider-endpoint.ts, apps/gateway/src/chat/tools/transform-streaming-to-openai.ts	Maps RanoAI to https://api.ranoai.com, the shared chat-completions endpoint, and OpenAI-style stream transformation.
Provider icon registration packages/shared/src/components/provider-icons.tsx, apps/ui/src/components/provider-keys/provider-logo.ts	Adds the RanoAI SVG icon and registers it in shared and UI logo lookups.
Configuration and e2e wiring .env.example, .env.unified.example, infra/helm/llmgateway/values.yaml, .github/workflows/e2e.yml	Adds the RanoAI API key configuration and passes its repository secret to e2e shard execution.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• theopenco/llmgateway#2875: Adds another OpenAI-compatible provider across streaming, endpoint, registry, and model handling.
• theopenco/llmgateway#3165: Extends the Gemini model catalogue in the same model definition file.
• theopenco/llmgateway#3206: Adds another provider to shared endpoint and provider-catalogue logic.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant getProviderEndpoint
  participant RanoAI
  participant transformStreamingToOpenai
  Client->>getProviderEndpoint: Resolve the RanoAI endpoint
  getProviderEndpoint->>RanoAI: Send chat completion request
  RanoAI-->>transformStreamingToOpenai: Return streaming chunks
  transformStreamingToOpenai-->>Client: Return OpenAI-style stream

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: adding the RanoAI provider, and is concise and specific.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch add-ranoai-provider

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}