Skip to content

feat: add StreamLake provider metadata - #41006

Open
ZHOUKAILIAN wants to merge 2 commits into
BerriAI:mainfrom
ZHOUKAILIAN:litellm_streamlake_provider_refresh
Open

ZHOUKAILIAN wants to merge 2 commits into
BerriAI:mainfrom
ZHOUKAILIAN:litellm_streamlake_provider_refresh

Conversation

@ZHOUKAILIAN

@ZHOUKAILIAN ZHOUKAILIAN commented Sep 13, 2026 •

Copy link
Copy Markdown

TLDR

Problem this solves:

  • StreamLake has no registered provider or model pricing
  • Provider discovery and dashboard setup need matching registrations

How it solves it:

  • Registers StreamLake as an OpenAI-compatible chat provider
  • Adds 15 public model records and provider setup fields
  • Publishes StreamLake in the bundled endpoint support matrix
  • Enables dashboard model selection and verifies deployment payloads
  • Tests synchronous and asynchronous completion requests through the SDK

This replaces #40354 at its author's request. The branch is rebased onto current main at bfc805b37279. The earlier dashboard schema sync commit was dropped because its change is already upstream. This does not include #40355

User Flow

Before: a developer cannot use StreamLake by provider name

  1. They configure streamlake/<model-or-endpoint> with their StreamLake API key
  2. They send POST https://litellm-domain/v1/chat/completions with that configured model and a user message
  3. The request fails because StreamLake is not a registered provider

After: the same developer can use StreamLake by provider name

  1. They configure streamlake/<model-or-endpoint> with their StreamLake API key
  2. They send POST https://litellm-domain/v1/chat/completions with that configured model and a user message
  3. With valid credentials and an accessible model, they can receive a chat completion from StreamLake

Provider discovery lists StreamLake under Chat Completions. Dashboard users can select StreamLake and a public model, enter their API key, and submit a deployment

These are the intended live flows. Tests cover request routing, endpoint discovery, and the dashboard submission payload, but live provider verification remains outstanding

Relevant issues

Replacement for #40354, which remains open for review history

The old proxy-behavior failure is the upstream auth-prefetch cache TTL flake fixed by #40996, now included through the rebase

Affected release

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally
  • My PR passes all required CI/CD checks
  • My PR's scope is as isolated as possible
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Local verification

At 2b17f1e, with PYTHON_DOTENV_DISABLED=1 and LITELLM_LOCAL_MODEL_COST_MAP=True:

uv run --no-sync pytest tests/test_litellm/llms/openai_like tests/test_litellm/litellm_core_utils/test_get_llm_provider_endpoint_match.py tests/test_litellm/proxy/public_endpoints/test_public_endpoints.py -q --tb=short -ra

Result: 296 passed, 5 skipped. The skips require PublicAI or Xiaomi credentials, not StreamLake credentials. All 50 StreamLake provider cases pass, including six HTTP-level completion cases: sync/async with the default base, a custom base, and a custom base ending in /

The completion tests call the public LiteLLM functions and the real OpenAI SDK, with only HTTP intercepted by respx. They assert the URL, method, authorization header, complete request body, model-prefix removal, token-parameter mapping, and parsed response content and usage

Mutation check: removing StreamLake from the bundled endpoint support file makes the new /public/endpoints regression fail. Restoring the entry makes it pass

cd ui/litellm-dashboard
npx vitest run src/components/provider_info_helpers.test.tsx 'src/app/(dashboard)/models-and-endpoints/panels/AddModelPanel.integration.test.tsx'

Result: 86 passed. The integration test selects StreamLake and a public model through the real form components, enters an API key, and checks the exact model-creation payload

The dashboard production build passed. Targeted ESLint passed with 0 errors and 11 existing warnings. Targeted Ruff checks, changed production Python/StreamLake-test formatting, test-quality and type-discipline gates against upstream/main, and git diff --check passed. No lint budgets were changed

Both cost maps preserve all upstream records and add the same 15 StreamLake records; the two files remain byte-identical. Both endpoint matrices preserve their existing entries and add the same StreamLake stanza

The checkout is sparse and uses locked Python dependencies without building the Rust extension. make lint BASE_REF=upstream/main could not complete its environment setup because the Rust build sources were not checked out. The Python-path tests and targeted gates above passed; full lint, native builds, coverage, and new-tip bot reviews remain for CI

Screenshots / Proof of Fix

Before (bfc805b)

  1. Live StreamLake request not run because provider credentials were unavailable
  2. No live response evidence is claimed

After (2b17f1e)

  1. Live StreamLake request not run because provider credentials were unavailable
  2. No live response or browser screenshot evidence is claimed; the automated tests above are not E2E proof

Type

New Feature

Bug Fix

Test

Caveats (if any)

Medium

Low

  • Live StreamLake verification still requires provider credentials
  • Full lint, native builds, coverage, and bot reviews await CI
  • Maintainer approval is still required

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@CLAassistant

CLAassistant commented Sep 13, 2026 •

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
3 out of 6 committers have signed the CLA.

✅ mateo-berri
✅ yuneng-berri
✅ ZHOUKAILIAN
❌ kerry-berri
❌ devin-ai-integration[bot]
❌ berriai-litellm-provider-info-sync[bot]
You have signed the CLA already but the status is still pending? Let us recheck it.

@codspeed

codspeed Bot commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing ZHOUKAILIAN:litellm_streamlake_provider_refresh (8ca411c) with litellm_internal_staging (c2c2a62)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with the previously missing behavioral coverage now added and no outstanding findings.

Findings

  1. P2 Behavior remains untested ▶

Summary

This PR registers StreamLake as an OpenAI-compatible chat provider and completes its pricing, discovery, dashboard, and behavioral test coverage.

  • Adds StreamLake provider resolution, environment configuration, URL construction, and token-parameter mapping
  • Registers 15 public models with pricing, limits, rate metadata, and supported capabilities
  • Exposes StreamLake through public endpoint discovery and dashboard model creation
  • Adds mocked synchronous and asynchronous completion coverage plus discovery and dashboard payload regressions

Reviews (2) · Last reviewed commit: "fix(streamlake): complete provider disco..."

}


class TestStreamlakeProviderConfig:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Behavior remains untested

These tests inspect registration and metadata without running a mocked chat completion. The repository requires behavioral tests before merging

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@codecov

codecov Bot commented Sep 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@ZHOUKAILIAN
ZHOUKAILIAN force-pushed the litellm_streamlake_provider_refresh branch from 8ca411c to 2b17f1e Compare September 20, 2026 13:19
@ZHOUKAILIAN
ZHOUKAILIAN changed the base branch from litellm_internal_staging to main September 20, 2026 13:20
@ZHOUKAILIAN
ZHOUKAILIAN requested a review from a team September 20, 2026 13:20
@ZHOUKAILIAN

Copy link
Copy Markdown
Author

@greptileai Please re-review 2b17f1e: added mocked completion coverage, fixed endpoint discovery and dashboard registration, and rebased onto main.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants