feat(scaledown): add native text provider support by soham96 · Pull Request #44168 · BerriAI/litellm

soham96 · 2026-10-02T10:59:26Z

TLDR

Problem this solves:

ScaleDown's native operations cannot run through LiteLLM
Missing pricing prevents accurate request cost tracking

How it solves it:

Adds classify, extract, summarize, decisions, and compress
Uses native endpoints with x-api-key authentication
Registers current text pricing and supported capabilities
Rejects unsupported inputs before contacting the provider

User Flow

Before, a developer cannot send ScaleDown requests through the proxy

Configure the ScaleDown models and provider key
POST text to http://127.0.0.1:4021/v1/chat/completions
Receive HTTP 400 because no ScaleDown deployment is available

After, the same task returns the native result and tracked cost

Configure the ScaleDown models and provider key
POST the same text to http://127.0.0.1:4019/v1/chat/completions
Receive HTTP 200, a JSON result in message content, and input usage
Read the cost from x-litellm-response-cost

In the Admin UI, open /ui/models-and-endpoints/, choose Add Model, and search for scaledown under Provider. Before, the search returns no providers. After, ScaleDown is selectable and exposes its API key field

Implementation and validation

scaledown/classify calls /classify with labels. scaledown/decisions calls /v1/scaledown with questions and the upstream model classify-1. Extraction combines scalar matches with nested results and removes provider span wrappers while retaining the untouched payload in SDK hidden parameters

The adapter uses the existing HTTP handler for sync, async, and simulated streaming. ScaleDown has its own dispatch entry. The existing xAI helper, xAI dispatch branch, and shared HTTP transport are unchanged. The provider registry input and ScaleDown model set are typed. Schema expansion bounds recursion, reference chains, and entity count

The focused provider and pricing suite passes 136 tests with 98% adapter coverage locally. The expanded run passes 501 tests, including 365 existing xAI, provider-routing, endpoint-matching, cost-map, and timeout tests. A baseline comparison confirms all 4,035 existing pricing entries, 2,558 existing models, and existing provider registries, endpoint declarations, and Admin UI credential fields are unchanged. Only ScaleDown entries are added. make check passes, including full type checking, test quality, formatting, and API schema sync. The adapter also passes strict type checking with zero errors. The recursion detector and CI shard inventory pass. The production docs build passes, and the page was inspected in a browser

Live checks also passed for mixed extraction, strict-schema rejection, image rejection, and streaming with extra_body.labels and stream_options.include_usage. Mixed extraction and nullable nested extraction both returned {"vendor":"Northwind","invoice":{"amount":500}}. Developer-message summarization returned the expected summary. An invalid question type and a nullable schema with sibling properties returned HTTP 400 and zero cost. A live request with a deliberately invalid key also invoked the SDK post-call logger with the provider's HTTP 402 response. Rejected requests returned HTTP 400 and zero cost. Streaming returned the classification result, 521 input tokens, and cost 0.00002605

The review follow-up preserves nullable nested objects and arrays, rejects ambiguous schema composition, treats malformed question types and unknown model names as HTTP 400, and forwards developer instructions for summarization. Malformed HTTP 200 responses become HTTP 502 errors

The provider model set is Final but stays mutable because pricing reloads update references held by existing callers. The bounded schema walker has explicit mutation exceptions to carry its entity budget and visited references. Adapter validation errors use one shared _reject function. The ScaleDown dispatch preserves the existing post-call hook when an upstream request fails

Four earlier regression cases failed before the fixes and pass afterward. Three reject sibling schema definitions beside nullable unions or $ref instead of dropping fields. The fourth checks that a failed upstream request reaches the caller's logger_fn as a post-call event

The latest review fix decodes URI fragments and JSON Pointer escapes before following local schema references. Unresolvable or unsupported references return HTTP 400 instead of silently dropping fields. All eight new regression cases failed on 3cdcd956a58b2436f692ba4f26580c5f664bd307 and pass on this tip. A fresh live request using the definition name Invoice/~1 Details returns both the vendor and amount. A missing definition returns HTTP 400 and zero cost

Merge dependencies

The docs PR is merged into main. The documentation and code-quality checks now pass against the merged docs, including the checks for SCALEDOWN_API_KEY and SCALEDOWN_API_BASE

The pricing PR contains the same ScaleDown registry and price entries. Both code PRs use $0.05 per million input tokens with output free, so either merge order preserves the rate

Both code PRs now include main at 16fcbffa7fd344c611c627ef720961b4cdd1c211. The pricing-file conflict retains both new Azure embedding models and all five ScaleDown entries. Both branches merge cleanly, including with each other. The updated cost-map guard passes locally

All four required checks pass on 0c942ec2794ba00fbe306e8b19c0ca82b5721e31, including lint, unit, smoke, and cost-map-guard. Greptile reviewed this commit at 5/5. Veria reports no open security concerns. No Bugbot result is available, so it is not recorded as a pass

Pre-Submission checklist

Meaningful regression tests added
Focused tests and local checks pass
All required CI/CD checks pass on the current tip
Scope is limited to ScaleDown provider support
Current-tip Greptile confidence is at least 4/5

Screenshots / Proof of Fix

Captured against a localhost proxy and the real ScaleDown API. The baseline is the merge base below. The fresh API run uses the provider code at the current tip. The Admin UI screenshots show the unchanged ScaleDown selector from the earlier validation. No mocked upstream responses were used

Use a real ScaleDown key in SCALEDOWN_API_KEY, set a local proxy master key, and use that master key as LITELLM_API_KEY in the curl commands. Start the baseline proxy on port 4021 and the updated proxy on port 4019 with this config

model_list:
  - model_name: scaledown-classify
    litellm_params:
      model: scaledown/classify
      api_key: os.environ/SCALEDOWN_API_KEY
  - model_name: scaledown-extract
    litellm_params:
      model: scaledown/extract
      api_key: os.environ/SCALEDOWN_API_KEY
  - model_name: scaledown-summarize
    litellm_params:
      model: scaledown/summarize
      api_key: os.environ/SCALEDOWN_API_KEY
  - model_name: scaledown-decisions
    litellm_params:
      model: scaledown/decisions
      api_key: os.environ/SCALEDOWN_API_KEY
  - model_name: scaledown-compress
    litellm_params:
      model: scaledown/compress
      api_key: os.environ/SCALEDOWN_API_KEY
litellm_settings:
  disable_aiohttp_transport: true
  telemetry: false
router_settings:
  num_retries: 0

Save these request bodies in the named files

classify.json

{"model":"scaledown-classify","messages":[{"role":"user","content":"My server has been down for three hours."}],"labels":[{"name":"urgent","rubric":"Service is unavailable."},{"name":"routine","rubric":"A question or cosmetic issue."}]}

extract.json

{"model":"scaledown-extract","messages":[{"role":"user","content":"Invoice from Northwind. Total: $500."}],"response_format":{"type":"json_schema","json_schema":{"name":"invoice","strict":false,"schema":{"type":"object","properties":{"vendor":{"type":"string","description":"company name"}}}}}}

summarize.json

{"model":"scaledown-summarize","messages":[{"role":"system","content":"Summarize in one sentence."},{"role":"user","content":"The team shipped the billing fix on Monday. Error rates fell. A follow-up review is scheduled for Friday."}],"max_tokens":64}

decisions.json

{"model":"scaledown-decisions","messages":[{"role":"user","content":"I was charged twice for my subscription this month."}],"questions":{"category":{"type":"choice","criteria":{"billing":"A charge or refund.","technical":"A bug."}}}}

compress.json

{"model":"scaledown-compress","messages":[{"role":"system","content":"The team shipped a billing fix. Error rates fell after the release."},{"role":"user","content":"What changed after the release?"}],"compression_rate":"auto"}

extract-escaped-ref.json

{"model":"scaledown-extract","messages":[{"role":"user","content":"Invoice from Northwind for $500."}],"response_format":{"type":"json_schema","json_schema":{"name":"invoice","strict":false,"schema":{"type":"object","$defs":{"Invoice/~1 Details":{"type":"object","properties":{"vendor":{"type":"string"},"amount":{"type":"number"}}}},"properties":{"invoice":{"$ref":"#/$defs/Invoice~1~01%20Details"}}}}}}

extract-missing-ref.json

{"model":"scaledown-extract","messages":[{"role":"user","content":"Invoice from Northwind for $500."}],"response_format":{"type":"json_schema","json_schema":{"name":"invoice","strict":false,"schema":{"type":"object","$defs":{"Invoice/~1 Details":{"type":"object","properties":{"vendor":{"type":"string"},"amount":{"type":"number"}}}},"properties":{"invoice":{"$ref":"#/$defs/Missing"}}}}}}

Before (`16fcbff`)

Classify

Send the request

curl -sS -D classify.headers http://127.0.0.1:4021/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @classify.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{
  "error": {
    "message": "litellm.BadRequestError: You passed in model=scaledown-classify. There are no healthy deployments for this model\n\nLiteLLM: model group 'scaledown-classify' failed with the error above. No fallback was attempted.",
    "type": "invalid_request_error",
    "param": null,
    "code": "400"
  }
}

Extract

Send the request

curl -sS -D extract.headers http://127.0.0.1:4021/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @extract.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{
  "error": {
    "message": "litellm.BadRequestError: You passed in model=scaledown-extract. There are no healthy deployments for this model\n\nLiteLLM: model group 'scaledown-extract' failed with the error above. No fallback was attempted.",
    "type": "invalid_request_error",
    "param": null,
    "code": "400"
  }
}

Summarize

Send the request

curl -sS -D summarize.headers http://127.0.0.1:4021/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @summarize.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{
  "error": {
    "message": "litellm.BadRequestError: You passed in model=scaledown-summarize. There are no healthy deployments for this model\n\nLiteLLM: model group 'scaledown-summarize' failed with the error above. No fallback was attempted.",
    "type": "invalid_request_error",
    "param": null,
    "code": "400"
  }
}

Decisions

Send the request

curl -sS -D decisions.headers http://127.0.0.1:4021/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @decisions.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{
  "error": {
    "message": "litellm.BadRequestError: You passed in model=scaledown-decisions. There are no healthy deployments for this model\n\nLiteLLM: model group 'scaledown-decisions' failed with the error above. No fallback was attempted.",
    "type": "invalid_request_error",
    "param": null,
    "code": "400"
  }
}

Compress

Send the request

curl -sS -D compress.headers http://127.0.0.1:4021/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @compress.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{
  "error": {
    "message": "litellm.BadRequestError: You passed in model=scaledown-compress. There are no healthy deployments for this model\n\nLiteLLM: model group 'scaledown-compress' failed with the error above. No fallback was attempted.",
    "type": "invalid_request_error",
    "param": null,
    "code": "400"
  }
}

Extraction with escaped references

Send the request

curl -sS -D extract-escaped-ref.headers http://127.0.0.1:4021/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @extract-escaped-ref.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{
  "error": {
    "message": "litellm.BadRequestError: You passed in model=scaledown-extract. There are no healthy deployments for this model\n\nLiteLLM: model group 'scaledown-extract' failed with the error above. No fallback was attempted.",
    "type": "invalid_request_error",
    "param": null,
    "code": "400"
  }
}

Unresolved extraction reference

Send the request

curl -sS -D extract-missing-ref.headers http://127.0.0.1:4021/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @extract-missing-ref.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{
  "error": {
    "message": "litellm.BadRequestError: You passed in model=scaledown-extract. There are no healthy deployments for this model\n\nLiteLLM: model group 'scaledown-extract' failed with the error above. No fallback was attempted.",
    "type": "invalid_request_error",
    "param": null,
    "code": "400"
  }
}

Admin UI provider selection

Open http://127.0.0.1:4021/ui/models-and-endpoints/, choose Add Model, and search for scaledown
The provider list says No providers found, shown in the Before screenshot under User Flow

After (`0c942ec`)

Classify

Send the request

curl -sS -D classify.headers http://127.0.0.1:4019/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @classify.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 200 OK
x-litellm-response-cost: 2.605e-05
{
  "result": {
    "top_label": "urgent",
    "scores": {
      "urgent": 0.9999785551727421,
      "routine": 2.144482725783596e-05
    },
    "labels": [
      {
        "label": "urgent",
        "score": 0.9999785551727421,
        "rubric": "Service is unavailable."
      },
      {
        "label": "routine",
        "score": 2.144482725783596e-05,
        "rubric": "A question or cosmetic issue."
      }
    ],
    "reasoning": "The text explicitly states the server has been down, which constitutes a service unavailability requiring urgent attention.",
    "results": [
      {
        "id": "0",
        "top_label": "urgent",
        "matched_labels": null,
        "scores": {
          "urgent": 0.9999785551727421,
          "routine": 2.144482725783596e-05
        },
        "labels": [
          {
            "label": "urgent",
            "score": 0.9999785551727421,
            "rubric": "Service is unavailable."
          },
          {
            "label": "routine",
            "score": 2.144482725783596e-05,
            "rubric": "A question or cosmetic issue."
          }
        ]
      }
    ],
    "chunk_count": 1,
    "ocr_text": null,
    "input_tokens": 521
  },
  "usage": {
    "completion_tokens": 0,
    "prompt_tokens": 521,
    "total_tokens": 521
  }
}

Extract

Send the request

curl -sS -D extract.headers http://127.0.0.1:4019/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @extract.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 200 OK
x-litellm-response-cost: 8.05e-06
{
  "result": {
    "vendor": "Northwind"
  },
  "usage": {
    "completion_tokens": 0,
    "prompt_tokens": 161,
    "total_tokens": 161
  }
}

Summarize

Send the request

curl -sS -D summarize.headers http://127.0.0.1:4019/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @summarize.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 200 OK
x-litellm-response-cost: 4.5e-06
{
  "result": {
    "summary": "After the team shipped the billing fix on Monday, which reduced error rates, a follow-up review is scheduled for Friday.",
    "input_chars": 105,
    "output_chars": 120,
    "latency_ms": 245,
    "ocr_text": null,
    "input_tokens": 90
  },
  "usage": {
    "completion_tokens": 0,
    "prompt_tokens": 90,
    "total_tokens": 90
  }
}

Decisions

Send the request

curl -sS -D decisions.headers http://127.0.0.1:4019/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @decisions.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 200 OK
x-litellm-response-cost: 1.1599999999999999e-05
{
  "result": {
    "category": {
      "type": "choice",
      "choice": "billing",
      "probabilities": {
        "billing": 0.9999898700214296,
        "technical": 1.012997857045329e-05
      },
      "confidence": 0.9999898700214296
    }
  },
  "usage": {
    "completion_tokens": 1,
    "prompt_tokens": 232,
    "total_tokens": 233
  }
}

Compress

Send the request

curl -sS -D compress.headers http://127.0.0.1:4019/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @compress.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 200 OK
x-litellm-response-cost: 1.2e-06
{
  "result": {
    "results": {
      "success": true,
      "compressed_prompt": "The team shipped a billing fix. Error rates fell after the release.",
      "original_prompt": "What changed after the release?",
      "original_prompt_tokens": 24,
      "compressed_prompt_tokens": 14,
      "compression_ratio": 0.5833333333333334
    },
    "model_used": "gpt-4o",
    "input_tokens": 24,
    "total_original_tokens": 24,
    "total_compressed_tokens": 14,
    "num_pairs_processed": 1,
    "num_successful": 1,
    "num_failed": 0,
    "successful": true,
    "partially_successful": false,
    "latency_ms": 45,
    "request_metadata": {
      "compression_time_ms": 45,
      "compression_rate": "auto",
      "average_compression_ratio": 0.5833333333333334
    }
  },
  "usage": {
    "completion_tokens": 0,
    "prompt_tokens": 24,
    "total_tokens": 24
  }
}

Extraction with escaped references

Send the request

curl -sS -D extract-escaped-ref.headers http://127.0.0.1:4019/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @extract-escaped-ref.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 200 OK
x-litellm-response-cost: 9.4e-06
{
  "result": {
    "invoice": {
      "vendor": "Northwind",
      "amount": 500
    }
  },
  "usage": {
    "completion_tokens": 0,
    "prompt_tokens": 188,
    "total_tokens": 188
  }
}

Unresolved extraction reference

Send the request

curl -sS -D extract-missing-ref.headers http://127.0.0.1:4019/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' --data-binary @extract-missing-ref.json \
  | jq 'if .error then {error} else {result:(.choices[0].message.content|fromjson),usage} end'

Observe the response and saved cost header

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{
  "error": {
    "message": "litellm.BadRequestError: ScaledownException - ScaleDown extraction cannot resolve $ref '#/$defs/Missing' to a schema object.\n\nLiteLLM: model group 'scaledown-extract' failed with the error above. No fallback was attempted.",
    "type": "invalid_request_error",
    "param": null,
    "code": "400"
  }
}

Admin UI provider selection

Open http://127.0.0.1:4019/ui/models-and-endpoints/, choose Add Model, and search for scaledown
Select ScaleDown and observe its API key field, shown in the After screenshot under User Flow

Type

New Feature

Caveats

Medium

Text only, document and image billing remains unsupported
Extraction hints do not enforce JSON Schema validation
Streaming is simulated after the native response completes
Classify, decisions, and extract reject system and developer messages

Low

Admin UI selection checked, database-backed model creation not exercised

CLAassistant · 2026-10-02T10:59:38Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ moe-berri
❌ soham96
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

greptile-apps · 2026-10-02T11:03:45Z

Confidence Score: 5/5

[Medium impact] The reviewed changes appear safe to merge.

Summary

Adds ScaleDown’s five native text operations, input-token pricing, provider registration, and an Admin UI credential field.

LiteLLM sends five ScaleDown text tasks to native endpoints.
LiteLLM tracks input-token costs for five ScaleDown models.
ScaleDown appears in LiteLLM’s provider setup and endpoint listings.

_{Reviews (16) · Last reviewed commit: "chore(scaledown): remove unused loop sup..." · Reviewed by Greptile}

veria-ai · 2026-10-02T11:07:53Z

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 5 · PR risk: 0/10

codecov · 2026-10-02T11:08:34Z

Codecov Report

❌ Patch coverage is 98.51190% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/scaledown/chat/transformation.py	98.43%	5 Missing ⚠️

📢 Thoughts on this report? Let us know!

codspeed · 2026-10-02T11:08:50Z

Merging this PR will not alter performance

⚠️

1 benchmark spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

test_completion_streaming

✅ 31 untouched benchmarks

_{Comparing soham96:scaledown-provider (0c942ec) with main (c327163)¹}

No successful run was found on main (f8d21a0) during the generation of this report, so c327163 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

Registers the five ScaleDown models in the pricing map. ScaleDown bills on input tokens only, so output_cost_per_token is zero for every entry. The classify and decisions models bill at $0.04 per million input tokens, and extract, summarize, and compress bill at $0.05 per million. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Removes the localStorage-backed ScaleDown mock now that the engine knows the provider (arthur-common#228). Every ScaleDown request goes to the real API: credentials, model whitelist, catalog, and model pickers, which offer ScaleDown once it's configured. The table still lists ScaleDown first. No backend code changes: ScaleDown is an API-key provider, which the engine already handles generically, and its catalog comes from litellm's cost map. Completions start working when litellm ships the provider (BerriAI/litellm#44168, #44167) and the pins are bumped. Regenerates staging.openapi.json and adds the API changelog entry for the new enum value. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…ontext limits Every ScaleDown model now bills $0.04 per million input tokens. The 32000-token max_input_tokens/max_output_tokens values were not confirmed against the service, so they are removed rather than registered. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This is the standard rate in ScaleDown's billing and matches the usage dashboard. The Decisions usage.cost field is not used for pricing. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

soham96 · 2026-10-09T23:32:30Z

@greptileai review

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

moe-berri · 2026-10-10T01:59:45Z

@greptileai

moe-berri · 2026-10-10T02:26:36Z

@greptileai

moe-berri · 2026-10-10T02:30:10Z

@greptileai

moe-berri · 2026-10-10T02:41:33Z

@greptileai

moe-berri · 2026-10-10T02:45:58Z

@greptileai

moe-berri · 2026-10-10T17:07:23Z

@greptileai Please review this updated commit, which resolves the merge conflict while preserving all existing provider pricing

moe-berri · 2026-10-10T17:15:38Z

@greptileai Please review the escaped-reference fix. Eight regression cases failed before this change and pass now, alongside the existing provider tests

moe-berri · 2026-10-10T17:17:27Z

@greptileai Please review the final reference fix. Escaped names now preserve nested fields, and unresolved references return HTTP 400 before contacting ScaleDown

moe-berri · 2026-10-10T17:22:05Z

@greptileai Please review the final commit. The full local check passes, with escaped-reference regression coverage and fresh live API verification

soham96 requested a review from a team October 2, 2026 10:59

soham96 requested review from litellm-stampbot, mateo-berri and ryan-crabbe-berri as code owners October 2, 2026 10:59

soham96 changed the title ~~Add ScaleDown as a chat provider~~ feat: add ScaleDown as a chat provider Oct 2, 2026

greptile-apps Bot reviewed Oct 2, 2026

View reviewed changes

veria-ai Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread litellm/llms/scaledown/chat/transformation.py Outdated

Comment thread litellm/llms/scaledown/chat/transformation.py Outdated

soham96 force-pushed the scaledown-provider branch from 72c8a3a to b657d96 Compare October 2, 2026 11:09

soham96 force-pushed the scaledown-provider branch from b657d96 to 7fb04fd Compare October 2, 2026 11:25

soham96 mentioned this pull request Oct 2, 2026

docs(scaledown): document native API integration BerriAI/litellm-docs#2016

Merged

devin-ai-integration Bot mentioned this pull request Oct 2, 2026

fix(model-prices): add missing xAI and Moonshot models and fix an OpenRouter context limit #44227

Closed

3 tasks done

This was referenced Oct 2, 2026

feat(models): add ScaleDown as a model provider arthur-ai/arthur-common#228

Draft

feat: add ScaleDown as a model provider arthur-ai/arthur-engine#2408

Draft

devin-ai-integration Bot mentioned this pull request Oct 8, 2026

fix(model_prices): add Bedrock flex prices, Anthropic web search flags, Gemini shutdown dates, OpenRouter alias drift, Mistral Large 4 context #44910

Merged

soham96 force-pushed the scaledown-provider branch from 7fb04fd to 1d60afa Compare October 9, 2026 20:14

soham96 requested a review from kerry-berri as a code owner October 9, 2026 20:14

Price all ScaleDown models at $0.05 per million input tokens

14169e9

This is the standard rate in ScaleDown's billing and matches the usage dashboard. The Decisions usage.cost field is not used for pricing. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

soham96 force-pushed the scaledown-provider branch from 1d60afa to 9f1a4da Compare October 9, 2026 22:32

veria-ai Bot reviewed Oct 9, 2026

View reviewed changes

Comment thread litellm/llms/scaledown/chat/transformation.py Outdated

Comment thread litellm/llms/scaledown/chat/transformation.py Outdated

Add regression tests for the ScaleDown pricing entries

a18239c

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

soham96 force-pushed the scaledown-provider branch from 6d4795e to 2904581 Compare October 9, 2026 23:39

moe-berri added 4 commits October 9, 2026 18:36

fix(scaledown): support native classify and preserve extraction results

a69da5b

test(scaledown): include provider regressions in CI shard

f235d5d

fix(scaledown): pass registry typing and bounded recursion checks

d38071c

docs(scaledown): capture provider setup before and after

12c8b6c

moe-berri changed the title ~~feat: add ScaleDown as a chat provider~~ feat(scaledown): add native text provider support Oct 10, 2026

moe-berri mentioned this pull request Oct 10, 2026

feat(models): register ScaleDown text pricing #44167

Merged

5 tasks done

greptile-apps Bot reviewed Oct 10, 2026

View reviewed changes

Comment thread litellm/llms/scaledown/chat/transformation.py

Comment thread litellm/llms/scaledown/chat/transformation.py

Comment thread litellm/llms/scaledown/chat/transformation.py

moe-berri added 2 commits October 9, 2026 19:17

fix(scaledown): preserve nullable schemas and reject malformed requests

c43a947

fix(scaledown): isolate provider dispatch from existing providers

bb3ebe4

fix(scaledown): preserve the shared dispatch return contract

e712f21

greptile-apps Bot reviewed Oct 10, 2026

View reviewed changes

Comment thread litellm/llms/scaledown/chat/transformation.py

Comment thread litellm/main.py Outdated

fix(scaledown): preserve failure logging and reject lossy schemas

858f1f3

fix(scaledown): keep timeout typing annotation on its expression

f3a2209

chore(scaledown): sync provider with main

3cdcd95

greptile-apps Bot reviewed Oct 10, 2026

View reviewed changes

Comment thread litellm/llms/scaledown/chat/transformation.py Outdated

fix(scaledown): resolve escaped extraction schema references

ac8366c

fix(scaledown): annotate the bounded reference walk

3bfe409

chore(scaledown): remove unused loop suppression

0c942ec

moe-berri approved these changes Oct 10, 2026

View reviewed changes

kerry-berri approved these changes Oct 10, 2026

View reviewed changes

kerry-berri merged commit 38bdda9 into BerriAI:main Oct 10, 2026
156 of 160 checks passed

Conversation

soham96 commented Oct 2, 2026 • edited by moe-berri Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Implementation and validation

Merge dependencies

Pre-Submission checklist

Screenshots / Proof of Fix

Before (16fcbff)

Classify

Extract

Summarize

Decisions

Compress

Extraction with escaped references

Unresolved extraction reference

Admin UI provider selection

After (0c942ec)

Classify

Extract

Summarize

Decisions

Compress

Extraction with escaped references

Unresolved extraction reference

Admin UI provider selection

Type

Caveats

Medium

Low

Uh oh!

CLAassistant commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

veria-ai Bot commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR overview

Security review

Uh oh!

codecov Bot commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed Bot commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

Uh oh!

Uh oh!

soham96 commented Oct 9, 2026

Uh oh!

moe-berri commented Oct 10, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

moe-berri commented Oct 10, 2026

Uh oh!

moe-berri commented Oct 10, 2026

Uh oh!

Uh oh!

Uh oh!

moe-berri commented Oct 10, 2026

Uh oh!

soham96 commented Oct 2, 2026 •

edited by moe-berri

Loading

Before (`16fcbff`)

After (`0c942ec`)

CLAassistant commented Oct 2, 2026 •

edited

Loading

greptile-apps Bot commented Oct 2, 2026 •

edited

Loading

veria-ai Bot commented Oct 2, 2026 •

edited

Loading

codecov Bot commented Oct 2, 2026 •

edited

Loading

codspeed Bot commented Oct 2, 2026 •

edited

Loading