Skip to content

fix(ui): keep completion-mode models in the playground chat dropdown (backport to rc/1.98.0) - #37955

Merged
yuneng-berri merged 2 commits into
rc/1.98.0from
litellm_rc_1_98_0_playground_completion_mode
Aug 22, 2026
Merged

fix(ui): keep completion-mode models in the playground chat dropdown (backport to rc/1.98.0)#37955
yuneng-berri merged 2 commits into
rc/1.98.0from
litellm_rc_1_98_0_playground_completion_mode

Conversation

@yuneng-berri

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Completion-mode models vanish from the playground model dropdown
  • rc/1.98.0 shipped that regression, so the line needs the fix
  • The line's committed UI bundle predates the fix

How it solves it:

User Flow

Before: an admin who registered a legacy text-completion deployment cannot select it in the playground, even though the proxy routes it fine

  1. They register davinci-002 and confirm it is live: POST https://litellm-domain/v1/chat/completions with "model": "davinci-002" returns 200 with a normal completion
  2. They open https://litellm-domain/ui/?page=llm-playground
  3. The endpoint selector is on /chat/completions
  4. They open the model dropdown and davinci-002 is absent, while their chat models are all listed
  5. They switch the endpoint to /responses and it is absent there too, so there is no way to try the model from the UI

After: the same deployment shows up and is selectable

  1. They register davinci-002 and confirm it is live: POST https://litellm-domain/v1/chat/completions with "model": "davinci-002" returns 200 with a normal completion
  2. They open https://litellm-domain/ui/?page=llm-playground
  3. The endpoint selector is on /chat/completions
  4. They open the model dropdown and davinci-002 is listed alongside their chat models
  5. They switch the endpoint to /responses and it is listed there too, and sending a prompt returns a completion
  6. Speech and transcription endpoints still do not list it, which is correct

Relevant issues

Backport of #37954

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Shared setup, run once against a proxy started from this branch:

cat > /tmp/completion_mode.yaml <<'EOF'
model_list:
  - model_name: davinci-002
    litellm_params:
      model: text-completion-openai/davinci-002
      api_key: os.environ/OPENAI_API_KEY
    model_info:
      mode: completion
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
EOF
python litellm/proxy/proxy_cli.py --config /tmp/completion_mode.yaml --detailed_debug 2>&1 | tee litellm.log

Confirm the deployment really routes, so the dropdown is the only thing at fault:

curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"davinci-002","messages":[{"role":"user","content":"say hi"}]}'

Before (bc6e7df)

Check out v1.98.0-rc.1, boot the proxy against the config above, then:

  1. Open http://localhost:4000/ui/?page=llm-playground
  2. Leave the endpoint selector on /chat/completions
  3. Open the model dropdown, screenshot it: davinci-002 is missing, gpt-4o is present
  4. Switch the endpoint selector to /responses, screenshot the dropdown: davinci-002 is missing there too

After (2f39f5c)

Check out this branch, boot the proxy against the same config, then:

  1. Open http://localhost:4000/ui/?page=llm-playground
  2. Leave the endpoint selector on /chat/completions
  3. Open the model dropdown, screenshot it: davinci-002 is now listed next to gpt-4o
  4. Select davinci-002, send "say hi", screenshot the reply
  5. Switch the endpoint selector to /responses, screenshot the dropdown: davinci-002 is listed
  6. Switch to /audio/speech, screenshot the dropdown: davinci-002 is correctly absent

Screenshots to follow on this PR once the runs above are captured

Type

🐛 Bug Fix

Caveats (if any)

  • Base is rc/1.98.0, the release line, not staging
  • Second commit is generated bundle output, no hand edits

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

…37954)

PR #36130 added a KNOWN_MODEL_MODES guard to isModelCompatibleWithEndpoint
that hides any model whose mode isn't in the ModelMode enum, to keep
rerank/ocr/batch/etc. models out of chat-style endpoints. mode: completion
(legacy text-completion models) wasn't in that enum, so it got caught by
the same guard and disappeared from every endpoint, including chat, where
it routes fine.

Add ModelMode.COMPLETION and map it to EndpointType.CHAT like the other
chat-compatible modes.

(cherry picked from commit 1c421f3)
@greptile-apps

greptile-apps Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review (445 files, 100 file limit).

Bypass the limit by tagging @greptile-apps to review.

@yuneng-berri
yuneng-berri merged commit d8f71d7 into rc/1.98.0 Aug 22, 2026
4 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_rc_1_98_0_playground_completion_mode branch August 22, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant