[litellm-agent] Staging → litellm_internal_staging (5/11/2026) - #27661
[litellm-agent] Staging → litellm_internal_staging (5/11/2026)#27661oss-pr-review-agent-shin[bot] wants to merge 5 commits into
Conversation
* Include provider in model names for ollama * Fix unit tests
* fix(health_check): skip max_tokens for image_generation mode
`_update_litellm_params_for_health_check` injected `max_tokens` for
every deployment. OpenAI `/v1/images/generations` strictly rejects
unknown fields, so health checks for dall-e-* and gpt-image-1 always
failed with `400 "Unknown parameter: 'max_tokens'"` even though the
actual image endpoint calls succeed. Skip the `max_tokens` injection
when `model_info.mode == "image_generation"`. `messages` still gets
injected (downstream `_filter_model_params` already strips it for
non-chat handlers).
* Switch to allow-list with per-deployment override
Per @krrishdholakia review: deny-listing image_generation only re-introduces
the same bug for every other non-chat mode (embedding, audio_*, rerank,
video_generation, ocr, search, moderation, ...).
Replace the single image_generation skip with `_MAX_TOKEN_SUPPORT_MODES =
{chat, completion, responses}`. Missing `mode` is treated as chat for
backward compatibility. New modes are safe by default.
Add `model_info.health_check_supports_max_tokens` as an operator escape
hatch — True forces injection on a non-listed deployment (operator wants
to bound probe tokens), False suppresses it on a chat-style deployment
behind a strict-schema provider.
Tests: parametrize over 3 chat-style + 10 non-chat modes, plus override
on/off and the no-mode legacy path.
… multipart uploads (#26718) Squash-merged by litellm-agent from dawidkulpa's PR.
|
@greptile please review |
Greptile SummaryThis automated staging PR bundles four independent bug fixes: crash prevention in
Confidence Score: 4/5Safe to merge with awareness that two changes alter observable output for existing users: Ollama model names now carry an Three of the four changes are clean, targeted bug fixes with matching tests and no observable regressions. The health-check change silently drops a previously-honoured config key (
|
| Filename | Overview |
|---|---|
| litellm/llms/custom_httpx/http_handler.py | Catches httpx.RequestNotRead when accessing streaming request body during error masking — clean defensive fix with matching test. |
| litellm/llms/ollama/chat/transformation.py | Changes elif to if so a chunk containing both thinking and content fields populates both reasoning_content and content on the delta — correct bug fix with new test. |
| litellm/llms/ollama/common_utils.py | Adds ollama/ prefix to model names returned by the live /api/tags endpoint, matching the static fallback path; any caller consuming bare model names from the dynamic path will see a breaking change. |
| litellm/proxy/health_check.py | Adds mode-based allow-list gating for max_tokens injection in health-check probes; fixes image/embedding 400 errors but silently drops health_check_max_tokens config for non-chat modes. |
| tests/test_litellm/proxy/test_health_check_max_tokens.py | Comprehensive new tests covering mode allow-list, per-deployment overrides, regression guards for chat/no-mode paths — all mock-only, no network calls. |
| tests/test_litellm/llms/custom_httpx/test_credential_leak_prevention.py | New test validates that streaming request bodies no longer crash MaskedHTTPStatusError construction and that credential masking still applies. |
| tests/test_litellm/llms/ollama/test_ollama_chat_transformation.py | New test for same-chunk thinking+content handling; existing test assertions are unchanged, no coverage weakening. |
| tests/test_litellm/llms/ollama/test_ollama_model_info.py | Existing assertions updated to expect ollama/-prefixed names, correctly reflecting the new consistent prefix behaviour. |
Reviews (1): Last reviewed commit: "fix(http_handler): handle RequestNotRead..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| def _should_inject_health_check_max_tokens(model_info: dict) -> bool: | ||
| """ | ||
| Whether the health-check probe should include `max_tokens`. | ||
|
|
||
| Order: | ||
| 1. `model_info.health_check_supports_max_tokens` (operator override). | ||
| 2. `_MAX_TOKEN_SUPPORT_MODES`. Missing `mode` is treated as `chat` | ||
| for backward compatibility. | ||
| """ | ||
| explicit = model_info.get("health_check_supports_max_tokens") | ||
| if explicit is not None: | ||
| return bool(explicit) | ||
| mode = model_info.get("mode") or "chat" | ||
| return mode in _MAX_TOKEN_SUPPORT_MODES |
There was a problem hiding this comment.
health_check_max_tokens no longer auto-enables injection on non-chat modes
Previously, setting health_check_max_tokens: 50 on any deployment (including mode: image_generation) would cause that value to be injected as max_tokens. After this change, _should_inject_health_check_max_tokens consults health_check_supports_max_tokens (a separate bool key) first, then the mode allow-list. An operator who relied on health_check_max_tokens to bound token usage on a non-chat endpoint that happens to accept the field will silently have it dropped — they now need the additional health_check_supports_max_tokens: true key. This is a backwards-incompatible behaviour change without a migration notice or a compatibility flag. The test at line 241–250 in the test file explicitly documents this dropped behaviour.
Rule Used: What: avoid backwards-incompatible changes without... (source)
…11_2026 Preserve both independent features added in parallel: - max_tokens mode-gating for health checks (skip injection for image_generation and other non-chat modes; per-deployment override via health_check_supports_max_tokens) - reasoning_effort forwarding for chat-style health-check probes (_HEALTH_CHECK_MODES_SUPPORTING_REASONING_EFFORT + health_check_reasoning_effort) Co-authored-by: Cursor <cursoragent@cursor.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
6 participants
Automated staging PR created by litellm-agent.
This branch collects PRs approved by the agent on 5/11/2026.