fix(streaming): map unknown finish_reason values to finish_reason_unspecified to prevent ValidationError in stream_chunk_builder

[xykong]

Problem

Some LLM providers return non-standard finish_reason values that are not in the OpenAIChatCompletionFinishReason Literal. For example, ZhipuAI/GLM returns "network_error" when a mid-stream network error occurs on their side.

map_finish_reason() handles many known provider-specific values but previously fell through with return finish_reason for unrecognized values. When this unknown value reaches Choices.__init__(), Pydantic validation fails, and stream_chunk_builder() catches the exception and raises a misleading generic error:

litellm.APIError: Error building chunks for logging/streaming usage calculation

The real root cause (a provider-specific finish_reason value) was hidden.

Full error traceback (production)

LiteLLM:ERROR: main.py:7498 - litellm.main.py::stream_chunk_builder() - Exception occurred - 1 validation error for Choices
finish_reason
  Input should be 'stop', 'content_filter', 'function_call', 'tool_calls', 'length',
  'guardrail_intervened', 'eos', 'finish_reason_unspecified' or 'malformed_function_call'
  [type=literal_error, input_value='network_error', input_type=str]

Traceback (most recent call last):
  File "litellm/litellm_core_utils/streaming_chunk_builder_utils.py", line 127, in build_base_response
    response = ModelResponse(...)
  File "litellm/types/utils.py", line 1805
    _new_choice = Choices(**choice)
pydantic_core.ValidationError: 1 validation error for Choices
  finish_reason Input should be ... [input_value='network_error']

Related issue: #22671

Solution

After all known provider-specific mappings in map_finish_reason(), validate the result against the set of valid OpenAIChatCompletionFinishReason values. Any unrecognized value is mapped to "finish_reason_unspecified" rather than being returned as-is.

This is consistent with:

• How Vertex AI's FINISH_REASON_UNSPECIFIED is already handled (maps to "finish_reason_unspecified")
• The approach in PR fix: handle empty string finish_reason in API responses #20732 (normalize at the mapping layer, not by relaxing the Pydantic type)

Changes

• litellm/litellm_core_utils/core_helpers.py: Add validation guard at the end of map_finish_reason() to return "finish_reason_unspecified" for any value not in the valid set

Testing

Added manual verification:

# Known values still pass through correctly
assert map_finish_reason("stop") == "stop"
assert map_finish_reason("tool_use") == "tool_calls"   # anthropic
assert map_finish_reason("MAX_TOKENS") == "length"      # cohere/vertex

# Unknown/provider-specific error values now map safely
assert map_finish_reason("network_error") == "finish_reason_unspecified"  # ZhipuAI/GLM
assert map_finish_reason("some_unknown") == "finish_reason_unspecified"

# Choices() construction no longer raises ValidationError
choice = Choices(finish_reason="network_error", index=0, message=Message(role="assistant", content=""))
assert choice.finish_reason == "finish_reason_unspecified"

Note: Please add at least 1 test in tests/litellm/ per the contribution guide — happy to add a unit test if you point me to the right test file for map_finish_reason.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Mar 4, 2026 5:16pm

[CLAassistant]

All committers have signed the CLA.

[greptile-apps]

Greptile Summary

This PR fixes a ValidationError crash in stream_chunk_builder() caused by non-standard finish_reason values from providers like ZhipuAI/GLM (e.g., "network_error"). The fix adds a fallback guard at the end of map_finish_reason() that maps any unrecognized value to "finish_reason_unspecified" with a warning log, preventing Pydantic validation failures in Choices.__init__().

• litellm/litellm_core_utils/core_helpers.py: Adds a module-level _VALID_OPENAI_FINISH_REASONS frozenset derived from get_args(OpenAIChatCompletionFinishReason) and a validation guard that logs and remaps unknown finish reasons
• tests/test_litellm/litellm_core_utils/test_core_helpers.py: Adds thorough unit tests covering all known pass-through values, provider-specific mappings, unknown values, empty strings, and the ZhipuAI regression case
• The approach is consistent with existing patterns (e.g., how FINISH_REASON_UNSPECIFIED from Vertex AI is already handled) and normalizes at the mapping layer rather than relaxing the Pydantic type

Confidence Score: 5/5

• This PR is safe to merge — it adds a defensive fallback that prevents crashes without changing any existing behavior for known finish reasons.
• The change is minimal, well-scoped, and backwards-compatible. It adds a fallback guard that only activates for previously-crashing unknown values. All known provider mappings continue to work as before. The valid set is derived from the source-of-truth Literal type using get_args(), ensuring it stays in sync. Comprehensive unit tests cover all paths.
• No files require special attention.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/core_helpers.py	Adds a fallback guard in map_finish_reason() that maps any unrecognized finish_reason value to "finish_reason_unspecified" with a warning log. Uses get_args() on the source-of-truth Literal type to derive the valid set at module level.
tests/test_litellm/litellm_core_utils/test_core_helpers.py	Adds comprehensive unit tests for map_finish_reason() covering all known pass-through values, provider-specific mappings (Anthropic, Cohere, Vertex AI), unknown values, empty strings, and a ZhipuAI regression case. All tests are pure function calls with no network access.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["map_finish_reason(finish_reason)"] --> B{Matches known\nprovider mapping?}
    B -->|Yes| C["Return mapped value\n(e.g. 'stop', 'length',\n'tool_calls', etc.)"]
    B -->|No| D{In valid\nOpenAI set?}
    D -->|Yes| E["Return finish_reason as-is"]
    D -->|No| F["Log warning\nReturn 'finish_reason_unspecified'"]
    F --> G["Choices.__init__() succeeds\n(no ValidationError)"]
    C --> G
    E --> G

Loading

_{Last reviewed commit: 7ea1202}

[greptile-apps]

_{1 file reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[xykong]

Thanks for the review @greptile-apps! Addressed in the follow-up commit:

• Replaced the hardcoded _valid_finish_reasons set with a module-level frozenset derived from get_args(OpenAIChatCompletionFinishReason) — stays in sync automatically with any future Literal updates
• Moved to module scope so the set is only computed once rather than on every streaming chunk call

Also updated the provider description in the comment from 'ZhipuAI/GLM' → 'ZhipuAI/GLM-5' to be more precise.

[greptile-apps]

Perfect! The changes look great:

✓ Using get_args(OpenAIChatCompletionFinishReason) as the single source of truth
✓ Module-level frozenset computed once instead of per-call
✓ Clear documentation explaining the performance benefit for streaming
✓ Updated provider detail to ZhipuAI/GLM-5

The implementation is clean and maintains sync with the Literal type automatically. Nice work!

[greptile-apps]

Consider logging unknown finish reasons

Silently mapping unknown values to "finish_reason_unspecified" is the right approach to prevent the ValidationError, but it may hide useful debugging information. Consider adding a verbose_logger.warning() (already imported) when an unknown finish reason is encountered so operators can track which providers are returning non-standard values:

Suggested change

	# Unknown finish_reason values (e.g. provider-specific error codes like
	# "network_error" from ZhipuAI/GLM-5) are not in OpenAIChatCompletionFinishReason
	# Literal and will cause a Pydantic ValidationError in Choices.__init__.
	# Map them to "finish_reason_unspecified" so the stream can be assembled
	# without raising an exception.
	if finish_reason not in _VALID_OPENAI_FINISH_REASONS:
	return "finish_reason_unspecified"
	# Unknown finish_reason values (e.g. provider-specific error codes like
	# "network_error" from ZhipuAI/GLM-5) are not in OpenAIChatCompletionFinishReason
	# Literal and will cause a Pydantic ValidationError in Choices.__init__.
	# Map them to "finish_reason_unspecified" so the stream can be assembled
	# without raising an exception.
	if finish_reason not in _VALID_OPENAI_FINISH_REASONS:
	verbose_logger.warning("Unknown finish_reason '%s' mapped to 'finish_reason_unspecified'", finish_reason)
	return "finish_reason_unspecified"

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

[Wonki4]

I'm waiting for this feature.

[xykong]

Thanks for the review, @krrishdholakia!

To clarify — the guard is inside map_finish_reason() itself (lines 110–111 of core_helpers.py), not a second copy of it. The full diff just adds a catch-all fallback at the tail of the existing function:

def map_finish_reason(finish_reason: str):
    # ... all existing provider-specific if/elif branches unchanged ...
    elif finish_reason == "compaction":
        return "length"

    # new: catch-all for anything not already mapped above
    if finish_reason not in _VALID_OPENAI_FINISH_REASONS:
        verbose_logger.warning(
            "Unknown finish_reason '%s' mapped to 'finish_reason_unspecified'", finish_reason
        )
        return "finish_reason_unspecified"
    return finish_reason

The _VALID_OPENAI_FINISH_REASONS frozenset is derived dynamically via get_args(OpenAIChatCompletionFinishReason), so it stays in sync with the Literal type automatically — no duplicate maintenance needed.

If you had a different spot in mind where you think the logic is duplicated, happy to take a look and consolidate. Just point me at the line!