Skip to content

fix(prompt_templates): do not mutate messages in-place in function_call_prompt - #41676

Open
mgeorgaklis wants to merge 5 commits into
BerriAI:mainfrom
mgeorgaklis:fix/function-call-prompt-mutation
Open

mgeorgaklis wants to merge 5 commits into
BerriAI:mainfrom
mgeorgaklis:fix/function-call-prompt-mutation

Conversation

@mgeorgaklis

@mgeorgaklis mgeorgaklis commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

TLDR: Shallow copies in message validation cause token inflation through duplicate tool definitions across multi-turn conversations

function_call_prompt(messages, functions) in litellm/litellm_core_utils/prompt_templates/factory.py previously mutated the caller's messages list, system message dictionary, or nested content list in-place

Because validate_and_fix_openai_messages(messages) performs a shallow copy of message dictionaries ({**m}), nested content lists remain shared with the caller's conversation history. In multi-turn tool-calling loops that use fallback tool prompting (such as Ollama tool calling or when add_function_to_prompt=True), each turn appended another duplicate copy of all tool definitions to the caller's retained system message, causing Turn N to send N copies of the tool schema prompt

Changes

This change updates function_call_prompt to construct and return fresh message dictionaries and content lists without mutating the caller's input objects in-place. It also adds parametrized multi-turn unit tests (test_function_call_prompt_does_not_inflate_multi_turn_history) built with immutable functional reductions and TypedDict bindings to verify that caller history is preserved and tool schemas are injected once per turn

@mgeorgaklis
mgeorgaklis requested a review from a team September 17, 2026 21:19
@greptile-apps

greptile-apps Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with the prior mutation-rule finding fully fixed and no new actionable defects identified

Summary

This PR prevents fallback function prompting from mutating retained conversation history and repeatedly inflating tool definitions

  • Builds fresh system-message dictionaries and content lists
  • Preserves the caller’s original messages and function definitions
  • Adds multi-turn regression coverage for validated and unvalidated messages
  • Refactors the test to avoid local list mutation

Reviews (2) · Last reviewed commit: "style(prompt_templates): eliminate based..."

@codspeed

codspeed Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing mgeorgaklis:fix/function-call-prompt-mutation (64b9daa) with main (356b8d4)

Open in CodSpeed

@codecov

codecov Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant