Skip to content

Conversation

@TheAli711
Copy link
Contributor

@TheAli711 TheAli711 commented Oct 4, 2025

Summary

This PR fixes an issue where certain models (like gpt-5) should not receive a temperature parameter.
Previously, temperature=0.0 was always passed in extra_kwargs, which caused problems for models that do not accept this argument.

Changes

  • Introduced excluded_models_temperature list (currently includes gpt-5).
  • Updated extra_kwargs logic to skip adding temperature if the model contains any excluded model name as substring.

Before

async for chunk in llm_v.chat(
    chat_ctx=chat_ctx,
    tools=[check_intent],
    tool_choice={"type": "function", "function": {"name": "check_intent"}},
    extra_kwargs={"temperature": 0.0},
):

After

extra_kwargs = {}
excluded_models_temperature = ["gpt-5"]  # Models excluded from temperature
if not any(excluded_model in llm_v.model for excluded_model in excluded_models_temperature):
    extra_kwargs["temperature"] = 0.0

async for chunk in llm_v.chat(
    chat_ctx=chat_ctx,
    tools=[check_intent],
    tool_choice={"type": "function", "function": {"name": "check_intent"}},
    extra_kwargs=extra_kwargs,
):

Test Cases

from livekit.agents import AgentSession
from livekit.plugins import openai
import pytest
from my_agent import Assistant


@pytest.mark.asyncio
async def test_assistant_greeting() -> None:
    async with (
        openai.LLM(model="gpt-5-nano") as llm,
        AgentSession(llm=llm) as session,
    ):
        await session.start(Assistant())

        result = await session.run(user_input="Hello")

        await result.expect.next_event().is_message(role="assistant").judge(
            llm, intent="Makes a friendly introduction and offers assistance."
        )
        
        result.expect.no_more_events()

Before Fix:

image

After Fix:

image

Issue Reference

Fixes: #3556

Notes

More models can be added to excluded_models_temperature in the future if needed.

This change ensures compatibility with gpt-5 while maintaining the original behavior for other models.

- Added a safeguard to avoid passing `temperature` for models that do not support it
- Introduced `excluded_models_temperature` list (currently includes `gpt-5`)
- Maintains existing behavior for other models
- Resolves issue livekit#3556

Reference: livekit#3556
@CLAassistant
Copy link

CLAassistant commented Oct 4, 2025

CLA assistant check
All committers have signed the CLA.

@Is44m
Copy link
Contributor

Is44m commented Oct 4, 2025

bumping up! 💯

@davidzhao davidzhao requested a review from theomonnom October 4, 2025 23:01
@theomonnom theomonnom merged commit 1169285 into livekit:main Oct 13, 2025
9 checks passed
akshaym1shra pushed a commit to akshaym1shra/agents that referenced this pull request Oct 28, 2025
akshaym1shra pushed a commit to akshaym1shra/agents that referenced this pull request Oct 28, 2025
meetakshay99 added a commit to meetakshay99/agents that referenced this pull request Oct 30, 2025
* staging: (114 commits)
  Added min_confidence_threshold for deepgram flux.
  livekit-agents 1.2.15 (livekit#3658)
  fix livekit#3650 cartesia version backward compatibility (livekit#3651)
  Unprompted STT Reconnection at startup (livekit#3649)
  enable zero retention mode in elevenlabs (livekit#3647)
  fix: heartbeat (livekit#3648)
  feat: Integrate streaming endpoints for Sarvam APIs (livekit#3498)
  turn_detection: reduce max_endpointing_delay to 3s (livekit#3640)
  fix: exclude temperature parameter for gpt-5 and similar models (livekit#3573)
  add backwards compatibility for google's realtime model (livekit#3630)
  Align Google STT plugin with official documentation (livekit#3628)
  feat(speechmatics): add max_speakers parameter for speaker diarization (livekit#3524)
  fix(deepgram): send CloseStream message before closing TTS WebSocket (livekit#3608)
  chore: Remove duplicate docstring for `preemptive_generation` parameter in AgentSession (livekit#3624)
  Add RTZR(ReturnZero) STT Plugin for LiveKit Agents (livekit#3376)
  feat(telemetry/utils): add ttft reporting to LangFuse (livekit#3594)
  catch delete_room errors and disable delete_room_on_close by default (livekit#3600)
  lift google realtime api out of beta (livekit#3614)
  fix: lock pyav to <16 due to build issue (livekit#3593)
  Updating Cartesia Version (livekit#3570)
  ...
akshaym1shra pushed a commit to akshaym1shra/agents that referenced this pull request Nov 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI GPT 5 family models cannot be used as judges

4 participants