ci(codspeed): re-enable benchmarks on litellm_internal_staging

[yassin-berriai]

Relevant issues

Linear ticket

Resolves LIT-4118

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests (not applicable; two-line workflow trigger change)
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

#31746 disabled CodSpeed on litellm_internal_staging because unpinned ubuntu-latest runners put BASE and HEAD on different machines, so CodSpeed reported "Different runtime environments detected" and sub-millisecond runner noise on 3-4 ms benchmarks flip-flopped results by 25-30% (for example the -25.2% false regression on #31684, which had no LLM code changes). The same commit that disabled the staging triggers also pinned the runner to ubuntu-24.04, which removes that noise source: CodSpeed runs in simulation mode (instruction counting), which is deterministic on a fixed runner image, so the flapping cannot recur once BASE and HEAD are both measured on the pinned image

The first push run after this merges re-baselines litellm_internal_staging on the pinned runner, so comparisons are clean from the first post-merge PR onward

Update: #32339 has merged and this branch is rebased on top of it, so the benchmarks job on this PR now runs against fixed code. Original note kept for context: #32339 fixes the fastapi import leak from #31576 that has failed every CodSpeed run since Jul 2 (ModuleNotFoundError: No module named 'fastapi' in test_completion_with_tools); re-enabling before it lands would bring the staging red X's straight back. With that fix applied, the exact command the workflow runs passes locally on this branch:

$ env PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 uv run --frozen --no-default-groups \
    --with pytest==8.3.5 --with pytest-codspeed==4.3.0 \
    --with "mcp>=1.26.0,<2.0" --with "a2a-sdk>=1.1.0,<2.0" \
    pytest -p pytest_codspeed.plugin tests/benchmarks/ --codspeed -q
30 passed, 2 warnings in 120.32s (0:02:00)

The job timeout is raised from 15 to 60 minutes because the staging suite runs much longer under CodSpeed's callgrind instrumentation than main's (~2 minute) runs; at 15 minutes the job was cancelled mid-measurement on every staging merge ref. The first completed run will show per-benchmark instruction counts on the CodSpeed dashboard, which pinpoints what grew on staging relative to main; the timeout can be tightened again once that is understood

Type

🚄 Infrastructure

Changes

Restores the litellm_internal_staging push and pull_request triggers in .github/workflows/codspeed.yml that #31746 removed, reverting the temporary disable now that its stated blocker (unpinned-runner noise) is fixed by the ubuntu-24.04 pin from that same PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

Re-enables CodSpeed benchmarks on litellm_internal_staging by adding that branch to the push and pull_request triggers in .github/workflows/codspeed.yml, and raises timeout-minutes from 15 to 60 to accommodate the ~21-minute callgrind-instrumented staging run.

• litellm_internal_staging added to both push and pull_request branch filters; the runner is already pinned to ubuntu-24.04 (from ci(codspeed): pin benchmark runner to ubuntu-24.04 #31746), so BASE and HEAD are measured on the same image, eliminating the runner-skew false regressions that caused the original disable.
• Timeout raised from 15 → 60 minutes; 15 minutes was insufficient for callgrind instrumentation of the full staging benchmark suite and was cancelling every run mid-measurement.

Confidence Score: 5/5

Safe to merge — the change is a two-line branch-filter addition and a one-line timeout bump in a CI workflow file with no application code touched.

Both modifications are mechanical and low-risk: adding a branch name to a trigger list and increasing a job timeout. All GitHub Actions are pinned to immutable SHAs, persist-credentials: false is in place, and the concurrency group with cancel-in-progress: true prevents queue pile-up even at the higher timeout. The PR description gives concrete timing evidence (21-minute callgrind runs vs. 15-minute cap) that fully justifies the timeout change.

No files require special attention.

Important Files Changed

Filename	Overview
.github/workflows/codspeed.yml	Adds litellm_internal_staging to push/pull_request branch triggers and raises timeout-minutes from 15 to 60; no logic changes, all actions remain pinned to verified SHAs.

_{Reviews (2): Last reviewed commit: "ci(codspeed): raise benchmark job timeou..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 30 untouched benchmarks

---

_{Comparing litellm_reenable_codspeed_internal_staging (34a3f42) with litellm_internal_staging (a0b26d2)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (ce2582e) during the generation of this report, so 0930ed7 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[yassin-berriai]

@greptileai please review the current head 34a3f42 (rebase onto staging after #32339 merged, plus a one-line timeout-minutes raise from 15 to 60; the staging suite takes ~21 minutes under callgrind instrumentation so the old 15 minute cap cancelled every run)