test: fix staging CI regressions from #38182, #38144, #38265, #37962, and #37969

[mateo-berri]

TLDR

Problem this solves:

• Staging CI is red with 9 test failures across 4 jobs
• All are stale tests or test-isolation bugs, not product bugs
• Every staging merge now looks broken and needs manual triage

How it solves it:

• Audio redaction test accepts content=None (redaction leaves None untouched)
• local_testing conftest drains the logging worker between tests
• Together test matches fix(together_ai): pass tools through for models missing from the registry #38265: tools always supported, response_format gated
• test_keys creates its team without models: null, so the team really exists
• Race tests use unique per-run ids, so xdist workers stop colliding

User Flow

Before: a maintainer merges an unrelated PR into litellm_internal_staging and the staging CircleCI run comes back red with test failures their change never touched

1. They merge their PR and open the staging pipeline at https://app.circleci.com/pipelines/gh/BerriAI/litellm for the new commit
2. local_testing_part1 is red: test_redacted_message_logging[True-False] fails asserting "redacted-by-litellm" == None on the gpt-audio response, and test_gcs_pub_sub fails with "carried 1 pending logging task" leaked from an earlier test
3. llm_translation_testing is red: test_get_supported_openai_params fails because tools now shows up for a Together model the test expected it stripped from
4. build_and_test is red: four test_key_model_list[/v1/models-key-*] cases fail with 404 "Team doesn't exist in db", because the test's /team/new call sends "models": null, gets a 422 it never checks, then issues a key against the team that was never created
5. auth_ui_unit_tests is red: the two team-race tests fail intermittently when parallel workers, all using the same hardcoded team id, delete each other's team mid-test
6. They spend time bisecting only to find every failure is a stale test, not their change

After: the same merge comes back green because the tests now match intended behavior and stop tripping over each other

1. They merge their PR and open the staging pipeline at https://app.circleci.com/pipelines/gh/BerriAI/litellm for the new commit
2. local_testing_part1 is green: the redaction test accepts the audio model's empty text content, and each test drains its own pending logging work before the next one starts
3. llm_translation_testing is green: the Together test expects tools to be supported and only response_format to be gated
4. build_and_test is green: the test creates its team without the null field, /team/new returns 200, and the key's /v1/models call returns the team's models
5. auth_ui_unit_tests is green: each race test run uses its own unique team and user ids, so parallel workers never collide
6. Only real regressions can turn the staging pipeline red now

Relevant issues

Regressions introduced by #38182, #38144, #38265, #37962, and #37969, first seen in the staging run for 72b8b47 (CircleCI workflow 7e7cff9b, jobs 2137083, 2137086, 2137096, and 2137108)

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

All runs below are against live services: real OpenAI calls for the redaction tests, a live proxy on postgresql://localhost/lit_keys for test_keys, and a shared local Postgres for the xdist race runs

Before (273b01a)

Audio redaction (real gpt-audio-1.5 call)

1. pytest tests/local_testing/test_custom_callback_input.py -k test_redacted_message_logging -q
2. FAILED ...[True-False] - AssertionError: assert None == 'redacted-by-litellm' (non-stream audio response has content=None)

Logging-worker leak into test_gcs_pub_sub

1. pytest <mocked-router-test> tests/local_testing/test_gcs_pub_sub.py -q (any earlier test that queues logging work, then the GCS test)
2. FAILED tests/local_testing/test_gcs_pub_sub.py::test_gcs_pub_sub - Event loop ... carried 1 pending logging task

Together supported params

1. pytest tests/llm_translation/test_together_ai.py -k test_get_supported_openai_params -q
2. FAILED ...[together_ai/Qwen/Qwen3-235B-A22B-fp8-tput-False] - AssertionError: assert 'tools' not in [...] ( fix(together_ai): pass tools through for models missing from the registry #38265 made tools always listed)

test_key_model_list against a live proxy

1. curl -X POST http://0.0.0.0:32963/team/new -H 'Authorization: Bearer sk-1234' -d '{"team_id":"proof-null-team","models":null}' returns HTTP 422 {"detail":[{"type":"list_type","loc":["body","models"],"msg":"Input should be a valid list","input":null}]}, which the test never checked
2. curl -X POST http://0.0.0.0:32963/key/generate ... -d '{"team_id":"proof-null-team","models":["gpt-4"]}' still hands back a key
3. curl http://0.0.0.0:32963/v1/models -H "Authorization: Bearer <that key>" returns HTTP 404 {"error":{"message":"{'error': \"Team doesn't exist in db. Team=proof-null-team...\"}"}} ( fix(auth): stop the team fallback from widening model access #37962 stopped vouching for unknown teams)
4. pytest tests/test_keys.py -k test_key_model_list -q (HEAD copy pointed at the live proxy): 4 failed, 4 passed with all four key-* cases failing on that 404

Team race tests under xdist

1. pytest tests/proxy_admin_ui_tests/test_team_delete_member_add_race.py -n 3 -q three times against one shared Postgres
2. 2 failed, 1 passed every run: workers share the hardcoded lit5544-race-team id and delete each other's team mid-race

After (0e999e3)

Audio redaction (real gpt-audio-1.5 call)

1. pytest tests/local_testing/test_custom_callback_input.py -k test_redacted_message_logging -q
2. 2 passed

Logging-worker leak into test_gcs_pub_sub

1. pytest <mocked-router-test> tests/local_testing/test_gcs_pub_sub.py -q
2. 2 passed; the teardown drain adds 0.29s to the leaking test and nothing measurable elsewhere

Together supported params

1. pytest tests/llm_translation/test_together_ai.py -k test_get_supported_openai_params -q
2. 2 passed: tools asserted present for both models, response_format only for the function-calling one

test_key_model_list against a live proxy

1. curl -X POST http://0.0.0.0:32963/team/new -H 'Authorization: Bearer sk-1234' -d '{"team_id":"proof-omit-team"}' (models omitted, as the test now sends) returns HTTP 200 with the created team
2. curl -X POST http://0.0.0.0:32963/key/generate ... -d '{"team_id":"proof-omit-team","models":["gpt-4"]}' hands back a key
3. curl http://0.0.0.0:32963/v1/models -H "Authorization: Bearer <that key>" returns HTTP 200 {"data":[{"id":"gpt-4",...}]}
4. pytest tests/test_keys.py -k test_key_model_list -q against the same proxy: 8 passed

Team race tests under xdist

1. pytest tests/proxy_admin_ui_tests/test_team_delete_member_add_race.py -n 3 -q three times against the same shared Postgres
2. 3 passed every run

Type

✅ Test

Caveats (if any)

Medium

• Two staging failures are deliberately not fixed here
  • vertex pass-through 401 is a real product regression from fix(passthrough): stop leaking the caller's virtual key on credential-less Vertex passthrough #38114, needs its own fix
  • Manus e2e failure is infra: the provider API key was revoked

Low

• The conftest drain runs for every local_testing test, measured at sub-second
• Race-test ids now vary per run, so leftover rows from aborted runs accumulate until cleaned

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 0e999e3 passes /live-pr-risk

[greptile-apps]

Greptile Summary

The PR updates staging tests and their isolation infrastructure to reflect current audio and team behavior and avoid shared-state collisions.

• Drains pending logging work during local-testing teardown.
• Adjusts audio redaction expectations for non-streaming responses.
• Gives database race tests unique team and user identifiers.
• Omits a null team-model field and verifies successful team creation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
tests/local_testing/conftest.py	Adds teardown-time draining of the global logging queue to prevent work leaking between tests.
tests/local_testing/test_custom_callback_input.py	Updates the audio logging assertion to distinguish streaming redacted content from non-streaming null content.
tests/proxy_admin_ui_tests/test_team_delete_member_add_race.py	Replaces shared database identifiers with per-test UUID-based identifiers and threads them through setup, assertions, and cleanup.
tests/test_keys.py	Omits models when unset during team creation and verifies that the requested team was created before generating its key.

_{Reviews (2): Last reviewed commit: "Merge branch 'litellm_internal_staging' ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai