Skip to content

test(bedrock): drop the leftover set_verbose from the embedding tests - #37844

Merged
yuneng-berri merged 2 commits into
litellm_internal_stagingfrom
litellm_bedrock_embedding_drop_set_verbose
Aug 22, 2026
Merged

test(bedrock): drop the leftover set_verbose from the embedding tests#37844
yuneng-berri merged 2 commits into
litellm_internal_stagingfrom
litellm_bedrock_embedding_drop_set_verbose

Conversation

@yuneng-berri

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Fifteen embedding tests turn on litellm.set_verbose and leave it on
  • Everything running after them in that process logs verbosely
  • Nothing in the file asserts on the output the flag produces

How it solves it:

  • The fifteen lines go, since they were left over from debugging
  • Restoring the flag instead would keep noise nobody reads
  • 15 TQ005 violations in the file drop to zero

User Flow

Before: a maintainer running the unit tier gets a wall of provider debug logging from tests that never asked for it

  1. They run the tier in one process, with the bedrock embedding tests in it
  2. The first embedding test turns litellm.set_verbose on and never turns it back off
  3. Every test after it in that process logs verbosely, which buries the failure they were reading the output for

After: the same run stays quiet, and a real failure is the loudest thing on screen

  1. They run the same tier in one process
  2. No embedding test touches litellm.set_verbose
  3. The flag reads the same after the run as before it, so later tests log the way they were written to

Relevant issues

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

litellm.set_verbose is not in the save/restore conftest snapshot, so nothing else puts it back. The probe sets it False, runs the file in the same interpreter, and reads it back.

uv run python -c '
import litellm, pytest
litellm.set_verbose = False
pytest.main(["tests/test_litellm/llms/bedrock/embed/test_bedrock_embedding.py", "-q", "-p", "no:randomly"])
print("litellm.set_verbose after the run:", litellm.set_verbose)
'
uv run python scripts/check_test_quality.py tests/test_litellm/llms/bedrock/embed/test_bedrock_embedding.py | grep -oE "TQ00[0-9]" | sort | uniq -c

Before (ff02d5c)

  1. The first command
29 passed, 2 warnings in 0.32s
litellm.set_verbose after the run: True
  1. The second command
   1 TQ003
  15 TQ005

After (324e1d8)

  1. The first command
29 passed, 2 warnings in 0.31s
litellm.set_verbose after the run: False
  1. The second command
   1 TQ003
  1. uv run python scripts/test_quality_gate.py --base origin/litellm_internal_staging
OK: every TQ rule is within its test-suite ceiling (base origin/litellm_internal_staging)

Type

🧹 Refactoring
✅ Test

Caveats (if any)

  • The file's one TQ003 is a sys.path.insert, left for its own change

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Fifteen tests opened with litellm.set_verbose = True and never put it back, so
the flag stayed on for everything that ran after them in the same process.
Nothing in the file reads the output it produces: there is no caplog, no capsys
and no assertion on a log line, so the flag was left over from debugging.
Deleting it beats restoring it, since restoring keeps the noise.
@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Removes fifteen leftover verbose-logging mutations from Bedrock embedding tests so they no longer leak global logging state into later tests.

  • Deletes fifteen litellm.set_verbose = True assignments without changing assertions, mocks, or tested behavior.
  • Ratchets the TQ005 quality budget from 2832 to 2817 to reflect the fifteen removed violations.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness, security, or test-coverage issues identified.

The removed flag assignments only controlled diagnostic output, and the quality-budget reduction exactly matches the fifteen eliminated TQ005 violations.

Important Files Changed

Filename Overview
tests/test_litellm/llms/bedrock/embed/test_bedrock_embedding.py Removes global verbose-logging mutations that only enabled diagnostic output and leaked state across tests.
test-quality-budget.json Correctly lowers the TQ005 ceiling by fifteen to match the removed global-mutation violations.

Reviews (1): Last reviewed commit: "test(bedrock): drop the leftover set_ver..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…itellm_bedrock_embedding_drop_set_verbose

# Conflicts:
#	test-quality-budget.json
@yuneng-berri
yuneng-berri merged commit b9bff09 into litellm_internal_staging Aug 22, 2026
68 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_bedrock_embedding_drop_set_verbose branch August 22, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants