Skip to content

test: add missing __init__.py files so duplicate tests basenames collect - #35609

Open
thejamesgore wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
thejamesgore:bugfix/test-collection-collisions
Open

test: add missing __init__.py files so duplicate tests basenames collect#35609
thejamesgore wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
thejamesgore:bugfix/test-collection-collisions

Conversation

@thejamesgore

@thejamesgore thejamesgore commented Aug 2, 2026

Copy link
Copy Markdown

TLDR

Problem this solves:

  • The make test-unit aborts at collection on a fresh clone
  • Three pairs of same named test files collide so multiple tests never run

How it solves it:

  • Adds __init__.py to the nine directories with colliding tests
  • This uses the same package convention already used elsewhere in the tree

Relevant issues

Fixes #35605

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

The bug is pytest collection itself so the collection output is the end to end behaviour and full repro is in #35605

On base 491eda3 make test-unit aborts at collection before running a single test

$ make test-unit
...
ERROR tests/test_litellm/proxy/client/test_models.py - import file mismatch:
imported module 'test_models' has this __file__ attribute:
  .../tests/test_litellm/models/test_models.py
which is not the same as the test file we want to collect:
  .../tests/test_litellm/proxy/client/test_models.py
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
================== 296 warnings, 1 error in 549.29s (0:09:09) ==================
make: *** [test-unit] Error 2

With 6398fbf collection succeeds and the suite actually runs. I have 4 test failures that also occur without this change due to a missing optional dependency and local CLI state issue #30175 fixes so the full suite passing belongs to CI

$ make test-unit
...
====== 4 failed, 770 passed, 2 skipped, 334 warnings in 571.92s (0:09:31) ======

Measured with collection only the three errors are gone and the 58 tests they shadowed now collect here

$ uv run pytest tests/test_litellm --collect-only -q     # with 491eda31
32856 tests collected, 3 errors in 202.50s

$ uv run pytest tests/test_litellm --collect-only -q     # with 6398fbf1
32914 tests collected in 171.30s

The six previously colliding files also pass together in one session with 6398fbf

$ uv run pytest tests/test_litellm/models/test_models.py tests/test_litellm/proxy/client/test_models.py \
    tests/test_litellm/realtime_api/test_main.py tests/test_litellm/responses/test_streaming_iterator.py \
    tests/test_litellm/proxy/hooks/litellm_skills/test_main.py \
    tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_streaming_iterator.py -q
118 passed, 72 warnings in 45.74s

Type

✅ Test

Changes

I added empty __init__.py files to the nine directories along the package chains of the three colliding pairs, no test code changes were necessary, and this allows the remaining tests to actually run.

I did consider switching pytest to --import-mode=importlib but that would change import semantics for test cases and that felt like a maintainer scale decision. Also considered renaming the files but this would go against the repos convention of test names mirroring the source tree. I also did find that there are multiple other instances of files with duplicated base names which could cause a problem in the future and can add those to this PR too or in a future one if it makes sense to.

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@greptile-apps

greptile-apps Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds empty package markers to affected test-directory chains so pytest assigns fully qualified module names to duplicate-basename tests.

  • Packages nine test directories involved in the collection collisions.
  • Leaves test behavior and production code unchanged.
  • Includes collection and targeted-suite evidence showing the previously shadowed tests now collect and execute.

Confidence Score: 5/5

The PR appears safe to merge and confines its effect to pytest module identity within the test tree.

The empty package markers complete valid test-package chains, follow the repository’s existing convention, and resolve duplicate-basename collection without changing test assertions or production imports.

Important Files Changed

Filename Overview
tests/test_litellm/llms/init.py Adds the package marker needed for qualified collection of tests beneath the LLM test hierarchy.
tests/test_litellm/llms/anthropic/experimental_pass_through/init.py Completes the package chain for Anthropic experimental pass-through tests.
tests/test_litellm/llms/anthropic/experimental_pass_through/messages/init.py Gives the messages test directory a qualified module namespace for collection.
tests/test_litellm/models/init.py Prevents the models test basename from being imported as an unqualified top-level test module.
tests/test_litellm/proxy/client/init.py Prevents the proxy-client test basename from colliding with the models test module.
tests/test_litellm/proxy/hooks/init.py Completes the parent package chain for hook tests.
tests/test_litellm/proxy/hooks/litellm_skills/init.py Gives LiteLLM skills tests a qualified module namespace.
tests/test_litellm/realtime_api/init.py Packages realtime API tests so their test_main module does not collide during collection.
tests/test_litellm/responses/init.py Packages response tests so duplicate streaming-iterator basenames remain distinct.

Reviews (1): Last reviewed commit: "test: add missing __init__.py files so d..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: make test-unit fails at collection on a fresh clone due to duplicate test basenames

1 participant