Skip to content

test: drop the cwd-relative sys.path.insert calls from the test suite - #37802

Merged
yuneng-berri merged 6 commits into
litellm_internal_stagingfrom
litellm_drop_cwd_relative_syspath
Aug 22, 2026
Merged

test: drop the cwd-relative sys.path.insert calls from the test suite#37802
yuneng-berri merged 6 commits into
litellm_internal_stagingfrom
litellm_drop_cwd_relative_syspath

Conversation

@yuneng-berri

@yuneng-berri yuneng-berri commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • 1,015 test files insert a path that never points at litellm
  • The argument resolves against the working directory, not the file
  • From the repo root it names the directory above the checkout
  • TQ003 has sat at its 1,077 seed since the rule shipped

How it solves it:

  • Delete those calls, and the imports they were the only user of
  • Ratchet the TQ003 ceiling from 1,077 to 62

User Flow

No end-user behavior changes and no test behavior changes. A contributor copying
an existing test file as a template stops inheriting a line that does nothing.

Relevant issues

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Before (ff02d5c)

What the line actually inserts

  1. Every job runs pytest from the repo root, so:
python -c "import os; print(os.path.abspath('../..'))"
/Users/yunengjiang/development
  1. That is the directory above the checkout, not litellm. The import works
    because the package is installed into the environment

The rule's count and shape

  1. python scripts/check_test_quality.py tests | grep -c ' TQ003 '
1077
  1. Grouped by the argument, 1,015 of the 1,077 resolve against the working
    directory:
  460  os.path.abspath('../..')
  211  os.path.abspath('../../..')
  153  os.path.abspath('../../../../..')
  128  os.path.abspath('../../../..')
   24  os.path.abspath('../../../../../..')
   15  os.path.abspath('../../../')
    ...

Collection baseline

  1. pytest <every touched tree> --collect-only -q
45871 tests collected, 51 errors in 8.91s

After (a704dfb)

The count drops to the calls that resolve against file

  1. python scripts/check_test_quality.py tests | grep -c ' TQ003 '
62
  1. Those 62 use os.path.dirname(__file__), Path(__file__) or a variable, so
    they are a different question and are untouched

Collection is unchanged

  1. Same command on this branch:
45871 tests collected, 51 errors in 23.02s
  1. diff of the two runs' ERROR lines is empty, so the same 51 pre-existing
    collection errors and no new ones

No new undefined names

  1. ruff check --select F821 tests/ before and after
  2. Both report the same single pre-existing finding, so nothing lost an import
    it was still using. One file keeps its import os because another test module
    imports the name back out of it

The gates agree

OK: every TQ rule is within its test-suite ceiling (base origin/litellm_internal_staging)
OK: every strict rule is within its codebase ceiling (base origin/litellm_internal_staging)
OK: every LIT rule is within its codebase ceiling (base origin/litellm_internal_staging)

Type

🧹 Refactoring
✅ Test

Caveats (if any)

  • 998 files, so it will conflict with anything in flight
  • Every removal is one of two shapes, so the diff reads at a glance
  • 62 __file__-relative call sites are deliberately left
  • Greptile skipped it: 999 files, past its 100 file limit

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.

Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.

Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review (999 files, 100 file limit).

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…itellm_drop_cwd_relative_syspath

# Conflicts:
#	test-quality-budget.json
#	tests/audio_tests/test_whisper.py
#	tests/enterprise/litellm_enterprise/integrations/test_prometheus_unit_tests.py
#	tests/llm_translation/test_infinity.py
#	tests/llm_translation/test_litellm_proxy_provider.py
#	tests/local_testing/test_blocked_user_list.py
#	tests/local_testing/test_braintrust.py
#	tests/local_testing/test_embedding.py
#	tests/local_testing/test_rules.py
#	tests/local_testing/test_update_spend.py
#	tests/logging_callback_tests/test_alerting.py
#	tests/logging_callback_tests/test_opentelemetry_unit_tests.py
#	tests/pass_through_unit_tests/test_assemblyai_unit_tests_passthrough.py
#	tests/proxy_admin_ui_tests/test_role_based_access.py
#	tests/proxy_unit_tests/test_key_generate_prisma.py
#	tests/proxy_unit_tests/test_unit_test_max_model_budget_limiter.py
#	tests/proxy_unit_tests/test_user_api_key_auth.py
#	tests/test_litellm/google_genai/test_google_genai_adapter.py
#	tests/test_litellm/google_genai/test_google_genai_main.py
#	tests/test_litellm/integrations/bitbucket/test_bitbucket_prompt_manager.py
#	tests/test_litellm/integrations/gitlab/test_gitlab_prompt_manager.py
#	tests/test_litellm/integrations/test_langfuse.py
#	tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py
#	tests/test_litellm/litellm_core_utils/test_token_counter.py
#	tests/test_litellm/llms/bedrock/test_cross_region_inference_profile_mapping.py
#	tests/test_litellm/proxy/guardrails/guardrail_hooks/test_onyx.py
#	tests/test_litellm/proxy/guardrails/test_prompt_security_guardrails.py
#	tests/test_litellm/proxy/management_endpoints/test_key_management_endpoints.py
#	tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py
#	tests/test_litellm/test_cost_calculator.py
#	tests/test_litellm/test_router_model_cost_isolation.py
@yuneng-berri
yuneng-berri merged commit 6a0d039 into litellm_internal_staging Aug 22, 2026
66 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_drop_cwd_relative_syspath branch August 22, 2026 16:26
yassin-berriai added a commit that referenced this pull request Aug 22, 2026
The two new CLI-session test files were added before #37802 swept the
suite's sys.path.insert(0, os.path.abspath(...)) calls, so rebasing onto
staging put them over the TQ003 test-quality-budget ceiling and failed lint.
shudonglin pushed a commit to rayward-external/litellm that referenced this pull request Aug 23, 2026
- restore MagicMock import dropped from the bedrock adaptive-thinking test
  (upstream BerriAI#37802 merged the import line to its own version)
- bump job-timeout-minutes 55->60 on 3 new test-unit.yml shards upstream
  added with insufficient headroom for their own startup-safety check
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants