ci: use --cov=./litellm so coverage paths resolve unambiguously in Codecov - #27960
Conversation
pytest-cov runs with --cov=litellm, which makes coverage.xml store paths relative to the package root (e.g. `proxy/proxy_server.py` instead of `litellm/proxy/proxy_server.py`). Codecov auto-resolves these only when the basename is unique in the repo. Files like proxy_server.py, router.py, utils.py, main.py, and constants.py — which have duplicates under enterprise/ or other subpackages — get silently dropped during ingest. The `fixes: ["::litellm/"]` rule prepends `litellm/` to every uploaded path so they resolve unambiguously. Confirmed against multiple recent coverage.xml artifacts that no uploader currently emits paths already prefixed with `litellm/`, so the rule is safe to apply universally. This restores Codecov visibility for the highest-fix-rate hotspots: proxy_server.py, router.py, proxy/utils.py, litellm_logging.py, constants.py, key_management_endpoints.py, utils.py, main.py, user_api_key_auth.py, team_endpoints.py, and litellm_pre_call_utils.py.
This reverts commit e25a988. The `fixes: ["::litellm/"]` rule turned out to be applied *after* Codecov's auto-resolution, not before. Files with unique basenames (which were auto-resolving correctly to `litellm/<path>`) got an extra `litellm/` prepended, producing `litellm/litellm/<path>` storage. Files with ambiguous basenames (the actual target of the fix) continued to be dropped because the auto-resolution still failed for them. Net result on the verification run: 1375 files now stored under unresolvable `litellm/litellm/...` paths, and the 11 originally-missing hotspots are still missing. Reverting before piling on further changes.
…decov pytest-cov treats --cov=<module-name> as a Python package and emits XML paths relative to the package root, stripping the litellm/ prefix (`proxy/proxy_server.py` instead of `litellm/proxy/proxy_server.py`). Codecov's auto-prefix heuristic then drops every file whose basename is ambiguous in the repo — `proxy_server.py` (3 copies under enterprise/), `router.py` (2 copies), `utils.py` (20+), `main.py` (20+), `constants.py` (2). The 11 highest-fix-rate hotspots have never appeared in Codecov. Switching to --cov=./litellm treats the argument as a path, which makes coverage.xml emit repo-relative paths (`litellm/proxy/proxy_server.py`). Each path is unambiguous, so Codecov resolves all files correctly. Verified locally: rerunning a single proxy_unit_tests test with --cov=./litellm produced `filename="litellm/proxy/proxy_server.py"`, `filename="litellm/router.py"`, and `filename="litellm/types/router.py"` as distinct entries — exactly the disambiguation Codecov needs. Touches every workflow that uploads coverage: the two reusable GHA workflows (_test-unit-base.yml, _test-unit-services-base.yml), test-mcp.yml, and all 14 invocations in .circleci/config.yml.
…itellm_/peaceful-jang-c0e43b
Greptile SummaryThis PR fixes Codecov path resolution for the
Confidence Score: 5/5Pure CI tooling change with no production code modifications; all 18 occurrences are updated and no other workflow files contain the old form. Every instance of No files require special attention.
|
| Filename | Overview |
|---|---|
| .circleci/config.yml | 14 occurrences of --cov=litellm replaced with --cov=./litellm to emit repo-relative paths in coverage.xml; changes are mechanical and consistent across all jobs. |
| .github/workflows/_test-unit-base.yml | Single --cov=litellm → --cov=./litellm replacement; base workflow used by all non-service unit test jobs. |
| .github/workflows/_test-unit-services-base.yml | Two --cov=litellm → --cov=./litellm replacements (one per branch of the if/else that controls parallelism); both branches updated correctly. |
| .github/workflows/test-mcp.yml | Single --cov=litellm → --cov=./litellm replacement in the MCP test run step. |
Reviews (1): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
fd92769
into
litellm_internal_staging
…c0e43b ci: use --cov=./litellm so coverage paths resolve unambiguously in Codecov
Summary
Switch
--cov=litellm→--cov=./litellmin every CI invocation.--cov=<module-name>makes pytest-cov treat the argument as a Python package and emit XML paths relative to the package root (proxy/proxy_server.py), which strips thelitellm/prefix. Codecov then drops any path whose basename is ambiguous in the repo.--cov=<./path>treats the argument as a directory and emits repo-relative paths (litellm/proxy/proxy_server.py), which Codecov resolves unambiguously.Why it matters
11 of the top-18 highest-fix-rate files have never appeared in Codecov despite being exercised by tests with coverage instrumentation:
litellm/proxy/proxy_server.pylitellm/router.pylitellm/proxy/utils.pylitellm/litellm_core_utils/litellm_logging.pylitellm/constants.pylitellm/proxy/management_endpoints/key_management_endpoints.pylitellm/utils.pylitellm/main.pylitellm/proxy/auth/user_api_key_auth.pylitellm/proxy/management_endpoints/team_endpoints.pylitellm/proxy/litellm_pre_call_utils.pyAll have ambiguous basenames in the repo (e.g.
proxy_server.pyexists 3 times underenterprise/andlitellm/proxy/;router.pyexists twice underlitellm/andlitellm/types/;utils.py,main.pyeach have 20+ copies across subpackages).Files like
_types.py(unique basename) were already resolving correctly, so the gap was invisible from looking at any single file.How the fix was found
coverage.xmlfrom atest-unit-proxy-dbartifact and confirmed it containedfilename="proxy/proxy_server.py"andfilename="router.py"— paths stripped of thelitellm/prefix.codecov.yaml: fixes: ["::litellm/"]to prepend the prefix server-side. It was applied after Codecov's auto-resolution, double-prefixing every previously-working file tolitellm/litellm/...while still dropping the ambiguous ones. Reverted.--cov=./litellm(path form, leading./) makes coverage.py treat the argument as a directory and emit repo-relative paths.Test plan
tests/proxy_unit_tests/test_proxy_server.pywith--cov=./litellm; verifiedcoverage.xmlemitsfilename="litellm/proxy/proxy_server.py",litellm/router.py, andlitellm/types/router.pyas distinct entries.Unit Tests: Proxy DB Operations(itspush: [main, "litellm_**"]trigger matches). Verifiedproxy_server.pyandrouter.pyappear under their properlitellm/...paths in Codecov for this branch's commit.mainmerge commit showsproxy_server.py+ siblings in Codecov.Scope
.github/workflows/_test-unit-base.yml(1).github/workflows/_test-unit-services-base.yml(2).github/workflows/test-mcp.yml(1).circleci/config.yml(14)