Re-add Codecov coverage reporting to GHA matrix workflow

[joereyna]

Summary

Coverage upload was lost on 2026-03-28 when CircleCI jobs were removed (7aec9101f). The codecov.yaml config was preserved but upload steps were never carried over to GitHub Actions.

• Add pytest-cov to dev dependencies and regenerate poetry.lock
• Collect coverage per matrix job with --cov=litellm --cov-report=xml
• Save each report as a GitHub Actions artifact — no secrets in scope during test execution
• Add isolated upload-coverage job that runs after all test jobs, checks out repo (so codecov.yaml is present), downloads artifacts, and uploads to Codecov via OIDC (use_oidc: true) with root_dir set so file paths resolve correctly
• No static CODECOV_TOKEN required — GitHub mints a short-lived OIDC token scoped to the job at runtime
• Add Enterprise component to existing codecov.yaml for separate enterprise coverage tracking
• Pin codecov-action to immutable SHA (v5.5.4)

Requires Codecov org "Global Upload Token" set to "Not Required" (already configured).

Test plan

• Coverage uploads successfully — 24.31% on litellm/ confirmed on Codecov dashboard
• File paths resolve correctly (litellm/... prefix)
• No CODECOV_TOKEN secret required
• Token never in scope during test execution

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 31, 2026 11:50pm

[greptile-apps]

Greptile Summary

This PR re-introduces Codecov coverage reporting to the GitHub Actions matrix workflow after it was lost when CircleCI jobs were removed. Coverage is collected per matrix job using pytest-cov, saved as individual artifacts, then merged and uploaded in an isolated upload-coverage job using OIDC tokenless authentication.

• pytest-cov = \"^5.0\" added to [tool.poetry.group.dev.dependencies] so it is installed by the existing poetry install --with dev step
• --cov=litellm --cov-report=xml:<name>.xml --cov-config=pyproject.toml appended to the pytest invocation in every matrix job; filename is parametrised correctly so artifacts never collide
• [tool.coverage.run] block added to pyproject.toml with relative_files = true, ensuring stored paths are portable across runners
• Isolated upload-coverage job gates on needs: test + if: always(), checks out the repo (so codecov.yaml is present), downloads all coverage-* artifacts into one directory, then uploads via codecov/codecov-action pinned to immutable SHA aa56896cf… (v5.5.4) using use_oidc: true — no static secret required
• Enterprise component added to codecov.yaml tracking enterprise/**; pre-existing coverage thresholds unchanged
• All previously raised concerns (mutable action tag, coverage filename mismatch) have been addressed

Confidence Score: 5/5

This PR is safe to merge — it adds only CI/coverage infrastructure with no changes to production code paths.

All previously flagged issues (mutable tag, coverage filename mismatch) have been resolved. The OIDC upload approach is correctly scoped, action SHAs are pinned, relative_files=true + root_dir ensure path portability, and the isolated upload job keeps any write-capable tokens away from test execution. No P0 or P1 findings remain.

No files require special attention.

Important Files Changed

Filename	Overview
.github/workflows/test-litellm-matrix.yml	Adds coverage collection flags to every matrix pytest invocation, saves per-job XML artifacts, and adds an isolated upload-coverage job using OIDC; all action SHAs pinned, filename parametrisation correct, and permissions scoped appropriately.
codecov.yaml	Adds Enterprise component tracking enterprise/** paths; existing coverage thresholds and layout unchanged.
pyproject.toml	Adds pytest-cov ^5.0 to dev dependencies and [tool.coverage.run] config block with source=["litellm"] and relative_files=true for portable path resolution.

Sequence Diagram

sequenceDiagram
    participant M as matrix: test (N jobs)
    participant AR as actions/upload-artifact
    participant U as upload-coverage job
    participant DA as actions/download-artifact
    participant CC as Codecov (OIDC)

    M->>M: pytest --cov=litellm --cov-report=xml:coverage-{name}.xml
    M->>AR: upload artifact coverage-{name} (retention 1 day)
    Note over M,AR: if: always() — captured even on partial failure

    M-->>U: needs: test (all matrix jobs)
    U->>U: actions/checkout (fetches codecov.yaml)
    U->>DA: download-artifact pattern=coverage-* → coverage-reports/
    DA-->>U: merged XML files
    U->>CC: codecov-action v5.5.4 (OIDC, directory=coverage-reports, root_dir=workspace)
    CC-->>U: upload ACK (fail_ci_if_error=false)

Loading

_{Reviews (19): Last reviewed commit: "Use unique filenames per matrix job to p..." | Re-trigger Greptile}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing joereyna:feat/add-codecov-to-ci (c903845) with main (94e0f44)}

[greptile-apps]

Coverage filename mismatch — artifacts will always be empty

--cov-report=xml writes to coverage.xml by default. The upload-artifact step then looks for coverage-${{ matrix.test-group.name }}.xml, which will never exist, so every artifact is uploaded empty and Codecov receives nothing.

The fix is to pass the target filename directly to the --cov-report option:

Suggested change

	--durations=20 \
	--cov=. \
	--cov-report=xml \
	--cov-config=pyproject.toml
	- name: Save coverage report
	if: always()
	uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
	with:
	name: coverage-${{ matrix.test-group.name }}
	path: coverage-${{ matrix.test-group.name }}.xml
	--durations=20 \
	--cov=. \
	--cov-report=xml:coverage-${{ matrix.test-group.name }}.xml \
	--cov-config=pyproject.toml

The "Debug coverage paths" step below also hard-codes coverage-root.xml as the file to inspect, which is only correct once the output filename is explicitly set this way.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!