ci: fix the litellm-tests unit job (sysmon, codecov on failure, env -i allowlist, selection errors, reruns param)

[devin-ai-integration]

Stack 1/5 of the split of #42831 (reference PR, stays open until the stack lands). Above: #42901. Carved from litellm_ci_migration_proxy_misc at c17f440 with git checkout -p, no rewrites

TLDR

Problem this solves:

• litellm-tests (ci: add tests-only CircleCI pipeline with coverage and docs validation #42773) is red on main: sysmon tracer missing, Vertex memory tests time out
• failing shards never upload coverage, relevance check runs after the slow install
• CircleCI injects every project secret into pytest; a selection error passes an empty shard

How it solves it:

• COVERAGE_CORE: sysmon on the unit job, install_codecov_cli with when: always
• checkout -> skip_unless_relevant -> setup_test_deps order
• pytest and run_integration.sh run under an env -i allowlist
• find tests/unit selection failure or empty selection fails the job
• reruns parameter, default 0, wired to --reruns/-p no:rerunfailures

Only .circleci/tests.yml changes (21 insertions, 11 deletions). Timeout stays 90s, coverage roots stay ./litellm and ./enterprise/litellm_enterprise, workers and dist stay parameterized, the default unit job keeps find tests/unit

Type

🚄 Infrastructure

Screenshots / Proof of Fix

litellm-tests on this head (definition a1e47579-d7f3-43f1-8c6c-b20f4c3b38f7, this branch as both config and checkout): https://app.circleci.com/pipelines/github/BerriAI/litellm/90270

Secret visibility inside pytest with the same allowlist: env | cut -d= -f1 | sort inside a unit test process lists only CI, COVERAGE_CORE, HOME, LC_CTYPE, LITELLM_LOCAL_MODEL_COST_MAP, PATH, UV, UV_RUN_RECURSION_DEPTH and VIRTUAL_ENV. None of OPENAI_API_KEY, AZURE_AI_API_KEY, DATABASE_URL, LITELLM_LICENSE, PROXY_MASTER_KEY or the Redis variables. The run_integration.sh step sees only PATH, HOME, CIRCLE_SHA1 and CIRCLE_WORKFLOW_ID and builds its own synthetic keys and URLs. No guard test is committed

Greptile follow-up in 8f84ef6: the circleci tests split output is captured into a variable first, so a failing split exits the shard non-zero instead of running nothing. 5553b66 then exits 0 on an empty shard string before mapfile, so a node that was assigned no files never hands pytest a blank path

Pre-Submission checklist

• I have added meaningful tests (CI only; the pipeline run above is the test)
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/d541aba7e5844703bab65085ef2d54b2
Open in Devin Desktop: https://app.devin.ai/desktop/session/d541aba7e5844703bab65085ef2d54b2?variant=devin

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge.

Summary

This PR updates the CircleCI unit and integration jobs to check test selection and splitting, isolate test environments, support configurable reruns, and install Codecov even after a failed step. Both previous findings are fixed in the current code.

_{Reviews (4) · Last reviewed commit: "ci: exit the unit shard cleanly when cir..."}

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai