[Infra] Remove CCI/GHA test duplication and semantically shard proxy DB tests

[yuneng-berri]

Summary

Two related CI cleanups:

1. Delete CCI jobs that duplicate GHA coverage. tests/mcp_tests/ ran in both CCI mcp_testing and GHA test-mcp.yml; tests/test_litellm/proxy/* ran in both CCI litellm_mapped_tests_proxy_part1/part2 and GHA test-unit-proxy-{auth,endpoints,infra}.yml. Removes the CCI copies so there is one source of truth per test directory, matching the design rule of CCI = integration / GHA = unit.
2. Re-shard test-unit-proxy-db.yml semantically. Replaces 4 alphabetic buckets with 8 named groups (auth-and-jwt, proxy-server, logging-and-callbacks, db-and-spend, guardrails-budget-hooks, endpoints-and-responses, plus the existing serial key-generation and test_proxy_utils.py shards). New test files are placed in whichever group they belong to. Adds a dist input to _test-unit-services-base.yml so test_proxy_utils.py can run with --dist=worksteal — the default --dist=loadscope pins a single file to a single xdist worker, which was the root cause of that shard running 10m+.

Changes

• .circleci/config.yml: remove mcp_testing, litellm_mapped_tests_proxy_part1, litellm_mapped_tests_proxy_part2 job definitions; remove their references from the build_and_test workflow, the upload-coverage dependency list, and the coverage-combine command.
• .github/workflows/test-unit-proxy-db.yml: expand the matrix from 4 alphabetic shards to 8 semantic shards; use dist: worksteal for the single-file proxy-utils shard.
• .github/workflows/_test-unit-services-base.yml: add dist input (default loadscope) and thread it through to pytest.
• .github/workflows/test-unit-proxy-endpoints.yml: add rag_endpoints and realtime_endpoints (previously only covered by the deleted CCI part2 job).

Testing

• YAML parse validation for all four files.
• circleci config pack .circleci/ succeeds.
• Verified every CCI job is still referenced by a workflow and every workflow reference resolves to a defined job (0 orphans, 0 dangling references).
• Cross-checked every subdirectory under tests/test_litellm/proxy/ against the remaining GHA workflows — rag_endpoints and realtime_endpoints were the only gaps and are added here; test_configs/ is fixture data.
• Cross-checked every test_*.py file under tests/proxy_unit_tests/ against the new semantic shards — all 58 test files assigned to exactly one shard (excluding test_configs/ which is fixture data).
• Real CI on this PR is the validation surface.

Type

🚄 Infrastructure
🧹 Refactoring

[veria-ai]

Low: CI infrastructure changes only

This PR removes duplicate CircleCI/GHA test jobs and reshards proxy DB tests into semantic groups. The hardcoded Postgres credentials (litellm:litellm) are for ephemeral localhost-only service containers on GHA runners, replacing equivalent GHA environment secrets. No production code is modified.

---

Status: 0 open
Risk: 1/10

Posted by Veria AI · 2026-04-23T23:33:13.324Z

[greptile-apps]

Greptile Summary

This PR removes three CircleCI jobs (mcp_testing, litellm_mapped_tests_proxy_part1/part2) that duplicated GHA coverage, re-shards test-unit-proxy-db.yml from 4 alphabetic buckets into 15 semantic groups, and replaces per-job GHA environment secret scoping with hardcoded ephemeral Postgres credentials (correctly documented as a throwaway localhost container). An assert-shard-coverage guard job is added to catch any test files not assigned to a shard, addressing a previously raised concern.

Confidence Score: 5/5

Safe to merge — all remaining findings are minor P2 style issues with no functional impact.

No P0 or P1 issues found. The secrets removal is safe (ephemeral localhost container, well-documented). The assert-shard-coverage guard correctly addresses the previously raised concern about uncovered test files. Both remaining comments are cosmetic: a stale header comment and a redundant line in a test fixture.

test-unit-proxy-db.yml (stale header comment); test_realtime_webrtc_endpoints.py (redundant master_key assignment in one test body)

Important Files Changed

Filename	Overview
.circleci/config.yml	Removes mcp_testing, litellm_mapped_tests_proxy_part1/part2 jobs and their workflow/coverage references — clean deduplication with no orphaned references.
.github/workflows/_test-unit-services-base.yml	Adds dist input, removes GHA environment secret scoping, replaces DATABASE_URL/POSTGRES_* secrets with hardcoded ephemeral localhost credentials for the throwaway service container; well-commented rationale included.
.github/workflows/test-unit-proxy-db.yml	Expands matrix from 4 alphabetic shards to 15 semantic shards and adds assert-shard-coverage guard; top-of-file comment still references the now-removed DATABASE_URL secret.
.github/workflows/test-unit-proxy-endpoints.yml	Adds rag_endpoints and realtime_endpoints to the test path, filling the coverage gap left by deleted CCI jobs.
.github/workflows/test-unit-security.yml	Updates stale comment and removes the now-unnecessary secrets block; the comment correctly explains why push-only trigger is retained.
tests/test_litellm/proxy/realtime_endpoints/test_realtime_webrtc_endpoints.py	Converts proxy_app fixture from return to yield with proper teardown to prevent master_key state from leaking across xdist workers; one test still sets master_key directly in the test body (redundant but harmless).

_{Reviews (7): Last reviewed commit: "[Infra] Stop attaching push-only postgre..." | Re-trigger Greptile}

[greptile-apps]

No catch-all shard for unassigned test files

The old remaining shard used tests/proxy_unit_tests --ignore=... as a directory target, so any new test_*.py file added to tests/proxy_unit_tests/ was automatically picked up. The new matrix replaces it with an exhaustive explicit list and no fallback directory shard. A developer who adds a new test file without also updating this matrix will have that file silently skipped by CI — no failure, no warning.

Consider keeping one "catch-all" shard (or adding a CI step that validates every test_*.py in the directory is covered by the matrix) to prevent future tests from going untested.

[greptile-apps]

dist: loadscope on a serial (workers: 0) shard is a no-op

The key-generation matrix entry sets dist: loadscope and workers: 0. In _test-unit-services-base.yml, the --dist flag is only passed in the else branch (i.e., when WORKERS != 0). The dist value is effectively ignored here. While harmless, it may mislead future maintainers into thinking --dist=loadscope is somehow enforced for this serial job.

Suggested change

	include:
	# Key generation tests must NOT run in parallel (event loop conflicts with logging worker)
	# Must run serially — event-loop conflict with the logging worker.
	- test-group: key-generation
	test-path: "tests/proxy_unit_tests/test_key_generate_prisma.py"
	workers: 0
	timeout: 30
	- test-group: auth-checks
	test-path: "tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py"
	workers: 8
	dist: loadscope
	- test-group: key-generation
	test-path: "tests/proxy_unit_tests/test_key_generate_prisma.py"
	workers: 0
	# dist is ignored when workers=0; serial run avoids event-loop conflicts
	timeout: 20

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
31539530	Triggered	Generic Password	66bf890	.github/workflows/_test-unit-services-base.yml	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!