fix(ci): support CircleCI rerun failed tests for local_testing jobs

[mateo-berri]

Relevant issues

CircleCI's "Rerun failed tests" feature has been broken for local_testing_part1, local_testing_part2, and litellm_router_testing jobs. Reruns collect 0 items and exit with code 123.

Pre-Submission checklist

• I have Added testing in the tests/test_litellm/ directory — N/A, this is a CI config change with no runtime code impact
• My PR passes all unit tests — no runtime code changed
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Before (rerun output):

[DEBU] Received: tests.local_testing.test_router
4 workers [0 items]
============================ no tests ran in 2.74s =============================
Error: exit status 123

Debug line from the plugin: "if all tests are being run instead of only failed tests, ensure your JUnit XML has a file or classname attribute."

Root cause: The circleci-tests-plugin-cli plugin, when rerunning failed tests, reads the JUnit XML classname attribute (dot notation) and pipes it into xargs pytest. pytest receives tests.local_testing.test_router instead of tests/local_testing/test_router.py, collects zero items, and fails with exit 123.

Fix verification:

$ printf 'tests/local_testing/test_router.py\ntests.local_testing.test_router\n' \
    | awk '/\.py/ {print; next} {gsub(/\./, "/"); print $0 ".py"}'
tests/local_testing/test_router.py
tests/local_testing/test_router.py

Both the normal initial-run input (file paths) and the rerun input (dot notation) are normalized to the same file path.

Type

🐛 Bug Fix
🚄 Infrastructure

Changes

Add an awk preprocessor between the circleci tests run stdin stream and xargs pytest in the three jobs that use this pattern:

• local_testing_part1
• local_testing_part2
• litellm_router_testing

The preprocessor:

• Passes through any line containing .py unchanged (normal file-path input)
• For lines without .py, replaces . with / and appends .py (converts tests.local_testing.test_router → tests/local_testing/test_router.py)

Diff:

-              --command="xargs uv run --no-sync python -m pytest \
+              --command="awk '/\\.py/ {print; next} {gsub(/\\./, \"/\"); print \$0 \".py\"}' | xargs uv run --no-sync python -m pytest \

Applied identically in all three jobs. No change to initial-run behavior; only fixes selective rerun.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[veria-ai]

Low: CI-only change to CircleCI test rerun logic

This PR adds an awk transformation in .circleci/config.yml to convert CircleCI test class identifiers back to file paths for rerunning failed tests. No runtime code, authentication, or production configuration is affected.

---

Status: 0 open
Risk: 1/10

Posted by Veria AI · 2026-04-24T23:43:08.636Z

[greptile-apps]

Greptile Summary

This PR fixes the CircleCI "Rerun failed tests" feature for local_testing_part1, local_testing_part2, and litellm_router_testing by inserting an awk preprocessor that normalises dot-notation classnames (e.g. tests.local_testing.test_router) emitted by the circleci-tests-plugin-cli into file paths (e.g. tests/local_testing/test_router.py) before they are piped to xargs pytest. The three-job change is applied identically and is a no-op for normal first-run execution where paths already contain .py.

Confidence Score: 5/5

Safe to merge — CI-only change with no runtime code impact and a well-reasoned fix for the selective rerun regression.

No P0/P1 issues found. The awk command correctly handles both normal file-path input and dot-notation rerun input, including the uppercase-class-name edge case via the sub guard. The fix is applied consistently across all three affected jobs. No runtime code is touched.

No files require special attention.

Important Files Changed

Filename	Overview
.circleci/config.yml	Adds an awk preprocessor before xargs pytest in three jobs (local_testing_part1, local_testing_part2, litellm_router_testing) to normalise CircleCI's dot-notation classnames to file paths on selective reruns; normal first-run paths pass through unchanged

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["circleci tests run stdin\n(split-by=timings output)"] --> B{Contains '.py'?}
    B -- Yes --> C["Pass through unchanged\n(e.g. tests/local_testing/test_router.py)"]
    B -- No --> D["sub: strip trailing .UppercaseSegment\n(removes test class name if present)"]
    D --> E["gsub: replace '.' with '/'"]
    E --> F["Append '.py'"]
    F --> G["Normalised file path\n(e.g. tests/local_testing/test_router.py)"]
    C --> H["xargs uv run python -m pytest"]
    G --> H
    style B fill:#f0f0f0,stroke:#333
    style H fill:#d4edda,stroke:#28a745

Loading

_{Reviews (2): Last reviewed commit: "fix(ci): strip trailing class segment fr..." | Re-trigger Greptile}

[greptile-apps]

Classnames with test class names will mis-convert

When a pytest test lives inside a class, pytest sets the JUnit XML classname to tests.local_testing.test_router.TestRouterClass (module + class). The awk preprocessor has no .py in that string, so it hits the gsub branch and produces tests/local_testing/test_router/TestRouterClass.py — a path that doesn't exist. If none of the three jobs use test classes this is a non-issue, but it's worth confirming. Consider stripping a trailing uppercase/class-name segment if classnames ever include one.

[mateo-berri]

This has been fixed

[ryan-crabbe-berri]

lgtm