Skip to content

ci: skip backend unit tests on ui-only PRs without stranding required checks - #32532

Merged
mateo-berri merged 1 commit into
litellm_internal_stagingfrom
litellm_ci_skip_backend_tests_ui_only
Jul 10, 2026
Merged

ci: skip backend unit tests on ui-only PRs without stranding required checks#32532
mateo-berri merged 1 commit into
litellm_internal_stagingfrom
litellm_ci_skip_backend_tests_ui_only

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Relevant issues

Re-does #32422 correctly and supersedes the straight revert in #32530

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

This is a CI trigger change, so there is no proxy or LLM surface to curl; the proof is the workflow config plus how the checks behave

The classification is the exact same script CircleCI uses (.circleci/scripts/classify_changes.sh backend), so the skip set is identical to CircleCI. Verified locally at HEAD:

$ printf 'ui/litellm-dashboard/src/x.tsx\n' | bash .circleci/scripts/classify_changes.sh backend
skip
$ printf 'README.md\ndocs/x.mdx\n'          | bash .circleci/scripts/classify_changes.sh backend
skip
$ printf 'ui/x.tsx\nlitellm/main.py\n'       | bash .circleci/scripts/classify_changes.sh backend
run

This PR itself only touches .github/, which classifies as run, so every test-unit-* job on this PR executes the full suite; that is the regression proof that backend PRs are unaffected. A UI-only PR like #32502 will now run each job, hit decision=skip, and report the required check green in seconds instead of leaving it stuck on "Expected; waiting for status to be reported"

Type

🚄 Infrastructure

Changes

#32422 added paths-ignore: [ui/**, **.md, **.mdx] to the pull_request trigger of the 12 test-unit-*.yml workflows so UI-only and docs-only PRs would not pay for the backend suites. The problem is that those job names (core-utils / Run tests, auth-checks / Run tests, and so on) are configured as required status checks in branch protection. paths-ignore stops the whole workflow from ever starting, so on a UI-only PR the required contexts are never reported and the PR sits forever on "Expected; waiting for status to be reported" (see #32502). GitHub only treats a required check as satisfied when the job actually runs and reports a conclusion; a workflow that never triggers is not the same as a skipped job

This is exactly the difference from CircleCI, where the equivalent filter runs inside the job (.circleci/scripts/path_filter.sh calls circleci-agent step halt), so the job still completes green. Mirroring that, this PR drops paths-ignore from all 12 workflows and instead short-circuits the expensive steps inside each job:

  • new composite action .github/actions/detect-backend-changes diffs the PR against github.event.pull_request.base.sha and pipes the changed files through the same classify_changes.sh backend script CircleCI uses, exposing decision=run|skip. It defaults to run for non pull_request events or whenever the base or diff cannot be resolved, so tests are never skipped when classification is uncertain
  • _test-unit-base.yml (the reusable base behind 10 of the workflows) runs the detect step right after checkout and gates Install dependencies, Generate Prisma client, Run tests and coverage on decision != 'skip'; the upload-coverage job is gated the same way
  • test-unit-documentation.yml and test-unit-proxy-legacy.yml do not use the reusable base, so they get the same detect step and step-level guards inline

The net effect is that the jobs always start and always report a conclusion, so required checks are satisfied on every PR, but on a UI-only or docs-only PR they skip the uv sync, prisma generate and pytest steps and finish in seconds. Backend PRs behave exactly as before

Link to Devin session: https://app.devin.ai/sessions/01543c7d72924959bdd52dc138bb4a20
Requested by: @mateo-berri

… checks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot requested a review from a team July 8, 2026 19:12
@mateo-berri mateo-berri self-assigned this Jul 8, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a branch-protection deadlock introduced by #32422: removing paths-ignore from all 12 test-unit-*.yml workflows and replacing it with a new composite action (.github/actions/detect-backend-changes) that classifies changed files inside the job using the same classify_changes.sh backend logic CircleCI uses. Jobs always start and always report a conclusion, so required status checks are never left in "Expected; waiting…" limbo, but expensive steps (uv sync, prisma generate, pytest) are gated on decision != 'skip' so UI-only and docs-only PRs still finish in seconds.

  • New composite action fetches the PR base SHA, diffs it against HEAD, and pipes the result through .circleci/scripts/classify_changes.sh backend; every failure path (non-PR event, unreachable base SHA, empty diff, script error) defaults to decision=run so tests are never incorrectly skipped.
  • _test-unit-base.yml propagates decision as a job output so the downstream upload-coverage job is also gated, and the two standalone workflows (test-unit-documentation.yml, test-unit-proxy-legacy.yml) receive inline copies of the same detect step.

Confidence Score: 5/5

Safe to merge — changes are confined to CI configuration, with no production code or test logic altered.

All 12 workflows are updated consistently: 10 via the reusable base and 2 inline. Every error path in the composite action defaults to run, so tests can never be silently skipped due to a classification failure. The fallback logic is also exercised by this PR itself (.github/** is classified as backend, so all jobs run normally). The upload-coverage gating via job outputs is wired correctly in the base workflow. No application code is touched.

No files require special attention; the most critical logic lives in .github/actions/detect-backend-changes/action.yml and .github/workflows/_test-unit-base.yml, both of which look correct.

Important Files Changed

Filename Overview
.github/actions/detect-backend-changes/action.yml New composite action that classifies PR changes via classify_changes.sh and exposes decision=run
.github/workflows/_test-unit-base.yml Adds detect-backend-changes step after checkout, gates Install dependencies / Generate Prisma / Run tests / Save coverage on decision != skip, and gates upload-coverage job via job outputs
.github/workflows/test-unit-documentation.yml Removes paths-ignore and adds inline detect-backend-changes step with step-level guards on install, prisma, and test steps
.github/workflows/test-unit-proxy-legacy.yml Removes paths-ignore and adds inline detect-backend-changes step with step-level guards; matrix jobs each independently run the detection
.github/workflows/test-unit-core-utils.yml paths-ignore block removed; skipping logic now lives entirely in the reusable _test-unit-base.yml
.github/workflows/test-unit-proxy-db.yml paths-ignore block removed; inherits skip logic from reusable base workflow

Reviews (1): Last reviewed commit: "ci: skip backend unit tests on ui-only P..." | Re-trigger Greptile

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 30 untouched benchmarks


Comparing litellm_ci_skip_backend_tests_ui_only (557770e) with litellm_internal_staging (b00877c)

Open in CodSpeed

@mateo-berri
mateo-berri self-requested a review July 8, 2026 19:51

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; thanks!

@mateo-berri
mateo-berri merged commit 45f9bee into litellm_internal_staging Jul 10, 2026
124 checks passed
@mateo-berri
mateo-berri deleted the litellm_ci_skip_backend_tests_ui_only branch July 10, 2026 03:50
devin-ai-integration Bot added a commit that referenced this pull request Jul 10, 2026
…only PRs

Extends the detect-relevant-changes gate from #32532 to the remaining slow PR checks so docs/ui-only PRs finish in seconds while still reporting green required checks. lint, code-quality and mcp use the backend category (skip on ui-only and docs-only); server-root-path uses the client category (skip only on pure-docs since it is a UI-serving e2e). Also guards the last unguarded CircleCI job, using_litellm_on_windows, which was running on every PR.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants