Skip to content

ci: skip slow lint/code-quality/mcp/server-root-path jobs on docs/ui-only PRs - #32546

Open
devin-ai-integration[bot] wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_ci_skip_slow_ui_docs_only
Open

ci: skip slow lint/code-quality/mcp/server-root-path jobs on docs/ui-only PRs#32546
devin-ai-integration[bot] wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_ci_skip_slow_ui_docs_only

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Relevant issues

Follow-up to #32532 (merged); rebased onto and targeting litellm_internal_staging

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

This is a CI-config change, so the proof is the classification the gate uses (.circleci/scripts/classify_changes.sh, the same script CircleCI and the composite action run) for representative diffs:

== docs-only (README.md) ==
  backend: skip
  client:  skip
== ui-only (ui/litellm-dashboard/foo.tsx) ==
  backend: skip
  client:  run
== backend (litellm/x.py) ==
  backend: run
  client:  run

So a docs-only PR skips every gated job; a ui-only PR skips the backend-only checks (lint, code-quality, mcp) but still runs the UI-serving server-root-path e2e; a backend PR runs everything, unchanged

Validated end-to-end on a throwaway docs-only PR (single README comment line, since closed): lint, budget-ratchet, code-quality, test (mcp) and both test-server-root-path legs all reported green in seconds with their heavy steps skipped, while secret-scan still ran. The server-root-path job log showed changed files vs <base>: README.md -> decision=skip -> straight to post-job cleanup (~5s instead of ~11m)

Type

🚄 Infrastructure

Changes

#32532 stopped stranding required checks on ui-only PRs for the 12 test-unit-* workflows, but the other slow PR checks never had any path filter and kept running on every PR (including docs-only ones); the timings on #32539 showed server-root-path at ~11m, plus lint/code-quality/mcp and the CircleCI Windows job all running for a one-line README change

This extends the exact same mechanism to those jobs. The steps still run inside a job that completes successfully (like CircleCI's step halt), so required checks report green rather than sitting on "waiting for status to be reported"

The detect-backend-changes composite action now takes a category input (default backend, so #32532's callers are unchanged) that it forwards to classify_changes.sh:

  • backend -> skip when only ui/**, **.md, **.mdx changed. Used by test-linting (the lint and non-gating budget-ratchet jobs; secret-scan is intentionally left running on every PR), test-code-quality, and test-mcp
  • client -> skip only when the change is pure docs, so ui/** changes still run. Used by test_server_root_path because it is a UI-serving e2e (builds the image, curls /ui/, runs Playwright); this mirrors CircleCI's e2e_ui_testing_server_root_path which is already category client

On the CircleCI side, using_litellm_on_windows was the only job without skip_if_unrelated_changes, so it ran on every PR (~5m45s). It now runs path_filter.sh backend first; since the Windows executor defaults to PowerShell, the guard step sets shell: bash.exe so the bash script and circleci-agent step halt resolve

Deliberate call worth a look: whether server-root-path should be client (skip only pure-docs, keep running on ui-only, as here) or backend (also skip on ui-only). I went with client since it validates UI serving, but it is a one-word flip if you'd rather skip it on ui-only too

Link to Devin session: https://app.devin.ai/sessions/01543c7d72924959bdd52dc138bb4a20
Requested by: @mateo-berri

@mateo-berri mateo-berri self-assigned this Jul 8, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR extends the docs/UI-only skip mechanism (from #32532) to four additional workflows — test-linting, test-code-quality, test-mcp, and test_server_root_path — plus the CircleCI Windows job, by adding a category input to the detect-backend-changes composite action and wiring it through classify_changes.sh.

  • The category: backend path skips when only ui/**, **.md, or **.mdx files changed; category: client skips only on pure-docs changes so UI-serving e2e tests still run on UI-only PRs. The classify_changes.sh logic is correct for both cases.
  • The CircleCI Windows job uses shell: bash.exe to run the bash path-filter script under PowerShell, which is the correct approach for that executor.
  • always() and failure() guards in test_server_root_path.yml are correctly narrowed with && steps.changes.outputs.decision != 'skip' so cleanup and artifact-upload steps don't attempt to operate on containers that were never started.

Confidence Score: 4/5

Safe to merge — the skip logic is correct and the fallback always defaults to run on uncertainty, so no legitimate test run can be accidentally suppressed.

The core classification logic in classify_changes.sh and the action wiring are correct. A handful of lightweight setup steps (Set up Python, Set up uv, Clean Python cache) are intentionally left ungated, adding a few seconds of overhead on skipped runs — minor efficiency gaps, not correctness issues. The secret-scan job remaining fully ungated is explicitly intentional.

No files require special attention. The ungated setup steps in test-mcp.yml and test-linting.yml are worth a look if CI minute costs become a concern, but they do not affect correctness.

Important Files Changed

Filename Overview
.circleci/config.yml Adds path_filter.sh backend guard to the Windows job with shell: bash.exe for correct PowerShell-to-bash bridging.
.github/actions/detect-backend-changes/action.yml Adds optional category input (default backend) passed through to classify_changes.sh; existing callers are unaffected.
.github/workflows/test-code-quality.yml Adds detect-backend-changes action; all expensive steps are gated, but uv and cache setup steps still run on skipped PRs (minor overhead).
.github/workflows/test-linting.yml Adds detect-backend-changes to both lint and budget-ratchet jobs; Set up Python, Set up uv, and Clean Python cache are ungated (minimal overhead); secret-scan intentionally left ungated.
.github/workflows/test-mcp.yml Adds detect-backend-changes; Install dependencies and Run MCP tests are gated; Set up Python, Set up uv, and Thank You Message still run unconditionally.
.github/workflows/test_server_root_path.yml Correctly uses category: client so UI-only PRs still run this e2e; always() and failure() guards are properly narrowed to avoid operating on non-existent containers when skipped.

Reviews (1): Last reviewed commit: "ci: skip slow lint/code-quality/mcp/serv..." | Re-trigger Greptile

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Base automatically changed from litellm_ci_skip_backend_tests_ui_only to litellm_internal_staging July 10, 2026 03:50
@tin-berri
tin-berri requested a review from a team July 10, 2026 03:50
…only PRs

Extends the detect-relevant-changes gate from #32532 to the remaining slow PR checks so docs/ui-only PRs finish in seconds while still reporting green required checks. lint, code-quality and mcp use the backend category (skip on ui-only and docs-only); server-root-path uses the client category (skip only on pure-docs since it is a UI-serving e2e). Also guards the last unguarded CircleCI job, using_litellm_on_windows, which was running on every PR.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot force-pushed the litellm_ci_skip_slow_ui_docs_only branch from cb5887b to 274f00f Compare July 10, 2026 03:51
@codspeed-hq

codspeed-hq Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_ci_skip_slow_ui_docs_only (274f00f) with litellm_internal_staging (45f9bee)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant