Skip to content

ci: enable ruff preview rules under the budgeted strict gate - #30352

Closed
mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
claude/typescript-python-type-inference-d37dm2
Closed

ci: enable ruff preview rules under the budgeted strict gate#30352
mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
claude/typescript-python-type-inference-d37dm2

Conversation

@mateo-berri

Copy link
Copy Markdown
Contributor

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

  • Branch creation CI run
    Link:
  • CI run for the last commit
    Link:
  • Merge / cherry-pick CI run
    Links:

Screenshots / Proof of Fix

The gate is deterministic, so the proof is the gate itself, run two ways. First, the codebase as-is passes the new ceiling:

$ uv run python scripts/ruff_strict_gate.py --base HEAD
OK: every strict rule is within its codebase ceiling (base HEAD)

The clean gate is untouched and still at zero, so make lint-ruff is unaffected:

$ cd litellm && uv run ruff check .
All checks passed!

And the ratchet update is idempotent against the committed baselines (no diff), confirming the budget matches the current tree:

$ uv run python scripts/ruff_strict_gate.py --update && git diff --stat ruff-strict-budget.json
Re-captured per-rule baselines from the current tree
(no change)

To see it actually catch a regression, add a single new banned-Any or a UP006-style legacy annotation on a fresh line, rerun python scripts/ruff_strict_gate.py, and the gate fails pointing at the new line.

Type

🚄 Infrastructure

Changes

This turns on ruff's preview rule set, but only in the budgeted strict lane, never in the clean gate.

Ruff has two lanes here: ruff.toml drives make lint-ruff (ruff check .) which must stay at zero, and ruff-strict.toml drives scripts/ruff_strict_gate.py, which counts violations against a baseline + slack ceiling in ruff-strict-budget.json. Preview belongs in the second lane because that is the only one with a grandfathering mechanism; flipping preview on in the clean gate would surface ~48k violations with no way to absorb them.

Enabling preview = true with non-explicit selection makes ruff pull in its broad preview catalog (415 rules enabled on this tree), which is both noisy and liable to shift on every ruff upgrade. Instead the config sets explicit-preview-rules = true and enumerates the exact 118 codes that fire today, so the gate is deterministic and a ruff bump never silently adds or drops a gated rule.

The 58438 existing violations across those 118 rules are grandfathered into ruff-strict-budget.json as per-rule baselines. The original ten rules keep their hand-tuned slack; each newly tracked rule gets slack 10 when its baseline is 50 or more and 3 otherwise, so there is a little headroom for churn while net-new violations still fail CI. Two pre-existing rules drifted up by one (ANN401 1885 -> 1886, TID251 2404 -> 2405) and are re-captured at their current counts.

No script or Makefile changes were needed; ruff_strict_gate.py already reads the rule set from ruff-strict.toml and the budget from the JSON, so make lint-strict-budget and make lint-strict-budget-update pick this up as-is.


Generated by Claude Code

Turn on ruff preview in the strict-budget lane (ruff-strict.toml) only,
leaving the clean gate (ruff.toml) untouched so make lint-ruff stays at
zero. Enumerate the 118 firing codes explicitly with
explicit-preview-rules so the gate is deterministic and stable across
ruff upgrades rather than depending on preview auto-selecting the broad
catalog.

Grandfather the existing 58438 violations into ruff-strict-budget.json
as per-rule baselines with headroom, so only net-new violations fail CI.
The existing ten rules keep their hand-tuned slack; the new rules get
slack 10 when the baseline is 50 or more and 3 otherwise.
@codecov

codecov Bot commented Jun 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mateo-berri

Copy link
Copy Markdown
Contributor Author

Superseded by #30379, which keeps this ruff-preview change and adds the ANN return-type rules plus the mypy and basedpyright baseline gates on a litellm_-prefixed branch. Closing this one in favor of that.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant