chore(typing): auto-tighten the basedpyright budget nightly

[mateo-berri]

TLDR

Problem this solves:

• The limits in basedpyright-code-budget.json sit far above the counts actually in the tree (63k errors of combined headroom before this PR), so reportAny/reportExplicitAny and every other gated rule can grow daily without any check going red
• Fixes that lower counts (like chore(typing): clear 2.7k basedpyright Any errors across 15 hotspot files #34745) only shrink the ceilings when the fixing branch remembers to ratchet, so cleared headroom silently reopens as free room to regress

How it solves it:

• Adds a --tighten mode to scripts/type_check_gate.py that lowers every rule's limit to its current head count plus 50 slack; limits never rise, rules outside the budget are not adopted, and the vacuous-run guard refuses to tighten when basedpyright emitted nothing
• Adds a nightly scheduled workflow that measures litellm_internal_staging, runs the tighten, and opens a PR only when there is stale headroom to remove
• Runs the first tighten in this PR: 63,290 errors of headroom removed across 23 of 48 rules; the reportAny limit falls 34,906 to 24,479 and reportExplicitAny 10,230 to 7,334

Friction stays where it was: the gate's delta-vs-base check spares any PR whose rule count does not exceed its merge-base count, so a PR that merely inherits drift can never go red from a tighten. Only a PR that pushes a rule past both its limit and its own base fails, and the 50 slack per rule is the shared allowance for net new errors between tightens

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

The tighten run that produced the budget change in this PR:

$ (NODE_OPTIONS=--max-old-space-size=8192 uv run --no-sync basedpyright --outputjson || true) \
    | uv run --no-sync python scripts/type_check_gate.py --tighten
Tightened basedpyright limits to head counts plus 50 slack, removing 63290 errors of stale headroom across 48 rules

The guard rails, exercised against the real CLI:

$ echo '{}' | uv run --no-sync python scripts/type_check_gate.py --tighten
FAIL: basedpyright produced no errors, but basedpyright-code-budget.json allows a nonzero total. The type checker almost certainly crashed or emitted nothing; refusing to tighten every limit down to the slack floor.
$ echo $?
1

$ echo '{}' | uv run --no-sync python scripts/type_check_gate.py --tighten --slack -5
type_check_gate.py: error: --slack must be >= 0

$ echo '{}' | uv run --no-sync python scripts/type_check_gate.py --tighten --update
type_check_gate.py: error: argument --update: not allowed with argument --tighten

Type

🚄 Infrastructure

Changes

scripts/type_check_gate.py gains tightened_budget() and cmd_tighten(), wired to a new --tighten flag that is mutually exclusive with --update and takes --slack (default 50). tightened_budget sets each budgeted rule's limit to min(existing limit, current count + slack), so a limit only ever falls and a rule that already sits close to its count keeps its limit. cmd_tighten reuses the existing vacuous-run guard so a crashed or empty basedpyright pass can never collapse every limit to the slack floor

.github/workflows/nightly_tighten_basedpyright_budget.yml checks out litellm_internal_staging on a nightly schedule (07:47 UTC, plus workflow_dispatch), installs the same environment the lint workflow uses, pipes a whole-tree basedpyright run into the tighten, and opens a PR against litellm_internal_staging only when basedpyright-code-budget.json changed. Branch names are date-suffixed and the job skips if the day's branch already exists. Note the schedule only fires from the workflow definition on the default branch; until it propagates there, it can be exercised with workflow_dispatch from a branch that contains it

basedpyright-code-budget.json carries the first tighten so the ceilings match the tree as of this branch instead of waiting for the first scheduled run

tests/test_litellm/test_type_check_gate.py adds six tests covering the tighten: headroom collapse to current plus slack, never raising a limit, flooring a cleared rule at the slack, not adopting rules outside the budget, writing only lowered limits through the CLI entry point, and refusing a vacuous run while leaving the budget file untouched

QA runbook

1. uv run --no-sync pytest tests/test_litellm/test_type_check_gate.py -q and expect all tests to pass
2. echo '{}' | uv run --no-sync python scripts/type_check_gate.py --tighten and expect a refusal with exit 1 and an unchanged basedpyright-code-budget.json
3. (NODE_OPTIONS=--max-old-space-size=8192 uv run --no-sync basedpyright --outputjson || true) | uv run --no-sync python scripts/type_check_gate.py --tighten and expect git diff basedpyright-code-budget.json to show only equal-or-lower limits
4. Once the workflow reaches the default branch, trigger Nightly Basedpyright Budget Tighten via workflow_dispatch and expect either "No stale headroom to remove" or a PR against litellm_internal_staging touching only basedpyright-code-budget.json

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Checker failures lose their status

When basedpyright terminates after emitting valid but incomplete nonempty JSON, || true discards the failure status and --tighten treats the partial diagnostics as a complete measurement. Missing rules are then counted as zero and lowered toward the slack floor, producing an invalid budget PR that can make subsequent type-check jobs fail.

[greptile-apps]

Branch check has a race

Two same-day workflow runs can both observe that the date-derived branch is absent before either pushes it. One run then receives a non-fast-forward rejection and fails instead of cleanly recognizing that the other run already created the nightly branch.

[greptile-apps]

Greptile Summary

This PR automates tightening the repository's basedpyright error budget.

• Adds a --tighten mode that lowers existing per-rule limits to measured counts plus configurable slack.
• Adds tests for monotonic tightening, omitted rules, CLI writes, and empty-run refusal.
• Adds a nightly workflow that measures staging and opens a budget-only pull request.
• Applies an initial tightening across the checked-in budget.

Confidence Score: 4/5

The PR is safe to merge with non-blocking robustness issues in the nightly workflow's checker-failure and concurrent-run handling.

The tightening logic preserves normal gate behavior, but the workflow accepts nonempty incomplete checker output and concurrent same-day runs can race while creating the shared date-derived branch.

Files Needing Attention: .github/workflows/nightly_tighten_basedpyright_budget.yml, scripts/type_check_gate.py

Important Files Changed

Filename	Overview
.github/workflows/nightly_tighten_basedpyright_budget.yml	Adds the scheduled measurement and PR workflow, but suppresses checker failure statuses and uses a race-prone check-before-push sequence.
scripts/type_check_gate.py	Adds monotonic budget-tightening logic and an empty-run guard, though the guard cannot distinguish nonempty partial results from a complete run.
tests/test_litellm/test_type_check_gate.py	Adds focused tests for tightening behavior and confirms that an entirely empty run leaves the budget untouched.
basedpyright-code-budget.json	Lowers 23 existing rule limits without changing the budget schema.

_{Reviews (1): Last reviewed commit: "chore(typing): auto-tighten the basedpyr..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!