perf(lint): schedule the lint fan-out around its long pole

[mateo-berri]

TLDR

Problem this solves:

• The lint fan-out ran cheap checks ahead of its long pole
• It spawned one job per core, starving that long pole
• Env sync and base fetch ran back to back for nothing
• Three budget gates re-resolved the same paths per violation
• A Python-only commit still needed the dashboard's node toolchain

How it solves it:

• Declare the checks longest-first, so basedpyright starts first
• Cap the fan-out at two jobs, still overridable
• Overlap the env sync with the base fetch
• Memoize path resolution in all three gates
• Provision the dashboard only when a node block runs

User Flow

Before: a contributor with a one-file Python change cannot run make check at all unless their node satisfies the dashboard's engines floor

1. They stage one Python file: git add litellm/_uuid.py
2. They run make check
3. Bootstrap runs npm install in ui/litellm-dashboard, even though no dashboard file is staged
4. On node below 24.14.1 with neither nvm nor fnm present, the run stops there with "does not meet ui/litellm-dashboard's engines floor" and exits 2, having linted nothing
5. On a machine that clears the floor, the run reaches the linters, spawns one job per core, and each of the three budget gates spends seconds re-resolving paths it has already resolved

After: the same contributor gets a full lint run, on two jobs rather than every core, and node's version has nothing to do with a change that never touches the dashboard

1. They stage the same Python file: git add litellm/_uuid.py
2. They run make check
3. Bootstrap provisions the Python environment only, so no npm install runs
4. The linters run and the command exits 0
5. Staging a dashboard file instead prints "check: provisioning the dashboard toolchain (make bootstrap-dashboard)" first, then lints the dashboard exactly as before
6. If that provisioning fails, the dashboard lint and API-type checks are skipped and the run reports the provisioning error, instead of a formatting error from tools run against an unprovisioned toolchain

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review)

Screenshots / Proof of Fix

This PR changes no runtime surface, so there is no proxy route to curl and no LLM call to bill. What a contributor runs here is make check, so that command is the proof, alongside the make lint fan-out underneath it. Both sides ran back to back on the same 4-core box with 15 GB RAM and node 22.22.2, which sits below the dashboard's engines floor:

$ nproc; node --version
4
v22.22.2

The make check case has exactly one Python file staged and nothing else, on both sides:

$ printf '\n\n# probe\n' >> litellm/_uuid.py && git add litellm/_uuid.py

The make lint case runs on a clean tree on both sides. Both arms resolve the same merge base, 973329e98, and both found its basedpyright counts already cached on disk, so neither one pays the cold recompute. The two runs below are one trial each, captured minutes apart, so read them as a demonstration rather than as the measurement. The multi-trial paired figures are in the benchmark section

Before (973329e)

make check with one Python file staged

1. Run it and time it:

$ S=$(date +%s); make check; echo "EXIT=$? elapsed=$(( $(date +%s) - S ))s"

2. Bootstrap reaches the dashboard's npm install, node is too old for it, and the run stops there having linted nothing:

cd ui/litellm-dashboard && ../../scripts/with_dashboard_node.sh npm install --no-audit --no-fund
with_dashboard_node: node 22.22.2 does not meet ui/litellm-dashboard's engines floor (>= 24.14.1) and neither nvm nor fnm is available to switch automatically.
Fix it with one of:
  - install nvm (https://github.com/nvm-sh/nvm) and re-run; it will pick up node 24.19.0 for you
  - or install/upgrade node yourself to >= 24.14.1 (e.g. brew install node), then re-run
make[1]: *** [Makefile:88: bootstrap] Error 1
make: *** [Makefile:260: check] Error 2
EXIT=2 elapsed=35s

make lint on a clean tree

1. Run it and time it:

$ S=$(date +%s.%N); make lint; echo "EXIT=$? elapsed=$(echo "$(date +%s.%N) - $S" | bc)s"

2. Setup runs as two serial steps, then the checks fan out one job per core with the cheap ones declared first:

uv sync --inexact --frozen --group proxy-dev --group e2e-dev
Audited 198 packages in 7ms
uv run --no-sync python scripts/prisma_generate_if_needed.py
Prisma client already generated for litellm/proxy/schema.prisma (prisma 0.11.0); skipping prisma generate
git fetch origin litellm_internal_staging
make -j 4 --output-sync=target LINT_DEP_INSTALL= LINT_E2E_DEP_INSTALL= LINT_DEP_BASE= lint-checks

3. The run passes:

uv run --no-sync python scripts/type_check_gate.py --base origin/litellm_internal_staging
OK: every rule is within its basedpyright limit or no higher than base (144743 errors total)
EXIT=0 elapsed=148.150560019s

After (7671b79)

make check with one Python file staged

1. Same command, same staged file:

$ S=$(date +%s); make check; echo "EXIT=$? elapsed=$(( $(date +%s) - S ))s"

2. Bootstrap provisions the Python half only, so no npm install runs, and the lint pass the before arm never reached completes:

uv sync --inexact --frozen --extra proxy --group proxy-dev --group e2e-dev
Audited 225 packages in 2ms
uv run --no-sync python scripts/prisma_generate_if_needed.py
Prisma client already generated for litellm/proxy/schema.prisma (prisma 0.11.0); skipping prisma generate
bootstrap: .env left untouched
./scripts/pre_commit_lint.sh
check: logging full output to /home/user/litellm/.git/pre_commit_lint.log
check: linting Python (make lint)
...
check: summary
    ran:     Python lint (make lint)
    skipped: tests/e2e checks (basedpyright + raw HTTP client ban) (no tests/e2e Python files in scope)
    skipped: dashboard lint (prettier + eslint + lint budgets) (no dashboard files in scope)
    skipped: dashboard API-type sync (npm run gen:api) (no litellm/proxy, litellm/types, or generator files in scope)
check: PASS
check: full log: /home/user/litellm/.git/pre_commit_lint.log
EXIT=0 elapsed=155s

make lint on a clean tree

1. Same command:

$ S=$(date +%s.%N); make lint; echo "EXIT=$? elapsed=$(echo "$(date +%s.%N) - $S" | bc)s"

2. The two setup steps now share one -j 2 sub-make, and the checks fan out at two jobs with basedpyright declared first:

make -j 2 --output-sync=target lint-install lint-fetch-base
uv sync --inexact --frozen --group proxy-dev --group e2e-dev
Audited 198 packages in 2ms
uv run --no-sync python scripts/prisma_generate_if_needed.py
Prisma client already generated for litellm/proxy/schema.prisma (prisma 0.11.0); skipping prisma generate
git fetch origin litellm_internal_staging
make -j 2 --output-sync=target LINT_DEP_INSTALL= LINT_E2E_DEP_INSTALL= LINT_DEP_BASE= lint-checks

3. The run passes, and reports the same total the before arm did, so the schedule changed and the verdict did not:

uv run --no-sync python scripts/type_check_gate.py --base origin/litellm_internal_staging
OK: every rule is within its basedpyright limit or no higher than base (144743 errors total)
EXIT=0 elapsed=141.435121390s

Type

🚄 Infrastructure

Benchmark

Method

One 4-core box, 15 GB RAM, node 22.22.2, warm caches, same tree throughout. Both arms were captured against 340ee50f, differing only in the Makefile under test: the before arm restores that file to its base-branch scheduling (nproc width, serial setup, cheap checks declared first) and the after arm is this PR's. Holding the tree fixed and swapping only the scheduling is the point, since anything else would change what the linters actually do

One disclosure about the after arm. The Makefile it ran had check-import-safety declared ahead of lint-e2e-basedpyright, and 0ce1359e swaps them, because the measurements in the next table put e2e at 7.03s and import-safety at 3.69s. That swap cannot move any number here. At two jobs both checks are third and fourth in line, they start only once lint-type-discipline frees the second slot at 34.4s, and the 14.2s of work behind that slot finishes with 76s of basedpyright still to run either way

Every end-to-end case is paired: the two arms alternate inside a trial and the arm that runs first flips on even trials, so page-cache warmth and background drift land on both arms equally. Percentages come from per-trial differences, not from two independently collected medians, because run-to-run spread on this box is 5 to 7%, wide enough to swamp the effect being measured. Absolute seconds are comparable within a case and not across cases collected in different sessions

Reproducing any row is one loop, for example the end-to-end case:

$ for trial in 1 2 3; do
    for arm in before after; do
      cp Makefile.$arm Makefile
      start=$(date +%s.%N)
      make lint > /dev/null 2>&1
      echo "$arm $(echo "$(date +%s.%N) - $start" | bc)"
    done
  done

Where the wall clock goes

Each check run on its own, back to back, median of two:

check	serial median	share
lint-basedpyright	124.38s	71.9%
lint-type-discipline	34.39s	19.9%
lint-e2e-basedpyright	7.03s	4.1%
check-import-safety	3.69s	2.1%
lint-gate	3.42s	2.0%
lint-ruff	0.05s	0.0%
check-circular-imports	0.04s	0.0%
lint-format-check-changed	0.02s	0.0%

Those sum to 173.02s, and a measured -j 1 run of lint-checks is 174.11s, so the breakdown accounts for the whole run. Two things follow. basedpyright is 72% of the serial work, and once the checks fan out it is the entire critical path, because everything else totals 48.6s and fits inside its 124s shadow twice over. So the makespan is basedpyright's runtime plus whatever the schedule wastes around it, and that is both what these changes attack and the ceiling on what they can win

Before and after

case	design	before	after	change	trials favouring after
make lint, end to end	9 paired trials	131.04s	127.12s	3.0% faster	8/9
lint-checks fan-out, -j 4 against -j 2	5 paired trials	136.42s	133.85s	inconclusive	3/5
setup phase, warm	9 paired trials	0.67s	0.52s	22.4% faster	9/9

The end-to-end row is the headline, and 3.0% is the honest size of it: 3.9s off a 131s run, mean paired difference 4.3s with a standard deviation of 3.6s. It is small because it has to be. A run whose critical path is one 124.4s single-threaded process cannot be shortened by scheduling below 124.4s, and make lint now finishes in 127.1s, which is 2.2% above that floor. The remaining 2.7s is make's own startup, the setup phase, the 98 MB of --outputjson the gate parses, and contention from the one check sharing the box. Anything further has to come out of basedpyright itself

The single pair in the proof section above, captured fresh at the current tip rather than in that batch, came in at 148.15s against 141.44s, so 4.5%. One trial each cannot separate 4.5% from 3.0% on a box with this much spread, which is the whole reason the table is paired and repeated. Take 3.0% as the number and that pair as a sanity check that the tip still behaves like the branch that was measured

All of that 3.9s is the ordering change, and it is arithmetic rather than statistics. The old line declared lint-checks: lint-format-check-changed lint-ruff lint-gate lint-type-discipline lint-basedpyright ..., so at two jobs basedpyright sat fifth and could not start until a slot freed after lint-gate, which is 3.42s. Add the 0.15s the warm setup overlap saves and the predicted win is 3.6s, against a measured paired median of 3.60s. Declaring the long pole first removes that delay by construction, on any host, at any width where basedpyright would otherwise queue behind something cheap

The width row is reported as inconclusive because that is what the paired trials showed. -j 4 came in at 130.15, 132.95, 136.42, 143.75 and 149.90s, -j 2 at 127.09, 126.39, 133.85, 145.45 and 169.23s. Median of medians favours -j 2 by 1.9%, the mean paired difference favours -j 4 by 1.3%, standard deviation is 10.25s, and only 3 of 5 trials favour -j 2. The box also drifted monotonically slower across the run, 130s to 150s on the -j 4 arm alone, which is most of that spread. So LINT_JOBS ?= 2 is not claimed as a speedup. It is a structural argument: every check other than basedpyright totals 48.6s against basedpyright's 124.4s, so a single spare slot absorbs all of them with 76s to spare, and every slot past the second can only take a core away from the critical path while heating the machine. ?= leaves it overridable for anyone who disagrees on their own hardware

The setup row is a large percentage of a tiny number and is worth exactly that warm. Its real payoff is the cold path: on a fresh clone or in CI the two steps measured 5.65s and 7.38s, and running them together bounds the pair at the slower one instead of their sum

How often each of those applies

The 3.0% is the common case rather than the whole story, so here is how the runs divide up. Only the dashboard share is measured: 477 of the last 2122 commits touch ui/litellm-dashboard or the API spec, which is 22.5%. The rest are estimates from how the cache paths behave, not measurements, and they are rounded to sum to 100%

situation	share	what the run looks like
warm cache, no dashboard file in scope, every budget within ceiling	70%	~127s, the 3.0% applies
warm cache, dashboard or API spec in scope	21%	the same lint plus the dashboard steps, which this PR provisions rather than skips
a rule over its ceiling, CI's base artifact downloadable	2%	one extra artifact fetch, seconds
a rule over its ceiling, base counts recomputed locally	4%	~250s, two full basedpyright passes
cold environment, fresh clone or new worktree	3%	dominated by uv sync and the Prisma client, where the setup overlap pays most

Gate memoization

Measured on the step it changes rather than end to end. Each gate resolved a path once per violation, and all three report far more violations than the tree has files, so Path.resolve() was making tens of thousands of filesystem round trips for the same couple of thousand paths. Best of three against one real 98 MB basedpyright --outputjson capture and live ruff and checker runs, cache off against cache on, identical input both ways:

step	before	after	saves
type_check_gate.count_basedpyright	6.78s	1.01s	5.77s
type_discipline_gate._check	36.38s	33.53s	2.84s
ruff_strict_gate.head_violations	1.23s	0.72s	0.52s

Those savings do not add up in wall clock and it would be wrong to quote their sum. The gates are separate make targets running concurrently, and only the type-check gate is on the critical path, so 5.77s of the 9.13s is real and the other two finish inside basedpyright's shadow either way

Measured and not landed

basedpyright --level error. The gate counts only severity == "error" diagnostics, so filtering at the source looked free. It is also pointless: the payload went from 98,239,712 bytes to 98,178,783, per-rule counts came back byte-identical, and wall clock was 121.80s against 122.55s across two paired runs, well inside noise. Nearly every diagnostic the tree emits is already an error, so there is nothing to filter

Threading. An earlier revision of this branch ran basedpyright across four pinned worker threads and claimed a 35% cut. That claim did not survive review, and the revert is in this branch. Recording why, so the next person does not spend the afternoon on it:

It changes the answer. Splitting files across threads reorders order-dependent inferences, so the tree's totals move with the width, 149330 serial against 149325 threaded. A gate whose entire job is counting violations must not have its verdict depend on the host

There is no portable width. A pinned constant keeps counts comparable across machines but tunes the gate for one of them. Following nproc is correct per host, yet the width has to live in the cache fingerprint, so every distinct core count mints its own key, nobody can reuse CI's precomputed base artifact, and everyone pays the cold path instead

The speed was noise. On four cores, --threads 4 ranged 84-114s against 137-151s serial, a spread wider than the median it was supposed to beat. On a contributor's laptop, width 8 took 724s against 113s serial, roughly 6x slower, and made the machine sluggish while it ran

scripts/type_check_gate.py documents this, and a test pins the absence of the flag, because a passing run shows neither the count drift nor the slowdown

The one lever left, deliberately not in this PR

basedpyright is 72% of the serial work and the whole critical path, so the only change that moves the number materially is not running it. Its head counts could be cached the way its base counts already are, keyed on the same environment fingerprints plus a content hash of litellm/**/*.{py,pyi}. 1147 of the last 2122 commits, 54.1%, touch nothing under litellm/, and hashing that tree costs 0.240s warm, so on more than half of runs make lint would drop from ~127s to roughly 37s. That is a much bigger change than anything here and it carries a real failure mode, since a wrong key means the gate certifies stale counts, so it belongs in its own PR with its own test proving a mutated litellm/ file always forces a recompute

Changes

Makefile schedules the lint fan-out around its long pole. lint-checks now declares its prerequisites longest-first, because make starts prerequisites in declared order as slots free, so a cheap check sitting ahead of lint-basedpyright pushes the makespan out by its own runtime at narrow widths. The width itself defaults to LINT_JOBS ?= 2 instead of one job per core: every check other than basedpyright totals 48.6s against its 124.4s, so one spare slot hides all of them, and slots beyond that only take cores away from the critical path. ?= keeps it overridable, so LINT_JOBS=8 make lint still does what it says, and dropping the old $(shell sysctl ... || nproc ...) also drops a subprocess from every parse of the Makefile. Finally lint runs lint-install and lint-fetch-base under their own -j 2 sub-make; one waits on the disk and the other on the network, so paying their sum bought nothing

tests/test_litellm/test_makefile_lint.py pins both of those, because reverting either still produces a completely correct lint run, just a slower one, and nothing but a test that inspects the schedule will notice. The overlap test copies the real Makefile into a sandbox whose PATH holds stub uv and git that each announce themselves and then block until the other has announced, so a serial setup deadlocks and fails. The ordering test walks make -n and asserts the declared order matches the measured one

scripts/type_check_gate.py, scripts/ruff_strict_gate.py and scripts/type_discipline_gate.py memoize their path resolution. The type-discipline gate stopped rebinding its root parameter along the way, which LIT011 bans anyway

Makefile also splits bootstrap into bootstrap-python and bootstrap-dashboard. make bootstrap still runs both and prints the same line, so provisioning a fresh worktree is unchanged, and check now depends on the Python half only. scripts/pre_commit_lint.sh provisions the dashboard itself when dashboard or api-spec files are in scope, placed after the Python block forks so the install overlaps that lint, and before both node blocks fork so two npm installs cannot race in one directory. When that provisioning fails, both node blocks are skipped and the run reports the real reason, rather than the misleading "format with npm run format" the tools emit when they run against an unprovisioned toolchain

This branch was merged with litellm_internal_staging after make pre-commit was renamed to make check there, so the provisioning trigger reads the new ui_prettier_changed / ui_eslint_changed / spec_files scope variables. Reading the _changed sets rather than the _files ones matters: a scope containing only a deleted dashboard file still runs the whole-folder eslint budget step, so it still needs a provisioned toolchain

That merge also brought scripts/gate_slot_lock.py and its test, which pinned lint-install and lint-fetch-base as declared prerequisites of lint-inner. Here they run from that target's recipe instead, under the -j 2 sub-make above, so the assertion now accepts either shape. What the test exists for is untouched: lint still declares no prerequisites of its own, so nothing runs before the slot is held

One drive-by fix that fell out of the above: the dashboard job leaked its eslint report on Ctrl-C. bash skips EXIT traps on an uncaught fatal signal, and on_interrupt reaches that job as a SIGTERM, so the temp file outlived the run. The existing interrupt test started failing once the dashboard block forked slightly later, which is how it surfaced. Staging's rewrite of lint_dashboard landed without those traps, so the merge restores them

Worth flagging separately, and deliberately left alone: check-circular-imports is a no-op. It runs cd litellm && python ../tests/documentation_tests/test_circular_imports.py, and that script scans a hardcoded ./litellm/, which from inside litellm/ resolves to litellm/litellm/ and does not exist. It always prints "No LiteLLM type hints found" and exits 0, in CI as well as locally, which is also why it clocks 0.04s in the table above. Fixing it belongs in its own PR, since it would turn a green check red rather than make anything faster

Same category, also left alone: provisioning goes through scripts/with_dashboard_node.sh, which enforces the dashboard's engines floor, while lint_dashboard then calls bare npx prettier, npx eslint and node. So the install refuses to run on old node and the lint that follows it does not care. That predates this PR

Caveats (if any)

• The 3.0% end-to-end win applies to roughly 70% of runs
• LINT_JOBS ?= 2 is structural, not a measured speedup
• Caching basedpyright head counts is the remaining win, deliberately deferred
• check-circular-imports stays a no-op; fixing it needs its own PR
• Dashboard provisioning enforces node's engines floor, the lint after it does not

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

The PR reorganizes local lint scheduling, conditionally provisions dashboard dependencies, and memoizes repeated path normalization.

• Runs lint setup concurrently and orders checks around the longest-running target.
• Splits Python and dashboard bootstrap paths for scoped pre-commit checks.
• Caches repeated path resolution in the three budget gates.
• Adds scheduling, provisioning, and cache-behavior tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
Makefile	Splits bootstrap targets and changes lint setup and fan-out scheduling.
scripts/pre_commit_lint.sh	Provisions dashboard dependencies only for scopes that execute Node-based checks.
scripts/type_check_gate.py	Memoizes diagnostic path normalization while retaining single-threaded basedpyright execution.
scripts/ruff_strict_gate.py	Memoizes Ruff diagnostic path normalization.
scripts/type_discipline_gate.py	Memoizes checker path normalization without rebinding the supplied root.
tests/test_litellm/test_makefile_lint.py	Adds regression coverage for concurrent setup and longest-first lint ordering.
tests/test_litellm/test_pre_commit_lint.py	Covers scoped dashboard provisioning, single provisioning, and failure handling.

_{Reviews (4): Last reviewed commit: "test: accept the parallel setup sub-make..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

proxy-infra is a pre-existing flake: same test and assertion failed on litellm_internal_staging at f05d4687. This PR touches no file that shard collects.

---

Generated by Claude Code

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[devin-ai-integration]

🚅 Hi, thanks for the PR! I'm Agent Shin, the automated triage bot for this repository. What's this and why am I getting it?

I read the description against our contribution rubric. Here's how it lined up:

What you got right:

• ✅ Clear problem description
• ✅ Expected vs. actual behavior

What's still missing:

• end-to-end QA proof with real command output, screenshot, or video
• a linked GitHub issue or equivalent issue reference is not present

The body clearly explains the scheduling/bootstrap problem and contrasts before vs after behavior, so context is present. However, the only evidence is narrative and benchmark/test claims; there is no screenshot/video or real end-to-end command output demonstrating the change, so triage must fail.

If the description isn't updated in the next 24 hours, I'll auto-close this PR. That's not us saying we don't care about the change; we want the open-PR list to mirror what a maintainer can act on right now, so contributors don't get lost in a backlog. A closed PR is a soft "park this for later," not a rejection. Take your time; everything below still works after the close.

During the grace period: just update the PR description with the missing pieces. No need to ping me; I'll re-check on the next sweep and skip the auto-close if it now passes. See what counts as QA proof for the full rubric (a linked issue alone isn't enough; it covers context, not proof).

If the PR does get auto-closed in 24 hours, you still have easy recovery paths:

• Comment @agent-shin reconsider after updating the description. I'll re-evaluate and reopen the PR if it now passes.
• Comment @greptileai to request a fresh Greptile review; that still works even after the PR is closed, and a stronger score is one of the signals that lifts the PR back into the queue. So a low Greptile score isn't a blocker either.

Internal BerriAI contributors: this rubric doesn't apply to you; ping a maintainer.

(I'm an LLM, so I'm not infallible. If you think I got this wrong, ping a maintainer; they'll override me.)

[mateo-berri]

@greptileai

Last review was two commits back. Since then: a staging merge, plus a test assertion that now accepts the parallel setup sub-make.

---

Generated by Claude Code