Skip to content

[Feat] Add support for 0 cost models - #20249

Merged
Sameerlite merged 2 commits into
mainfrom
litellm_add_0_budget_model_bypass2
Feb 2, 2026
Merged

Sameerlite merged 2 commits into
mainfrom
litellm_add_0_budget_model_bypass2

Conversation

@Sameerlite

Copy link
Copy Markdown
Contributor

Relevant issues

Adding back reverted #19027

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

@vercel

vercel Bot commented Feb 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 2, 2026 8:01am

Request Review

@greptile-apps

greptile-apps Bot commented Feb 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Overview

Greptile Summary

This PR adds support for zero-cost models that bypass budget checks when both input_cost_per_token and output_cost_per_token are explicitly set to 0. This allows on-premises or free models to remain accessible even when users exceed their budget limits.

Key changes:

  • Added _is_model_cost_zero() helper function in auth_checks.py that checks if a model has zero cost by querying the router's model group info
  • Modified common_checks() to accept a skip_budget_checks parameter that conditionally wraps all budget validation logic (team, organization, user, team member, end-user, and global proxy budgets)
  • Integrated zero-cost check in two places in user_api_key_auth.py (lines 608-620 and 1027-1039) before calling common_checks()
  • Added comprehensive test suite with 590 lines covering all budget bypass scenarios
  • Updated documentation explaining the feature, configuration, and behavior

Implementation approach:
The implementation follows a conservative approach: only models with both costs explicitly set to 0 (not None) will bypass budget checks. The llm_router.get_model_group_info() method is called in the request path to determine model cost status.

Issues found:

  • Code duplication: The zero-cost check logic appears twice in user_api_key_auth.py (could be extracted to a helper)
  • The comparison input_cost > 0 or output_cost > 0 doesn't handle negative costs (though unlikely in practice)

Confidence Score: 4/5

  • This PR is generally safe to merge with minor considerations around performance and code quality
  • Score reflects solid implementation with comprehensive tests and documentation, but concerns about code duplication and potential performance impact from calling llm_router.get_model_group_info() in the critical request path. The logic is sound and conservative (requiring explicit zero costs), but the duplication should be addressed and performance should be monitored.
  • Pay attention to litellm/proxy/auth/user_api_key_auth.py for code duplication and performance monitoring of the router call in the request path

Important Files Changed

Filename Overview
docs/my-website/docs/proxy/custom_pricing.md Added comprehensive documentation for zero-cost models feature, explaining configuration and behavior
litellm/proxy/auth/auth_checks.py Adds _is_model_cost_zero helper and wraps budget checks in conditional block based on skip_budget_checks flag
litellm/proxy/auth/user_api_key_auth.py Integrates zero-cost check in two authentication paths with duplicated code blocks; wraps budget checks conditionally
tests/proxy_unit_tests/test_zero_cost_model_budget_bypass.py Comprehensive test suite covering zero-cost model scenarios across user, team, end-user budgets, and edge cases

Sequence Diagram

sequenceDiagram
    participant C as Client
    participant U as user_api_key_auth
    participant A as auth_checks
    participant R as Router
    participant CC as common_checks

    C->>U: Request
    U->>U: Extract model
    U->>A: Check cost
    A->>R: Get pricing
    R-->>A: Pricing data
    
    alt Zero cost
        A-->>U: Bypass enabled
    else Has cost
        A-->>U: Normal flow
    end
    
    U->>CC: Common checks
    
    alt Normal flow
        CC->>CC: All budget checks
    else Bypass
        CC->>CC: Skip budget checks
    end
    
    CC-->>U: Result
    U-->>C: Response
Loading

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 files reviewed, 3 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +608 to +620
# Check if model has zero cost - if so, skip all budget checks
model = get_model_from_request(request_data, route)
skip_budget_checks = False
if model is not None and llm_router is not None:
from litellm.proxy.auth.auth_checks import _is_model_cost_zero

skip_budget_checks = _is_model_cost_zero(
model=model, llm_router=llm_router
)
if skip_budget_checks:
verbose_proxy_logger.info(
f"Skipping all budget checks for zero-cost model: {model}"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code block for checking zero-cost models is duplicated at lines 1027-1039. Consider extracting this logic into a helper function to reduce duplication and maintain consistency.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix With AI
This is a comment left during a code review.
Path: litellm/proxy/auth/user_api_key_auth.py
Line: 608:620

Comment:
This code block for checking zero-cost models is duplicated at lines 1027-1039. Consider extracting this logic into a helper function to reduce duplication and maintain consistency.

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

for model_name in model_list:
try:
# Use router's get_model_group_info method directly for better reliability
model_group_info = llm_router.get_model_group_info(model_group=model_name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verify that llm_router.get_model_group_info() uses caching and doesn't cause performance degradation in the request path, as per the custom rule about avoiding expensive operations.

Context Used: Rule from dashboard - What: Avoid creating new database requests or Router objects in the critical request path.

Why: Cre... (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: litellm/proxy/auth/auth_checks.py
Line: 103:103

Comment:
Verify that `llm_router.get_model_group_info()` uses caching and doesn't cause performance degradation in the request path, as per the custom rule about avoiding expensive operations.

**Context Used:** Rule from `dashboard` - What: Avoid creating new database requests or Router objects in the critical request path.

Why: Cre... ([source](https://app.greptile.com/review/custom-context?memory=0c2a17ad-5f29-423f-a48b-371852ac4169))

How can I resolve this? If you propose a fix, please make it concise.

return False

# If either cost is non-zero, return False
if input_cost > 0 or output_cost > 0:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition input_cost > 0 or output_cost > 0 doesn't handle negative costs (though unlikely in practice). Consider using input_cost != 0 or output_cost != 0 for completeness, or add validation that costs cannot be negative.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix With AI
This is a comment left during a code review.
Path: litellm/proxy/auth/auth_checks.py
Line: 126:126

Comment:
The condition `input_cost > 0 or output_cost > 0` doesn't handle negative costs (though unlikely in practice). Consider using `input_cost != 0 or output_cost != 0` for completeness, or add validation that costs cannot be negative.

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

@Sameerlite
Sameerlite merged commit 19f6640 into main Feb 2, 2026
60 of 65 checks passed
@ishaan-berri
ishaan-berri deleted the litellm_add_0_budget_model_bypass2 branch March 26, 2026 22:29
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
…del_bypass2

[Feat] Add support for 0 cost models

This branch was successfully deployed

1 active deployment
Preview — 14c29323 Deployed Feb 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant