feat(workflow): daily monitor of Databricks Foundation Model Serving pricing by leecoder · Pull Request #38950 · BerriAI/litellm

leecoder · 2026-08-31T09:33:26Z

Summary

Adds a GitHub Action (daily 02:00 UTC) that keeps the Databricks Foundation Model Serving rates in model_prices_and_context_window.json fresh - the one provider family the existing auto_update_price_and_context_window.yml does not cover (it only syncs OpenRouter and Vercel AI Gateway, both JSON APIs).

How it works (registry-wide, not a fixed list)

The monitor parses every Standard Pay Per Token DBU table on both pricing pages (open, proprietary) and maps numeric columns by header text (Input / Output / Cache read / Cache write - column order differs per table). It then compares against every mapped databricks/* registry entry and reports:

UPDATED - list-rate entry drifted from the page: rate fields refreshed in place (metadata/context windows untouched)
PROMO_SKIPPED / PROMO_ON_PAGE - promotional pricing (x0.8) involved; left for a human (e.g. gemini-2.5, gpt-5.6-terra, grok-4.6 promos)
RATES_AVAILABLE / NOT_IN_REGISTRY / UNMAPPED_PAGE_MODEL - new-model signal
MISSING_FROM_PAGE - retirement signal

Cache rates: page value wins; when a cache column is n/a the entry bills cache at the input rate (repo convention); custom conventions (gemini 0.1x reads) are preserved. Long-context tiers and image/audio token sub-rows are skipped. NO_CHANGE short-circuits PR creation.

Runs where it lives - no PAT, no secrets

PRs are created in the repository the workflow runs in (GITHUB_REPOSITORY), so the built-in github.token covers push and gh pr create - the same pattern as this repo's own auto_update_price_and_context_window workflow
Before this PR merges, workflow_dispatch / repository_dispatch on the fork runs the whole pipeline against the upstream checkout (verified live: run 34196076779 - upstream checkout + script bootstrap + monitor, NO_CHANGE green in 1m05s)
After the merge, upstream's cron owns the schedule (job guard pins schedule to BerriAI/litellm); the fork keeps manual triggers only, so the two never race on the same base
repository_dispatch (event_type: dbx-monitor) lets external price-drift alerting kick off a check

Verified locally and in Actions

Live run against the current upstream registry: NO_CHANGE with correct classifications (PROMO_SKIPPED x2, PROMO_ON_PAGE for gpt-5.6-terra's live promo, 8 retired models flagged MISSING_FROM_PAGE)
Planted-drift simulation: removed cache fields from gpt-5/gemma-3-12b in a scratch copy - monitor restored them to exactly the upstream-stored values and wrote the PR body report; second run was idempotent (NO_CHANGE)

greptile-apps · 2026-08-31T09:36:18Z

Greptile Summary

The PR adds a scheduled and manually dispatchable workflow that scrapes Databricks model-serving rates and updates both model-price maps when rates change.

Pins checkout and Python setup actions to immutable revisions.
Propagates scraper pipeline failures with pipefail.
Pushes changed pricing files through an authenticated branch and opens a pull request.
Adds the pricing scraper and JSON update logic.

Confidence Score: 4/5

The PR is not yet safe to merge because detected pricing changes still cannot reach the pull-request creation step.

The monitor writes a step output without declaring id: monitor, while the PR step reads steps.monitor.outputs.changed; therefore changed runs silently skip the update PR.

Files Needing Attention: .github/workflows/monitor_databricks_pricing.yml

Important Files Changed

Filename	Overview
.github/workflows/monitor_databricks_pricing.yml	Adds the daily monitor and authenticated PR workflow, but the previously reported missing monitor step ID remains.
scripts/monitor_databricks_pricing.py	Adds fixed-page DBU parsing, price derivation, dual-map updates, and lint-compliant stdout output.

_{Reviews (4): Last reviewed commit: "style(workflow): replace print with sys...." | Re-trigger Greptile}

codecov · 2026-08-31T09:38:16Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

veria-ai · 2026-08-31T09:38:41Z

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 2 · PR risk: 0/10

leecoder · 2026-09-01T01:45:17Z

@greptileai Please re-review the latest commit — the three flagged items (pipefail, PR-create failure propagation, action SHA pinning) are fixed in d66a4ea and 14439f1.

leecoder · 2026-09-01T02:18:40Z

@greptileai Re-review head d31b62b — git push now authenticates with GITHUB_TOKEN via x-access-token URL (persist-credentials: false was blocking the push). Please confirm the workflow wiring is correct.

leecoder · 2026-09-01T03:20:02Z

@greptileai Re-review head 1bf8f61 — T201 (print) violations fixed with sys.stdout.write and ruff format applied; YAML/scripts relocated per lint gates. All local lint checks pass. Please confirm.

…pricing Adds a GitHub Action (daily 02:00 UTC + manual dispatch) that: - Scrapes the official DBU rates for DeepSeek V4 Flash (0731) / V4 Pro (0813) from https://www.databricks.com/product/pricing/foundation-model-serving - Derives per-token USD at $0.07/DBU and updates both model_prices_and_context_window.json and the packaged backup - Opens a PR (base litellm_internal_staging) only when a rate actually changed Runs on the leecoder fork (no repo guard) so the fork stays current and can feed an upstream PR on demand.

…s to SHAs - Add set -o pipefail so a scraper exception fails the job instead of being misread as a price change - Drop the || true suppression so a failed PR creation fails the run - Pin actions/checkout and actions/setup-python to immutable commit SHAs (repo convention, addresses Greptile/veria-ai review comments)

…ntials: false) actions/checkout runs with persist-credentials: false, so the previous 'git push origin' would fail with auth error. Push over HTTPS with an x-access-token URL using GITHUB_TOKEN. Addresses Greptile review: 'authenticated Git push configuration'.

…scalar - .github/scripts/assert_workflow_dir_hygiene requires only .yml files in .github/workflows/; relocate monitor_databricks_pricing.py to scripts/ - Fix YAML ScannerError by turning the multi-line git commit -m block into a single line (YAML block scalar containment issue) - update the run step path to scripts/monitor_databricks_pricing.py

…01 gate) scripts/monitor_databricks_pricing.py violated T201 (print) which the LiteLLM Linting workflow gates; switch to sys.stdout.write and run ruff format so lint checks pass locally before CI re-run.

…pair The monitor hard-coded MONITORED = {deepseek-v4-flash, deepseek-v4-pro}, so new page models were never detected and column reordering could silently mis-map rates. - Parse every "Standard Pay Per Token" table on both pricing pages (open FMS + proprietary FMS); map numeric columns by header text (Input / Output / Cache read / Cache write), which differs per table - Compare against every mapped registry entry and report UPDATED / PROMO_SKIPPED / PROMO_ON_PAGE / REVIEW / RATES_AVAILABLE / NOT_IN_REGISTRY / UNMAPPED_PAGE_MODEL / MISSING_FROM_PAGE - Refresh only rate fields in place; metadata (context windows, capabilities, deprecation dates) is preserved - Cache fields: page value wins; when the page shows n/a, entries bill cache at the input rate; custom conventions (gemini 0.1x reads) kept - Dash/n-a cells are placeholders, not row qualifiers (gpt-oss / bge / gemma rows were silently dropped before) - Skip long-context tier rows and image/audio token sub-rows - Workflow: PR body now embeds the full monitor report from /tmp/dbx_monitor_pr_body.md Verified live: on the current branch registry the monitor reports 10 cache-field UPDATEDs matching the values upstream already stores, 8 retired models flagged MISSING_FROM_PAGE, 2 PROMO_SKIPPED (gemini 2.5), and NOT_IN_REGISTRY for every new model awaiting BerriAI#39714.

The monitor now executes on the leecoder/litellm fork: - checkout pins the upstream repo and its litellm_internal_staging base, so the monitor always audits upstream's registry, not the fork's - schedule guard skips upstream's own checkout (only the fork runs it); manual workflow_dispatch still works on upstream - changed rates push to a dated branch on the fork and open the PR against BerriAI/litellm with head leecoder:<branch> - PR identity: leecoder / leecoder@aol.com - auth uses the DBX_MONITOR_TOKEN secret (leecoder PAT with repo scopes); the default GITHUB_TOKEN cannot create PRs across forks - permissions trimmed to contents: write

Two fixes for the fork-run setup: - the monitor audits upstream's registry (checkout pins BerriAI/litellm@litellm_internal_staging), where the script does not exist until BerriAI#38950 lands - bootstrap it from the fork branch when missing - cron would fire on both repos once merged; guard the job to the fork so only leecoder/litellm opens PRs and upstream never races

The registry prices both models from rows shared with siblings, but the mapping only listed one key per row, so the monitor flagged them MISSING_FROM_PAGE every run: - 'Gemini 3.0 Pro, 3.1 Pro' also prices databricks-gemini-3-pro - 'Claude Sonnet 4' also prices databricks-claude-sonnet-4-1 Verified against upstream/staging values: all four entries match the page list rates (35.714/214.286/3.571 and 42.857/214.286/4.286 DBU).

The monitor used to run on the fork and open PRs against upstream, which structurally requires a cross-repo PAT. Two changes remove it: - PRs are created in whichever repo the workflow runs in (GITHUB_REPOSITORY), so the built-in github.token covers both push and pr create - same pattern as upstream's auto_update_price_and_context_window workflow - the job guard flips: upstream owns the merged schedule (github.repository == BerriAI/litellm), the fork stays dispatch-only, so the two never race on the same base The script bootstrap from the fork branch stays until BerriAI#38950 lands upstream. DBX_MONITOR_TOKEN secret is no longer referenced.

…on the fork - repository_dispatch (event_type: dbx-monitor) provides an external trigger for price-drift alerting to kick off a check - the job guard previously allowed only workflow_dispatch on the fork, which would have skipped repository_dispatch runs; now only the schedule event is pinned to upstream so cron never races, while both manual trigger types run on either repo

Two fixes for the fork-run setup: - the monitor audits upstream's registry (checkout pins BerriAI/litellm@litellm_internal_staging), where the script does not exist until BerriAI#38950 lands - bootstrap it from the fork branch when missing - cron would fire on both repos once merged; guard the job to the fork so only leecoder/litellm opens PRs and upstream never races

The monitor used to run on the fork and open PRs against upstream, which structurally requires a cross-repo PAT. Two changes remove it: - PRs are created in whichever repo the workflow runs in (GITHUB_REPOSITORY), so the built-in github.token covers both push and pr create - same pattern as upstream's auto_update_price_and_context_window workflow - the job guard flips: upstream owns the merged schedule (github.repository == BerriAI/litellm), the fork stays dispatch-only, so the two never race on the same base The script bootstrap from the fork branch stays until BerriAI#38950 lands upstream. DBX_MONITOR_TOKEN secret is no longer referenced.

This reverts commit affdef5.

leecoder requested a review from a team August 31, 2026 09:33

greptile-apps Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread .github/workflows/monitor_databricks_pricing.yml Outdated

Comment thread .github/workflows/monitor_databricks_pricing.yml Outdated

Comment thread .github/workflows/monitor_databricks_pricing.yml Outdated

veria-ai Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread .github/workflows/monitor_databricks_pricing.yml Outdated

greptile-apps Bot reviewed Sep 1, 2026

View reviewed changes

Comment thread .github/workflows/monitor_databricks_pricing.yml

Comment thread .github/workflows/monitor_databricks_pricing.yml

leecoder added 7 commits September 8, 2026 09:34

style(workflow): trailing newline

937ed96

style(workflow): replace print with sys.stdout.write, ruff format (T2…

ca0375a

…01 gate) scripts/monitor_databricks_pricing.py violated T201 (print) which the LiteLLM Linting workflow gates; switch to sys.stdout.write and run ruff format so lint checks pass locally before CI re-run.

leecoder force-pushed the feat/dbx-pricing-monitor-clean branch from 640c2bd to 5434a47 Compare September 8, 2026 02:33

veria-ai Bot reviewed Sep 8, 2026

View reviewed changes

Comment thread .github/workflows/monitor_databricks_pricing.yml Outdated

leecoder added 4 commits September 8, 2026 12:51

leecoder force-pushed the feat/dbx-pricing-monitor-clean branch from 56d7dff to c0d96ec Compare September 9, 2026 05:41

leecoder force-pushed the feat/dbx-pricing-monitor-clean branch 2 times, most recently from 7c1cea8 to 56d7dff Compare September 9, 2026 07:06

yuneng-berri deleted the branch BerriAI:main September 13, 2026 04:47

yuneng-berri closed this Sep 13, 2026

yuneng-berri reopened this Sep 13, 2026

leecoder added 2 commits September 22, 2026 13:24

feat(macos-usage): add gateway quota menu bar app

affdef5

Revert "feat(macos-usage): add gateway quota menu bar app"

cf2f311

This reverts commit affdef5.

devin-ai-integration Bot changed the base branch from litellm_internal_staging to main September 23, 2026 14:50

Navigation Menu

Repository navigation

feat(workflow): daily monitor of Databricks Foundation Model Serving pricing - #38950

feat(workflow): daily monitor of Databricks Foundation Model Serving pricing#38950
leecoder wants to merge 14 commits into
BerriAI:mainfrom
leecoder:feat/dbx-pricing-monitor-clean

leecoder commented Aug 31, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Aug 31, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Aug 31, 2026 •

edited

Loading

Uh oh!

Uh oh!

veria-ai Bot commented Aug 31, 2026 •

edited

Loading

Uh oh!

leecoder commented Sep 1, 2026

Uh oh!

Uh oh!

Uh oh!

leecoder commented Sep 1, 2026

Uh oh!

leecoder commented Sep 1, 2026

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

leecoder commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How it works (registry-wide, not a fixed list)

Runs where it lives - no PAT, no secrets

Verified locally and in Actions

Uh oh!

greptile-apps Bot commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

veria-ai Bot commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR overview

Security review

Uh oh!

leecoder commented Sep 1, 2026

Uh oh!

Uh oh!

Uh oh!

leecoder commented Sep 1, 2026

Uh oh!

leecoder commented Sep 1, 2026

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

leecoder commented Aug 31, 2026 •

edited

Loading

greptile-apps Bot commented Aug 31, 2026 •

edited

Loading

codecov Bot commented Aug 31, 2026 •

edited

Loading

veria-ai Bot commented Aug 31, 2026 •

edited

Loading