Skip to content

test(together_ai): stop pinning successor deprecation status - #41635

Merged
kerry-berri merged 1 commit into
mainfrom
litellm_together_successor_test_drop_deprecation_pin
Sep 17, 2026
Merged

kerry-berri merged 1 commit into
mainfrom
litellm_together_successor_test_drop_deprecation_pin

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Together successor test asserted the successor model has no deprecation date
  • That is a Together fact, not ours: it broke when Together dated DeepSeek-V4-Pro-0813
  • The price sync writes provider dates, so it cannot keep this test green

How it solves it:

  • Keep the check that every successor key exists in the catalog
  • Drop the "successor must not be deprecated" assertion

User Flow

This does not apply to a test-only change

Relevant issues

Unblocks the price sync PR #41570, which fails only on this test

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

This does not apply to a test-only change. The provider evidence that the old assertion pins an outside fact is the raw Together deprecations page, fetched with curl -sL "https://docs.together.ai/docs/deprecations.md?x=$RANDOM":

| 2026-09-29   | `deepseek-ai/DeepSeek-V4-Pro-0813`   | `deepseek-ai/DeepSeek-V4.1-Flash` | Yes |
| 2026-09-29   | `deepseek-ai/DeepSeek-V4-Flash-0731` | `deepseek-ai/DeepSeek-V4.1-Flash` | Yes |

Three catalog rows (DeepSeek-R1, DeepSeek-V3, DeepSeek-V3.1) name DeepSeek-V4-Pro-0813 as their successor, so the old test fails as soon as the sync writes that date

Type

Test

Caveats (if any)

Low

  • The three metadata.successor pointers to DeepSeek-V4-Pro-0813 are now stale against Together's replacement (DeepSeek-V4.1-Flash); the sync does not maintain that field

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/70717d80beaa4b07b193697a69f1e6a7
Open in Devin Desktop: https://app.devin.ai/desktop/session/70717d80beaa4b07b193697a69f1e6a7?variant=devin


Note

Low Risk
Test-only change; production catalog behavior is unchanged and only removes an assertion on provider deprecation state.

Overview
Relaxes the Together AI metadata.successor catalog test so it only verifies that every successor key exists in model_prices_and_context_window.json, instead of also requiring successors to be undeprecated.

The test is renamed to test_together_successor_metadata_points_at_known_models to match that narrower contract. This avoids CI failures when Together deprecates a model that older entries still point at as a successor (e.g. after price sync writes deprecation_date from provider docs).

Reviewed by Cursor Bugbot for commit 427d084. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration
devin-ai-integration Bot requested a review from a team September 17, 2026 19:40
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 427d084. Configure here.

@greptile-apps

greptile-apps Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge because it removes a brittle provider-fact assertion without weakening the catalog membership invariant

Summary

This test-only PR stops requiring Together AI successor models to have no deprecation date while preserving validation that every successor exists in the model catalog

  • Renames the test to reflect its catalog-membership contract
  • Retains the minimum successor-count check
  • Avoids failing when provider-managed lifecycle dates change

Reviews (1) · Last reviewed commit: "test(together_ai): stop pinning successo..."

@codecov

codecov Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@kerry-berri
kerry-berri merged commit decbb96 into main Sep 17, 2026
86 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant