Repository navigation
chore(decisions): remove the System One converters and OpenAI spec types left dead by #45214 - #45442
Merged
Merged
Conversation
…pes left dead by #45214 #45214 routed both decision formats through the shared decisions IR in litellm/llms/base_llm/decisions/transformation.py and litellm/types/decisions.py. That left litellm/llms/base_llm/decisions/systemone.py (to_system_one_request, question_keys, to_decisions_response, the SystemOne* models and their adapter) and litellm/types/openai_decisions.py with no importer outside their own unit tests, so both modules and both test files go.
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
Contributor
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Contributor
Contributor
|
bugbot run |
Contributor
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit b9cbbe4. Configure here.
DeoLeung
pushed a commit
to aiyou178/litellm
that referenced
this pull request
Oct 11, 2026
…pes left dead by BerriAI#45214 (BerriAI#45442) BerriAI#45214 routed both decision formats through the shared decisions IR in litellm/llms/base_llm/decisions/transformation.py and litellm/types/decisions.py. That left litellm/llms/base_llm/decisions/systemone.py (to_system_one_request, question_keys, to_decisions_response, the SystemOne* models and their adapter) and litellm/types/openai_decisions.py with no importer outside their own unit tests, so both modules and both test files go. Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> (cherry picked from commit 0e48048)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
How it solves it:
litellm/llms/base_llm/decisions/systemone.pyandlitellm/types/openai_decisions.pyUser Flow
Before: a developer classifying a support ticket gets answers from both decision routes, and nothing here is broken
"model": "gpt-6-luna", aninputstring, and aquestionslist, and get 200 with one answer per question"model": "typesafe/jev-1.13.0", astatestring, and aquestionsmap, and get 200 with one answer per question nameAfter: the same two calls answer the same way; the only change is on the maintainer's side, where two unused modules are gone
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
This PR deletes code nothing imports, so the expected result is that both legs answer the same. Both did, on real OpenAI and real TypeSafe calls
Shared setup,
config.yaml(the env holdsOPENAI_API_KEY,TYPESAFE_API_KEY, and a randomLITELLM_MASTER_KEY, and noDATABASE_URL):decisions_openai.json:{"model": "gpt-6-luna", "input": "Ticket (billing): The export job hangs at 99% and never finishes", "questions": [{"type": "predicate", "name": "is_defect", "instructions": "Is this a defect?"}, {"type": "choice", "name": "severity", "instructions": "How severe is it?", "choices": [{"value": "low", "description": "cosmetic"}, {"value": "high", "description": "blocks users"}]}]}systemone_typesafe.json:{"model": "typesafe/jev-1.13.0", "state": "Ticket (billing): The export job hangs at 99% and never finishes", "questions": {"defect": {"type": "noul", "instructions": "Is this a defect?"}, "severity": {"type": "choice", "instructions": "How severe is it?", "criteria": {"low": "cosmetic", "high": "blocks users"}}}}Before (0721cff)
Proxy booted from a worktree at
0721cffab2:python litellm/proxy/proxy_cli.py --config config.yaml --host 127.0.0.1 --port 42027 --num_workers 2 --timeout_worker_healthcheck 60, one instance, 2 uvicorn workers, no databasePOST /v1/decisions on an OpenAI model (gpt-6-luna)
curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:42027/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" --data @decisions_openai.json{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_defect","probability":0.84},{"type":"choice","name":"severity","choice":"high","probabilities":[{"value":"low","probability":0.02},{"value":"high","probability":0.98}],"confidence":0.96}],"usage":{"input_tokens":275,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":275}} HTTP 200POST /v1/systemone on a TypeSafe model (jev-1.13.0)
curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:42027/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" --data @systemone_typesafe.json{"model":"jev-1.13.0","answers":{"defect":{"type":"noul","noul":0.87},"severity":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"high":1.0,"low":0.0}}},"usage":{"input_tokens":336,"output_tokens":48}} HTTP 200After (b9cbbe4)
Proxy booted from a worktree at
b9cbbe4aa3:python litellm/proxy/proxy_cli.py --config config.yaml --host 127.0.0.1 --port 41758 --num_workers 2 --timeout_worker_healthcheck 60, one instance, 2 uvicorn workers, no databasePOST /v1/decisions on an OpenAI model (gpt-6-luna)
curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:41758/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" --data @decisions_openai.json{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_defect","probability":0.84},{"type":"choice","name":"severity","choice":"high","probabilities":[{"value":"low","probability":0.02},{"value":"high","probability":0.98}],"confidence":0.96}],"usage":{"input_tokens":275,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":275}} HTTP 200POST /v1/systemone on a TypeSafe model (jev-1.13.0)
curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:41758/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" --data @systemone_typesafe.json{"model":"jev-1.13.0","answers":{"defect":{"type":"noul","noul":0.87},"severity":{"type":"choice","choice":"high","confidence":0.99,"probabilities":{"low":0.0,"high":1.0}}},"usage":{"input_tokens":336,"output_tokens":48}} HTTP 200Observations:
Both routes answer 200 with the same answer shape on both legs
The TypeSafe confidence moved 1.0 to 0.99 between calls: provider variance, same on both legs
/v1/decisionsusage reportsoutput_tokens: 0on both legs; this PR leaves it aloneb9cbbe4 passes /audit (skipped: no product code on a request path, a refactor deleting unimported modules)
Type
🧹 Refactoring
Caveats (if any)
Low
tests/unit/llms/base_llm/decisions/andtests/unit/llms/openai/decisions/litellm.types.openai_decisionsandlitellm.llms.base_llm.decisions.systemonestop importing; they only ever shipped in the dev tag v1.106.0-dev.2, never in a stable or rc release, and removing them is the point of this PRFinal Attestation
The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
b9cbbe4 passes /live-pr-risk