Skip to content

chore(decisions): remove the System One converters and OpenAI spec types left dead by #45214 - #45442

Merged
mateo-berri merged 1 commit into
mainfrom
litellm_remove_dead_systemone_converters
Oct 8, 2026
Merged

mateo-berri merged 1 commit into
mainfrom
litellm_remove_dead_systemone_converters

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

How it solves it:

  • Deletes litellm/llms/base_llm/decisions/systemone.py and litellm/types/openai_decisions.py
  • Deletes the two unit test files that only exercised those modules
  • Changes no request path: nothing imported either module outside its own tests

User Flow

Before: a developer classifying a support ticket gets answers from both decision routes, and nothing here is broken

  1. They send POST https://litellm-domain/v1/decisions with "model": "gpt-6-luna", an input string, and a questions list, and get 200 with one answer per question
  2. They send POST https://litellm-domain/v1/systemone with "model": "typesafe/jev-1.13.0", a state string, and a questions map, and get 200 with one answer per question name

After: the same two calls answer the same way; the only change is on the maintainer's side, where two unused modules are gone

  1. They send the same POST https://litellm-domain/v1/decisions and get the same 200 with one answer per question
  2. They send the same POST https://litellm-domain/v1/systemone and get the same 200 with one answer per question name

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests (this PR only deletes tests that covered the deleted modules)
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

This PR deletes code nothing imports, so the expected result is that both legs answer the same. Both did, on real OpenAI and real TypeSafe calls

Shared setup, config.yaml (the env holds OPENAI_API_KEY, TYPESAFE_API_KEY, and a random LITELLM_MASTER_KEY, and no DATABASE_URL):

model_list:
  - model_name: gpt-6-luna
    litellm_params:
      model: openai/gpt-6-luna
      api_key: os.environ/OPENAI_API_KEY
  - model_name: typesafe/jev-1.13.0
    litellm_params:
      model: typesafe/jev-1.13.0
      api_key: os.environ/TYPESAFE_API_KEY
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

decisions_openai.json:

{"model": "gpt-6-luna", "input": "Ticket (billing): The export job hangs at 99% and never finishes", "questions": [{"type": "predicate", "name": "is_defect", "instructions": "Is this a defect?"}, {"type": "choice", "name": "severity", "instructions": "How severe is it?", "choices": [{"value": "low", "description": "cosmetic"}, {"value": "high", "description": "blocks users"}]}]}

systemone_typesafe.json:

{"model": "typesafe/jev-1.13.0", "state": "Ticket (billing): The export job hangs at 99% and never finishes", "questions": {"defect": {"type": "noul", "instructions": "Is this a defect?"}, "severity": {"type": "choice", "instructions": "How severe is it?", "criteria": {"low": "cosmetic", "high": "blocks users"}}}}

Before (0721cff)

Proxy booted from a worktree at 0721cffab2: python litellm/proxy/proxy_cli.py --config config.yaml --host 127.0.0.1 --port 42027 --num_workers 2 --timeout_worker_healthcheck 60, one instance, 2 uvicorn workers, no database

POST /v1/decisions on an OpenAI model (gpt-6-luna)

  1. curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:42027/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" --data @decisions_openai.json
  2. Observed:
{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_defect","probability":0.84},{"type":"choice","name":"severity","choice":"high","probabilities":[{"value":"low","probability":0.02},{"value":"high","probability":0.98}],"confidence":0.96}],"usage":{"input_tokens":275,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":275}}
HTTP 200

POST /v1/systemone on a TypeSafe model (jev-1.13.0)

  1. curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:42027/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" --data @systemone_typesafe.json
  2. Observed:
{"model":"jev-1.13.0","answers":{"defect":{"type":"noul","noul":0.87},"severity":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"high":1.0,"low":0.0}}},"usage":{"input_tokens":336,"output_tokens":48}}
HTTP 200

After (b9cbbe4)

Proxy booted from a worktree at b9cbbe4aa3: python litellm/proxy/proxy_cli.py --config config.yaml --host 127.0.0.1 --port 41758 --num_workers 2 --timeout_worker_healthcheck 60, one instance, 2 uvicorn workers, no database

POST /v1/decisions on an OpenAI model (gpt-6-luna)

  1. curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:41758/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" --data @decisions_openai.json
  2. Observed:
{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_defect","probability":0.84},{"type":"choice","name":"severity","choice":"high","probabilities":[{"value":"low","probability":0.02},{"value":"high","probability":0.98}],"confidence":0.96}],"usage":{"input_tokens":275,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":275}}
HTTP 200

POST /v1/systemone on a TypeSafe model (jev-1.13.0)

  1. curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:41758/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" --data @systemone_typesafe.json
  2. Observed:
{"model":"jev-1.13.0","answers":{"defect":{"type":"noul","noul":0.87},"severity":{"type":"choice","choice":"high","confidence":0.99,"probabilities":{"low":0.0,"high":1.0}}},"usage":{"input_tokens":336,"output_tokens":48}}
HTTP 200

Observations:

  • Both routes answer 200 with the same answer shape on both legs

  • The TypeSafe confidence moved 1.0 to 0.99 between calls: provider variance, same on both legs

  • /v1/decisions usage reports output_tokens: 0 on both legs; this PR leaves it alone

  • b9cbbe4 passes /audit (skipped: no product code on a request path, a refactor deleting unimported modules)

Type

🧹 Refactoring

Caveats (if any)

Low

  • No before=fail leg exists: the PR changes nothing a caller can see, so the proof is a same-answer A/B over both decision routes
  • The deleted unit tests covered only the deleted modules; the live System One translation keeps its tests under tests/unit/llms/base_llm/decisions/ and tests/unit/llms/openai/decisions/
  • litellm.types.openai_decisions and litellm.llms.base_llm.decisions.systemone stop importing; they only ever shipped in the dev tag v1.106.0-dev.2, never in a stable or rc release, and removing them is the point of this PR

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

  • b9cbbe4 passes /live-pr-risk

…pes left dead by #45214

#45214 routed both decision formats through the shared decisions IR in
litellm/llms/base_llm/decisions/transformation.py and litellm/types/decisions.py.
That left litellm/llms/base_llm/decisions/systemone.py (to_system_one_request,
question_keys, to_decisions_response, the SystemOne* models and their adapter)
and litellm/types/openai_decisions.py with no importer outside their own unit
tests, so both modules and both test files go.
@devin-ai-integration

devin-ai-integration Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Low impact] This PR appears safe to merge because live Decisions calls do not use the deleted modules

Summary

Removes unused System One converters, old OpenAI Decisions types, and their two test files

  • Live Decisions calls already use the retained types and shared conversion code
  • Retained tests cover both request formats and their replies
  • No actionable issues found. No runtime tests were run

Reviews (1) · Last reviewed commit: "chore(decisions): remove the System One ..." · Reviewed by Greptile

@codecov

codecov Bot commented Oct 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_remove_dead_systemone_converters (b9cbbe4) with main (4ed0267)

Open in CodSpeed

@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit b9cbbe4. Configure here.

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mateo-berri
mateo-berri merged commit 0e48048 into main Oct 8, 2026
112 checks passed
@mateo-berri
mateo-berri deleted the litellm_remove_dead_systemone_converters branch October 8, 2026 21:34
DeoLeung pushed a commit to aiyou178/litellm that referenced this pull request Oct 11, 2026
…pes left dead by BerriAI#45214 (BerriAI#45442)

BerriAI#45214 routed both decision formats through the shared decisions IR in
litellm/llms/base_llm/decisions/transformation.py and litellm/types/decisions.py.
That left litellm/llms/base_llm/decisions/systemone.py (to_system_one_request,
question_keys, to_decisions_response, the SystemOne* models and their adapter)
and litellm/types/openai_decisions.py with no importer outside their own unit
tests, so both modules and both test files go.

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 0e48048)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant