Skip to content

feat(signals): accept the decisions model mode alongside evaluation - #67

Merged
moe-berri merged 1 commit into
mainfrom
litellm_accept_decisions_mode
Oct 12, 2026
Merged

moe-berri merged 1 commit into
mainfrom
litellm_accept_decisions_mode

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Summary

LiteLLM is adding cost map rows for decision models with mode: decisions. The existing rows keep the legacy mode: evaluation (BerriAI/litellm#45474 makes the gateway accept both). Lens only matched evaluation, so a decisions model was never discovered for Signals, and saving a Signals config with one was rejected

This PR treats both values the same through one predicate, lens_contract::activity::is_decisions_mode, used in gateway discovery (gateway/discovery.rs) and the Signals save check (server/src/signals.rs). The UI filter systemOneModels uses a matching isDecisionsMode, and docs/signals.md names both modes

Tests

discovery.rs has a new test that discovers a decisions model and an evaluation model into the evaluation deployments. gateway_http.rs now refreshes a gateway reporting one of each, saves a Signals config on the decisions model and still rejects a chat model. There are also unit tests for both predicates

Caveats

The bundled model_catalog.json parses with litellm-model-catalog pinned at 3969348, which has no Mode::Decisions. One decisions row makes Catalog::parse fail for the whole file, so refreshing that snapshot after LiteLLM adds decisions rows needs the pin bumped past BerriAI/litellm#45474 first. The same bump should let decisions/src/transport.rs accept Mode::Decisions next to Mode::Evaluation

Link to Devin session: https://app.devin.ai/sessions/290da858f1bd412dbb3b36b7d880974c
Open in Devin Desktop: https://app.devin.ai/desktop/session/290da858f1bd412dbb3b36b7d880974c?variant=devin
Requested by: @kerry-berri


Note

Devin errored when opening this Pull Request as kerry-berri.
As a fallback, Devin opened this PR as itself.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@moe-berri
moe-berri merged commit e7f871c into main Oct 12, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants