Repository navigation
Conversation
Decision signals read only the current user turn. prior_user_turns: N puts up to N earlier user turns, oldest first, before it in the state. On MT-Bench follow-ups that raised right answers from 41-55% to 71-84% on Decision 2.0 and from 35-51% to 67.5-87.5% on Vela 2.0. Related vllm-project#3771 Signed-off-by: subin9 <101092510+subin9@users.noreply.github.com>
65b5f8b to
ad94ec5
Compare
There was a problem hiding this comment.
Thanks for clarifying the long-input behavior. My earlier statement that the current turn was truncated, and the reference to a blocking review, were incorrect: Decision rejects an over-limit state. The revised shared history budget preserves the current turn; both classification regressions, config/DSL checks, and all three Decision long-input rejection controls pass. I found no remaining source blocker in this change.
Eight earlier turns of 1,760 characters each could push a state past the model's input limit, and Decision models refuse such an input. The earlier turns now share 1,760 characters, filled from the latest turn. Signed-off-by: subin9 <101092510+subin9@users.noreply.github.com>
…turns Signed-off-by: subin9 <101092510+subin9@users.noreply.github.com>
The overview now says the standalone frontend or an ExtProc gateway presents the request, so the contract looked for a sentence that main no longer has and Router Contracts failed on every pull request. Signed-off-by: subin9 <101092510+subin9@users.noreply.github.com>
The Chinese guide now has to show the English page's snippets, so it gets the prior_user_turns section too. Signed-off-by: subin9 <101092510+subin9@users.noreply.github.com>
|
You're right. Eight earlier turns of up to 1,760 characters each could take the state past the model's input limit, and the Decision models refuse such an input instead of cutting it, so the question failed even when the current turn alone fit. The earlier turns now share 1,760 characters, filled from the latest turn back. I couldn't find the blocking review you mentioned, so if your regression shows a different case, could you paste it here? |
Purpose
Decision signals read only the current user turn, so a follow-up such as "Rewrite your previous answer in fewer words" routes on almost nothing. This implements Option A of #3771 for decision-model questions.
prior_user_turns: N(0 to 8) on arouting.signals.decisionrule makes its question read up to N earlier user turns, oldest first, then the current turn, one blank line apart. The earlier turns share 1,760 characters, filled from the latest turn back, so however many turns a question reads they add at most that much to the state. Questions to one deployment share a call only when they read the same number of turns, so single-turn requests still make one call. Assistant and tool messages are left out: on the measurement below, the whole conversation scored no better than the user turns on all but the smallest model, at about twice the tokens. The decision selector and the built-in signals are unchanged, and #3771's Option B (a continuity signal family) is not covered.On MT-Bench's 80 two-turn conversations (prompts Apache-2.0), an eight-way Choice over its categories, asked on the second turn:
Related #3771
Test Plan
Unit tests for the state, the oldest-first order, the shared limit, the call grouping and the first-turn case. The field reaches the validator, the DSL decompiler, the CLI model, the generated schema, the dashboard type and the decision signal tutorial. The new E2E case
decision-prior-user-turnsruns in themodel-runtimeprofile: it finds a first turn for which the fixture runtime answers the follow-up differently alone and after that turn, then checks that the Router routes on the second answer.Test Result
The
model-runtimeprofile passed locally with all 13 cases on main with #4734. In the new case the fixture answeredcodefor the follow-up alone andchatafter "Prove that the square root of two is irrational.", and the Router matchedfollow_up_kind:chat. The E2E case asks a Decision 2.0 fixture; the Vela 2.0 numbers above come from the runtime directly, not through the Router.