Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions docs-site/src/content/docs/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -512,12 +512,13 @@ target/effort, a coarse decision gate, latency, optional confidence/probability,
Automated tests use mocked TypeSafe responses plus a no-key fail-open smoke; a live TypeSafe decision
requires an operator-supplied key and is not run implicitly.

When JEV's selected effort is applied to the Responses child, request and attempt logs show a
differing effort as `high->xhigh` or `high->low`. Later effort adjustments remain in the chain,
such as `high->xhigh->medium`. An unchanged effort or a target without effort control keeps the
existing label; fallback attempts do not inherit the initial JEV effort. With the opt-in
`protocols.rollout.nativeChatCombos` path, native Chat children currently do not apply JEV's
selected effort; their labels retain the caller's effort and any native child adjustments instead.
JEV's selected effort applies to the first Responses child and to native Chat children on the opt-in
`protocols.rollout.nativeChatCombos` path. It replaces caller effort and removes conflicting thinking
controls and caller `service_tier`; an explicit no-effort choice removes effort even when the Combo has
a forced default. Provider effort pins, caps and wire normalization still apply afterward.
Request and attempt logs show a differing applied effort as `high->xhigh` or `high->low`, retaining later
adjustments such as `high->xhigh->medium`. An unchanged effort or a no-effort choice keeps the existing
label. Later fallback targets use ordinary Combo policy from the original request, without another JEV call.

After the Combo has served requests, open **Models → Combos → jev-auto → Stats** to inspect JEV's
picks without replacing the normal model picker or Usage page. The tab separates backend-reported decision
Expand Down
2 changes: 1 addition & 1 deletion scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,7 @@
"protocol-direct-encoders-stall-disabled.test.ts": "responses",
"protocol-direct-encoders-local-upstream.test.ts": "responses",
"protocol-direct-encoders-messages.test.ts": "responses",
"chat-native-combo.test.ts": "responses",
"chat-native-combo.test.ts": "responses", "jev-initial-effort-wire.test.ts": "responses",
"messages-native-eligibility.test.ts": "responses",
"messages-native-bridge-policy.test.ts": "responses",
"messages-native-oauth-eligibility.test.ts": "responses",
Expand Down
23 changes: 20 additions & 3 deletions src/server/responses/core-combo-native.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ import {
type RequestLogContext,
} from "../request-log";
import { captureRouteStaticPolicy, routeConcreteModel } from "../../router";
import { comboDefaultEffort, concreteComboRequestBody, getCombo } from "../../combos";
import { comboDefaultEffort, concreteComboRequestBody, getCombo, type JevDecision } from "../../combos";
import { supportedLadderFor } from "../effort-policy";
import { applyDroidReasoningDefault } from "../droid-reasoning-default";
import { resolveWireProtocolOverride } from "../adapter-resolve";
Expand Down Expand Up @@ -94,13 +94,15 @@ export interface ComboProtocolLanes {
target: ComboTarget,
targetRoute: RouteResult,
targetSendBudget: TransientSendBudget | undefined,
initialDecision?: Pick<JevDecision, "effort">,
): NativeComboChildPlan | undefined;
}

interface CandidateVerdict {
skip?: ProtocolFeature[];
}

/** Build opt-in native eligibility and child plans; only the initial plan applies a JEV choice. */
export function createComboProtocolLanes(input: {
source: ComboProtocolSource | undefined;
req: Request;
Expand Down Expand Up @@ -203,7 +205,7 @@ export function createComboProtocolLanes(input: {
400, unrepresentableMessage(features), "invalid_request_error", "unsupported_feature",
), "chat");
},
nativeChild: (target, targetRoute, targetSendBudget) => {
nativeChild: (target, targetRoute, targetSendBudget, initialDecision) => {
// Without the request's execution budget a native child could not share its sends, so it
// keeps the bridge rather than opening a tracker of its own.
if (!isRequestExecutionBudget(targetSendBudget)) return undefined;
Expand All @@ -214,7 +216,22 @@ export function createComboProtocolLanes(input: {
provider: targetRoute.provider,
modelId: targetRoute.modelId,
});
applyComboEffort(body, config, comboId, target, targetRoute);
if (initialDecision) {
// A validated JEV choice owns only this first dispatch, including explicit no-effort.
// Do not run ordinary force defaults for null: they require a non-null default.
delete body.service_tier;
delete body.thinking_budget;
delete body.thinking;
const reasoning = body.reasoning;
if (reasoning && typeof reasoning === "object" && !Array.isArray(reasoning)) {
const preserved = { ...reasoning as Rec };
delete preserved.effort;
if (Object.keys(preserved).length > 0) body.reasoning = preserved;
else delete body.reasoning;
}
if (initialDecision.effort === null) delete body.reasoning_effort;
else body.reasoning_effort = initialDecision.effort;
} else applyComboEffort(body, config, comboId, target, targetRoute);
return { route, body, sendBudget: targetSendBudget };
},
};
Expand Down
8 changes: 4 additions & 4 deletions src/server/responses/core-combo.ts
Original file line number Diff line number Diff line change
Expand Up @@ -764,7 +764,7 @@ export async function executeComboResponses(
initialJevDecision ? initialJevDecision.effort : comboDefaultEffort(config, comboId),
initialJevDecision?.effort === null ? [] : targetReasoningEfforts,
combo.reasoningEffortMode,
initialJevDecision !== undefined && initialJevDecision.effort !== null ? "force" : combo.defaultEffortMode,
initialJevDecision ? (initialJevDecision.effort === null ? "fallback" : "force") : combo.defaultEffortMode,
);
if (initialJevDecision) {
delete childBody.service_tier;
Expand Down Expand Up @@ -860,7 +860,7 @@ export async function executeComboResponses(
&& targetEligible(target)
&& !isComboTargetInCooldown(comboId, target),
);
const nativeChild = protocolLanes?.nativeChild(pick.target, targetRoute, targetSendBudget);
const nativeChild = protocolLanes?.nativeChild(pick.target, targetRoute, targetSendBudget, initialJevDecision);
response = nativeChild ? await dispatchNativeComboChild({
source: options.protocolSource!,
plan: nativeChild,
Expand Down Expand Up @@ -909,8 +909,8 @@ export async function executeComboResponses(
onNativePassthroughCancel: callbackGate.onCancel,
onResponseComplete: callbackGate.onResponseComplete,
});
// Native Chat rebuilds its own body and does not apply the initial JEV effort.
restoreOriginalRequestedEffort(childLog, nativeChild ? undefined : initialJevDecision?.effort);
// Both lanes applied the initial choice before provider pins/caps; retain those transitions.
restoreOriginalRequestedEffort(childLog, initialJevDecision?.effort);
} catch (error) {
callbackGate.discard();
if (options.abortSignal?.aborted) {
Expand Down
2 changes: 1 addition & 1 deletion structure/clients/claude-desktop.md
Original file line number Diff line number Diff line change
Expand Up @@ -514,7 +514,7 @@ The account history response can include a [low-confidence effective capacity es

Account quota surfaces use [safe probe diagnostics](../transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](../runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Expand Down
2 changes: 1 addition & 1 deletion structure/dashboard-and-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -497,7 +497,7 @@ The existing dashboard file-client maps include Cline CLI and reuse its committe
Native Chat applies qualifying effort ceilings independently of model pins; pin selection precedes the cap and only pins or cap rewrites enter wire mapping. The [catalog effort contract](catalog.md#ultra-reasoning-level) records the V1/compaction exemptions and caller-preservation boundary.
Pool quota producers and account commands follow the [bounded raw-observation contract](providers/openai-accounts.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. The account history response can include a [low-confidence effective capacity estimate](providers/openai-accounts.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. Account quota surfaces use [safe probe diagnostics](transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.
Dashboard overview polling observes authorization failures independently of stalled or rejected peer requests, cancels remaining child requests after a decisive result, and exposes resource-level deadline failures without rewriting them as authentication failures.
Expand Down
2 changes: 1 addition & 1 deletion structure/data-planes/images.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ Pool quota producers and account commands follow the [bounded raw-observation co

Account quota surfaces use [safe probe diagnostics](../transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](../runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Expand Down
2 changes: 1 addition & 1 deletion structure/ops/service-and-sidecars.md
Original file line number Diff line number Diff line change
Expand Up @@ -332,7 +332,7 @@ Pool quota producers and account commands follow the [bounded raw-observation co

Account quota surfaces use [safe probe diagnostics](../transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](../runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Expand Down
17 changes: 9 additions & 8 deletions structure/providers/jev-decision.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,16 @@ target/effort allowlist stay authoritative. The note reaches TypeSafe with each
applicable decision, so operators must keep secrets and private paths out of it.
An absent note leaves the prior decision payload shape intact.

`src/server/responses/core-combo.ts` computes current eligibility, asks JEV once for the initial pick,
applies the validated effort, and removes caller `service_tier` on the Responses child body. The
opt-in native Chat lane builds a separate body in `src/server/responses/core-combo-native.ts` and
does not apply that JEV effort override. A retryable child failure re-enters the ordinary Combo
fallback loop from the untouched request without another JEV call. Request and attempt
`requestedEffort` labels retain an applied differing JEV effort as `caller->forced`, followed by
later child transitions; native children omit the unapplied override and preserve their own
transitions. Absent or unchanged forced efforts keep the existing label.
`src/server/responses/core-combo.ts` computes current eligibility and asks JEV once for the initial pick.
The validated effort shapes the first Responses child or the separate opt-in native Chat body in
`src/server/responses/core-combo-native.ts`; both remove caller `service_tier` and conflicting effort controls.
Explicit null strips effort using a non-force path, even when the Combo has a forced default; the ordinary
force-default invariant is unchanged. Provider pins, caps and wire normalization still run afterward.
A retryable failure rebuilds later targets from the untouched request under ordinary Combo policy, without
another JEV call. Request and attempt `requestedEffort` labels retain a differing applied JEV effort as
`caller->forced`, followed by child transitions on either lane; null or unchanged effort keeps the existing label.
`src/server/responses/combo-requested-effort.ts` owns this pure label calculation.
`tests/responses/jev-initial-effort-wire.test.ts` captures both lanes across all three decision backends.
Each target may carry an optional non-empty `reasoningEfforts` allowlist. Omission keeps the
backward-compatible all-advertised behavior; a present list is intersected with current capabilities,
and an empty intersection removes that target from the JEV choice map rather than broadening it.
Expand Down
2 changes: 1 addition & 1 deletion structure/providers/xai-grok.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ do not alter xAI discovery, model metadata, or transport behavior.

Native Chat applies qualifying effort ceilings independently of model pins; pin selection precedes the cap and only pins or cap rewrites enter wire mapping. The [catalog effort contract](../catalog.md#ultra-reasoning-level) records the V1/compaction exemptions and caller-preservation boundary.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

The [explicit model-capability contract](../config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior.

Expand Down
2 changes: 1 addition & 1 deletion structure/subagents.md
Original file line number Diff line number Diff line change
Expand Up @@ -506,7 +506,7 @@ The account history response can include a [low-confidence effective capacity es

Account quota surfaces use [safe probe diagnostics](transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Expand Down
6 changes: 3 additions & 3 deletions structure/transports/responses.md
Original file line number Diff line number Diff line change
Expand Up @@ -541,9 +541,9 @@ not `openai-chat` is decided without a copy. An eligible child is dispatched thr
already opened (its opening stays hand-rolled: the ordinal comes from the parent context while the
active attempt and requested effort land on the child context, which `beginInferenceAttempt`
does not express). The child gets the combo's per-target send budget, the client's abort signal
and the turn lease, and the configured combo effort policy mapped onto `reasoning_effort` through `concreteComboRequestBody`.
The initial JEV override changes only the separate Responses body; native labels omit it and keep native child transitions
([JEV Decision Routing](../providers/jev-decision.md)). Its attempt path is native (`[chat, chat]`) and its answer is marked `chat`.
and the turn lease. Ordinary Combo policy maps onto `reasoning_effort` through `concreteComboRequestBody`; the first JEV choice overrides it, including explicit null.
Initial shaping removes conflicting controls and caller `service_tier` before provider pins/caps and wire normalization; labels retain applied transitions ([JEV Decision Routing](../providers/jev-decision.md)).
Later targets use ordinary policy from the original body. Its attempt path is native (`[chat, chat]`) and its answer is marked `chat`.

Send accounting: the combo's hop reservation already booked the target's first send, so the native
child opens no spend tracker; it reports each physical send to the target budget (the first
Expand Down
Loading
Loading