Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions docs-site/src/content/docs/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -512,12 +512,13 @@ target/effort, a coarse decision gate, latency, optional confidence/probability,
Automated tests use mocked TypeSafe responses plus a no-key fail-open smoke; a live TypeSafe decision
requires an operator-supplied key and is not run implicitly.

When JEV's selected effort is applied to the Responses child, request and attempt logs show a
differing effort as `high->xhigh` or `high->low`. Later effort adjustments remain in the chain,
such as `high->xhigh->medium`. An unchanged effort or a target without effort control keeps the
existing label; fallback attempts do not inherit the initial JEV effort. With the opt-in
`protocols.rollout.nativeChatCombos` path, native Chat children currently do not apply JEV's
selected effort; their labels retain the caller's effort and any native child adjustments instead.
JEV's selected effort applies to the first Responses child and to native Chat children on the opt-in
`protocols.rollout.nativeChatCombos` path. It replaces caller effort and removes conflicting thinking
controls and caller `service_tier`; an explicit no-effort choice removes effort even when the Combo has
a forced default. Provider effort pins, caps and wire normalization still apply afterward.
Request and attempt logs show a differing applied effort as `high->xhigh` or `high->low`, retaining later
adjustments such as `high->xhigh->medium`. An unchanged effort or a no-effort choice keeps the existing
label. Later fallback targets use ordinary Combo policy from the original request, without another JEV call.

After the Combo has served requests, open **Models → Combos → jev-auto → Stats** to inspect JEV's
picks without replacing the normal model picker or Usage page. The tab separates backend-reported decision
Expand Down
2 changes: 1 addition & 1 deletion scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,7 @@
"protocol-direct-encoders-stall-disabled.test.ts": "responses",
"protocol-direct-encoders-local-upstream.test.ts": "responses",
"protocol-direct-encoders-messages.test.ts": "responses",
"chat-native-combo.test.ts": "responses",
"chat-native-combo.test.ts": "responses", "jev-initial-effort-wire.test.ts": "responses",
"messages-native-eligibility.test.ts": "responses",
"messages-native-bridge-policy.test.ts": "responses",
"messages-native-oauth-eligibility.test.ts": "responses",
Expand Down
4 changes: 3 additions & 1 deletion src/server/responses/combo-requested-effort.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,7 @@ export function comboRequestedEffortLabel(
const forcedTransition = forcedEffort != null && forcedEffort !== originalRequestedEffort
? `->${forcedEffort}`
: "";
return `${originalRequestedEffort}${forcedTransition}${laterTransitions}`;
const segments = `${originalRequestedEffort}${forcedTransition}${laterTransitions}`.split("->");
// A pin that kept the effort it was given is not a transition; drop the repeated segment.
return segments.filter((segment, index) => index === 0 || segment !== segments[index - 1]).join("->");
}
23 changes: 20 additions & 3 deletions src/server/responses/core-combo-native.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ import {
type RequestLogContext,
} from "../request-log";
import { captureRouteStaticPolicy, routeConcreteModel } from "../../router";
import { comboDefaultEffort, concreteComboRequestBody, getCombo } from "../../combos";
import { comboDefaultEffort, concreteComboRequestBody, getCombo, type JevDecision } from "../../combos";
import { supportedLadderFor } from "../effort-policy";
import { applyDroidReasoningDefault } from "../droid-reasoning-default";
import { resolveWireProtocolOverride } from "../adapter-resolve";
Expand Down Expand Up @@ -94,13 +94,15 @@ export interface ComboProtocolLanes {
target: ComboTarget,
targetRoute: RouteResult,
targetSendBudget: TransientSendBudget | undefined,
initialDecision?: Pick<JevDecision, "effort">,
): NativeComboChildPlan | undefined;
}

interface CandidateVerdict {
skip?: ProtocolFeature[];
}

/** Build opt-in native eligibility and child plans; only the initial plan applies a JEV choice. */
export function createComboProtocolLanes(input: {
source: ComboProtocolSource | undefined;
req: Request;
Expand Down Expand Up @@ -203,7 +205,7 @@ export function createComboProtocolLanes(input: {
400, unrepresentableMessage(features), "invalid_request_error", "unsupported_feature",
), "chat");
},
nativeChild: (target, targetRoute, targetSendBudget) => {
nativeChild: (target, targetRoute, targetSendBudget, initialDecision) => {
// Without the request's execution budget a native child could not share its sends, so it
// keeps the bridge rather than opening a tracker of its own.
if (!isRequestExecutionBudget(targetSendBudget)) return undefined;
Expand All @@ -214,7 +216,22 @@ export function createComboProtocolLanes(input: {
provider: targetRoute.provider,
modelId: targetRoute.modelId,
});
applyComboEffort(body, config, comboId, target, targetRoute);
if (initialDecision) {
// A validated JEV choice owns only this first dispatch, including explicit no-effort.
// Do not run ordinary force defaults for null: they require a non-null default.
delete body.service_tier;
delete body.thinking_budget;
delete body.thinking;
const reasoning = body.reasoning;
if (reasoning && typeof reasoning === "object" && !Array.isArray(reasoning)) {
const preserved = { ...reasoning as Rec };
delete preserved.effort;
if (Object.keys(preserved).length > 0) body.reasoning = preserved;
else delete body.reasoning;
}
if (initialDecision.effort === null) delete body.reasoning_effort;
else body.reasoning_effort = initialDecision.effort;
Comment on lines +232 to +233

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Normalize initial JEV effort on the native wire

When native Chat is enabled and JEV selects an effort that has a provider wire alias—such as ultra, which must become max, or xhigh with reasoningEffortMap: { xhigh: "high" }—this assignment sends the semantic value directly. normalizePinnedChatEffort invokes mapReasoningEffort only after a pin or cap rewrite, while buildOpenAIChatPassthroughRequest deliberately preserves the remaining raw value, so an unpinned JEV choice can reach the provider unsupported and be rejected. Normalize this policy-produced effort before dispatch while retaining the semantic value for logging; the updated structure docs currently promise that wire normalization runs afterward.

AGENTS.md reference: structure/AGENTS.md:L9-L10

Useful? React with 👍 / 👎.

} else applyComboEffort(body, config, comboId, target, targetRoute);
return { route, body, sendBudget: targetSendBudget };
},
};
Expand Down
8 changes: 4 additions & 4 deletions src/server/responses/core-combo.ts
Original file line number Diff line number Diff line change
Expand Up @@ -764,7 +764,7 @@ export async function executeComboResponses(
initialJevDecision ? initialJevDecision.effort : comboDefaultEffort(config, comboId),
initialJevDecision?.effort === null ? [] : targetReasoningEfforts,
combo.reasoningEffortMode,
initialJevDecision !== undefined && initialJevDecision.effort !== null ? "force" : combo.defaultEffortMode,
initialJevDecision ? (initialJevDecision.effort === null ? "fallback" : "force") : combo.defaultEffortMode,
);
if (initialJevDecision) {
delete childBody.service_tier;
Expand Down Expand Up @@ -860,7 +860,7 @@ export async function executeComboResponses(
&& targetEligible(target)
&& !isComboTargetInCooldown(comboId, target),
);
const nativeChild = protocolLanes?.nativeChild(pick.target, targetRoute, targetSendBudget);
const nativeChild = protocolLanes?.nativeChild(pick.target, targetRoute, targetSendBudget, initialJevDecision);
response = nativeChild ? await dispatchNativeComboChild({
source: options.protocolSource!,
plan: nativeChild,
Expand Down Expand Up @@ -909,8 +909,8 @@ export async function executeComboResponses(
onNativePassthroughCancel: callbackGate.onCancel,
onResponseComplete: callbackGate.onResponseComplete,
});
// Native Chat rebuilds its own body and does not apply the initial JEV effort.
restoreOriginalRequestedEffort(childLog, nativeChild ? undefined : initialJevDecision?.effort);
// Both lanes applied the initial choice before provider pins/caps; retain those transitions.
restoreOriginalRequestedEffort(childLog, initialJevDecision?.effort);
Comment on lines +912 to +913

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve a pin transition after a null JEV choice

For a native request where JEV explicitly selects null and a downstream provider pin then supplies a different effort, this restoration drops the pin from both request and attempt labels. For example, caller low plus JEV null plus provider pin high sends high, but comboRequestedEffortLabel("low", "high", null) returns only low because a child label without -> is discarded when the forced effort is null. Record the newly introduced pinned base as a transition so the logs describe the actual wire behavior promised by the updated structure contract.

AGENTS.md reference: structure/AGENTS.md:L9-L10

Useful? React with 👍 / 👎.

} catch (error) {
callbackGate.discard();
if (options.abortSignal?.aborted) {
Expand Down
2 changes: 1 addition & 1 deletion structure/clients/claude-desktop.md
Original file line number Diff line number Diff line change
Expand Up @@ -514,7 +514,7 @@ The account history response can include a [low-confidence effective capacity es

Account quota surfaces use [safe probe diagnostics](../transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](../runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Expand Down
2 changes: 1 addition & 1 deletion structure/dashboard-and-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -497,7 +497,7 @@ The existing dashboard file-client maps include Cline CLI and reuse its committe
Native Chat applies qualifying effort ceilings independently of model pins; pin selection precedes the cap and only pins or cap rewrites enter wire mapping. The [catalog effort contract](catalog.md#ultra-reasoning-level) records the V1/compaction exemptions and caller-preservation boundary.
Pool quota producers and account commands follow the [bounded raw-observation contract](providers/openai-accounts.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. The account history response can include a [low-confidence effective capacity estimate](providers/openai-accounts.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. Account quota surfaces use [safe probe diagnostics](transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.
Dashboard overview polling observes authorization failures independently of stalled or rejected peer requests, cancels remaining child requests after a decisive result, and exposes resource-level deadline failures without rewriting them as authentication failures.
Expand Down
2 changes: 1 addition & 1 deletion structure/data-planes/images.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ Pool quota producers and account commands follow the [bounded raw-observation co

Account quota surfaces use [safe probe diagnostics](../transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](../runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Expand Down
2 changes: 1 addition & 1 deletion structure/ops/service-and-sidecars.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ Pool quota producers and account commands follow the [bounded raw-observation co

Account quota surfaces use [safe probe diagnostics](../transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](../runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Expand Down
17 changes: 9 additions & 8 deletions structure/providers/jev-decision.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,16 @@ target/effort allowlist stay authoritative. The note reaches TypeSafe with each
applicable decision, so operators must keep secrets and private paths out of it.
An absent note leaves the prior decision payload shape intact.

`src/server/responses/core-combo.ts` computes current eligibility, asks JEV once for the initial pick,
applies the validated effort, and removes caller `service_tier` on the Responses child body. The
opt-in native Chat lane builds a separate body in `src/server/responses/core-combo-native.ts` and
does not apply that JEV effort override. A retryable child failure re-enters the ordinary Combo
fallback loop from the untouched request without another JEV call. Request and attempt
`requestedEffort` labels retain an applied differing JEV effort as `caller->forced`, followed by
later child transitions; native children omit the unapplied override and preserve their own
transitions. Absent or unchanged forced efforts keep the existing label.
`src/server/responses/core-combo.ts` computes current eligibility and asks JEV once for the initial pick.
The validated effort shapes the first Responses child or the separate opt-in native Chat body in
`src/server/responses/core-combo-native.ts`; both remove caller `service_tier` and conflicting effort controls.
Explicit null strips effort using a non-force path, even when the Combo has a forced default; the ordinary
force-default invariant is unchanged. Provider pins, caps and wire normalization still run afterward.
A retryable failure rebuilds later targets from the untouched request under ordinary Combo policy, without
another JEV call. Request and attempt `requestedEffort` labels retain a differing applied JEV effort as
`caller->forced`, followed by child transitions on either lane; null or unchanged effort keeps the existing label.
`src/server/responses/combo-requested-effort.ts` owns this pure label calculation.
`tests/responses/jev-initial-effort-wire.test.ts` captures both lanes across all three decision backends.
Each target may carry an optional non-empty `reasoningEfforts` allowlist. Omission keeps the
backward-compatible all-advertised behavior; a present list is intersected with current capabilities,
and an empty intersection removes that target from the JEV choice map rather than broadening it.
Expand Down
2 changes: 1 addition & 1 deletion structure/providers/xai-grok.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ do not alter xAI discovery, model metadata, or transport behavior.

Native Chat applies qualifying effort ceilings independently of model pins; pin selection precedes the cap and only pins or cap rewrites enter wire mapping. The [catalog effort contract](../catalog.md#ultra-reasoning-level) records the V1/compaction exemptions and caller-preservation boundary.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](../transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

The [explicit model-capability contract](../config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior.

Expand Down
2 changes: 1 addition & 1 deletion structure/subagents.md
Original file line number Diff line number Diff line change
Expand Up @@ -503,7 +503,7 @@ The account history response can include a [low-confidence effective capacity es

Account quota surfaces use [safe probe diagnostics](transports/inventory.md#account-quota-failure-diagnostics) separately from quota validity, credential health and routing authority.

Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](transports/responses.md) documents this boundary, and native Chat removes effort only for an explicit empty declaration or no-reasoning model.
Combo child requests normalize effort and thinking controls against the selected target while retaining reasoning summaries; strict unknown targets preserve caller controls. The [Responses transport owner](transports/responses.md) documents this boundary, and native Chat capability stripping applies only for an explicit empty declaration or no-reasoning model; an initial JEV null choice separately strips caller effort.

Live sideband admission and its bounded upstream handshake follow the [runtime contract](runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Expand Down
6 changes: 3 additions & 3 deletions structure/transports/responses.md
Original file line number Diff line number Diff line change
Expand Up @@ -541,9 +541,9 @@ not `openai-chat` is decided without a copy. An eligible child is dispatched thr
already opened (its opening stays hand-rolled: the ordinal comes from the parent context while the
active attempt and requested effort land on the child context, which `beginInferenceAttempt`
does not express). The child gets the combo's per-target send budget, the client's abort signal
and the turn lease, and the configured combo effort policy mapped onto `reasoning_effort` through `concreteComboRequestBody`.
The initial JEV override changes only the separate Responses body; native labels omit it and keep native child transitions
([JEV Decision Routing](../providers/jev-decision.md)). Its attempt path is native (`[chat, chat]`) and its answer is marked `chat`.
and the turn lease. Ordinary Combo policy maps onto `reasoning_effort` through `concreteComboRequestBody`; the first JEV choice overrides it, including explicit null.
Initial shaping removes conflicting controls and caller `service_tier` before provider pins/caps and wire normalization; labels retain applied transitions ([JEV Decision Routing](../providers/jev-decision.md)).
Later targets use ordinary policy from the original body. Its attempt path is native (`[chat, chat]`) and its answer is marked `chat`.

Send accounting: the combo's hop reservation already booked the target's first send, so the native
child opens no spend tracker; it reports each physical send to the target budget (the first
Expand Down
Loading
Loading