feat(combo): add sticky weighted selection limit with exhaustion-aware renormalization

config/quality/file-size-baseline.json

@@ -9,6 +9,7 @@
   "_rebaseline_2026_06_21_4427_low_noise_catalog": "PR #4427 (Rahulsharma0810) own growth: src/app/api/v1/models/catalog.ts 1478->1486 (+8). The opt-in MODELS_CATALOG_PREFIX_MODE (dual default | alias | canonical, with ?prefix= per-request override) gates the dual alias+canonical model emission at the three /v1/models push sites (static, synced, custom) behind includeAlias/includeCanonical, suppressing the duplicate cross-prefix entries (net +3 from the feature). On review, 4 incidental explanatory comments removed by the PR were restored (synced-models resolve, skip-static, try-block intent, strip-modelIdPrefix; +5) since their code is unchanged — useful docs on a non-trivial catalog function. Default `dual` keeps byte-identical output; request-side alias resolution unchanged. Structural shrink of this route tracked in #3789. Covered by tests/unit/models-catalog-low-noise-flag.test.ts.",
   "_rebaseline_2026_06_21_phase2_active_selector": "Compression Phase 2 (named profiles + active selector) own growth: chatCore.ts 5110->5125 (+15 at the existing compression-dispatch chokepoint, just before the selectCompressionStrategy call). chatCore now loads the operator's named compression combos once into a `namedCombos` map (a best-effort try/catch dynamic-import of listCompressionCombos, debug-logged on failure) and passes it as the new `combos` arg to selectCompressionStrategy + selectCompressionPlan, plus an `&& !activeComboResolves(config, namedCombos)` guard term on the legacy default-combo block so the seeded default cannot shadow the operator's active profile. The resolver itself stays pure (the `combos` threading + activeComboResolves live in open-sse/services/compression/strategySelector.ts, <cap). The two UI files net SHRINK or stay flat: CompressionHub.tsx 521->277 (master toggle/mode/reorder removed, replaced by the thin active-profile selector + preview) and CompressionCombosPageClient.tsx grows a few lines under its frozen for the active badge. Cohesive wiring at the existing compression chokepoint, mirroring the prior compression rebaselines (#4217/#4210/#3890); not extractable without hiding the dispatch boundary. Structural shrink of chatCore.ts tracked in #3501. Covered by tests/unit/compression/active-combo-dispatch.test.ts + active-combo-integration.test.ts + tests/unit/ui/compressionHub-active-selector.test.tsx + namedCombos-active-badge.test.tsx.",
   "_rebaseline_2026_06_21_qg9_chatcore_service_tier": "QG v2 Fase 9 T5 (#3501) — chatCore.ts 5137->5110 (shrink -27). The two inline Codex service-tier resolvers (resolveEffectiveServiceTier / resolveReportedServiceTier, ~36 LOC) were extracted byte-identically into the new pure leaf open-sse/handlers/chatCore/serviceTier.ts; the handler now keeps a `let effectiveServiceTier` + two thin binding closures that pass provider/credentials?.providerSpecificData to the extracted functions, so every call site stays unchanged. The orphaned getCodexRequestDefaults/normalizeCodexServiceTier/CodexServiceTier imports moved to the leaf. Ratcheted the frozen value down to lock the freed budget. Covered by tests/unit/chatcore-service-tier.test.ts.",
+  "_rebaseline_2026_06_21_4489_sticky_weighted_limit": "PR #4489 (adivekar-utexas) own growth: src/app/(dashboard)/dashboard/combos/page.tsx 4385->4386 (+1 = the new Sticky Weighted Limit input in the weighted advanced section, a single FieldLabelWithHelp + number <input> block gated to strategy===weighted, analogous to the existing stickyRoundRobinLimit field for round-robin) and open-sse/services/combo.ts ->2761 (sticky-weighted feature: weightedStickyTargets state wiring + the isTargetSelectableForWeighted availability pre-filter + exhaustion-aware sticky clearing/migration in handleComboChat + the round-robin sticky pre-filter in handleRoundRobinCombo + the 4 gemini-review fixes — provider guard, startsWith-separator key match, stepGroups dedup, stale-sticky cleanup). Frozen to the merge-tree measurement (release/v3.8.33 base 2649 + the feature) since CI measures the merged tree, not the v3.8.32-based branch tip. Cohesive routing logic at the existing combo dispatch chokepoints; not a movable block. Structural shrink of combo.ts tracked in #3501. Covered by tests/unit/combo-strategy-fallbacks.test.ts (sticky-weighted batching/fallback-migration/stale-clear/nested-availability + RR sticky-clear) and tests/unit/combo-config.test.ts (stickyWeightedLimit schema).",
   "_rebaseline_2026_06_20_reviewprs_mine_r2_filesize": "Reconciliacao file-size pos-lote /review-prs 'apenas minhas' r2: dois frozen cresceram cumulativamente sem bump (cada PR media OK na sua base, mas o crescimento empilhou acima do frozen no tip de merge; o fast-path do release nao roda check:file-size, so release->main). (1) src/shared/constants/pricing.ts 1620->1623 (+3 = linhas de pricing Claude Code (cc) do #4440, sobre o 1620 que o #4447 ja setara para gpt-4.1-mini/nano + o3/o4-mini). (2) open-sse/executors/base.ts 1399->1407 (+8 = handling granular de reasoning_effort para Claude no Copilot do #4443). Ambos dados/wiring coesos nos chokepoints existentes; nao extraiveis. Cobertos por tests/unit (claude-code pricing / base-executor-sanitize-effort + github-claude-reasoning-effort-granular).",
   "_rebaseline_2026_06_20_4023_web_cookie_noauth_validation": "PR #4023 (oyi77) own growth: src/lib/providers/validation.ts 4450->4518 (+68 = a new validateWebCookieProvider that probes the provider's /models endpoint — 401/403 => AUTH_007 SESSION_EXPIRED, any other status => valid session, empty cookie => invalid, provider-not-in-registry => unsupported — plus a local STANDARD_USER_AGENT const for the probe). Cohesive validator at the validateProviderApiKey dispatch; not extractable. Covered by tests/unit/provider-validation-web-cookie-auth007.test.ts. Heavily curated on merge — the PR's branch was badly stale-based (squash-base-stale), so its tree was DESTRUCTIVE: providers/index.ts deleted live providers openadapter/dit/tokenrouter (added by #4313) and the executor/base.ts edits reverted release fixes (#4037 duckduckgo host, theoldllm gpt5 models, base.ts fetch-start-timeout). Only the purely-additive validation feature was kept (validation.ts validateWebCookieProvider + errorCodes AUTH_007 + the test). Dropped: 5 malformed new registry entries (used non-RegistryEntry fields defaultModel/auth + referenced non-existent executors -> tsc TS2353), the destructive providers/index.ts + executor reverts, the unrelated pr-*.sh automation scripts, and evals/types.ts (belongs to the deferred evals modularization #4422). Also removed the PR's fragile 'Phase 2' executor probe (ran a live upstream chat during validation + classified any 'auth'-containing error as SESSION_EXPIRED) and rewrote the test to install its fetch mock before module load (the original mocked too late and silently hit live chatgpt.com).",
   "_rebaseline_2026_06_20_1308_model_lockout_honors_reset": "port from 9router#1308 own growth: open-sse/services/accountFallback.ts 1731->1752 (+21 = the new exported pure helper selectLockoutCooldownMs + its doc comment — picks the parsed upstream reset as the model-lockout exactCooldownMs when it exceeds the base cooldown, e.g. Antigravity \"Resets in 160h\", else preserves the existing 0/base behavior) and open-sse/executors/antigravity.ts 1680->1686 (this PR +1 = parseRetryFromErrorMessage regex `reset` -> `resets?` so plural \"Resets in 160h27m24s\" matches, plus a comment line; frozen set to the SUM 1686 with the concurrent #1944 which adds +5 at the disjoint passthroughFields region of the same file, so either merge order passes — pair-file rule). The combo lockout call sites in combo.ts now pass selectLockoutCooldownMs(cooldownMs, mlSettings) instead of always base/exponential, so an exhausted model honors the real upstream reset instead of being retried within minutes. Both edits are cohesive at the existing lockout/parse chokepoints; the helper is its own pure function (not extractable further). Covered by tests/unit/combo-model-lockout-honors-reset-1308.test.ts.",
@@ -129,7 +130,7 @@
     "open-sse/services/batchProcessor.ts": 828,
     "open-sse/services/browserBackedChat.ts": 850,
     "open-sse/services/claudeCodeCompatible.ts": 1202,
-    "open-sse/services/combo.ts": 2701,
+    "open-sse/services/combo.ts": 2818,
     "open-sse/services/rateLimitManager.ts": 1035,
     "open-sse/services/tokenRefresh.ts": 1997,
     "open-sse/services/usage.ts": 3450,
@@ -144,7 +145,7 @@
     "src/app/(dashboard)/dashboard/cache/page.tsx": 841,
     "src/app/(dashboard)/dashboard/cli-code/components/CodexToolCard.tsx": 900,
     "src/app/(dashboard)/dashboard/cloud-agents/page.tsx": 922,
-    "src/app/(dashboard)/dashboard/combos/page.tsx": 4385,
+    "src/app/(dashboard)/dashboard/combos/page.tsx": 4406,
     "src/app/(dashboard)/dashboard/costs/CostOverviewTab.tsx": 1495,
     "src/app/(dashboard)/dashboard/costs/quota-share/components/PoolWizard.tsx": 1007,
     "src/app/(dashboard)/dashboard/endpoint/EndpointPageClient.tsx": 2570,

open-sse/services/combo.ts

@@ -99,16 +99,21 @@ import type {
   HandleRoundRobinOptions,
   ResolvedComboTarget,
   AutoProviderCandidate,
+  ComboRuntimeStep,
   HistoricalLatencyStatsEntry,
 } from "./combo/types.ts";
 
 import {
   MAX_RR_COUNTERS,
   rrCounters,
   rrStickyTargets,
+  weightedStickyTargets,
   clampStickyRoundRobinTargetLimit,
+  clampStickyWeightedTargetLimit,
   getStickyRoundRobinStartIndex,
   recordStickyRoundRobinSuccess,
+  getStickyWeightedExecutionKey,
+  recordStickyWeightedSuccess,
 } from "./combo/rrState.ts";
 import { validateResponseQuality, toRetryAfterDisplayValue } from "./combo/validateQuality.ts";
 import {
@@ -143,6 +148,7 @@ import {
   getModelContextLimitForModelString,
   resolveComboTargets,
   resolveWeightedTargets,
+  resolveWeightedStepGroups,
   sortTargetsByContextSize,
 } from "./combo/comboStructure.ts";
 import {
@@ -697,10 +703,31 @@ export async function handleComboChat({
   const maxSetRetries = config.maxSetRetries ?? 0;
   const setRetryDelayMs = resolveDelayMs(config.setRetryDelayMs, 2000);
 
+  const isTargetSelectableForWeighted = async (target: ResolvedComboTarget): Promise<boolean> => {
+    const rawModel = parseModel(target.modelStr).model || target.modelStr;
+    if (target.provider && getCircuitBreaker(target.provider).getStatus().state === "OPEN") return false;
+    if (
+      resilienceSettings.providerCooldown.enabled &&
+      Boolean(target.provider && target.provider !== "unknown") &&
+      isProviderInCooldown(target.provider, target.connectionId ?? undefined, resilienceSettings)
+    ) {
+      return false;
+    }
+    if (
+      target.provider &&
+      rawModel &&
+      isModelLocked(target.provider, target.connectionId || "", rawModel)
+    ) {
+      return false;
+    }
+    return isModelAvailable ? await isModelAvailable(target.modelStr, target) : true;
+  };
+
   // #2562: Expand provider-wildcard steps (e.g. `fta/*`, `openai/gpt-4*`) into
   // concrete model entries sourced from the live synced-models catalog + registry.
-  // Must run before resolveComboTargets so that wildcard-originated steps are
-  // treated identically to hand-authored entries by all downstream logic.
+  // Must run before any step-group / target resolution so that wildcard-originated
+  // steps are treated identically to hand-authored entries by all downstream logic
+  // (including the sticky-weighted eligibility pass below).
   const expandedCombo = await expandProviderWildcardsInCombo(combo);
   const expandedAllCombos = allCombos
     ? Array.isArray(allCombos)
@@ -713,21 +740,66 @@ export async function handleComboChat({
         }
     : allCombos;
 
-  let orderedTargets =
+  const stickyWeightedLimit = clampStickyWeightedTargetLimit(
+    (config as Record<string, unknown>).stickyWeightedLimit
+  );
+  if (
+    strategy === "weighted" &&
+    !weightedStickyTargets.has(combo.name) &&
+    weightedStickyTargets.size >= MAX_RR_COUNTERS
+  ) {
+    const oldest = weightedStickyTargets.keys().next().value;
+    if (oldest !== undefined) weightedStickyTargets.delete(oldest);
+  }
+  let stepGroups: Array<{ step: ComboRuntimeStep; targets: ResolvedComboTarget[] }> | undefined;
+  const weightedEligibleKeys = new Set<string>();
+  if (strategy === "weighted") {
+    stepGroups = resolveWeightedStepGroups(expandedCombo, expandedAllCombos);
+    for (const group of stepGroups) {
+      const availability = await Promise.all(group.targets.map(isTargetSelectableForWeighted));
+      if (availability.some(Boolean)) weightedEligibleKeys.add(group.step.executionKey);
+    }
+  }
+  const rawStickyWeightedKey =
+    strategy === "weighted" ? getStickyWeightedExecutionKey(combo.name, stickyWeightedLimit) : null;
+  const stickyWeightedKey =
+    rawStickyWeightedKey && weightedEligibleKeys.has(rawStickyWeightedKey)
+      ? rawStickyWeightedKey
+      : null;
+  if (strategy !== "weighted" || stickyWeightedLimit <= 1) {
+    weightedStickyTargets.delete(combo.name);
+  } else if (rawStickyWeightedKey && !stickyWeightedKey) {
+    weightedStickyTargets.delete(combo.name);
+  }
+  const weightedResolution =
     strategy === "weighted"
-      ? resolveWeightedTargets(expandedCombo, expandedAllCombos)?.orderedTargets || []
-      : resolveComboTargets(
+      ? resolveWeightedTargets(
           expandedCombo,
           expandedAllCombos,
-          clampComboDepth(config.maxComboDepth)
-        );
+          stickyWeightedKey,
+          weightedEligibleKeys,
+          stepGroups
+        )
+      : null;
+  const getWeightedStepKeyForTarget = (target: ResolvedComboTarget): string | null => {
+    if (!weightedResolution?.orderedSteps) return null;
+    const step = weightedResolution.orderedSteps.find((entry) =>
+      target.executionKey === entry.executionKey ||
+      target.executionKey.startsWith(entry.executionKey + ">")
+    );
+    return step?.executionKey || null;
+  };
+  let orderedTargets =
+    strategy === "weighted"
+      ? weightedResolution?.orderedTargets || []
+      : resolveComboTargets(expandedCombo, expandedAllCombos, clampComboDepth(config.maxComboDepth));
 
   orderedTargets = await applyRequestTagRouting(orderedTargets, body, log);
 
   if (strategy === "weighted") {
     log.info(
       "COMBO",
-      `Weighted selection${allCombos ? " with nested resolution" : ""}: ${orderedTargets.length} total targets`
+      `Weighted selection${stickyWeightedKey ? " (sticky)" : ""}${allCombos ? " with nested resolution" : ""}: ${orderedTargets.length} total targets`
     );
   } else if (allCombos) {
     log.info("COMBO", `${strategy} with nested resolution: ${orderedTargets.length} total targets`);
@@ -1575,6 +1647,12 @@ export async function handleComboChat({
             if (provider && provider !== "unknown") {
               recordProviderSuccess(provider, target.connectionId ?? undefined);
             }
+            if (strategy === "weighted" && stickyWeightedLimit > 1) {
+              const stickySuccessKey = getWeightedStepKeyForTarget(target);
+              if (stickySuccessKey) {
+                recordStickyWeightedSuccess(combo.name, stickySuccessKey, stickyWeightedLimit);
+              }
+            }
             // Webhook fan-out: best-effort, never blocks the response stream.
             notifyWebhookEvent("request.completed", {
               combo: combo.name,
@@ -2216,6 +2294,45 @@ async function handleRoundRobinCombo({
       : (settings as Record<string, unknown> | null)?.stickyRoundRobinLimit
   );
   const stickyRoundRobinEnabled = stickyLimit > 1;
+  // Exhaustion-aware sticky: if the currently sticky target is no longer
+  // available (circuit breaker OPEN, provider cooldown, model lockout, or
+  // isModelAvailable returns false), clear the sticky record so the rotation
+  // starts at the counter position instead of probing a dead target.
+  if (stickyRoundRobinEnabled) {
+    const sticky = rrStickyTargets.get(combo.name);
+    if (sticky) {
+      const stickyTarget = filteredTargets.find(
+        (target) => target.executionKey === sticky.executionKey
+      );
+      if (stickyTarget) {
+        const rawModel = parseModel(stickyTarget.modelStr).model || stickyTarget.modelStr;
+        const stickyAvailable =
+          (!stickyTarget.provider || getCircuitBreaker(stickyTarget.provider).getStatus().state !== "OPEN") &&
+          !(
+            resilienceSettings.providerCooldown.enabled &&
+            Boolean(stickyTarget.provider && stickyTarget.provider !== "unknown") &&
+            isProviderInCooldown(
+              stickyTarget.provider,
+              stickyTarget.connectionId ?? undefined,
+              resilienceSettings
+            )
+          ) &&
+          !(
+            stickyTarget.provider &&
+            rawModel &&
+            isModelLocked(stickyTarget.provider, stickyTarget.connectionId || "", rawModel)
+          ) &&
+          (isModelAvailable ? await isModelAvailable(stickyTarget.modelStr, stickyTarget) : true);
+        if (!stickyAvailable) {
+          log.info(
+            "COMBO-RR",
+            `Clearing stale sticky target ${stickyTarget.modelStr} — unavailable`
+          );
+          rrStickyTargets.delete(combo.name);
+        }
+      }
+    }
+  }
   if (
     !rrCounters.has(combo.name) &&
     !rrStickyTargets.has(combo.name) &&

open-sse/services/combo/comboStructure.ts

@@ -607,38 +607,66 @@ export function resolveComboTargets(
     : getDirectComboTargets(combo);
 }
 
-export function resolveWeightedTargets(
+export function resolveWeightedStepGroups(
   combo: ComboLike,
   allCombos: ComboCollectionLike
+): Array<{ step: ComboRuntimeStep; targets: ResolvedComboTarget[] }> {
+  return getOrderedTopLevelRuntimeSteps(combo, allCombos)
+    .map((step) => ({
+      step,
+      targets: !allCombos
+        ? step.kind === "model"
+          ? [step]
+          : []
+        : expandRuntimeStep(step, allCombos, new Set([combo.name])),
+    }))
+    .filter((group) => group.targets.length > 0);
+}
+
+export function resolveWeightedTargets(
+  combo: ComboLike,
+  allCombos: ComboCollectionLike,
+  preferredExecutionKey: string | null = null,
+  eligibleExecutionKeys: ReadonlySet<string> | null = null,
+  stepGroups?: Array<{ step: ComboRuntimeStep; targets: ResolvedComboTarget[] }>
 ): {
   orderedTargets: ResolvedComboTarget[];
   selectedStep: ComboRuntimeStep | null;
+  orderedSteps: ComboRuntimeStep[];
 } {
-  const topLevelSteps = getOrderedTopLevelRuntimeSteps(combo, allCombos);
+  const topLevelSteps = getOrderedTopLevelRuntimeSteps(combo, allCombos).filter((step) =>
+    eligibleExecutionKeys ? eligibleExecutionKeys.has(step.executionKey) : true
+  );
   if (topLevelSteps.length === 0) {
-    return { orderedTargets: [], selectedStep: null };
+    return { orderedTargets: [], selectedStep: null, orderedSteps: [] };
   }
 
-  const selectedStep = selectWeightedTarget(topLevelSteps);
+  const preferredStep = preferredExecutionKey
+    ? topLevelSteps.find((step) => step.executionKey === preferredExecutionKey) || null
+    : null;
+  const selectedStep = preferredStep || selectWeightedTarget(topLevelSteps);
   if (!selectedStep) {
-    return { orderedTargets: [], selectedStep: null };
+    return { orderedTargets: [], selectedStep: null, orderedSteps: [] };
   }
 
   const orderedSteps = orderTargetsForWeightedFallback(
     topLevelSteps,
     selectedStep.executionKey,
     hasCompositeTierRuntimeOrder(combo)
   );
-  const expandedTargets = orderedSteps.flatMap((step) => {
-    if (!step) return [];
-    if (!allCombos) {
-      return step.kind === "model" ? [step] : [];
-    }
-    return expandRuntimeStep(step, allCombos, new Set([combo.name]));
-  });
+  const targetsByStep = new Map(
+    (stepGroups || resolveWeightedStepGroups(combo, allCombos)).map((group) => [
+      group.step.executionKey,
+      group.targets,
+    ])
+  );
+  const expandedTargets = orderedSteps.flatMap(
+    (step) => targetsByStep.get(step.executionKey) || []
+  );
 
   return {
     orderedTargets: dedupeTargetsByExecutionKey(expandedTargets),
     selectedStep,
+    orderedSteps,
   };
 }