[adivekar-utexas]

PR: Nested Combo Execution (nestedComboMode) -- Stage 1

Summary

Add nestedComboMode: "flatten" | "execute" (default "flatten") as a per-combo config toggle. In "execute" mode, combo-ref members stay as opaque black-box runtime targets: the parent strategy selects the child combo as one unit, and the child runs its own strategy, retries, sticky, and validation via a recursive handleComboChat call. This fixes the flattening problem where nested combo boundaries were lost and inner strategy/retry/sticky behavior never executed.

Fixes: Cache locality loss and cache-control marker stripping in round-robin-over-priority topologies (see the detailed design doc at docs/routing/NESTED-COMBO-EXECUTION.md).

Changes

Core Runtime

• comboConfig.ts: nestedComboMode: "flatten" in DEFAULT_COMBO_CONFIG.
• types.ts: NestedComboMode, ComboNestingContext, ResolvedComboUnit, ResolvedComboRefTarget, ComboRuntimeStep alias, effectiveComboStrategy on SingleModelTarget, nesting on HandleComboChatOptions.
• comboStructure.ts: resolveComboRuntimeUnits -- mode-aware resolver that delegates to legacy flattening in flatten mode and returns top-level runtime steps (including combo-ref units) in execute mode. validateComboDAG reused for static cycle detection.
• runtimeUnits.ts: executeRuntimeUnitCombo (per-unit loop with attempt-budget enforcement, retries, metrics), executeRuntimeUnit (dispatch by kind), executeModelUnit, executeComboRefUnit (recursive handleComboChat call with depth/cycle guard), buildChildNestingContext.
• combo.ts (dispatch block ~551-671): Nesting context construction, normalizeNestedComboMode, resolveComboRuntimeUnits, simpleExecuteStrategies set, strategy-specific unit ordering (sticky rotation, weighted sticky, fisher-yates shuffle, deck draw), executeRuntimeUnitCombo call, post-success sticky recording. effectiveComboStrategy threaded at non-RR leaf call (~1495) and round-robin handler (~2421).
• rrState.ts: getStickyRoundRobinStartIndex and recordStickyRoundRobinSuccess retyped from ResolvedComboTarget to ResolvedComboUnit so sticky works over combo-ref units.

Schema & Integration

• schema/combo.ts: nestedComboMode: z.enum(["flatten", "execute"]).optional() in comboRuntimeConfigSchema.
• chat.ts: Both handleSingleModel closures (main combo path ~600-645, auto-combo redirect ~796-829) read target?.effectiveComboStrategy ?? combo.strategy and pass it as comboStrategy to handleSingleModelChat.
• page.tsx: UI select bound to config.nestedComboMode (~3867) with help text (~172). sanitizeComboRuntimeConfig passes through non-null keys, so no save-path change is needed.

Tests

• combo-config.test.ts: 3 new tests (default, schema accept/reject, cascade).
• combo-strategy-fallbacks.test.ts: 5 new tests (black-box priority, parent fallback, 3-level nesting, cycle detection, flatten unchanged default).

Test Results

• All 35 combo-config tests pass.
• 22/24 strategy-fallback tests pass (including all 5 new nested-execute tests).
• 2 failures are pre-existing flaky weighted sticky limit tests from the base feat/combo-sticky-weighted-clean commit (fail identically on the base commit and are unrelated to this work).

Backward Compatibility

• nestedComboMode defaults to "flatten" in DEFAULT_COMBO_CONFIG.
• Existing combos without this key resolve to "flatten" and follow the legacy path unchanged.
• Execute mode only engages when ALL three conditions hold: nestedComboMode === "execute", the strategy is in simpleExecuteStrategies, AND the unit list contains at least one combo-ref.
• effectiveComboStrategy is optional and falls back to combo.strategy.
• Legacy flattening resolvers and strategy loops are untouched.

Notes

• This branch stacks on feat/combo-sticky-weighted-clean (requires weighted-sticky helpers from that PR).
• A detailed design doc is at docs/routing/NESTED-COMBO-EXECUTION.md.

Closes #4536

Made with Cursor

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[diegosouzapw]

Update: #4489 (sticky-weighted) is now merged into release/v3.8.33, so the path to land this is clear. Right now #4537 is still stacked on the full sticky-weighted change (commits da31fc4db/4b72342d8/ebf96043d are #4489's), and ebf96043d mixes sticky-weighted + nested-dispatch in one commit.

Could you rebase onto the current release/v3.8.33 tip? The sticky-weighted commits are now upstream, so the rebase should drop them and leave only the nestedComboMode delta. You'll likely need to split ebf96043d so the nested-dispatch lines land without the sticky-weighted ones. Once the diff is nested-combo-only and CI is green, I'll merge it — the recursion-safety caps (depth/cycle/budget) are a solid design. 🙏

[diegosouzapw]

Thanks @adivekar-utexas! Now that #4489 (sticky-weighted) is merged, I rebased this onto release/v3.8.33 and resolved the overlap. The nestedComboMode: execute path — black-box combo-ref execution via resolveComboRuntimeUnits with the recursion-safety caps (depth/cycle/budget) — is preserved and layered on top of release's reconciled sticky-weighted + provider-wildcard (#4545) code; I dropped the branch's pre-#4489 sticky-weighted duplicate and the old Math.random test mocks (release uses the secureRandom source). Two nested-execute sites read stickyWeightedLimit via the (config as Record<string, unknown>) cast (mirrors stickyRoundRobinLimit) since the combo-config type tightened. 59/59 combo tests pass on the merged tip, typecheck:core is clean, baselines reconciled. Merging into release/v3.8.33. 🙏