Add expert combo configuration mode - #1547
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an opt-in “Expert” combo configuration mode (defaulting to Guided) that reuses the existing combo builder UI as a single-page editor, supports manual provider-qualified model entry, and adjusts combo live-test behavior to avoid cooldown-aware retry fan-out.
Changes:
- Introduces
comboConfigModesetting (guided/expert) with validation, defaults, and UI control in Appearance settings. - Updates the combos create/edit modal to support an expert single-page layout and manual model entry (with provider alias resolution).
- Disables cooldown-aware request retry for combo live tests and aligns isolated Next build bundler selection.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/settings-schema-routing-strategies.test.ts | Adds coverage for comboConfigMode schema validation. |
| tests/unit/db-settings-crud.test.ts | Asserts comboConfigMode default is exposed via settings DB API. |
| tests/unit/combo-builder-draft.test.ts | Adds unit tests for manual model parsing + provider alias resolution. |
| tests/unit/chat-combo-live-test.test.ts | Verifies combo live tests don’t perform cooldown-aware retries on upstream 503s. |
| tests/e2e/combos-flow.spec.ts | Adds E2E coverage for expert mode single-page form + manual model entry. |
| src/sse/handlers/chat.ts | Disables cooldown-aware retry when forceLiveComboTest is active. |
| src/shared/validation/settingsSchemas.ts | Allows comboConfigMode in settings route schema. |
| src/shared/validation/schemas.ts | Allows comboConfigMode in shared settings schema. |
| src/shared/constants/comboConfigMode.ts | Adds constants + normalization helper for combo config mode. |
| src/lib/db/settings.ts | Adds comboConfigMode: "guided" to defaults. |
| src/lib/dataPaths.js | Removes stale compiled artifact. |
| src/lib/combos/builderDraft.ts | Adds manual model step builder + provider-id resolution helper. |
| src/app/api/settings/tests/settings.test.ts | Adds API test ensuring PATCH updates comboConfigMode. |
| src/app/(dashboard)/dashboard/settings/components/AppearanceTab.tsx | Adds Appearance UI for selecting guided vs expert combo mode. |
| src/app/(dashboard)/dashboard/combos/page.tsx | Fetches comboConfigMode and renders expert-mode single-page combo form + manual model input. |
| scripts/build-next-isolated.mjs | Adds bundler flag selection for isolated Next builds. |
| README.md | Updates documentation wording to mention guided vs expert builder + direct model entry. |
| ); | ||
| }); | ||
|
|
||
| return toTrimmedString(matchedProvider?.providerId) || normalizedProviderId; |
There was a problem hiding this comment.
resolveComboBuilderProviderId currently falls back to returning the raw providerIdOrAlias when it doesn’t match any entry in providers. That means buildManualComboModelStep can produce steps for unknown providers (e.g. foo/bar), which will later fail during model resolution/execution and lets users save broken combos.
Consider returning null when no provider match is found (or adding an explicit allowUnknownProvider flag defaulting to false) so the UI can surface a clear “unknown provider” validation error instead of creating an invalid step.
| return toTrimmedString(matchedProvider?.providerId) || normalizedProviderId; | |
| return toTrimmedString(matchedProvider?.providerId) || null; |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces an "Expert Mode" for the Combo Builder, allowing users to view all configuration options on a single page and manually enter models. The changes span the UI, settings management, and backend validation, supported by new E2E and unit tests. Review feedback points out that hiding the readiness panel and validation error messages in Expert mode may cause confusion when the "Save" button is disabled, as users will lack feedback on which validation rules are failing.
| {!isExpertMode && ( | ||
| <> | ||
| <div className="mt-2"> | ||
| <ComboReadinessPanel checks={readinessChecks} blockers={saveBlockers} /> | ||
| </div> |
There was a problem hiding this comment.
In Expert mode, the ComboReadinessPanel is hidden. While Expert mode aims to reduce clutter, this panel provides a critical summary of whether the combo is valid (e.g., weights totaling 100%, pricing coverage). Without it, users may find the "Save" button disabled without a clear indication of which validation rule is failing, especially since the review stage is also skipped.
| )} | ||
|
|
||
| {builderStage !== "review" && !canAdvanceFromCurrentStage && ( | ||
| {!isExpertMode && builderStage !== "review" && !canAdvanceFromCurrentStage && ( |
There was a problem hiding this comment.
The validation warning message is currently hidden in Expert mode. This message is essential for providing feedback when the "Save" button is disabled due to validation errors (like an invalid name or missing steps). Hiding it in Expert mode degrades the user experience as it leaves the user guessing why they cannot persist their changes.
| {!isExpertMode && builderStage !== "review" && !canAdvanceFromCurrentStage && ( | |
| {(isExpertMode || (builderStage !== "review" && !canAdvanceFromCurrentStage)) && ( |
1d90c80 to
5d33a79
Compare
5d33a79 to
4137ec3
Compare
Summary
cx/gpt-5.5andopenrouter/openai/gpt-5.5, using dynamic account selection.src/lib/dataPaths.jsartifact that shadowed the TypeScript source during builds.Impact
Validation
git diff --checknode --import tsx/esm --test tests/unit/combo-builder-draft.test.ts tests/unit/settings-schema-routing-strategies.test.ts tests/unit/db-settings-crud.test.tsnode --import tsx/esm --test --test-force-exit tests/unit/chat-combo-live-test.test.tsvitest run src/app/api/settings/__tests__/settings.test.tstests/e2e/combos-flow.spec.tstsc --pretty false -p tsconfig.typecheck-core.jsonscripts/build-next-isolated.mjs