Skip to content

feat(combo): Auto-Combo snapshot generation/duplication in the UX - #10354

Merged
diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.50from
swingtempo:swingtempo/duplicate-auto-combo2
Aug 21, 2026
Merged

diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.50from
swingtempo:swingtempo/duplicate-auto-combo2

Conversation

@swingtempo

@swingtempo swingtempo commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • AutoComboCatalog - each autocombo card (e.g. auto/best-coding) has a button to generate a snapshot of the combo so a user can see and edit it
  • Upon clicking, the combo gets generated
  • window scrolls to the newly created card

Related Issues

What Changed

1. Snapshot Weight Computation (open-sse/services/autoCombo/virtualFactory.ts)

  • New function computeSnapshotWeights() scores candidates at combo creation time using:
    • taskFit: reasoning & vision-capable models score higher when taskFit is weighted
    • stability: richer capability profiles assumed more stable
    • tierPriority / costInv: premium tiers and free models get bonuses based on the weight pack
    • latencyInv / health / quota: neutral baseline (no runtime telemetry at snapshot time)
  • Replaces hardcoded weight: 1 in createVirtualAutoComboFromPrepared() with computed scores normalized to [0, 1].

2. Duplicate Endpoint (src/app/api/combos/duplicate/route.ts) — NEW

  • POST /api/combos/duplicate resolves an auto/* template (e.g. auto/best-coding, auto/glm) into a static combo snapshot.
  • Reuses the same candidate resolution logic as createVirtualAutoCombo().
  • Normalizes weights to sum exactly 100, applies naming convention (static-{name} copy), and preserves the original weight pack in config.
  • Returns 201 on success with the created combo object; validates input (400) and unknown templates (422).

3. Dashboard UI Updates

File Change
src/app/(dashboard)/dashboard/combos/page.tsx Added handleComboCreated() callback to auto-scroll newly created combo cards; wired it into <AutoComboCatalog />. Refactored ComboFormModal provider builder grid layout.
src/app/(dashboard)/dashboard/combos/AutoComboCatalog.tsx Accepts optional onComboCreated prop for post-creation scroll behavior.

4. Tests (Rule #18)

Test File Coverage
tests/unit/snapshot-weights.test.ts Capability differentiation, tier bonuses, neutral baselines, score clamping
tests/unit/combos-duplicate-route.test.ts Auth gate, input validation (400), success response shape, weight distribution, naming convention, error sanitization
tests/unit/combos-duplicate-resolution-audit.test.ts Audit-style checks for duplicate resolution edge cases

5. Utilities & i18n

  • scripts/ad-hoc/dump-auto-combos.ts: CLI helper to dump resolved auto-combo candidates (debug / inspection).
  • ~47 locale files: minor string updates in src/i18n/messages/*.json.

Why This Matters

Problem Solution
Auto-combos always gave equal weight (1) to every model, ignoring capability differences. Snapshot weights differentiate reasoning/vision-capable models at creation time based on the active weight pack.
No way to "freeze" an auto-combo into a static combo for manual tweaking or versioning. /api/combos/duplicate materializes any auto/* template as a persistent combo with preserved weights and metadata.

API Reference — Duplicate Endpoint

POST /api/combos/duplicate
Authorization: Bearer <management-api-key>   (if requireLogin is true)
Content-Type: application/json

{
  "name": "auto/best-coding",    // required — any auto/* template
  "strategy": "priority"         // optional — defaults to "priority"
}

Response (201):

{
  "id": "...",
  "name": "static-best-coding copy",
  "models": [
    { "model": "anthropic/claude-sonnet-4-20250514", "weight": 35 },
    { "model": "openai/gpt-4.1",                    "weight": 28 }
  ],
  "strategy": "priority",
  "config": {
    "sourceAutoCombo": "auto/best-coding",
    "weightPack": { "taskFit": 0.3, "stability": 0.25, ... }
  }
}

Error codes: 400 (missing/invalid name), 401/403 (auth), 422 (unknown template or no matching models), 500 (server error).


Files Changed (51 files, +1412 / −249)

Area Files
Core logic open-sse/services/autoCombo/virtualFactory.ts
API route src/app/api/combos/duplicate/route.ts (new)
Dashboard UI src/app/(dashboard)/dashboard/combos/page.tsx, AutoComboCatalog.tsx
Tests tests/unit/snapshot-weights.test.ts (new), tests/unit/combos-duplicate-route.test.ts (new), tests/unit/combos-duplicate-resolution-audit.test.ts (new)
i18n ~47 locale files in src/i18n/messages/
Scripts scripts/ad-hoc/dump-auto-combos.ts (new)

Testing

# Snapshot weights
node --import tsx/esm --test tests/unit/snapshot-weights.test.ts

# Duplicate route
node --import tsx/esm --test tests/unit/combos-duplicate-route.test.ts

# All unit tests
npm run test:unit

Notes for Reviewers

NOTE! Prettifier page.tsx impact there are too many changes in page.tsx because Prettifier made a bunch of formatting changes. Main change is the OnComboCreated handler passed to AutoComboCatalog! Confirmed with AI that the large page.tsx changes are purely formatting.

  1. Snapshot weights are static — they use capabilities and tier info available at combo creation time, not runtime telemetry (p95 latency, quota remaining). This is intentional; the function docstring explains the trade-off.
  2. Weight normalization: snapshot scores are clamped to [0, 1], then normalized so all model weights sum exactly to 100 in the duplicate endpoint (with remainder distributed round-robin).
  3. Error sanitization follows Rule fix(ui): fix Select dropdown dark theme inconsistency #12 — no raw stack traces in responses.

}

const providerPool = [...new Set(effectivePool.map((c) => c.provider))];
const snapshotScores = computeSnapshotWeights(effectivePool, weights);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diegosouzapw
Need confirmation from product team that this is desired and is a bug that was fixed. This isn't an area I'm familiar with.

@@ -3203,213 +3212,232 @@ function ComboFormModal({ isOpen, combo, onClose, onSave, activeProviders, combo
{builderSelectionMode === "step" && (
<>
<div className="grid grid-cols-1 md:grid-cols-3 gap-2 mt-3">

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diegosouzapw These huge sections here are due to reformatting from Prettifier. I think this is automatic.

@swingtempo
swingtempo marked this pull request as ready for review August 14, 2026 06:16
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Deferido para revalidação do head atual. A rodada anterior registrou um hold de ownership, mas a inspeção atual não confirmou esse worktree; por favor, reconfirme o head e os checks antes da decisão final.

@diegosouzapw diegosouzapw added deferred-v3.8.50 Adiada para o ciclo v3.8.50 (validacao VPS, refactor, ou escopo grande) merge-train-deferred PR ejetada do merge-train — triagem separada labels Aug 17, 2026
@diegosouzapw diegosouzapw removed merge-train-deferred PR ejetada do merge-train — triagem separada deferred-v3.8.50 Adiada para o ciclo v3.8.50 (validacao VPS, refactor, ou escopo grande) labels Aug 17, 2026
…y term

- Route /api/combos/duplicate read request.json() without validateBody()/
  .safeParse(), failing the t06 CI gate (Hard Rule #7). Add
  duplicateAutoComboSchema and route the body through validateBody().
- zh-CN string for duplicateAutoComboSnapshotMsg used the glossary synonym
  提供商 instead of the canonical 提供者 for "provider", failing the
  i18n glossary consistency gate.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Babysit summary (Unit Tests fast-path 4/4)

Root cause (from run 31853607931, job Unit Tests fast-path (4/4), tests/unit/i18n-glossary-consistency-check.test.ts + tests/unit/api-route-body-validation.test.ts / t06 gate):

  1. POST /api/combos/duplicate (src/app/api/combos/duplicate/route.ts) read request.json() and did hand-rolled typeof checks instead of routing through validateBody()/Zod — fails the t06 "every API route reading request.json() validates it with Zod" gate (Hard Rule chore(env): sync .env.example with current .env structure #7).
  2. duplicateAutoComboSnapshotMsg in src/i18n/messages/zh-CN.json used the retired glossary synonym 提供商 instead of the canonical 提供者 for "provider" — fails the i18n glossary consistency regression guard (post-feat(i18n): add native-quality zh-CN terminology and visual QA gates #8038 normalization).

Fix (commit 4f911f6, authored by @swingtempo, co-authored by @diegosouzapw):

  • Added duplicateAutoComboSchema (src/shared/validation/schemas/combo.ts) and routed the route body through validateBody()/isValidationFailure() (src/shared/validation/helpers.ts), preserving the same error message/shape for existing tests.
  • Swapped 提供商 → 提供者 in the zh-CN string.

Verification:

  • node --import tsx/esm --test tests/unit/combos-duplicate-route.test.ts — 10/10 pass.
  • node --import tsx/esm --test tests/unit/combos-duplicate-resolution-audit.test.ts — 1/1 pass.
  • npx eslint src/app/api/combos/duplicate/route.ts src/shared/validation/schemas/combo.ts — clean.
  • Pushed to swingtempo/duplicate-auto-combo2; fresh CI run triggered (run 32438457440 at time of writing).

Threads left unresolved (not addressed by this fix, out of scope):

  • virtualFactory.ts — product-confirmation question, unrelated to CI failure.
  • dashboard/combos/page.tsx — Prettier reformatting note, unrelated to CI failure.

dast-smoke was previously cancelled/showed fail after a 30m timeout window in the prior run — no assertion failure observed; re-run triggered along with the rest of the fresh CI run. Not modified.

Ready for CI to finish + human review/merge.

@diegosouzapw

Copy link
Copy Markdown
Collaborator

⚠️ base-red inherited: #9985

New CI run (32438457440, after the fix push) shows 2 additional reds unrelated to this PR's diff — both are inherited base drift, not caused by this branch:

  • Docs Gates (fast-path): provider-count drift (docs say 346, live code now defines 347) + stale SVG diagrams — a base-tip content change, not touched by this PR.
  • No new ESLint warnings: eslintErrors: 0, eslintWarnings: 0 in the job's own metrics — the failure is the quality-ratchet step (openapiCoverage.pct: 38.6 < baseline 39.2), also base-tip drift per the open base-red issue 🔴 Release branch not green: release/v3.8.50 #9985 ("ratchet drift ... is expected mid-cycle and is rebaselined at release — it is NOT a contributor concern").

Original target ("Unit Tests fast-path (4/4)") is fixed by commit 4f911f6 — see prior comment. Not chasing the base-red items per project policy (Hard Rule #21 / base-green protocol); they'll clear on the next base merge or release rebaseline.

@diegosouzapw
diegosouzapw merged commit 74c9e7e into cheaper-inference:release/v3.8.50 Aug 21, 2026
9 of 16 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…eaper-inference#10354)

Adds a snapshot-generation button to each Auto-Combo catalog card: computeSnapshotWeights() scores candidates (taskFit/stability/tierPriority/costInv) at combo-creation time instead of the previous hardcoded weight:1, and the new POST /api/combos/duplicate endpoint materializes any auto/* template into a persistent, editable static combo with normalized weights. Closes cheaper-inference#10231.

Validated in an isolated worktree boarded onto origin/release/v3.8.50 (0 conflicts, 51 files):
- 19/19 focused tests pass (snapshot-weights, combos-duplicate-route, combos-duplicate-resolution-audit) — covers auth gate (401/403), input validation (400/422), success shape, weight normalization, naming/dedup, and error-response sanitization (no stack traces).
- check-file-size, check-changelog-integrity: OK.
- typecheck:core: clean.
- check-complexity / check-cognitive-complexity: OK, both under baseline.

Co-authored-by: swingtempo <swingtempo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Allow duplication of auto-combos, materialize the list into a snapshot for easy editing

2 participants