Skip to content

fix(api): prepare the auto-combo candidate pool once in GET /api/combos/auto - #14925

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
developerjillur:fix/combos-auto-prepare-once
Sep 29, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
developerjillur:fix/combos-auto-prepare-once

Conversation

@developerjillur

Copy link
Copy Markdown
Contributor

Summary

  • GET /api/combos/auto built every listed auto variant with createVirtualAutoCombo(), which runs prepareVirtualAutoComboInputs() on every call, so one request prepared the candidate pool once per variant (47 times on this branch).
  • The handler now prepares the pool once and builds each variant with createVirtualAutoComboFromPrepared(), the same prepare-once pattern catalog.ts already uses. The prepared inputs do not depend on the variant or spec, and createVirtualAutoComboFromPrepared() works on a copy of the candidates, so the response is unchanged.
  • Scope is only the repeated preparation. The empty catch {} blocks and the other items listed in the issue are left for separate changes.

Related Issues

Validation

  • Change type: routing (API route)
  • Focused tests: the new test, plus every tests/unit/*.test.ts with auto or combo in its name (316 files): 2,079 pass, 7 skipped, 1 local-only failure. combos-quota-protected.test.ts opens a source file through a URL path and gets ENOENT because my checkout path contains a space (%20); it does not touch this route.
  • npm run lint: exits 2 with "suppressions left that do not occur anymore", the same as on the base branch without this change. npx eslint on the two changed files is clean, and npm run typecheck:core is clean.
  • Reconciled with the current active release base (release/v3.8.51 at a58000c7)
  • Production-code changes include a new automated test

⚠️ base-red inherited: #14866

Tests Added Or Updated

  • tests/unit/14889-combos-auto-prepare-once.test.ts (new): seeds six API-key providers, counts the preparation yields (setImmediate, one every four candidates) for one prepareVirtualAutoComboInputs() call and for one GET /api/combos/auto, and asserts they are equal. Before this change it fails with 658 against 14 (47 preparations).

Coverage Notes

  • src/app/api/combos/auto/route.ts is covered by the new test and by the existing auto-combos-free-models-routes.test.ts (6 pass) and auto-combo-context-advertising.test.ts (16 pass).

Reviewer Notes

  • Output check: on a test DB with 12 API-key providers (107 regular candidates), the old and the new handler, run alternately in the same process, return identical JSON for all 47 combos when each candidate pool is compared as a set. The candidate order, and auto/chaos's judgeModel when the top candidates tie on weight, already differ from one process to the next on the base branch. In three separate processes the two handlers agreed each time (on kimi-k3, kimi-k3 and zai-glm-4.7), so this change does not alter either.
  • Error handling: a throw inside the pool preparation used to be swallowed by each variant's own catch {}, and the request ended with { combos: [] } after 47 identical failures. Now the single preparation fails once, reaches the outer catch, is logged with console.error, and the response is still { combos: [] }.
  • Numbers from that DB: preparation yields per request drop from 1,128 to 24, and wall time from about 500 ms to about 400 ms. Preparation is cheap there because the DB has no synced models. On the pool in the issue, where one preparation took about a second, the saving grows with the number of listed variants.

…os/auto

The route listed every auto variant by calling createVirtualAutoCombo() for
each one, and createVirtualAutoCombo() runs prepareVirtualAutoComboInputs()
every time. One request therefore rebuilt the whole candidate pool once per
listed variant (47 times on this branch).

Prepare the pool once at the top of the handler and build every variant with
createVirtualAutoComboFromPrepared(), the same prepare-once pattern the model
catalog already uses. The prepared inputs do not depend on the variant or
spec, and createVirtualAutoComboFromPrepared() works on a copy of the
candidates, so the listed combos are unchanged.

Closes diegosouzapw#14889
@diegosouzapw
diegosouzapw merged commit e59590d into diegosouzapw:release/v3.8.51 Sep 29, 2026
11 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(api): GET /api/combos/auto rebuilds the full provider pool 40x serially (~49s per request)

2 participants