Skip to content

feat(dashboard): models and combos catalog with filters and health tests - #14887

Merged
diegosouzapw merged 18 commits into
diegosouzapw:release/v3.8.52from
fouadSalkini:feat/models-combos-catalog
Oct 8, 2026
Merged

diegosouzapw merged 18 commits into
diegosouzapw:release/v3.8.52from
fouadSalkini:feat/models-combos-catalog

Conversation

@fouadSalkini

@fouadSalkini fouadSalkini commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

What

Turns /dashboard/models into a Models & Combos catalog with filters and health tests. The page gets two
tabs: the model catalog and a new combos catalog. UI only: no API route, DB or schema changes.

Why

Admins have no single place to browse every model and combo, narrow them down by capability or health, and check
which ones actually answer. Today that means opening provider pages one at a time or testing from the combo editor.

How

  • Tabs: Models (GET /api/models/catalog) and Combos (GET /api/combos), deep-linkable with ?tab=,
    keyboard accessible (Arrow/Home/End, roving tabindex, labelled tab panels).
  • Filters (all kept in the URL query string, shareable, with result counts and "Clear filters"):
    • Models: search (id/name/provider), provider, type, subtype, capabilities (derived from the catalog data), min
      context length, min max-output tokens, free/paid, provider health (GET /api/providers/health-matrix), last
      test result (untested/ok/slow/error).
    • Combos: search, strategy, status, member-count range (nested combo steps count as members), last test result.
    • URL and stored state are read after mount, so server and first client render match.
  • Health tests:
    • Per-row Test: models use POST /api/models/test, combos use POST /api/combos/test.
    • Bulk "Test selected" / "Test all filtered": models are grouped per provider and sent to
      POST /api/models/test-all (at most 100 ids per call, at most 2 calls in flight). Combos run at most 2 at a
      time. Runs above 50 models or 20 combos ask for confirmation, and exact duplicates are removed first.
    • A progress bar plus Cancel. Cancel stops scheduling and aborts in-flight combo tests, and a cancelled run can't
      take over a newer run. Leaving the page aborts the run. Models the server skipped (stoppedEarly) stay
      untested.
    • Results show a status badge (ok / slow / error class: rate-limited, quota, timeout, other), latency and test
      time. The sanitized API error text is available on hover and to screen readers.
    • Results are kept in the browser only (localStorage, versioned key, newest 2,000 entries), with a
      "Clear results" action.
  • Sidebar label becomes "Models & Combos" through i18n (route unchanged). New labels are translated in all 66
    locales.

Tests

Vitest (10 files, 111 tests): filter/sort utils for models and combos, URL state parsing (invalid numbers
ignored), bulk batching (>100 ids split, at most 2 concurrent requests), result storage (cap, versioning, corrupt
JSON), response classification including stoppedEarly, tab switching and deep link, filter → URL + rows, single
model and combo test flows, cancel then restart, abort on unmount, and results restored from storage.

Commands run locally: eslint (with suppressions) and prettier on changed files, npm run check:file-size, the
vitest files above, and the i18n checks. check-key-completeness fails the same way on the base:
featureFlagProxyPoolSharedEgressOrderDescription (added in #14657) is missing from 65 locales. This PR does not
touch it.

⚠️ base-red inherited: #15306

Test runner (useCatalogTestRunner):
- Abort the active bulk run and any per-row tests on unmount; no further
  scheduling, state updates or localStorage writes afterwards.
- Load stored results in a mount effect instead of a useState initializer.
- Bulk combo tests use the run's AbortSignal, so Cancel also aborts tests in
  flight. Only the run that still owns the runner may advance progress or
  reset running state, so a cancelled run's late completions cannot end or
  hijack the next run.
- Honour stoppedEarly from /api/models/test-all: models the server never
  tested stay untested instead of being stored as errors.
- Response mapping moved to catalogTestResponses.ts; no English fallback
  text is stored, the badge renders translated messages instead.

Page and components:
- URL filters and the active tab are applied after mount (SSR-safe);
  deep links and Back/Forward keep working.
- Tabs extracted to CatalogTabs with Arrow/Home/End keys, roving tabindex
  and focus following selection.
- Confirm dialog before bulk runs of more than 50 models or 20 combos;
  exact duplicate targets are dropped before counting and batching.
- New "min max output" filter bound to the existing minOutput param.
- Numeric URL params and inputs accept whole numbers >= 0 only.
- Per-row Test buttons are disabled while a bulk run is active; the
  progress bar has an accessible name and value text; error details are
  available to screen readers; test time is shown as a relative age.
- combo-ref steps count as combo members.

Tests: fake timers and deferred promises instead of real waits; new unit
suites for bulk utils, URL state and response mapping; UI coverage for
unmount abort, cancel/restart, stoppedEarly, >100-model batching with at
most 2 calls in flight, confirm dialog, keyboard tabs, filter URL sync,
single combo test and unsafe numeric params.
…cales

Adds the 16 new modelCatalog keys (min max output filter, bulk progress
and cancel status, bulk confirm dialog, error detail and HTTP fallback
text, additional capabilities label, nested combo member label) to every
locale with real translations; no __MISSING__ markers remain.
…loads

- Write the URL from one effect only, skip the write when pathname and
  search are unchanged, and ignore a refused history.replaceState call
  (browsers rate-limit it), so a throttled update never takes the page down.
- Check free-form URL filters against loaded options only while options
  exist (models/combos loaded), so a Refresh no longer flashes an empty
  table and a failed load no longer drops valid filters from the URL.
- Match the capability URL param case-insensitively, like the filter.
- Drop the redundant lg/xl grid-cols-3 classes on the model filter grid.
… secrets gate

gitleaks' generic-api-key rule matched `CATALOG_TEST_RESULTS_KEY = "..."`
as a credential. Rename the identifier to CATALOG_TEST_RESULTS_STORAGE_NAME
and keep the localStorage entry name "omniroute.catalogTestResults.v1"
unchanged, so results users already stored keep loading. The two tests that
import the constant only follow the rename.
… complexity gate

Behaviour-preserving split of the new catalog helpers so each function stays
under the cyclomatic and cognitive limits:

- catalogTestStorage: classifyError reads named rate-limit, quota and timeout
  checks; loadCatalogTestResults delegates the stored-entry validation.
- comboCatalogUtils: flattenCombos builds each row and member step through
  small parsers; the combo filter uses named member-bound and test-result
  predicates.
- modelCatalogUtils: capability collection and each optional model filter
  are named predicates. flattenCatalog is unchanged from the base branch.
…maller helpers for the complexity gate

Behaviour-preserving split with the same props, DOM, classes and i18n keys:

- CatalogFilterFields: shared search row, select, number input and option
  lists that the model and combo filter panels used to repeat inline.
- ModelCatalogFilters groups its grid into catalog-option selects and
  fixed-choice / numeric filters; ComboCatalogFilters uses the shared fields.
- CatalogTestBadge renders its testing, untested and result states through
  three small components.
- CatalogBulkActionBar moves the progress bar and cancelled notice into
  CatalogBulkProgress; the action buttons stay where they were.
…e complexity gate

Behaviour-preserving split with the same props, DOM, classes and i18n keys:

- CatalogTableParts: plain and actions headings, selection checkbox cells
  and the paging footer that both tables rendered inline.
- ModelCatalogTable renders its header and rows through
  ModelCatalogTableHead and ModelCatalogTableRow, with the provider, type,
  capabilities and flags cells as small components. Its props block keeps
  its place; only the labels shape gets a name.
- ComboCatalogTable does the same with ComboCatalogTableHead,
  ComboCatalogTableRow and ComboMembersCell; its props move to the
  ComboCatalogTableProps interface with the same members.
@diegosouzapw diegosouzapw added the deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52 label Sep 28, 2026
@diegosouzapw diegosouzapw changed the title feat(dashboard): models and combos catalog with filters and health tests [defer] feat(dashboard): models and combos catalog with filters and health tests Sep 28, 2026
@diegosouzapw
diegosouzapw changed the base branch from release/v3.8.51 to release/v3.8.52 September 29, 2026 11:18
@diegosouzapw

Copy link
Copy Markdown
Owner

Re-homed to release/v3.8.52: v3.8.51 entered its release freeze, so the branch now belongs to the release captain and development continues on the next cycle. Nothing is wrong with this PR — it just needed a live base. No action needed from you; CI will re-run against the new base.

@diegosouzapw diegosouzapw changed the title [defer] feat(dashboard): models and combos catalog with filters and health tests feat(dashboard): models and combos catalog with filters and health tests Oct 1, 2026
@diegosouzapw diegosouzapw removed the deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52 label Oct 1, 2026
…ical zh provider term

The catalog page re-added sidebar.modelCatalog and sidebar.modelCatalogSubtitle while the base already had them later in the same object, so JSON.parse kept the base copy and the new 'Models & Combos' label never showed. Each locale now has one copy carrying the new label. zh-CN catalog strings use the glossary's canonical 提供者 instead of the retired 提供商.
@diegosouzapw
diegosouzapw merged commit 814ca10 into diegosouzapw:release/v3.8.52 Oct 8, 2026
31 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants