Skip to content

fix(api): self-hydrate model aliases from DB on GET after restart - #5777

Merged
diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.43from
jleonar2:fix/model-aliases-startup-persistence
Jul 1, 2026
Merged

diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.43from
jleonar2:fix/model-aliases-startup-persistence

Conversation

@jleonar2

@jleonar2 jleonar2 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Problem

In the standalone production build, webpack creates two separate copies of modelDeprecation.ts:

  • Module 498996 (chunk 73671): hydrated by the startup path (applyRuntimeSettings) — used for request routing
  • Module 32041 (chunk 33364): used by GET /api/settings/model-aliases — starts empty after each restart

Each module instance has its own _customAliases state. The startup code hydrates module 498996 from the DB, but the API route reads from module 32041 which was never initialized. This causes the Settings → Routing UI to show "No exact-match aliases configured" even though the aliases are persisted in the DB and work fine for request routing.

Verification (before fix, after restart):

Check Result
GET /api/settings → modelAliases ✅ present (reads directly from DB)
GET /api/settings/model-aliases → custom ❌ {} (reads from empty in-memory state)
DB key_value row ✅ data intact

Fix

The GET handler in /api/settings/model-aliases now detects an empty _customAliases state and reads the modelAliases key from the settings blob in the DB, calling setCustomAliases() to hydrate this module instance. This is a best-effort fallback — when _customAliases is already populated (e.g. dev mode where only one module instance exists), no DB read occurs.

~15 lines changed in route.ts.

Regression test

tests/unit/model-aliases-settings-route-selfheal.test.ts — 3/3 passing:

  1. Hydrates from DB when in-memory state is empty
  2. Skips hydration when in-memory state is already populated (no overwrite)
  3. Handles missing modelAliases in DB gracefully

Live verification

After restart with the fix applied:

curl -s http://localhost:20128/api/settings/model-aliases | jq .custom
# Returns stored aliases instead of {}

chirag127 and others added 2 commits June 30, 2026 22:52
…ate is empty

In the standalone production build, webpack creates two separate copies of
modelDeprecation.ts — one hydrated by the startup path (used for request
routing) and one used by the /api/settings/model-aliases API route.  The
API route's copy starts with an empty _customAliases after each server
restart, causing the Settings → Routing UI to show 'No exact-match aliases
configured' even though the aliases are persisted in the DB.

The GET handler now detects an empty _customAliases state and reads the
modelAliases key from the settings blob in the DB, calling
setCustomAliases() to hydrate this module instance.  This is a best-effort
fallback — when _customAliases is already populated (e.g. by the startup
path in dev mode), no DB read occurs.

Regression test: tests/unit/model-aliases-settings-route-selfheal.test.ts
- Verifies hydration from DB when in-memory state is empty
- Verifies no hydration when in-memory state is already populated
- Verifies graceful handling when no modelAliases exist in DB
@jleonar2
jleonar2 requested a review from diegosouzapw as a code owner July 1, 2026 06:10
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@diegosouzapw
diegosouzapw changed the base branch from main to release/v3.8.43 July 1, 2026 07:08
@diegosouzapw
diegosouzapw merged commit e2d2c27 into cheaper-inference:release/v3.8.43 Jul 1, 2026
diegosouzapw added a commit that referenced this pull request Jul 1, 2026
…llow-up) (#5821)

#5777 self-healed the GET /api/settings/model-aliases symptom at the route layer,
but the root cause remained: modelDeprecation.ts held _customAliases in a plain
module-level let, which webpack duplicates across the startup and app-route module
graphs (same class as #5312). Startup hydration landed on one copy; the API route
read the other (empty) one.

Back the store with globalThis (__omniroute_customAliases__) so both instances share
one store — the exact pattern already used by thinkingBudget.ts/backgroundTaskDetector.ts
(#5312). The route-layer DB self-heal from #5777 stays as a harmless fallback.

Extends #5777 (thanks @jleonar2). Regression: tests/unit/model-aliases-globalthis-5777.test.ts
(fails on the plain-let store: never populates globalThis, never reads a sibling
instance's write).
@diegosouzapw diegosouzapw mentioned this pull request Jul 2, 2026
tkgo11 pushed a commit to tkgo11/OmniRoute that referenced this pull request Sep 23, 2026
…eaper-inference#5777)

* Fix grammatical errors in readme (cheaper-inference#5738)

* fix(api): self-hydrate model aliases from DB on GET when in-memory state is empty

In the standalone production build, webpack creates two separate copies of
modelDeprecation.ts — one hydrated by the startup path (used for request
routing) and one used by the /api/settings/model-aliases API route.  The
API route's copy starts with an empty _customAliases after each server
restart, causing the Settings → Routing UI to show 'No exact-match aliases
configured' even though the aliases are persisted in the DB.

The GET handler now detects an empty _customAliases state and reads the
modelAliases key from the settings blob in the DB, calling
setCustomAliases() to hydrate this module instance.  This is a best-effort
fallback — when _customAliases is already populated (e.g. by the startup
path in dev mode), no DB read occurs.

Regression test: tests/unit/model-aliases-settings-route-selfheal.test.ts
- Verifies hydration from DB when in-memory state is empty
- Verifies no hydration when in-memory state is already populated
- Verifies graceful handling when no modelAliases exist in DB

---------

Co-authored-by: Chirag Singhal <76880977+chirag127@users.noreply.github.com>
Co-authored-by: marcelpeterson <marcelpeterson@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
tkgo11 pushed a commit to tkgo11/OmniRoute that referenced this pull request Sep 23, 2026
…inference#5777 follow-up) (cheaper-inference#5821)

cheaper-inference#5777 self-healed the GET /api/settings/model-aliases symptom at the route layer,
but the root cause remained: modelDeprecation.ts held _customAliases in a plain
module-level let, which webpack duplicates across the startup and app-route module
graphs (same class as cheaper-inference#5312). Startup hydration landed on one copy; the API route
read the other (empty) one.

Back the store with globalThis (__omniroute_customAliases__) so both instances share
one store — the exact pattern already used by thinkingBudget.ts/backgroundTaskDetector.ts
(cheaper-inference#5312). The route-layer DB self-heal from cheaper-inference#5777 stays as a harmless fallback.

Extends cheaper-inference#5777 (thanks @jleonar2). Regression: tests/unit/model-aliases-globalthis-5777.test.ts
(fails on the plain-let store: never populates globalThis, never reads a sibling
instance's write).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants