feat(services): promote Bifrost to embedded/supervised service (#5670) by diegosouzapw · Pull Request #5817 · cheaper-inference/OmniRoute

diegosouzapw · 2026-07-01T11:54:19Z

What

Promotes Bifrost (@maximhq/bifrost — the Go AI-gateway) from an env-only relay sidecar to a first-class embedded/supervised service, using the exact same model already shipped for cliproxy and 9router: installer → supervisor → lifecycle APIs → dashboard tab → health/auto-start/logs/version row → local-only route guard.

This implements item #2 of #5670. The broader RouterBackend contract (items #1, #3–#5) and the native hot-path migration stay out of scope. The existing env-configured relay path (/v1/relay/.../bifrost, OMNIROUTE_RELAY_BACKEND=bifrost) is kept unchanged as a compatibility layer.

Implementation (8 steps, mapped to the design spec)

Installer src/lib/services/installers/bifrost.ts — npm-style (mirrors ninerouter.ts, since @maximhq/bifrost is a thin wrapper that lazily downloads the Go binary): install/update/getInstalledVersion/getLatestVersion (1h cache) + resolveSpawnArgs(port) (Go single-dash flags -port/-host 127.0.0.1/-app-dir/-log-level warn, pinned BIFROST_TRANSPORT_VERSION). needsApiKey=false.
Bootstrap src/lib/services/bootstrap.ts — SERVICES[] entry (healthPath: /v1/models, 5s health interval, 15s stop timeout, 5 MiB log ring) + buildSpawnArgsFactory branch.
Migration 113_bifrost_service.sql — seeds the version_manager row (not_installed, port 8080, auto_update=1, provider_expose=1).
API src/app/api/services/bifrost/ — _lib.ts + 7 lifecycle routes (install/start/stop/restart/update/status/auto-start), verbatim from cliproxy, all errors via createErrorResponse/sanitizeErrorMessage.
Shared logs [name]/logs/route.ts — bifrost branch.
Route guard — no code change (/api/services/ is already LOCAL_ONLY); a unit test asserts isLocalOnlyPath("/api/services/bifrost/*") === true.
Dashboard BifrostServiceTab.tsx + registration in the services page.tsx shell (Tab type + TABS[] + render block).
Docs + tests — EMBEDDED-SERVICES.md (§1 table + §4.3 endpoint table) + openapi.yaml (7 paths); unit tests + gated integration lifecycle.

Relay auto-wiring: when BIFROST_BASE_URL is unset and the supervised instance is running, getBifrostRoutingConfig() defaults the relay base URL to http://127.0.0.1:{port}. Explicit BIFROST_BASE_URL always wins.

Tests

Unit (19, all green): installer version-pattern + spawn-args shape (8), route-guard classification (6), §4b routing-backend defaulting incl. env-precedence + stopped cases (5).
Integration: full bifrost lifecycle added to full-lifecycle.int.test.ts, gated by RUN_SERVICES_INT=1 (does not run or download the Go binary in the default suite).
Local gates green for the feature: typecheck:core, lint (0 errors), check:cycles, test:unit + test:vitest (no feature-attributable failures).

⚠️ Merge gate — VPS live-test required (Hard Rule #18 / spec §7)

The actual Go-binary install → start → health path can only be validated live (it downloads a per-platform binary from downloads.getmaxim.ai on first start). Per Hard Rule #18, this must not be merged on "worked locally" alone — run the gated integration harness on the VPS and record the result before merge:

RUN_SERVICES_INT=1 node --import tsx/esm --test tests/integration/services/full-lifecycle.int.test.ts

Open questions to close during that run: Windows binary download; headless boot with an empty -app-dir.

Out of scope

RouterBackend unification, CLIProxyAPI ecosystem parity, and the hot-path migration (#5670 items #1, #3–#5).

Part of #5670.

chatgpt-codex-connector · 2026-07-01T11:54:25Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

gemini-code-assist

Code Review

This pull request integrates Bifrost as a new supervised embedded service, introducing lifecycle API endpoints, UI components, database migrations, and comprehensive tests. The review feedback highlights several critical issues: potential TypeErrors in routingBackend.ts and the update route due to unsafe chaining on sup?.getStatus(), a missing stateChange event listener on the dynamically initialized supervisor that prevents model synchronization, hardcoded Portuguese error messages in the start and restart routes, a type mismatch returning a number instead of a boolean for autoStart, and a violation of Repository Hard Rule #8 by defining a Zod schema locally.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-07-01T11:56:50Z

+  if (!resolvedBaseUrl) {
+    const sup = getSupervisor("bifrost");
+    if (sup?.getStatus().state === "running") {
+      resolvedBaseUrl = `http://127.0.0.1:${sup.getStatus().port}`;
+    }
+  }


If getSupervisor("bifrost") returns undefined (e.g., if the service has not been initialized or started yet), sup?.getStatus() will evaluate to undefined. Attempting to access .state on undefined will throw a TypeError: Cannot read properties of undefined (reading 'state').

To prevent this, store the status in a variable and use optional chaining to safely check the state and retrieve the port.

Suggested change

if (!resolvedBaseUrl) {

const sup = getSupervisor("bifrost");

if (sup?.getStatus().state === "running") {

resolvedBaseUrl = `http://127.0.0.1:${sup.getStatus().port}`;

}

}

if (!resolvedBaseUrl) {

const sup = getSupervisor("bifrost");

const status = sup?.getStatus();

if (status?.state === "running") {

resolvedBaseUrl = `http://127.0.0.1:${status.port}`;

}

}

gemini-code-assist · 2026-07-01T11:56:50Z

+    const sup = getSupervisor("bifrost");
+    const wasRunning = sup?.getStatus().state === "running";


If getSupervisor("bifrost") returns undefined, sup?.getStatus() will evaluate to undefined. Accessing .state on undefined will throw a TypeError. Use optional chaining to safely access the state.

Suggested change

const sup = getSupervisor("bifrost");

const wasRunning = sup?.getStatus().state === "running";

const sup = getSupervisor("bifrost");

const wasRunning = sup?.getStatus()?.state === "running";

gemini-code-assist · 2026-07-01T11:56:50Z

+export async function getOrInitSupervisor(): Promise<ServiceSupervisor> {
+  const existing = getSupervisor(TOOL);
+  if (existing) return existing;
+
+  const sup = new ServiceSupervisor({
+    tool: TOOL,
+    port: PORT,
+    spawnArgs: () => resolveSpawnArgs(PORT),
+    healthUrl: () => `http://127.0.0.1:${PORT}/v1/models`,
+    healthIntervalMs: 5_000,
+    stopTimeoutMs: 15_000,
+    logsBufferBytes: 5_242_880,
+  });
+
+  registerSupervisor(sup);
+  return sup;
+}


When the supervisor is initialized on demand via getOrInitSupervisor(), it is registered in the registry but lacks the "stateChange" event listener that is attached during bootstrapEmbeddedServices().

Because Bifrost starts as not_installed by default, it is skipped during the initial bootstrap. When a user later installs and starts the service, the supervisor is created here on demand. Without the "stateChange" listener, transitioning to "running" will never trigger model synchronization (scheduleServiceModelSync), and stopping or encountering an error will never stop synchronization or mark models as unavailable.

We should dynamically import the required helpers and attach the "stateChange" listener here as well.

export async function getOrInitSupervisor(): Promise<ServiceSupervisor> { const existing = getSupervisor(TOOL); if (existing) return existing; const sup = new ServiceSupervisor({ tool: TOOL, port: PORT, spawnArgs: () => resolveSpawnArgs(PORT), healthUrl: () => `http://127.0.0.1:${PORT}/v1/models`, healthIntervalMs: 5_000, stopTimeoutMs: 15_000, logsBufferBytes: 5_242_880, }); const { scheduleServiceModelSync, stopServiceModelSync } = await import("@/lib/services/modelSync"); const { markAllUnavailable } = await import("@/lib/db/serviceModels"); const baseUrl = `http://127.0.0.1:${PORT}`; sup.on("stateChange", (status) => { if (status.state === "running") { scheduleServiceModelSync(TOOL, baseUrl, ""); } else if (status.state === "stopped" || status.state === "error") { stopServiceModelSync(TOOL); markAllUnavailable(TOOL); } }); registerSupervisor(sup); return sup; }

gemini-code-assist · 2026-07-01T11:56:50Z

+    if (!row || row.status === "not_installed") {
+      return createErrorResponse({ status: 409, message: "Bifrost não está instalado." });
+    }


The error message "Bifrost não está instalado." is hardcoded in Portuguese, which is inconsistent with the rest of the API routes and the OpenAPI specification (Bifrost is not installed). It should be in English.

Suggested change

if (!row || row.status === "not_installed") {

return createErrorResponse({ status: 409, message: "Bifrost não está instalado." });

}

if (!row || row.status === "not_installed") {

return createErrorResponse({ status: 409, message: "Bifrost is not installed." });

}

gemini-code-assist · 2026-07-01T11:56:50Z

+    if (!row || row.status === "not_installed") {
+      return createErrorResponse({ status: 409, message: "Bifrost não está instalado." });
+    }


The error message "Bifrost não está instalado." is hardcoded in Portuguese, which is inconsistent with the rest of the API routes and the OpenAPI specification (Bifrost is not installed). It should be in English.

Suggested change

if (!row || row.status === "not_installed") {

return createErrorResponse({ status: 409, message: "Bifrost não está instalado." });

}

if (!row || row.status === "not_installed") {

return createErrorResponse({ status: 409, message: "Bifrost is not installed." });

}

gemini-code-assist · 2026-07-01T11:56:50Z

+      installedVersion: installedVersion ?? row?.installedVersion ?? null,
+      latestVersion,
+      updateAvailable: !!installedVersion && !!latestVersion && installedVersion !== latestVersion,
+      autoStart: row?.autoStart ?? false,


Since SQLite stores boolean values as integers (0 or 1), row?.autoStart will return a number. Returning a number directly violates the OpenAPI schema which expects autoStart to be a boolean. Coerce it to a boolean explicitly.

Suggested change

autoStart: row?.autoStart ?? false,

autoStart: !!row?.autoStart,

gemini-code-assist · 2026-07-01T11:56:50Z

+import { createErrorResponse } from "@/lib/api/errorResponse";
+import { sanitizeErrorMessage } from "@omniroute/open-sse/utils/error";
+
+const BodySchema = z.object({ enabled: z.boolean() });


Defining the Zod schema BodySchema locally violates Hard Rule #8 of the Repository Style Guide, which states: 'Always validate inputs with Zod schemas from src/shared/validation/schemas.ts'.

Please move this schema to src/shared/validation/schemas.ts and import it here.

References

Always validate inputs with Zod schemas from src/shared/validation/schemas.ts. ^(link)

diegosouzapw · 2026-07-02T04:16:38Z

Re-stacked on the provider-manifest stack (#5869 → #5870)

This PR was rebased on top of #5870 (feat(relay): gate bifrost auto routing by provider manifest), which is itself stacked on #5869 (feat(providers): add sidecar plugin manifest). Base retargeted release/v3.8.43 → main to match the stack.

Why: both this PR and #5870 edit src/app/api/v1/relay/chat/completions/routingBackend.ts. Rebasing here reconciles them in one place instead of leaving a conflict for whoever merges second. The two changes are orthogonal and compose:

This PR (feat(backend): unified RouterBackend contract across CLIProxyAPI, Bifrost, VibeProxy, and native hot paths #5670 item build(deps): bump actions/github-script from 7 to 8 #2) — getBifrostRoutingConfig() now defaults the relay base URL to the supervised Bifrost instance (http://127.0.0.1:{port}) when BIFROST_BASE_URL is unset and the service is running.
feat(relay): gate bifrost auto routing by provider manifest #5870 — shouldTryBifrostForRequest() gates auto routing by provider-manifest eligibility.

Together: starting the supervised Bifrost service auto-enables the auto backend (this PR), and the per-request manifest gate (#5870) keeps only sidecar-eligible providers on the Bifrost path. Explicit OMNIROUTE_RELAY_BACKEND / BIFROST_BASE_URL still win.

Merge order: #5869 → #5870 → this PR. Until #5869/#5870 land on main, this PR's diff includes their commits (30 files); once they merge, it auto-reduces to the 22 Bifrost-service files.

Validation on the combined tree: typecheck:core clean · check:cycles clean · eslint 0 errors · tests green — relay-routing-backend (11) + bifrost-routing-backend (5) + installer/route-guard (14) + provider-plugin-manifest (4).

Merge gate unchanged: the Go-binary install/start/health path still needs the RUN_SERVICES_INT=1 VPS live-test before merge (Hard Rule #18).

github-actions · 2026-07-02T05:07:08Z

CI Coverage Report

Coverage job: skipped
PR test policy: success

Coverage artifact was not available for this run.

Promotes Bifrost (@maximhq/bifrost — Go AI-gateway) from an env-only relay sidecar to a first-class embedded/supervised service, matching the existing cliproxy/9router model. Implements item #2 of #5670; the broader RouterBackend contract (items #1, #3-#5) stays out of scope. - Installer (npm-style, ninerouter model): install/update/getInstalledVersion/ getLatestVersion (1h cache)/resolveSpawnArgs (Go single-dash flags, pinned BIFROST_TRANSPORT_VERSION), needsApiKey=false - Bootstrap SERVICES entry (healthPath /v1/models) + spawn-args factory branch - Migration 113 seeds the version_manager row (not_installed, port 8080, auto_update=1, provider_expose=1) - 7 lifecycle API routes under /api/services/bifrost/ (verbatim from cliproxy, errors sanitized) — loopback-only via existing LOCAL_ONLY_API_PREFIXES - Shared [name]/logs branch for bifrost - Dashboard tab + registration in the services page shell - Relay auto-wiring: getBifrostRoutingConfig defaults BIFROST_BASE_URL to the supervised port when the instance is running; explicit env still wins; the env-only relay path (/v1/relay/.../bifrost) stays unchanged (compat layer) - Docs (EMBEDDED-SERVICES, openapi) + unit tests (installer/route-guard/routing, 19 tests) + RUN_SERVICES_INT-gated integration lifecycle Note: the actual Go-binary install/start/health path requires a documented VPS live-test before merge (Hard Rule #18 / spec section 7); the gated integration harness is the vehicle for that run.

…er-inference#5670) (cheaper-inference#5817) Promotes Bifrost (@maximhq/bifrost — Go AI-gateway) from an env-only relay sidecar to a first-class embedded/supervised service, matching the existing cliproxy/9router model. Implements item cheaper-inference#2 of cheaper-inference#5670; the broader RouterBackend contract (items cheaper-inference#1, cheaper-inference#3-cheaper-inference#5) stays out of scope. - Installer (npm-style, ninerouter model): install/update/getInstalledVersion/ getLatestVersion (1h cache)/resolveSpawnArgs (Go single-dash flags, pinned BIFROST_TRANSPORT_VERSION), needsApiKey=false - Bootstrap SERVICES entry (healthPath /v1/models) + spawn-args factory branch - Migration 113 seeds the version_manager row (not_installed, port 8080, auto_update=1, provider_expose=1) - 7 lifecycle API routes under /api/services/bifrost/ (verbatim from cliproxy, errors sanitized) — loopback-only via existing LOCAL_ONLY_API_PREFIXES - Shared [name]/logs branch for bifrost - Dashboard tab + registration in the services page shell - Relay auto-wiring: getBifrostRoutingConfig defaults BIFROST_BASE_URL to the supervised port when the instance is running; explicit env still wins; the env-only relay path (/v1/relay/.../bifrost) stays unchanged (compat layer) - Docs (EMBEDDED-SERVICES, openapi) + unit tests (installer/route-guard/routing, 19 tests) + RUN_SERVICES_INT-gated integration lifecycle Note: the actual Go-binary install/start/health path requires a documented VPS live-test before merge (Hard Rule cheaper-inference#18 / spec section 7); the gated integration harness is the vehicle for that run.

gemini-code-assist Bot reviewed Jul 1, 2026

View reviewed changes

diegosouzapw mentioned this pull request Jul 2, 2026

[codex] Add router backend registry #5868

Closed

diegosouzapw changed the base branch from release/v3.8.43 to main July 2, 2026 04:15

diegosouzapw force-pushed the feat/bifrost-supervised-service branch from 8d08f1a to b83d351 Compare July 2, 2026 04:15

This was referenced Jul 2, 2026

question: clarify 9router/cliproxy backend model and service-route routing contract #5603

Closed

docs(architecture): Router Backends & Embedded Services ADR (#5603) #5891

Merged

diegosouzapw changed the base branch from main to release/v3.8.44 July 2, 2026 16:18

diegosouzapw force-pushed the feat/bifrost-supervised-service branch from b83d351 to 0ceab1c Compare July 3, 2026 12:05

diegosouzapw merged commit 556f191 into release/v3.8.44 Jul 3, 2026
1 of 3 checks passed

diegosouzapw deleted the feat/bifrost-supervised-service branch July 3, 2026 12:12

diegosouzapw mentioned this pull request Jul 4, 2026

Release v3.8.44 #5925

Merged

diegosouzapw mentioned this pull request Jul 20, 2026

feat(backend): unified RouterBackend contract across CLIProxyAPI, Bifrost, VibeProxy, and native hot paths #5670

Closed

Navigation Menu

Repository navigation

feat(services): promote Bifrost to embedded/supervised service (#5670) - #5817

feat(services): promote Bifrost to embedded/supervised service (#5670)#5817
diegosouzapw merged 1 commit into
release/v3.8.44from
feat/bifrost-supervised-service

diegosouzapw commented Jul 1, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jul 1, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Uh oh!

diegosouzapw commented Jul 2, 2026

Uh oh!

github-actions Bot commented Jul 2, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		const sup = getSupervisor("bifrost");
		const wasRunning = sup?.getStatus().state === "running";

	autoStart: row?.autoStart ?? false,
	autoStart: !!row?.autoStart,

Conversation

diegosouzapw commented Jul 1, 2026

What

Implementation (8 steps, mapped to the design spec)

Tests

⚠️ Merge gate — VPS live-test required (Hard Rule #18 / spec §7)

Out of scope

Uh oh!

chatgpt-codex-connector Bot commented Jul 1, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

diegosouzapw commented Jul 2, 2026

Re-stacked on the provider-manifest stack (#5869 → #5870)

Uh oh!

github-actions Bot commented Jul 2, 2026

CI Coverage Report

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant