Repository navigation
feat(gateway): dynamic routes via dynamic/<name> - #3432
Conversation
Add Cloudflare-AI-Gateway-style named dynamic routes: versioned per-project routing graphs invoked by putting dynamic/<name> in the model field. Graphs support conditional nodes (header/body/metadata), session-sticky percentage splits, model nodes with optional provider allowlists, and end nodes. Resolution hands off to the existing auto-routing scoring and provider fallback machinery. - packages/shared: dynamic-route graph Zod schema + validation - packages/db: dynamic_route + dynamic_route_version tables (draft + published version pointer, instant rollback) - gateway: parse dynamic/ prefix, SWR-cached route loader, pure graph evaluator, chat.ts handoff into the auto-routing block - api: enterprise-gated CRUD with draft/publish/rollback/versions - ui: Dynamic Routes settings page with JSON editor - reserve "dynamic" as a custom provider name Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds project-scoped dynamic routes with validated graphs, drafts, publishing, rollback, version history, gateway execution, routing metadata, and an enterprise dashboard management page. ChangesDynamic Routes
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Dashboard
participant DynamicRoutesAPI
participant Database
participant ChatGateway
participant RouteEvaluator
Dashboard->>DynamicRoutesAPI: create or update route draft
DynamicRoutesAPI->>Database: store route graph
Dashboard->>DynamicRoutesAPI: publish draft
DynamicRoutesAPI->>Database: create version and update published pointer
ChatGateway->>Database: load enabled published route
ChatGateway->>RouteEvaluator: evaluate graph with request context
RouteEvaluator-->>ChatGateway: selected model and traversal path
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1⚔️ Resolve merge conflicts 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Adds “dynamic routes” to the gateway: requests can pass model: "dynamic/<name>" and the gateway resolves that name to a published, versioned routing graph at request time, then reuses existing auto-routing/scoring/fallback for the resolved target.
Changes:
- Introduces a shared dynamic-route graph schema + evaluator (conditional/percentage/model/end) and dynamic model parsing.
- Adds DB tables + API CRUD/publish/rollback/version endpoints for managing dynamic routes (enterprise-gated) and a gateway loader for published routes.
- Adds a UI “Dynamic Routes” project settings page for editing JSON graphs and publishing/rollback/version history.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/shared/src/index.ts | Re-exports new reserved custom-provider constants. |
| packages/shared/src/dynamic-route.ts | Adds dynamic-route prefix/name parsing and Zod graph schema validated against model catalog. |
| packages/shared/src/custom-providers.ts | Adds reserved custom-provider name list (e.g. "dynamic"). |
| packages/shared/package.json | Exposes ./dynamic-route entrypoint for shared consumers. |
| packages/db/src/schema.ts | Adds dynamic_route + dynamic_route_version tables/columns for draft + published snapshots. |
| packages/db/src/relations.ts | Adds Drizzle relations for dynamic routes and versions. |
| packages/db/migrations/meta/_journal.json | Records the new migration. |
| packages/db/migrations/1785948725_funny_triathlon.sql | Creates dynamic route tables + indexes + FKs. |
| packages/actions/src/get-cheapest-from-available-providers.ts | Extends RoutingMetadata with dynamicRoute (name/version/path). |
| apps/ui/src/components/dashboard/dashboard-sidebar.tsx | Adds “Dynamic Routes” link under project settings. |
| apps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/page.tsx | New settings page wrapper for the dynamic routes client UI. |
| apps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsx | New dynamic routes management UI (list/create/edit draft/publish/rollback/version history). |
| apps/gateway/src/lib/dynamic-route-loader.ts | Adds SWR-cached loader for published route graph by project/name. |
| apps/gateway/src/chat/tools/parse-model-input.ts | Tags dynamic/<name> model inputs and defers to auto placeholder. |
| apps/gateway/src/chat/tools/parse-model-input.spec.ts | Adds unit coverage for dynamic route parsing and validation. |
| apps/gateway/src/chat/tools/evaluate-dynamic-route.ts | Implements graph evaluation (conditions + deterministic percentage splits + end/model). |
| apps/gateway/src/chat/tools/evaluate-dynamic-route.spec.ts | Adds evaluator + schema validation unit tests. |
| apps/gateway/src/chat/chat.ts | Resolves dynamic route before auto-routing; narrows candidates; records routing metadata; fails closed on no match. |
| apps/api/src/routes/routing-config.ts | Exports checkProjectEnterpriseAccess for reuse by dynamic routes. |
| apps/api/src/routes/keys-provider.ts | Prevents creating custom providers with reserved names (e.g. "dynamic"). |
| apps/api/src/routes/index.ts | Registers /dynamic-routes routes. |
| apps/api/src/routes/dynamic-routes.ts | Adds dynamic routes CRUD + draft/publish/rollback + version history endpoints. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // When free_models_only is true, only consider models marked as free | ||
| // Otherwise, only consider hardcoded allowed models | ||
| if (effectiveFreeModelsOnly) { | ||
| else if (effectiveFreeModelsOnly) { | ||
| if (!("free" in modelDef && modelDef.free)) { | ||
| continue; |
| const version = await db.query.dynamicRouteVersion.findFirst({ | ||
| where: { id: { eq: body.versionId }, routeId: { eq: route.id } }, | ||
| }); | ||
| if (!version) { | ||
| throw new HTTPException(404, { | ||
| message: "Version not found for this route", | ||
| }); | ||
| } | ||
|
|
| const conditionFieldSchema = z.object({ | ||
| /** | ||
| * - `header`: request header (path = header name, case-insensitive) | ||
| * - `body`: dot-path into the JSON request body (e.g. "metadata.plan") | ||
| * - `metadata`: gateway request context; path is one of "orgId", | ||
| * "projectId", "apiKeyId", "plan" | ||
| */ | ||
| source: z.enum(["header", "body", "metadata"]), | ||
| path: z.string().min(1).max(256), | ||
| }); |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/api/src/routes/routing-config.ts (1)
33-38: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winThe error messages now reach dynamic-route callers and name the wrong feature.
apps/api/src/routes/dynamic-routes.tscalls this helper for every dynamic-route endpoint. A user who lacks access there receives "Only owners and admins can manage routing configuration" or "Routing configuration requires an enterprise plan". Both messages name routing configuration, not dynamic routes. Pass the feature name so each caller reports the correct resource.♻️ Proposed change
export async function checkProjectEnterpriseAccess( userId: string, projectId: string, + feature = "routing configuration", ): Promise<{ project: { id: string; organizationId: string }; }> { @@ if (userOrg.role !== "owner" && userOrg.role !== "admin") { throw new HTTPException(403, { - message: "Only owners and admins can manage routing configuration", + message: `Only owners and admins can manage ${feature}`, }); } @@ if (userOrg.organization?.plan !== "enterprise") { throw new HTTPException(403, { - message: "Routing configuration requires an enterprise plan", + message: `${feature} requires an enterprise plan`, }); }Then call
checkProjectEnterpriseAccess(user.id, projectId, "dynamic routes")from the dynamic-route handlers.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/api/src/routes/routing-config.ts` around lines 33 - 38, Update checkProjectEnterpriseAccess to accept a feature-name parameter and use it in both access-denied error messages instead of hardcoding routing configuration. Update routing-config callers to pass the routing configuration feature name, and update every dynamic-route handler calling this helper to pass "dynamic routes".
🧹 Nitpick comments (7)
apps/gateway/src/chat/tools/evaluate-dynamic-route.spec.ts (1)
305-337: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a schema test for cycle rejection.
The comment on line 306 states that a two-node cycle bypasses save-time validation. That is accurate today, because
dynamicRouteGraphSchemadoes not detect cycles. Keep this hop-bound test as a runtime safeguard. Add adynamicRouteGraphSchemacase that asserts the same graph fails validation, after cycle detection is added inpackages/shared/src/dynamic-route.ts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/chat/tools/evaluate-dynamic-route.spec.ts` around lines 305 - 337, Add a schema-validation test for the same two-node cycle used in the hop-bound test, asserting that dynamicRouteGraphSchema rejects it after cycle detection is implemented in dynamic-route.ts. Keep the existing evaluateDynamicRoute runtime safeguard test unchanged and reuse the graph structure and validation assertion conventions already present in the surrounding tests.apps/gateway/src/lib/dynamic-route-loader.ts (1)
37-58: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider the relations query API for consistency.
apps/api/src/routes/dynamic-routes.tslines 67-92 read the same data withdb().query.dynamicRoute.findFirst({ with: { publishedVersion: true } }). This loader uses the select-builder join instead. The behavior is equivalent, and project scoping is correct here. Aligning on the relations API would keep both readers on one shape and avoid drift if the version pointer changes.The Drizzle read convention in the coding guidelines targets
packages/db/src/**, so this is a consistency suggestion, not a violation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/lib/dynamic-route-loader.ts` around lines 37 - 58, Update the dynamic-route lookup in the loader to use the Drizzle relations API, specifically dynamicRoute.findFirst with publishedVersion included, instead of the select-builder innerJoin. Preserve the existing projectId, name, and enabled filters, limit behavior, and null result when no route is found; adapt the returned fields to the loader’s current shape.apps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsx (2)
26-40: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
STARTER_GRAPHpins two model ids that the catalog can retire.
gpt-5-nanoandgemini-2.5-flashare literals.dynamicRouteGraphSchemarejects an unknown model, so if either id leaves the catalog, both the Create button at line 388 and the default draft text stop validating for every user. Derive the starter models from the model catalog, or keep the starter graph in@llmgateway/sharednext to the schema so both move together.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsx around lines 26 - 40, Update STARTER_GRAPH to avoid hard-coded model IDs that can become invalid when the catalog changes. Derive its model values from the shared model catalog, or move the starter graph alongside dynamicRouteGraphSchema in `@llmgateway/shared` so the defaults and validation remain synchronized.
284-300: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReplace the
as nevercasts with client-side parsing through the shared schema.
handleSaveDraftvalidates only that the text is JSON, then casts withgraph as never. Line 392 castsSTARTER_GRAPH as neverfor the same reason. Both casts defeat the typed client and defer every structural error to the API round trip.dynamicRouteGraphSchemais already exported from@llmgateway/shared/dynamic-routeand is the same schema the API applies. Parse with it in the client to get a typed value and a specific message for a bad node reference or an unknown model.As per coding guidelines: "This is a pure TypeScript project: never use
anyoras anyunless absolutely necessary."♻️ Proposed refactor
+import { dynamicRouteGraphSchema } from "`@llmgateway/shared/dynamic-route`"; + +import type { DynamicRouteGraph } from "`@llmgateway/shared/dynamic-route`";const handleSaveDraft = () => { if (!selectedName) { return; } - let graph: unknown; + let graph: DynamicRouteGraph; try { - graph = JSON.parse(effectiveDraftText); + graph = dynamicRouteGraphSchema.parse(JSON.parse(effectiveDraftText)); } catch (error) { - setDraftError(`Invalid JSON: ${String(error)}`); + setDraftError( + error instanceof Error ? error.message : "Invalid graph", + ); return; } setDraftError(null); draftMutation.mutate({ params: { path: { projectId, name: selectedName } }, - body: { graph: graph as never }, + body: { graph }, }); };Type
STARTER_GRAPHasDynamicRouteGraphat line 26 so the create call at line 392 also drops its cast. Confirm that@llmgateway/shared/dynamic-routeis importable fromapps/ui, and that pulling in the schema's model and provider catalog does not grow the client bundle beyond what is acceptable.Also applies to: 387-395
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsx around lines 284 - 300, Update handleSaveDraft to validate effectiveDraftText with dynamicRouteGraphSchema after JSON parsing, surface the schema validation message through setDraftError, and pass the resulting typed graph to draftMutation without an as never cast. Type STARTER_GRAPH as DynamicRouteGraph and remove its corresponding cast in the create flow around the STARTER_GRAPH usage, reusing the shared `@llmgateway/shared/dynamic-route` exports without introducing any or as any.Source: Coding guidelines
packages/db/src/schema.ts (1)
3746-3749: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low valueThe
project_idindex duplicates the unique constraint prefix.
unique("dynamic_route_project_id_name_idx")on(project_id, name)already creates a B-tree index whose leading column isproject_id. Postgres can use that index forproject_id-only lookups, soindex("dynamic_route_project_id_idx")adds write cost without a new access path. Consider dropping it.♻️ Proposed change
(table) => [ unique("dynamic_route_project_id_name_idx").on(table.projectId, table.name), - index("dynamic_route_project_id_idx").on(table.projectId), ],Also remove the matching
CREATE INDEXfrompackages/db/migrations/1785948725_funny_triathlon.sql, and regenerate withpnpm migrationsinstead of hand-editing the journal/snapshot.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/src/schema.ts` around lines 3746 - 3749, Remove the redundant projectId-only index declaration from the dynamic route table definition, keeping the existing unique constraint on projectId and name. Regenerate migrations using pnpm migrations so the matching CREATE INDEX is removed from the generated migration and related metadata remains consistent; do not hand-edit the journal or snapshot.apps/api/src/routes/dynamic-routes.ts (2)
496-510: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winVersion history returns every version with its full graph and no limit.
Each publish appends a row, and each row carries a complete graph up to
DYNAMIC_ROUTE_MAX_NODESnodes. A long-lived route makes this response grow without bound, and the UI atapps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsxrenders all of them. Add alimitwith an optional cursor, and consider omittinggraphfrom the list while serving it from a per-version endpoint.♻️ Minimal change
const versions = await db.query.dynamicRouteVersion.findMany({ where: { routeId: { eq: route.id } }, orderBy: { version: "desc" }, + limit: 50, });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/api/src/routes/dynamic-routes.ts` around lines 496 - 510, Update the version-history query and response in the dynamic route handler around findMany to support a bounded limit and optional cursor, returning only the fields needed for the list instead of each full version.graph. Add or reuse a per-version endpoint to fetch an individual graph, and update the client’s version-history flow to request additional pages and fetch graph data only when needed.
439-456: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winRollback skips the catalog re-validation that publish performs.
Lines 376-383 re-validate the draft at publish time so a graph that references a removed model can never become the published version. Rollback re-points
publishedVersionIdto a stored version with no equivalent check. An operator can therefore restore a version whosemodelorprovidersno longer exist in the catalog, and the gateway then fails at request time instead of at rollback time. Apply the samedynamicRouteGraphSchema.safeParseguard toversion.graph.♻️ Proposed fix
if (!version) { throw new HTTPException(404, { message: "Version not found for this route", }); } + + // Same guard as publish: a stored version can reference a model or provider + // that has since left the catalog. + const parsed = dynamicRouteGraphSchema.safeParse(version.graph); + if (!parsed.success) { + throw new HTTPException(400, { + message: `Version ${version.version} is no longer valid: ${parsed.error.issues[0]?.message}`, + }); + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/api/src/routes/dynamic-routes.ts` around lines 439 - 456, Before updating publishedVersionId in the rollback flow, apply the same dynamicRouteGraphSchema.safeParse validation used by the publish path to version.graph. Reject invalid graphs, including references to unavailable models or providers, and only perform the existing update when validation succeeds.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/api/src/routes/dynamic-routes.ts`:
- Around line 178-195: Update the dynamic-route creation flow around the
existing check and cdb.insert call so the insert catches a database
unique-constraint violation for the project/name pair and converts it to
HTTPException 409 with the existing conflict message. The pre-check may remain
as a fast path, but no concurrent duplicate insert should propagate an unhandled
driver error.
- Around line 385-403: The publish flow around dynamicRouteVersion insertion and
dynamicRoute publishedVersionId updating must run inside a single
cdb.transaction() so both writes commit or roll back together. Within that
transaction, derive the next version using a database-side atomic operation or
locking strategy rather than the cached latest read, preventing concurrent
publishes from selecting the same route version; keep the existing graph and
creator data and repoint the route only after the version insert succeeds.
In `@apps/gateway/src/chat/chat.ts`:
- Around line 3231-3240: Update the model candidate filtering around
dynamicRouteSelection and effectiveFreeModelsOnly so free_models_only is
enforced after dynamic-route resolution, including for the resolved model. If
the resolved target is paid or no free matching model remains, return the
existing 400 error path instead of allowing an upstream request; retain the
hardcoded allowlist behavior for non-dynamic routes.
In `@apps/gateway/src/chat/tools/evaluate-dynamic-route.ts`:
- Around line 58-59: Restrict the metadata path accepted by conditionFieldSchema
to the four documented metadata keys, using an enum or equivalent literal-union
constraint instead of a general string. Update the schema definition in
conditionFieldSchema so invalid or prototype keys are rejected during
validation, while resolveFieldValue continues resolving the permitted keys
unchanged.
In
`@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsx:
- Around line 439-457: Add an accessible aria-label to the icon-only delete
Button in the dynamic-routes client component, clearly identifying that it
deletes the corresponding route. Keep the existing click confirmation and
mutation behavior unchanged.
- Around line 278-282: Update the effectiveDraftText fallback near
detail.draftGraph to use the published graph from detail when no draft exists,
before falling back to STARTER_GRAPH. Preserve draftText precedence and the
existing JSON formatting, ensuring routes with a published graph display that
graph rather than the starter template.
- Around line 222-254: Add a shared error-toast handler and assign it as onError
for rollbackMutation, patchMutation, and deleteMutation, matching the
destructive failure behavior used by createMutation, draftMutation, and
publishMutation. Reuse the helper to replace those existing duplicated onError
bodies without changing the mutations’ success behavior.
In `@packages/shared/src/dynamic-route.ts`:
- Around line 223-248: dynamicRouteGraphSchema currently permits cyclic graphs,
so move cycle failures to validation time. In
packages/shared/src/dynamic-route.ts lines 223-248, extend the superRefine logic
beside the reachability check to detect cycles and add a validation issue
identifying the node that closes each cycle. In
apps/gateway/src/chat/tools/evaluate-dynamic-route.spec.ts lines 305-337,
preserve the existing hop-bound runtime test and add a
dynamicRouteGraphSchema.safeParse assertion showing the same two-node cycle is
rejected.
---
Outside diff comments:
In `@apps/api/src/routes/routing-config.ts`:
- Around line 33-38: Update checkProjectEnterpriseAccess to accept a
feature-name parameter and use it in both access-denied error messages instead
of hardcoding routing configuration. Update routing-config callers to pass the
routing configuration feature name, and update every dynamic-route handler
calling this helper to pass "dynamic routes".
---
Nitpick comments:
In `@apps/api/src/routes/dynamic-routes.ts`:
- Around line 496-510: Update the version-history query and response in the
dynamic route handler around findMany to support a bounded limit and optional
cursor, returning only the fields needed for the list instead of each full
version.graph. Add or reuse a per-version endpoint to fetch an individual graph,
and update the client’s version-history flow to request additional pages and
fetch graph data only when needed.
- Around line 439-456: Before updating publishedVersionId in the rollback flow,
apply the same dynamicRouteGraphSchema.safeParse validation used by the publish
path to version.graph. Reject invalid graphs, including references to
unavailable models or providers, and only perform the existing update when
validation succeeds.
In `@apps/gateway/src/chat/tools/evaluate-dynamic-route.spec.ts`:
- Around line 305-337: Add a schema-validation test for the same two-node cycle
used in the hop-bound test, asserting that dynamicRouteGraphSchema rejects it
after cycle detection is implemented in dynamic-route.ts. Keep the existing
evaluateDynamicRoute runtime safeguard test unchanged and reuse the graph
structure and validation assertion conventions already present in the
surrounding tests.
In `@apps/gateway/src/lib/dynamic-route-loader.ts`:
- Around line 37-58: Update the dynamic-route lookup in the loader to use the
Drizzle relations API, specifically dynamicRoute.findFirst with publishedVersion
included, instead of the select-builder innerJoin. Preserve the existing
projectId, name, and enabled filters, limit behavior, and null result when no
route is found; adapt the returned fields to the loader’s current shape.
In
`@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsx:
- Around line 26-40: Update STARTER_GRAPH to avoid hard-coded model IDs that can
become invalid when the catalog changes. Derive its model values from the shared
model catalog, or move the starter graph alongside dynamicRouteGraphSchema in
`@llmgateway/shared` so the defaults and validation remain synchronized.
- Around line 284-300: Update handleSaveDraft to validate effectiveDraftText
with dynamicRouteGraphSchema after JSON parsing, surface the schema validation
message through setDraftError, and pass the resulting typed graph to
draftMutation without an as never cast. Type STARTER_GRAPH as DynamicRouteGraph
and remove its corresponding cast in the create flow around the STARTER_GRAPH
usage, reusing the shared `@llmgateway/shared/dynamic-route` exports without
introducing any or as any.
In `@packages/db/src/schema.ts`:
- Around line 3746-3749: Remove the redundant projectId-only index declaration
from the dynamic route table definition, keeping the existing unique constraint
on projectId and name. Regenerate migrations using pnpm migrations so the
matching CREATE INDEX is removed from the generated migration and related
metadata remains consistent; do not hand-edit the journal or snapshot.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 43090ac0-742f-4f15-a2c0-d387cf21e164
📒 Files selected for processing (23)
apps/api/src/routes/dynamic-routes.tsapps/api/src/routes/index.tsapps/api/src/routes/keys-provider.tsapps/api/src/routes/routing-config.tsapps/gateway/src/chat/chat.tsapps/gateway/src/chat/tools/evaluate-dynamic-route.spec.tsapps/gateway/src/chat/tools/evaluate-dynamic-route.tsapps/gateway/src/chat/tools/parse-model-input.spec.tsapps/gateway/src/chat/tools/parse-model-input.tsapps/gateway/src/lib/dynamic-route-loader.tsapps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsxapps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/page.tsxapps/ui/src/components/dashboard/dashboard-sidebar.tsxpackages/actions/src/get-cheapest-from-available-providers.tspackages/db/migrations/1785948725_funny_triathlon.sqlpackages/db/migrations/meta/1785948725_snapshot.jsonpackages/db/migrations/meta/_journal.jsonpackages/db/src/relations.tspackages/db/src/schema.tspackages/shared/package.jsonpackages/shared/src/custom-providers.tspackages/shared/src/dynamic-route.tspackages/shared/src/index.ts
- enforce free_models_only (incl. test wallets) on route-selected models - reject cyclic graphs, unknown metadata paths, and array values for non-in operators at validation time - guard prototype keys in metadata/body field resolution - re-validate version graphs on rollback like publish does - make create rely on the unique constraint (409 instead of racy 500) - wrap publish in a transaction with DB-derived version numbers - UI: error toasts for rollback/patch/delete, published-graph draft fallback, aria-label on delete button Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Addressed all review feedback in 0886704:
All 30 unit tests pass (4 new); gateway/api/ui builds green. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/api/src/routes/dynamic-routes.ts`:
- Around line 388-405: Serialize per-route version allocation in the transaction
around the publish flow: lock or otherwise serialize the corresponding
dynamicRoute row before the dynamicRouteVersion insert derives max(version) + 1.
Update the transaction beginning before the insert, while preserving the
existing version insert and publishedVersionId update behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 97f74e03-53f1-42c0-8b86-fb0493a0d0eb
📒 Files selected for processing (6)
apps/api/src/routes/dynamic-routes.tsapps/gateway/src/chat/chat.tsapps/gateway/src/chat/tools/evaluate-dynamic-route.spec.tsapps/gateway/src/chat/tools/evaluate-dynamic-route.tsapps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsxpackages/shared/src/dynamic-route.ts
🚧 Files skipped from review as they are similar to previous changes (4)
- apps/gateway/src/chat/chat.ts
- apps/gateway/src/chat/tools/evaluate-dynamic-route.spec.ts
- apps/gateway/src/chat/tools/evaluate-dynamic-route.ts
- apps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsx
Add a Visual/JSON tab pair to the dynamic-route editor. The visual tab is a React Flow canvas with drag-and-drop node palette (model, conditional, percentage, end), per-branch connection handles, a node inspector (model picker from the catalog, ordered provider fallback list, condition and split editing), and a synthetic Start node for the entry edge. Edges are derived from branch targets so connect/delete operations stay consistent, and a deterministic layered layout places loaded graphs. Both tabs share one draft: visual edits sync to the JSON text and switching to Visual re-reads it (blocked while the JSON is invalid). Saving is gated on the same shared Zod schema the API enforces, with friendly structural errors (unconnected branches, unset models) shown before catalog-level issues. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (3)
apps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/flow-graph.ts (1)
96-101: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueDrop the
as DynamicRouteGraphcast.The cast suppresses type checking on the returned object. If
DynamicRouteGraphlater gains a required field, this function keeps compiling and produces an invalid graph. Type the return through the object literal instead.♻️ Proposed refactor
export function editorToGraph(state: EditorState): DynamicRouteGraph { - return { - entry: state.entry, - nodes: state.nodes.map((n) => n.node), - } as DynamicRouteGraph; + return { + entry: state.entry, + nodes: state.nodes.map((n) => n.node), + }; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/flow-graph.ts around lines 96 - 101, Update editorToGraph so the returned object literal is type-checked as DynamicRouteGraph without using an `as DynamicRouteGraph` assertion. Preserve the existing `entry` and mapped `nodes` fields while allowing missing required fields to surface as compile-time errors.apps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/route-flow-editor.tsx (2)
294-303: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winRun the Zod validation only when the graph content changes.
state.nodesgets a new array reference on every position change. Each drag frame therefore triggersvalidateEditorState, which runs a fulldynamicRouteGraphSchema.safeParse. Positions are not part of the serialized graph, so this work has no effect on the reported errors. Reuse the serialization guard for validation as well.♻️ Proposed refactor
useEffect(() => { const graph = editorToGraph(state); const serialized = JSON.stringify(graph); - if (serialized !== initialSerialized.current) { - initialSerialized.current = serialized; - onGraphChange(graph); - } - onValidationChange(validateEditorState(state)); + if (serialized === initialSerialized.current) { + return; + } + initialSerialized.current = serialized; + onGraphChange(graph); + onValidationChange(validateEditorState(state)); // eslint-disable-next-line react-hooks/exhaustive-deps }, [state.entry, state.nodes]);Keep an initial
onValidationChangecall on mount so the parent receives the errors of the loaded draft.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/route-flow-editor.tsx around lines 294 - 303, Update the useEffect in the route-flow editor to run validateEditorState and call onValidationChange only when the serialized graph content differs from initialSerialized.current, preventing validation during position-only updates. Preserve an initial validation call on mount so the parent receives errors for the loaded draft, while keeping onGraphChange behavior tied to actual graph changes.
750-768: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low valueNumeric condition values collapse to
0while the user types.
parseConditionValuereturns0for any non-finite input, andNumber("")is0. The user cannot clear the field, and an intermediate value such as-becomes0. Keep the raw text in component state, and convert only when the value is finite and non-empty.Also applies to: 892-909
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/route-flow-editor.tsx around lines 750 - 768, Update parseConditionValue and the associated condition editor state to preserve raw numeric text while the user is typing, including empty and intermediate values such as "-". For "gt" and "lt", convert the text only when it is non-empty and Number(text) is finite; otherwise retain the raw text until a valid numeric value is entered, and apply the same behavior in the alternate code path around the condition editor handlers.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/flow-graph.ts:
- Around line 1-6: Add exported DynamicRouteGraph and DynamicRouteNode type
definitions to the shared dynamic-route module alongside
dynamicRouteGraphSchema, then expose the DynamicRoute* symbols from the shared
package root index consistently with its existing exports so the UI import and
root consumers type-check.
In
`@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/route-flow-editor.tsx:
- Around line 487-502: Update onDrop to validate the value returned by
getData(DND_MIME) against the known DynamicRouteNode types before calling
addNode. Return early for any unsupported string, ensuring createNode receives
only valid types and the editor never stores an undefined node.
- Around line 427-440: Update nodeIdSchema to reject node IDs containing “:”
while preserving its existing non-empty, 1–64 character constraints, before
graph state can be changed. This keeps edge IDs produced by the route-flow
editor unambiguous for onEdgesChange and its colon-based splitting.
---
Nitpick comments:
In
`@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/flow-graph.ts:
- Around line 96-101: Update editorToGraph so the returned object literal is
type-checked as DynamicRouteGraph without using an `as DynamicRouteGraph`
assertion. Preserve the existing `entry` and mapped `nodes` fields while
allowing missing required fields to surface as compile-time errors.
In
`@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/route-flow-editor.tsx:
- Around line 294-303: Update the useEffect in the route-flow editor to run
validateEditorState and call onValidationChange only when the serialized graph
content differs from initialSerialized.current, preventing validation during
position-only updates. Preserve an initial validation call on mount so the
parent receives errors for the loaded draft, while keeping onGraphChange
behavior tied to actual graph changes.
- Around line 750-768: Update parseConditionValue and the associated condition
editor state to preserve raw numeric text while the user is typing, including
empty and intermediate values such as "-". For "gt" and "lt", convert the text
only when it is non-empty and Number(text) is finite; otherwise retain the raw
text until a valid numeric value is entered, and apply the same behavior in the
alternate code path around the condition editor handlers.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 98dc7637-de5e-4eed-8181-ccd576d4cb69
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (4)
apps/ui/package.jsonapps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsxapps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/flow-graph.tsapps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/route-flow-editor.tsx
🚧 Files skipped from review as they are similar to previous changes (1)
- apps/ui/src/app/dashboard/[orgId]/[projectId]/settings/dynamic-routes/_components/dynamic-routes-client.tsx
| import { dynamicRouteGraphSchema } from "@llmgateway/shared/dynamic-route"; | ||
|
|
||
| import type { | ||
| DynamicRouteGraph, | ||
| DynamicRouteNode, | ||
| } from "@llmgateway/shared/dynamic-route"; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Check the shared package export map and the dynamic-route entry point.
fd -t f 'package.json' packages/shared --max-depth 2 --exec jq '{name, exports, main, types}'
fd -t f 'dynamic-route.ts' packages/shared/src
rg -n 'dynamic-route' --glob '**/package.json'Repository: theopenco/llmgateway
Length of output: 1487
Share route schema types and export the submodule consistently.
@llmgateway/shared/dynamic-route points at dist/dynamic-route.{js,d.ts}, but packages/shared/src/dynamic-route.ts contains only schema values without exported DynamicRouteGraph/DynamicRouteNode types. Add the type exports there, otherwise this UI import cannot type-check, and add the DynamicRoute* exports to packages/shared/src/index.ts if callers should import them from the package root.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@apps/ui/src/app/dashboard/`[orgId]/[projectId]/settings/dynamic-routes/_components/flow-graph.ts
around lines 1 - 6, Add exported DynamicRouteGraph and DynamicRouteNode type
definitions to the shared dynamic-route module alongside
dynamicRouteGraphSchema, then expose the DynamicRoute* symbols from the shared
package root index consistently with its existing exports so the UI import and
root consumers type-check.
- lock the route row during publish so concurrent publishes serialize instead of racing max(version)+1 into a unique-constraint 500 - restrict node ids to [a-zA-Z0-9_-] so composite edge ids stay parseable - split edge ids on the last colon when clearing branch targets - validate the drag-and-drop payload type before creating a node Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Addressed the second review round in a3dcc20:
23 unit tests pass; api/gateway/ui builds green. |
…uting # Conflicts: # pnpm-lock.yaml
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/content/features/dynamic-routes.mdx`:
- Around line 15-20: Replace leading spaces with tabs in the continued curl
command and JSON body at apps/docs/content/features/dynamic-routes.mdx lines
15-20 and apps/ui/src/content/changelog/2026-08-07-dynamic-routes.md lines
18-23. Run pnpm format and pnpm build before committing.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 7afea89f-5672-4fbd-bf98-683b6769abb7
⛔ Files ignored due to path filters (1)
apps/ui/public/changelog/dynamic-routes.pngis excluded by!**/*.png
📒 Files selected for processing (2)
apps/docs/content/features/dynamic-routes.mdxapps/ui/src/content/changelog/2026-08-07-dynamic-routes.md
| -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \ | ||
| -H "x-user-tier: paid" \ | ||
| -d '{ | ||
| "model": "dynamic/support", | ||
| "messages": [{"role": "user", "content": "Hello!"}] | ||
| }' |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use tabs in the fenced shell examples.
Both shell examples use spaces for indentation. Replace the leading spaces with tabs. Run pnpm format and pnpm build before committing.
apps/docs/content/features/dynamic-routes.mdx#L15-L20: use tabs for the continuedcurlcommand and JSON body.apps/ui/src/content/changelog/2026-08-07-dynamic-routes.md#L18-L23: use tabs for the continuedcurlcommand and JSON body.
As per coding guidelines, **/*.{ts,tsx,js,jsx,json,md,yml,yaml} must use tabs for indentation.
📍 Affects 2 files
apps/docs/content/features/dynamic-routes.mdx#L15-L20(this comment)apps/ui/src/content/changelog/2026-08-07-dynamic-routes.md#L18-L23
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/docs/content/features/dynamic-routes.mdx` around lines 15 - 20, Replace
leading spaces with tabs in the continued curl command and JSON body at
apps/docs/content/features/dynamic-routes.mdx lines 15-20 and
apps/ui/src/content/changelog/2026-08-07-dynamic-routes.md lines 18-23. Run pnpm
format and pnpm build before committing.
Source: Coding guidelines
Also fix the flow editor to follow the app theme instead of the OS setting, so React Flow's chrome renders dark in app dark mode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Images automagically compressed by Calibre's image-actions ✨ Compression reduced images by 66.2%, saving 1.6 MB.
|
|
Images automagically compressed by Calibre's image-actions ✨ Compression reduced images by 15.2%, saving 109.5 KB.
2 images did not require optimisation. |
|
Images automagically compressed by Calibre's image-actions ✨ Compression reduced images by 13.3%, saving 72.4 KB.
4 images did not require optimisation. |
|
Images automagically compressed by Calibre's image-actions ✨ Compression reduced images by 11.1%, saving 52.3 KB.
4 images did not require optimisation. |
Replace the plain datalist input and native select in the model-node inspector with the searchable ModelSelector (family logos, pricing, capabilities; rootOnly) and MultiProviderSelector (provider logos, options limited to providers serving the chosen model) used elsewhere in the dashboard. The selector emits provider/model ids, so the handler normalizes to the root catalog id the graph schema expects. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… cloudflare-dynamic-routing
|
The model-node inspector now reuses the dashboard's rich pickers instead of a plain text input and native select (commit 9595a48):
Verified end-to-end in the browser: picking a model via search stores the root id, adding a provider syncs into the JSON tab, and Save draft passes API validation.
|
Add request-path integration tests (gateway harness) covering the enterprise gate, 404 semantics, conditional/body/metadata evaluation, provider restriction vs natural routing, session-sticky splits, end nodes, free_models_only, and the recorded dynamicRoute metadata; plus API lifecycle tests for create/draft/publish/rollback/versions, validation rejections, and 409 conflicts. Also fix a bug the tests surfaced: evaluation received the schema-validated body, which strips unknown keys, so body conditions on caller-defined fields (e.g. metadata.segment) could never match. Evaluation now reads the raw request body. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Test coverage review for the routing feature, and the gaps are now filled in 7e9349c. Coverage by layer:
Writing the matrix also surfaced a real bug, fixed in the same commit: body conditions evaluated against the schema-validated body, which strips unknown keys — so a condition on a caller-defined field like 48 dynamic-route tests total across the four suites, all green. |
|
Reviewed the latest round: no critical/major findings are outstanding. The single new comment (tabs inside the fenced Status of all earlier major/high findings: free_models_only bypass, publish version race (row lock), cycle detection, metadata-path allowlist, rollback re-validation, create-race 409, node-id charset, and the raw-body evaluation bug are all fixed with tests; the two incorrect findings (missing type exports, shared-selector claims) were skipped with reasons in the earlier comments. |
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…uting # Conflicts: # packages/shared/package.json
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Problem
There is no way to define reusable, versioned routing logic. Callers must hardcode a model (or use
auto), so A/B tests, gradual rollouts, and per-segment model selection all require client-side changes and redeploys. Cloudflare AI Gateway ships this as "dynamic routing" (https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/).Approach
Named, versioned dynamic routes: put
dynamic/<name>in the OpenAI-compatiblemodelfield and the gateway evaluates a per-project routing graph at request time, then hands the resolved target to the existing scoring + fallback machinery (nothing re-implemented).Graph nodes (Zod-validated in
packages/shared/src/dynamic-route.ts, shared by API + gateway):conditional— if/else on request headers, body dot-paths, or request metadata (orgId/projectId/apiKeyId/plan); ops: eq, neq, in, contains, gt, lt, existspercentage— weighted splits, deterministic per session (x-session-idet al hashed via FNV-1a) so A/B assignments stay stable across a conversation; per-request otherwisemodel— target catalog model + optional provider allowlist; provider selection/fallback within the target still uses weighted smart routingend— explicit reject branch (400)Integration seam:
parseModelInputonly tags thedynamic/prefix and piggybacks the existingautoplaceholder model; the graph is evaluated inchat.tsright before the auto-routing block (where project/org context exists). The resolved model narrows the auto-candidate loop to exactly that model/providers, so IAM, compliance, capability filters, dev-plan gating, scoring, sticky sessions, and cross-provider retry all apply unchanged. A route never falls back to the hardcoded default model — it 400s instead.routingMetadata.dynamicRouterecords{name, version, path}for observability.Versioning:
dynamic_routeholds a mutable draft +published_version_idpointer; publishing snapshots the draft into an immutabledynamic_route_versionrow (re-validated against the current model catalog at publish time). Rollback just re-points the pointer. Gateway reads the published version through an SWR-cached loader; API mutations write viacdbso caches bust centrally.Access: enterprise-only (same gate as routing config).
dynamicis now a reserved custom-provider name so BYOK providers can't shadow the prefix.Surfaces:
/dynamic-routes/*CRUD with draft/publish/rollback/versions (Hono + Zod + OpenAPI)Verification
fallback.spec.tslog-wait timeout passes in isolation)gemini-2.5-flash✔x-session-id✔DATABASE_URL(past the 60s Drizzle-cache TTL, so the SWR stale mirrors were doing the work).dynamic/supportkept serving HTTP 200 with correct conditional routing and sticky percentage assignment; the log showsserving SWR stale fallbackfordynamicRoute:<project>:<name>alongside apiKey/project/org/IAM/provider-key/routing-config mirrors. Route lookups ride the same two-layer store as every other gateway config read (Redis query cache → Postgres → 4h stale mirror on failure), and API mutations invalidate both layers viacdb.pnpm format,pnpm build(17/17), migration generated via drizzle-kitDocs
Ships with a changelog entry (
/changelog/dynamic-routes, with generated OG image) and a docs page (docs.llmgateway.io/features/dynamic-routes) covering the graph format, node types, condition operators, validation rules, versioning/rollback, and the caching/stale-fallback behavior.Deliberate MVP omissions (follow-ups)
Screenshots
Route list with published/unpublished states:
Editing the
supportroute (conditional + percentage graph, publish controls, version history with one-click rollback):Visual flow editor (second commit): the route editor now has Visual/JSON tabs. The visual tab is a React Flow canvas with a drag-and-drop node palette, per-branch connection handles (each condition, else, and split is its own handle), and a node inspector with a catalog model picker and an ordered provider-fallback list. Both tabs edit the same draft; saving is blocked (with a readable error list) until the graph passes the same shared Zod validation the API enforces — unconnected branches, unset models, unknown catalog ids, and cycles all surface before save.
Example configs
Invoke any of these with
{"model": "dynamic/<name>", ...}on/v1/chat/completions.Tiered support — paid users (via header) get Claude, everyone else is A/B split 80/20:
{ "entry": "tier", "nodes": [ { "id": "tier", "type": "conditional", "conditions": [ { "field": { "source": "header", "path": "x-user-tier" }, "op": "eq", "value": "paid", "next": "premium" }, { "field": { "source": "body", "path": "metadata.beta" }, "op": "eq", "value": true, "next": "split" } ], "else": "split" }, { "id": "premium", "type": "model", "model": "claude-sonnet-4-6" }, { "id": "split", "type": "percentage", "splits": [ { "weight": 80, "next": "stable" }, { "weight": 20, "next": "experiment" } ] }, { "id": "stable", "type": "model", "model": "gpt-5-nano", "providers": ["openai"] }, { "id": "experiment", "type": "model", "model": "gemini-2.5-flash" } ] }Canary rollout — 95/5 gradual rollout, sticky per session so a conversation never flip-flops:
{ "entry": "rollout", "nodes": [ { "id": "rollout", "type": "percentage", "splits": [ { "weight": 95, "next": "current" }, { "weight": 5, "next": "canary" } ] }, { "id": "current", "type": "model", "model": "gpt-4o-mini" }, { "id": "canary", "type": "model", "model": "gpt-5-nano" } ] }Metadata gate — allow only enterprise-plan traffic, explicitly reject the rest with a 400:
{ "entry": "gate", "nodes": [ { "id": "gate", "type": "conditional", "conditions": [ { "field": { "source": "metadata", "path": "plan" }, "op": "eq", "value": "enterprise", "next": "allowed" } ], "else": "reject" }, { "id": "allowed", "type": "model", "model": "claude-sonnet-4-6" }, { "id": "reject", "type": "end" } ] }🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
dynamic/<route-name>support in chat completions.Validation
Documentation