docs: add Routing Plugins page (SDK + complexity-router proxy YAML) - #572
Conversation
…xy YAML Documents the Router routing-plugin pipeline (#32972) and the proxy YAML support for the complexity auto-router (#33251): what plugins can and can't do, RoutingContext shape, request lifecycle, and both SDK and config.yaml wiring for combining plugins with the complexity router.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6362cdd3a8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }, | ||
| }, | ||
| ], | ||
| plugins=[cost_ceiling_plugin], |
There was a problem hiding this comment.
In the complexity-router SDK example, passing the cost ceiling as a top-level Router(plugins=...) does not run it against the classified tier pool; that constructor pipeline only sees the outer auto_router/complexity_router deployment, while tier-pool filtering is wired through complexity_router_config.plugins. Users copying this snippet can still route to gpt-4o, so the plugin instance should be placed in the complexity_router_config dict instead.
Useful? React with 👍 / 👎.
|
|
||
| ## Reference | ||
|
|
||
| Config: [`router_settings.plugins`](./proxy/config_settings#router_settings---reference). |
There was a problem hiding this comment.
This reference points readers to router_settings.plugins, but the proxy YAML support documented above is complexity_router_config.plugins; the target row labels router_settings.plugins as SDK-only. Proxy users following this reference will put dotted paths under a key the proxy does not resolve for complexity-router tier filtering, so the final reference should point to/add the complexity-router config key instead.
Useful? React with 👍 / 👎.
2 participants
Summary
Adds a Routing Plugins doc page under Routing & Load Balancing, covering both the SDK entry point (
Router(plugins=[...]), BerriAI/litellm#32972) and the proxy YAML support that landed for the complexity auto-router (complexity_router_config.plugins, BerriAI/litellm#33251).The page reuses jeann2013's framing from discussion BerriAI/litellm#32168 (pipeline that enriches routing context; concrete e2e scenario with language, domain, tenant, budget plugins) and adds:
RoutingContextshape with per-field notes on read vs writecomplexity_router_config.plugins, dotted paths resolved relative toconfig.yaml) with theCostCeilingPluginexample from feat(router): resolve auto-router routing plugins from proxy YAML config litellm#33251's proof-of-fixsession_affinitydisabled when plugins configured,adaptive=True + pluginsraises at config-validation, tier narrowing that empties the pool raises rather than falling back todefault_modelusage-based-routingv1), proxy YAML wired only for complexity router today, include/exclude-only filtering (no weighted scoring yet)Also adds a sidebar entry for the new page under Routing & Load Balancing.
Test plan
npm run startlocally and confirm/docs/routing_pluginsrenders with correct code highlighting, admonition, and sidebar placementrouter_settings.pluginsindocs/proxy/config_settings.mdresolves