(MOT-3882) feat(providers): provider-owned system prompts, operator-overridable - #430
Merged
Merged
Conversation
…via llm-router config
Providers now author and declare their own identity prompt; the llm-router
serves it and lets operators override it per provider. The harness keeps only
a fallback default prompt and fetches the effective prompt at turn creation.
llm-router:
- ProviderDeclaration.system_prompt carries the provider-authored identity prompt
- new router::system_prompt::get resolves {provider?} -> operator override (config
slice, when set) -> provider-declared -> null; reads the config entry live per
request (no restart, no debounce)
- per-provider config slice gains a nullable system_prompt (format: "textarea");
set overrides, unset/null serves the provider default
providers (anthropic, openai, openai-codex, xai):
- each ships prompts/identity.txt and declares it; per-crate invariant test
harness:
- prompt module fetches identity from the router (RouterClient::system_prompt_get)
and composes locally (mode paragraph + identity + enrich/override); falls back
to the embedded default.txt when the router serves nothing
- drop the hardcoded provider->prompt family map and the anthropic/gpt/kimi bodies
- fix stale README claim (enrich, not override, is the default strategy)
console:
- schema-form String fields honor format: "textarea"; the env-pill Lexical editor
gains a multiline mode so ${VAR} pills work in long-form values
Verified live end-to-end on a dev engine: override reaches the model, unset
falls back to the declared prompt, and the console renders the multiline pill editor.
…on unset Two config-editor UX fixes for the per-provider system_prompt field: - The router now carries each provider's declared prompt as the config slice's system_prompt.default (via a testable provider_entry_schema helper). The console's NullableField already seeds the schema default on "set", so flipping an unset field to "set" now pre-fills the editor with the provider-declared prompt — an editable starting point instead of a blank box. - NullableField stashes the last set value in a ref and restores it when the operator flips unset -> set, so toggling "unset" no longer discards what was typed. The saved value is still null while unset (router serves the provider default); only the UI draft is preserved. Verified live: set pre-fills the declared prompt, and a custom edit survives an unset/set round-trip.
…e textarea is cleared Clearing the system_prompt editor left it empty, so StringField fell back to showing schema.default (the full ~21k provider-declared prompt) as a placeholder. The placeholder div is position:absolute; inset:0 with no clip, so the multi-KB string overflowed the box and painted over the fields below. - StringField no longer uses schema.default as a placeholder for textarea fields — there the default is the seed-on-set value, not a hint. - .env-lexical-placeholder gets overflow:hidden as a defensive clip so no long placeholder can ever spill past its box.
Same content and rules, ~70% fewer lines: prose collapsed into dense single-paragraph sections, discovery reference tightened to one line per function, examples kept.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
skill-check — worker0 verified, 33 skipped (no docs/).
Four for four. Nicely done. |
📝 WalkthroughWalkthroughThis PR adds a ChangesRouter System Prompt Feature
Estimated code review effort: 4 (Complex) | ~75 minutes Console Multiline Env Editor
Estimated code review effort: 2 (Simple) | ~15 minutes Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
andersonleal
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
console/web/src/pages/Configuration/tabs/WorkersTab/schema-form/env-lexical/EnvLexicalInput.tsx (1)
239-258: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winGate newline seeding on
multiline.seedEditorFromTemplateturns every\nintoLineBreakNodes for bothinitialConfig.editorStateandExternalValueSyncPlugin, so a single-line field with embedded newlines can still render as multiline. Threadmultilinethrough the seed path and keep line breaks only for textarea mode.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@console/web/src/pages/Configuration/tabs/WorkersTab/schema-form/env-lexical/EnvLexicalInput.tsx` around lines 239 - 258, The newline handling in seedEditorFromTemplate is unconditional, so single-line fields can still render with LineBreakNode content. Thread the multiline flag through the seeding path used by initialConfig.editorState and ExternalValueSyncPlugin, and only split text on "\n" / append $createLineBreakNode when multiline is true; otherwise keep the seeded value on one line. Use the seedEditorFromTemplate helper and its callers in EnvLexicalInput.tsx to gate this behavior consistently.harness/src/functions/send.rs (1)
119-147: 🚀 Performance & Scalability | 🟡 Minor | ⚡ Quick winMove the idempotency check before the router prompt fetch
harness/src/functions/send.rs:119-147
system_prompt_getstill runs before the dedupe lookup, so redelivered webhook retries pay for a router round-trip that gets thrown away.seed_or_mergestill needsoptions, so this only saves the duplicate-request path, but that’s a cheap win on a documented retry path.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@harness/src/functions/send.rs` around lines 119 - 147, The idempotency lookup in start currently happens after deps.router().await.system_prompt_get, so duplicate requests still pay for an unnecessary router round-trip. Reorder start in send.rs so the req.idempotency_key check and crate::state::get_idem happen immediately after cfg/session setup, before fetching the provider identity prompt, while keeping build_options and seed_or_merge using the resolved identity/options only on the non-deduplicated path.
🧹 Nitpick comments (4)
provider-xai/prompts/identity.txt (1)
1-260: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winFully duplicated identity prompt with
provider-openai.This file is identical to
provider-openai/prompts/identity.txt. Since both are embedded verbatim viainclude_str!in their respective crates, a future edit to one (e.g. fixing a rule, updating a doc URL, adding a caveat) can silently drift from the other unless the author remembers to mirror it in both places.Consider extracting the shared text into a single location both crates can
include_str!from (e.g. a sharedprompts/directory at the workspace root, or a small shared crate exposing the constant), keepingprovider-anthropic's condensed prompt separate since it's intentionally different.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@provider-xai/prompts/identity.txt` around lines 1 - 260, The identity prompt text is duplicated between the OpenAI and XAI providers, so edits can drift out of sync. Refactor the shared prompt content into one source that both provider crates load via include_str!, and update the relevant prompt-loading code to point at that shared location while keeping provider-anthropic’s shorter prompt separate. Use the existing prompt constants/loaders around the provider prompt files to locate and switch both consumers to the shared file.console/web/src/index.css (1)
321-330: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReplace deprecated
word-break: break-word.Per CSS Text spec,
word-break: break-wordis a deprecated legacy alias foroverflow-wrap: anywhere+word-break: normal. Static analysis correctly flags this.♻️ Proposed fix
.workers-tab .env-lexical-editor--multiline { white-space: pre-wrap; - word-break: break-word; + overflow-wrap: anywhere; + word-break: normal; overflow-x: visible; overflow-y: auto; max-height: 24rem; }Based on static analysis hints:
Deprecated keyword "break-word" for property "word-break".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@console/web/src/index.css` around lines 321 - 330, The multiline textarea styling in the `.workers-tab .env-lexical-editor--multiline` rule uses the deprecated `word-break: break-word` value; update this CSS to use the modern wrapping behavior instead by replacing that legacy keyword with the spec-compliant combination in the same selector. Keep the existing `white-space`, `overflow-x`, `overflow-y`, and `max-height` behavior intact while fixing the text-wrapping rule.Source: Linters/SAST tools
llm-router/src/register.rs (1)
205-215: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDuplicate provider-schema-construction block.
This loop is byte-for-byte identical to the one in
registry/register.rs(lines 90-99, inside the entry-write-lock block ofmake_provider_register). Consider extracting a shared helper (e.g.fn provider_schemas_from_registry(records: &[Record]) -> BTreeMap<String, Value>) to keep both call sites in sync as the schema-building contract evolves.♻️ Sketch of shared helper
fn provider_schemas(records: &[RegistryRecord]) -> BTreeMap<String, Value> { records .iter() .map(|rec| { let schema = provider_entry_schema( rec.declaration.config_schema.as_ref(), &serde_json::to_value(rec.declaration.defaults.clone()).unwrap_or(Value::Null), rec.declaration.system_prompt.as_deref(), ); (rec.declaration.id.clone(), schema) }) .collect() }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@llm-router/src/register.rs` around lines 205 - 215, The provider-schema construction logic is duplicated here and in make_provider_register, so extract the shared BTreeMap-building loop into a helper (for example, provider_schemas_from_registry or provider_schemas) and call it from register_entry flow and the registry/register.rs path. Keep the helper responsible for iterating registry.list() records, calling provider_entry_schema with config_schema/defaults/system_prompt, and inserting by declaration.id so both call sites stay synchronized.llm-router/src/system_prompt.rs (1)
46-51: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winEmpty-string
providerbypasses thedefault_providerfallback.
req.provider.or_else(...)only falls back todefault_providerwhenproviderisNone. An explicitprovider: ""(e.g. a serialization artifact or a caller bug) would be used as-is, skipping the fallback and very likely returning a null prompt for a provider id that was never intended. Giveneffective_promptalready filters empty strings symmetrically for the override/declared values, treating an emptyproviderthe same way keeps the precedence logic consistent.🩹 Proposed fix
- let provider = req.provider.or_else(|| { + let provider = req.provider.filter(|p| !p.is_empty()).or_else(|| { entry .get("default_provider") .and_then(Value::as_str) .map(String::from) });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@llm-router/src/system_prompt.rs` around lines 46 - 51, The provider selection in system prompt resolution treats only None as missing, so an explicit empty string can bypass the default_provider fallback. Update the provider lookup in the system_prompt logic to treat empty strings the same as absent values by filtering out blank req.provider before falling back to entry.get("default_provider"), keeping the precedence consistent with effective_prompt’s empty-string handling.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@llm-router/src/config/schema.rs`:
- Around line 61-95: The `system_prompt` field is only added when `properties`
already exists, so custom provider schemas like `validate_custom_schema` allows
for `{"type":"object"}` will miss the override knob entirely. Update
`provider_entry_schema` and `with_prompt_fields` to ensure a `properties` object
exists before inserting `system_prompt`, so minimal custom schemas still receive
the prompt field. Use the existing helpers and symbols (`provider_entry_schema`,
`with_prompt_fields`, `system_prompt_schema`) to centralize the fix and preserve
declared defaults.
- Around line 87-95: The custom config schema can currently shadow the
router-owned system_prompt field because with_prompt_fields only inserts it when
missing. Update validate_custom_schema and/or the schema composition path to
reserve system_prompt explicitly, rejecting it in user-provided schemas or
always overwriting it with system_prompt_schema(None) in with_prompt_fields.
Make sure the fix is applied around the validate_custom_schema and
with_prompt_fields logic so the console’s set/unset behavior remains controlled
by the router.
---
Outside diff comments:
In
`@console/web/src/pages/Configuration/tabs/WorkersTab/schema-form/env-lexical/EnvLexicalInput.tsx`:
- Around line 239-258: The newline handling in seedEditorFromTemplate is
unconditional, so single-line fields can still render with LineBreakNode
content. Thread the multiline flag through the seeding path used by
initialConfig.editorState and ExternalValueSyncPlugin, and only split text on
"\n" / append $createLineBreakNode when multiline is true; otherwise keep the
seeded value on one line. Use the seedEditorFromTemplate helper and its callers
in EnvLexicalInput.tsx to gate this behavior consistently.
In `@harness/src/functions/send.rs`:
- Around line 119-147: The idempotency lookup in start currently happens after
deps.router().await.system_prompt_get, so duplicate requests still pay for an
unnecessary router round-trip. Reorder start in send.rs so the
req.idempotency_key check and crate::state::get_idem happen immediately after
cfg/session setup, before fetching the provider identity prompt, while keeping
build_options and seed_or_merge using the resolved identity/options only on the
non-deduplicated path.
---
Nitpick comments:
In `@console/web/src/index.css`:
- Around line 321-330: The multiline textarea styling in the `.workers-tab
.env-lexical-editor--multiline` rule uses the deprecated `word-break:
break-word` value; update this CSS to use the modern wrapping behavior instead
by replacing that legacy keyword with the spec-compliant combination in the same
selector. Keep the existing `white-space`, `overflow-x`, `overflow-y`, and
`max-height` behavior intact while fixing the text-wrapping rule.
In `@llm-router/src/register.rs`:
- Around line 205-215: The provider-schema construction logic is duplicated here
and in make_provider_register, so extract the shared BTreeMap-building loop into
a helper (for example, provider_schemas_from_registry or provider_schemas) and
call it from register_entry flow and the registry/register.rs path. Keep the
helper responsible for iterating registry.list() records, calling
provider_entry_schema with config_schema/defaults/system_prompt, and inserting
by declaration.id so both call sites stay synchronized.
In `@llm-router/src/system_prompt.rs`:
- Around line 46-51: The provider selection in system prompt resolution treats
only None as missing, so an explicit empty string can bypass the
default_provider fallback. Update the provider lookup in the system_prompt logic
to treat empty strings the same as absent values by filtering out blank
req.provider before falling back to entry.get("default_provider"), keeping the
precedence consistent with effective_prompt’s empty-string handling.
In `@provider-xai/prompts/identity.txt`:
- Around line 1-260: The identity prompt text is duplicated between the OpenAI
and XAI providers, so edits can drift out of sync. Refactor the shared prompt
content into one source that both provider crates load via include_str!, and
update the relevant prompt-loading code to point at that shared location while
keeping provider-anthropic’s shorter prompt separate. Use the existing prompt
constants/loaders around the provider prompt files to locate and switch both
consumers to the shared file.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 1e06f8c0-5b18-49a0-9e5f-9ef8bc4ae102
📒 Files selected for processing (36)
console/web/src/index.cssconsole/web/src/pages/Configuration/tabs/WorkersTab/schema-form/NullableField.tsxconsole/web/src/pages/Configuration/tabs/WorkersTab/schema-form/StringField.tsxconsole/web/src/pages/Configuration/tabs/WorkersTab/schema-form/env-lexical/EnvLexicalInput.tsxharness/README.mdharness/prompts/anthropic.txtharness/prompts/kimi.txtharness/src/clients/router.rsharness/src/functions/send.rsharness/src/prompt/family.rsharness/src/prompt/mod.rsharness/src/prompt/tests.rsharness/src/prompt/variants.rsharness/src/subagent.rsllm-router/README.mdllm-router/src/config/schema.rsllm-router/src/lib.rsllm-router/src/register.rsllm-router/src/registry/register.rsllm-router/src/surface.rsllm-router/src/system_prompt.rsllm-router/src/types/router.rsllm-router/tests/golden/schemas/router.provider.register.jsonllm-router/tests/golden/schemas/router.system_prompt.get.jsonllm-router/tests/integration.rsllm-router/tests/schemas.rsprovider-anthropic/prompts/identity.txtprovider-anthropic/src/register.rsprovider-openai-codex/prompts/identity.txtprovider-openai-codex/src/register.rsprovider-openai/prompts/identity.txtprovider-openai/src/register.rsprovider-xai/prompts/identity.txtprovider-xai/src/register.rstech-specs/2026-06-agentic/harness.mdtech-specs/2026-06-agentic/llm-router.md
💤 Files with no reviewable changes (3)
- harness/prompts/kimi.txt
- harness/src/prompt/family.rs
- harness/prompts/anthropic.txt
sergiofilhowz
approved these changes
Jul 6, 2026
ytallo
approved these changes
Jul 6, 2026
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
llm-routerexposes it viarouter::system_prompt::getand lets operators override it per-provider through config. The harness drops its bundled prompt files and fetches the effective prompt at turn creation, falling back to a minimal default.system_promptfield to "set" now pre-fills the editor with the provider-declared prompt instead of a blank box, and toggling back to "unset" preserves the typed draft instead of discarding it.system_prompttextarea fell back to rendering the ~21k-char provider default as an unclipped placeholder, which visually overflowed past the field and painted over the UI below it.provider-anthropic's identity prompt (~70% fewer lines, same rules) as a proof point for the new provider-owned prompt format.Test plan
cargo test -p llm-router(schema/integration tests forsystem_promptresolution andprovider_entry_schema)cargo test -p harness(prompt resolution/fallback tests)system_promptfield unset → set → unset for a provider and confirm the declared prompt pre-fills and edits survive the round-tripsystem_prompttextarea and confirm no placeholder overflowSummary by CodeRabbit
New Features
Bug Fixes