Skip to content

fix: translate "developer" role to "system" for chat template compatibility #20079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
marksverdhei wants to merge 1 commit into from
Closed

fix: translate "developer" role to "system" for chat template compatibility #20079

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions common/chat.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,11 @@ std::vector<common_chat_msg> common_chat_msgs_parse_oaicompat(const json & messa
throw std::invalid_argument("Missing 'role' in message: " + message.dump());
}
msg.role = message.at("role");
// OpenAI uses "developer" as an alias for "system" in the Responses API;
// remap it so chat templates that only handle "system" don't break.
if (msg.role == "developer") {
msg.role = "system";
}

auto has_content = message.contains("content");
auto has_tool_calls = message.contains("tool_calls");
Expand Down
40 changes: 38 additions & 2 deletions tools/server/server-common.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -944,6 +944,11 @@ json oaicompat_chat_params_parse(
}
for (auto & msg : messages) {
std::string role = json_value(msg, "role", std::string());
// Remap "developer" → "system" for chat template compatibility
if (role == "developer") {
msg["role"] = "system";
role = "system";
}
if (role != "assistant" && !msg.contains("content")) {
throw std::invalid_argument("All non-assistant messages must contain 'content'");
}
Expand Down Expand Up @@ -1239,8 +1244,39 @@ json convert_responses_to_chatcmpl(const json & response_body) {
item.erase("status");
}
item["content"] = chatcmpl_content;

chatcmpl_messages.push_back(item);
// Remap "developer" → "system" for chat template compatibility.
// Also ensure system messages end up at position 0 (merge with
// any existing system message created from the "instructions" field).
if (item.at("role") == "developer" || item.at("role") == "system") {
item["role"] = "system";
// Extract plain text from the content parts
std::string sys_text;
for (const auto & part : chatcmpl_content) {
if (part.value("type", "") == "text" && part.contains("text")) {
if (!sys_text.empty()) {
sys_text += "\n";
}
sys_text += part.at("text").get<std::string>();
}
}
if (!chatcmpl_messages.empty() && chatcmpl_messages.front().value("role", "") == "system") {
// Merge into existing system message
std::string existing = chatcmpl_messages.front().value("content", "");
if (!existing.empty() && !sys_text.empty()) {
existing += "\n\n";
}
existing += sys_text;
chatcmpl_messages.front()["content"] = existing;
} else {
// Insert system message at position 0
chatcmpl_messages.insert(chatcmpl_messages.begin(), {
{"role", "system"},
{"content", sys_text},
});
}
} else {
chatcmpl_messages.push_back(item);
}
} else if (exists_and_is_array(item, "content") &&
exists_and_is_string(item, "role") &&
item.at("role") == "assistant" &&
Expand Down