common/chat, server: refactor, move all conversion functions to common, add tests by pwilkin · Pull Request #20690 · ggml-org/llama.cpp

pwilkin · 2026-03-17T16:36:11Z

Unifies the conversion for Chat Completions, Responses and Anthropic Messages endpoints by moving the conversion functions to a dedicated conversion file in common.

Adds test for Responses API (currently one test for #20280) + fixes issue with type: message string-only content.

Naster17 · 2026-03-17T17:02:31Z

hm, give a shot for this

Naster17 · 2026-03-17T17:11:44Z

Finally its work Thank you. Dziękuję ;)

pwilkin · 2026-03-19T16:53:57Z

Bumpy bump? @ngxson @CISC maybe?

ngxson

IMO it's a bit strange to move the conversion functions to common, as they are technically only used by server.

Just wondering, it is possible to confine them inside server, maybe: server/server-chat.cpp ?

test-chat can be linked against libserver, the way as llama-cli

aldehir · 2026-03-19T19:21:02Z

I'm in agreement with @ngxson. I would expect interface choices to be made at the API boundary, instead of propagating internally.

In a way the chat completions API is our "internal" representation, for better or worse.

What are we trying to solve?

pwilkin · 2026-03-19T20:15:57Z

I'm trying to make the conversion clearer (and more testable), possibly in preparation for migration to a different system than Chat Completions as our base.

Okay, I'll make a separate class in server/ instead for the other conversions.

ngxson · 2026-03-20T09:38:21Z

+using json = nlohmann::ordered_json;
+
+// Convert OpenAI Responses API format to OpenAI Chat Completions API format
+json common_chat_convert_responses_to_chatcmpl(const json & body);


Suggested change

json common_chat_convert_responses_to_chatcmpl(const json & body);

json server_chat_convert_responses_to_chatcmpl(const json & body);

ngxson · 2026-03-20T09:38:53Z

+
+#include "chat.h"
+
+#include <nlohmann/json.hpp>


It should be possible to use json_fwd here, right?

pwilkin · 2026-04-03T11:22:45Z

Okay, errors fixed, bumping since there are already further fixes to the Anthropic API people want.

alejrobinson · 2026-04-17T13:55:24Z

This commit seems to solve the 400 error currently happening in N8N, are there plans to merge it soon?

Thanks and great work!

pwilkin · 2026-04-17T15:49:43Z

@ggml-org/maintainers need two approvals after merge please.

alejrobinson · 2026-04-20T09:57:38Z

Just had the time to properly pull and build this PR to test it and it works! All problems I had with N8N are gone.

Hope it get's merged soon, thanks again for the great work!

pwilkin · 2026-04-20T10:13:35Z

@ggml-org/maintainers bumpy bump PTAL :)

pwilkin · 2026-04-22T08:11:23Z

@CISC could I get a reapproval pls? This has been stuck for so long now :/

CISC

@CISC could I get a reapproval pls? This has been stuck for so long now :/

Sure, but it won't help, you still need a second approval. :P

pwilkin · 2026-04-22T08:21:09Z

@ggml-org/maintainers pretty please :)

Upstream llama.cpp refactor (ggml-org/llama.cpp#20690) moved the OAI/Anthropic/Responses and transcription conversion helpers out of server-common.cpp into a new server-chat.cpp, and server-task.cpp and server-context.cpp now call those symbols (convert_transcriptions_to_chatcmpl, server_chat_convert_responses_to_chatcmpl, server_chat_convert_anthropic_to_oai, server_chat_msg_diff_to_json_oaicompat) via server-chat.h. grpc-server.cpp builds as a single translation unit by #include-ing the upstream .cpp files directly. Without including server-chat.cpp, the declarations are satisfied at compile time via server-chat.h but the link step fails with undefined references once LLAMA_VERSION crosses the refactor commit (134d6e54). Guard the include with __has_include so the same source stays buildable on older LLAMA_VERSION pins that predate the refactor (where prepare.sh won't copy server-chat.cpp into tools/grpc-server/). Assisted-by: Claude:claude-opus-4-7 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

…it (#9511) * fix(llama-cpp): include server-chat.cpp in grpc-server translation unit Upstream llama.cpp refactor (ggml-org/llama.cpp#20690) moved the OAI/Anthropic/Responses and transcription conversion helpers out of server-common.cpp into a new server-chat.cpp, and server-task.cpp and server-context.cpp now call those symbols (convert_transcriptions_to_chatcmpl, server_chat_convert_responses_to_chatcmpl, server_chat_convert_anthropic_to_oai, server_chat_msg_diff_to_json_oaicompat) via server-chat.h. grpc-server.cpp builds as a single translation unit by #include-ing the upstream .cpp files directly. Without including server-chat.cpp, the declarations are satisfied at compile time via server-chat.h but the link step fails with undefined references once LLAMA_VERSION crosses the refactor commit (134d6e54). Guard the include with __has_include so the same source stays buildable on older LLAMA_VERSION pins that predate the refactor (where prepare.sh won't copy server-chat.cpp into tools/grpc-server/). Assisted-by: Claude:claude-opus-4-7 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(llama-cpp): bump LLAMA_VERSION to 0d0764dfd Bump to ggml-org/llama.cpp@0d0764d. Paired with the preceding grpc-server server-chat.cpp include so the refactor at 134d6e54 links cleanly. Supersedes PR #9494. Assisted-by: Claude:claude-opus-4-7 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

…n, add tests (ggml-org#20690) * Refactor conversion functions

…n, add tests (#20690) * Refactor conversion functions

…n, add tests (ggml-org#20690) * Refactor conversion functions

pwilkin requested review from a team as code owners March 17, 2026 16:36

github-actions Bot added testing Everything test related examples server labels Mar 17, 2026

ngxson reviewed Mar 19, 2026

View reviewed changes

pwilkin force-pushed the refactor-chat-conversion branch from 4d6942c to 8686b80 Compare March 19, 2026 20:17

pwilkin requested a review from ggerganov as a code owner March 19, 2026 20:51

ngxson reviewed Mar 20, 2026

View reviewed changes

pwilkin force-pushed the refactor-chat-conversion branch from fcad53c to 2438739 Compare March 31, 2026 20:07

pwilkin requested review from CISC, aldehir and ngxson March 31, 2026 20:08

pwilkin force-pushed the refactor-chat-conversion branch from 76212e4 to d0bf6ac Compare April 3, 2026 11:21

CISC approved these changes Apr 3, 2026

View reviewed changes

Comment thread common/chat.cpp Outdated

pwilkin added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Apr 3, 2026

pwilkin force-pushed the refactor-chat-conversion branch from 0a5bf62 to 531b71b Compare April 17, 2026 14:01

Squash - refactor conversion functions

889121d

pwilkin force-pushed the refactor-chat-conversion branch from 531b71b to 889121d Compare April 17, 2026 14:05

Huh

3c4eaa9

CISC approved these changes Apr 22, 2026

View reviewed changes

am17an approved these changes Apr 22, 2026

View reviewed changes

pwilkin merged commit 134d6e5 into ggml-org:master Apr 22, 2026
49 of 51 checks passed

mudler mentioned this pull request Apr 23, 2026

fix(llama-cpp): include server-chat.cpp in grpc-server translation unit mudler/LocalAI#9511

Merged

my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026

common/chat, server: refactor, move all conversion functions to commo…

e971edc

…n, add tests (ggml-org#20690) * Refactor conversion functions

baramofme pushed a commit to baramofme/llama-cpp-turboquant that referenced this pull request May 23, 2026

common/chat, server: refactor, move all conversion functions to commo…

70db1b9

…n, add tests (ggml-org#20690) * Refactor conversion functions

fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request May 30, 2026

common/chat, server: refactor, move all conversion functions to commo…

221e2b1

…n, add tests (ggml-org#20690) * Refactor conversion functions

MrLordCat referenced this pull request in MrLordCat/llama.cpp-rdna-lab Jul 16, 2026

common/chat, server: refactor, move all conversion functions to commo…

56b8db0

…n, add tests (#20690) * Refactor conversion functions

zommiommy pushed a commit to zommiommy/llama.cpp that referenced this pull request Aug 18, 2026

common/chat, server: refactor, move all conversion functions to commo…

34d7f39

…n, add tests (ggml-org#20690) * Refactor conversion functions

	json common_chat_convert_responses_to_chatcmpl(const json & body);
	json server_chat_convert_responses_to_chatcmpl(const json & body);


		#include "chat.h"

		#include <nlohmann/json.hpp>

Uh oh!

Conversation

pwilkin commented Mar 17, 2026

Uh oh!

Naster17 commented Mar 17, 2026

Uh oh!

Naster17 commented Mar 17, 2026

Uh oh!

pwilkin commented Mar 19, 2026

Uh oh!

ngxson left a comment

Choose a reason for hiding this comment

Uh oh!

aldehir commented Mar 19, 2026

Uh oh!

pwilkin commented Mar 19, 2026

Uh oh!

ngxson Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

ngxson Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

pwilkin commented Apr 3, 2026

Uh oh!

Uh oh!

alejrobinson commented Apr 17, 2026

Uh oh!

pwilkin commented Apr 17, 2026

Uh oh!

alejrobinson commented Apr 20, 2026

Uh oh!

pwilkin commented Apr 20, 2026

Uh oh!

pwilkin commented Apr 22, 2026

Uh oh!

CISC left a comment

Choose a reason for hiding this comment

Uh oh!

pwilkin commented Apr 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants