Refactor chat and server file

[firecoperana]

This PR refactors chat.cpp and server.cpp for better readability and faster compilation. In my test, it reduced 15s of compile time for my machine. It also ports log function from mainline. Didn't modify any existing log functions, but for future ports, we can use new log functions.
For chat.cpp, move all common_chat_parse_* to chat-parser.cpp
For server.cpp, split server.cpp code into server/common/task/queue/context files.
Some other minor changes like ggml-org/llama.cpp#17738 and ggml-org/llama.cpp#17357
Should not cause any regression.

[hksdpc255]

Seems this PR revert some of the fix for Kimi-K2 tool call:

ik_llama.cpp/common/chat-parser.cpp

Lines 914 to 930 in f506f5d

	static void common_chat_parse_kimi_k2(common_chat_msg_parser & builder) {
	static const xml_tool_call_format form = ([]() {
	xml_tool_call_format form {};
	form.scope_start = "<|tool_calls_section_begin|>";
	form.tool_start = "<|tool_call_begin|>";
	form.tool_sep = "<|tool_call_argument_begin|>{";
	form.key_start = "\"";
	form.key_val_sep = "\": ";
	form.val_end = ", ";
	form.tool_end = "}<|tool_call_end|>";
	form.scope_end = "<|tool_calls_section_end|>";
	form.raw_argval = false;
	form.last_val_end = "";
	return form;
	})();
	builder.consume_reasoning_with_xml_tool_calls(form, "<think>", "</think>");
	}

[firecoperana]

#1070
Thanks. Didn't realize this has not been merged in mainline as I was cherrypicking that commit.

[ikawrakow]

@firecoperana This PR completely bricks llama-cli. I now see stuff such as

n_past = 125
 Aslast: [ ''s':594, ' cultural':12746, ' background':4004, ',':11, ' or':476, ' someone':4325, ' with':448, ' Balk':59454, 'an':276, ' roots':19666, ' exploring':23899, ' their':862, ' heritage':27753, '.':13, ' The':576, ' word':3409, ' "':330, 'extended':40868, '"':1, ' suggests':13222, ' they':807, ' expect':1720, ' depth':7989, ',':11, ' not':537, ' just':1101, ' surface':7329, ' facts':13057, '.':382, 'Hmm':79380, ',':11, ' Bulgaria':58314, ''s':594, ' history':3840, ' is':374, ' dense':27855, ' and':323, ' spans':44038, ' centuries':23566, '.':13, ' I':358, ' should':1265, ' start':1191, ' with':448, ' the':279, ' earliest':29545, ' settlements':44346, ' to':311, ' show':1473, ' continuity':48695, ',':11, ' then':1221, ' highlight':11162, ' key':1376, ' turning':13047, ' points':3501, ':':25, ' the':279, ' First':5512, ' Bulgarian':87892, ' Empire':20406, ' under':1212, ' Khan':24218, ' As':1634 ]
n_remain: 8066
eval: [ ' As':1634 ]
n_past = 126
parlast: [ ' cultural':12746, ' background':4004, ',':11, ' or':476, ' someone':4325, ' with':448, ' Balk':59454, 'an':276, ' roots':19666, ' exploring':23899, ' their':862, ' heritage':27753, '.':13, ' The':576, ' word':3409, ' "':330, 'extended':40868, '"':1, ' suggests':13222, ' they':807, ' expect':1720, ' depth':7989, ',':11, ' not':537, ' just':1101, ' surface':7329, ' facts':13057, '.':382, 'Hmm':79380, ',':11, ' Bulgaria':58314, ''s':594, ' history':3840, ' is':374, ' dense':27855, ' and':323, ' spans':44038, ' centuries':23566, '.':13, ' I':358, ' should':1265, ' start':1191, ' with':448, ' the':279, ' earliest':29545, ' settlements':44346, ' to':311, ' show':1473, ' continuity':48695, ',':11, ' then':1221, ' highlight':11162, ' key':1376, ' turning':13047, ' points':3501, ':':25, ' the':279, ' First':5512, ' Bulgarian':87892, ' Empire':20406, ' under':1212, ' Khan':24218, ' As':1634, 'par':1732 ]
n_remain: 8065
eval: [ 'par':1732 ]
n_past = 127
uhlast: [ ' background':4004, ',':11, ' or':476, ' someone':4325, ' with':448, ' Balk':59454, 'an':276, ' roots':19666, ' exploring':23899, ' their':862, ' heritage':27753, '.':13, ' The':576, ' word':3409, ' "':330, 'extended':40868, '"':1, ' suggests':13222, ' they':807, ' expect':1720, ' depth':7989, ',':11, ' not':537, ' just':1101, ' surface':7329, ' facts':13057, '.':382, 'Hmm':79380, ',':11, ' Bulgaria':58314, ''s':594, ' history':3840, ' is':374, ' dense':27855, ' and':323, ' spans':44038, ' centuries':23566, '.':13, ' I':358, ' should':1265, ' start':1191, ' with':448, ' the':279, ' earliest':29545, ' settlements':44346, ' to':311, ' show':1473, ' continuity':48695, ',':11, ' then':1221, ' highlight':11162, ' key':1376, ' turning':13047, ' points':3501, ':':25, ' the':279, ' First':5512, ' Bulgarian':87892, ' Empire':20406, ' under':1212, ' Khan':24218, ' As':1634, 'par':1732, 'uh':12534 ]
n_remain: 8064
eval: [ 'uh':12534 ]
n_past = 128
,last: [ ',':11, ' or':476, ' someone':4325, ' with':448, ' Balk':59454, 'an':276, ' roots':19666, ' exploring':23899, ' their':862, ' heritage':27753, '.':13, ' The':576, ' word':3409, ' "':330, 'extended':40868, '"':1, ' suggests':13222, ' they':807, ' expect':1720, ' depth':7989, ',':11, ' not':537, ' just':1101, ' surface':7329, ' facts':13057, '.':382, 'Hmm':79380, ',':11, ' Bulgaria':58314, ''s':594, ' history':3840, ' is':374, ' dense':27855, ' and':323, ' spans':44038, ' centuries':23566, '.':13, ' I':358, ' should':1265, ' start':1191, ' with':448, ' the':279, ' earliest':29545, ' settlements':44346, ' to':311, ' show':1473, ' continuity':48695, ',':11, ' then':1221, ' highlight':11162, ' key':1376, ' turning':13047, ' points':3501, ':':25, ' the':279, ' First':5512, ' Bulgarian':87892, ' Empire':20406, ' under':1212, ' Khan':24218, ' As':1634, 'par':1732, 'uh':12534, ',':11 ]
n_remain: 8063

in my terminal, instead of just seeing the output of the model.

Can you fix it?

[firecoperana]

Yes, maybe it's the log file I changed.

[firecoperana]

Does #1071 fix for you?