Why server is not provided with binaries ?

[mirek190]

Question ... why server is not provided with binaries ?

[BarfingLemurs]

there are: https://github.com/ggerganov/llama.cpp/releases/tag/master-7d87381

but you need to build the arm versions yourself though.

[noprotocolunit]

Perhaps I'm missing something, but I downloaded llama-master-7d87381-bin-win-openblas-x64.zip and the server executable is not among them. Am I doing it wrong?

[maddes8cht]

Still there is no server.exe in none of the provided binaries, in none of the latest releases.

[mirek190]

Is not present . That's why I'm asking l.
I can build by myself but still ...

[FSSRepo]

The LLAMA_BUILD_SERVER option is OFF by default.

[mirek190]

..yes.. but why?

[jp-aa]

I don't mean to be annoying, but the server doesn't get built by default when using make, nor appears on the Makefile either ( i'm not really good with build systems and stuff). It does seem to be included when building with cmake, but passing the NVCC flags I need to cmake isn't working for me. Is there a reason why it's not getting built by default? Or at least included in the Makefile?

[KerfuffleV2]

Looks like you can only build the server via cmake.

From poking around a bit, it's actually quite large compared to all the other examples and includes a copy of a HTTP and JSON parsing library. I'd guess it's disabled by default because it's relatively slow to compile.

Just for comparison, the main example (basically what people call llama.cpp) is around 25k with about another 33k of common code which is shared with all the other examples. For the server, the JSON parsing and HTTP support come to about 1MB of source code.

This answer doesn't necessarily help someone that just wants that feature but hopefully at least makes it seem like less of an arbitrary decision.

[jp-aa]

Looks like you can only build the server via cmake.

From poking around a bit, it's actually quite large compared to all the other examples and includes a copy of a HTTP and JSON parsing library. I'd guess it's disabled by default because it's relatively slow to compile.

Just for comparison, the main example (basically what people call llama.cpp) is around 25k with about another 33k of common code which is shared with all the other examples. For the server, the JSON parsing and HTTP support come to about 1MB of source code.

This answer doesn't necessarily help someone that just wants that feature but hopefully at least makes it seem like less of an arbitrary decision.

Well it's not that bad. I managed to compile the server with make just by adding "server" to the default target, clean target, and an entry in the examples section of the Makefile which looks the same as most of the stuff in there.
My only problem now is offloading model layers to the GPU while running the server binary. I have NO IDEA if it's even a feature yet. If someone knows how please tell me

[SlyEcho]

It needs to be added to the workflow, nothing special.