server : use different seeds for child completions

[ggerganov]

rel #18663

I realized that we want to have different RNG seeds for the n_cmpl completions - otherwise we will generate the same result for each child. This makes the assert in the test_chat_completions_multiple_choices() test invalid.

[ngxson]

Should we also handle the case where default seed is used?

[ngxson]

Ah nevermind, I think we don't need to care about that case here

[ggerganov]

Hm, you might be right. If we don't handle the default seed here, then 1 child will have random seed and the rest will get the same seeds.

[CISC]

I think server-windows CI possibly broke:
https://github.com/ggml-org/llama.cpp/actions/runs/20844687013/job/59885629295
https://github.com/ggml-org/llama.cpp/actions/runs/20847242141/job/59893626577

[ggerganov]

These 2 test expect a failure due to the cache (256 tokens) getting filled, but the assumption of the test is that the 4 parallel requests would start together at the same time:

llama.cpp/tools/server/tests/unit/test_completion.py

Lines 377 to 378 in 53eb943

	(256, 4, [70, 70, 70, 70], [False, False, False, False]),
	(256, 4, [90, 90, 40, 90], [False, False, True, False]),

So the logic is flaky - if the runner is under load like in these 2 cases, the first request starts much earlier than the others.

[CISC]

@ggerganov BTW, ggml-cis having issues again:
https://github.com/ggml-org/llama.cpp/actions?page=2&query=is%3Aqueued

[ggerganov]

@CISC Do you mean that the workflows are not processed fast enough?

[CISC]

@CISC Do you mean that the workflows are not processed fast enough?

There were several queued from yesterday, though it seems to have cleared somewhat, now there are just 2 left.

[CISC]

@CISC Do you mean that the workflows are not processed fast enough?

There were several queued from yesterday, though it seems to have cleared somewhat, now there are just 2 left.

Ah, I think that may just be because they have timed out.

[CISC]

@ggerganov As you can see the nvidia runners seem to be stuck/extremely slow:
https://github.com/ggml-org/llama.cpp/actions?query=is%3Aqueued