Skip to content

server : use pytest-xdist for server tests - #28298

Merged
danbev merged 6 commits into
ggml-org:masterfrom
danbev:server-tests-parallel
Sep 3, 2026
Merged

server : use pytest-xdist for server tests#28298
danbev merged 6 commits into
ggml-org:masterfrom
danbev:server-tests-parallel

Conversation

@danbev

@danbev danbev commented Sep 3, 2026

Copy link
Copy Markdown
Member

Overview

This commit adds pytest-xdist to the server tests. This is pytest plugin that distributes test execution across multiple CPU cores.

Additional information

server.yml workflow comparision:

Platform Master Fork (PR)
Ubuntu 8m43s 7m45s
Windows 12m46s 6m12s

Refs: #26734 (comment)

Requirements

@danbev
server : use pytest-xdist for server tests
6dc4377 
This commit adds pytest-xdist to the server tests. This is pytest
plugin that distributes test execution across multiple CPU cores.

Assisted-by: pi:llama.cpp/qwen3.8-27B

Refs: ggml-org#26734 (comment)
@github-actions github-actions Bot added devops improvements to build systems and github actions server labels Sep 3, 2026
ggerganov
ggerganov reviewed Sep 3, 2026
View reviewed changes
Comment thread tools/server/tests/unit/test_compat_anthropic.py Outdated
@danbev

danbev commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

I think we might be able to use --dist=worksteal instead of loadfile. The motivation for this is that loadfile will send all the tests from one file to the same worker, serializing them which may not be required. For example, it looks like this is the case for test_router.py. But there might be cases like test_slot_save.py which need to use paths specific to the worker to not conflict with each other. But it might be worth doing that.

@danbev
danbev marked this pull request as ready for review September 3, 2026 12:07
@danbev
danbev requested review from a team and CISC as code owners September 3, 2026 12:07
@danbev
danbev merged commit 42f0225 into ggml-org:master Sep 3, 2026
13 checks passed
@danbev
danbev deleted the server-tests-parallel branch September 3, 2026 13:04
@ggerganov ggerganov mentioned this pull request Sep 4, 2026
Merged
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request Sep 5, 2026
@danbev @fewtarius
server : use pytest-xdist for server tests (ggml-org#28298)
b6e8d9a 
* server : use pytest-xdist for server tests

This commit adds pytest-xdist to the server tests. This is pytest
plugin that distributes test execution across multiple CPU cores.

Assisted-by: pi:llama.cpp/qwen3.8-27B

Refs: ggml-org#26734 (comment)

* remove server_base_port and BASE_PORT

* use worksteal and pytest builting tmp_path
thecodacus pushed a commit to thecodacus/llama.cpp that referenced this pull request Sep 7, 2026
@danbev
server : use pytest-xdist for server tests (ggml-org#28298)
c75784e 
* server : use pytest-xdist for server tests

This commit adds pytest-xdist to the server tests. This is pytest
plugin that distributes test execution across multiple CPU cores.

Assisted-by: pi:llama.cpp/qwen3.8-27B

Refs: ggml-org#26734 (comment)

* remove server_base_port and BASE_PORT

* use worksteal and pytest builting tmp_path
@danbev danbev mentioned this pull request Sep 8, 2026
Merged
danbev added a commit that referenced this pull request Sep 8, 2026
@danbev
ci : add PYTEST_WORKERS=1 to fix server-self-hosted job (#28603)
1744c6b 
* ci : add PYTEST_WORKERS=1 to fix server-self-hosted job

This commit adds the `PYTEST_WORKERS=1` environment variable to the
hf-jobs-t4-small:cuda13 runner steps.

This is an attempt to address CI failure of this job that I might have
introduced in Commit 42f0225
("server : use pytest-xdist for server tests (#28298)").

Refs: https://github.com/ggml-org/llama.cpp/actions/runs/34126971262/job/101757819134

* apply same changes to server-metal steps
x1250 pushed a commit to x1250/llama.cpp that referenced this pull request Sep 9, 2026
@danbev
server : use pytest-xdist for server tests (ggml-org#28298)
e9d43f4 
* server : use pytest-xdist for server tests

This commit adds pytest-xdist to the server tests. This is pytest
plugin that distributes test execution across multiple CPU cores.

Assisted-by: pi:llama.cpp/qwen3.8-27B

Refs: ggml-org#26734 (comment)

* remove server_base_port and BASE_PORT

* use worksteal and pytest builting tmp_path
x1250 pushed a commit to x1250/llama.cpp that referenced this pull request Sep 9, 2026
@danbev
ci : add PYTEST_WORKERS=1 to fix server-self-hosted job (ggml-org#28603)
8d6d1cb 
* ci : add PYTEST_WORKERS=1 to fix server-self-hosted job

This commit adds the `PYTEST_WORKERS=1` environment variable to the
hf-jobs-t4-small:cuda13 runner steps.

This is an attempt to address CI failure of this job that I might have
introduced in Commit 42f0225
("server : use pytest-xdist for server tests (ggml-org#28298)").

Refs: https://github.com/ggml-org/llama.cpp/actions/runs/34126971262/job/101757819134

* apply same changes to server-metal steps
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov left review comments
@CISC CISC CISC approved these changes
@ServeurpersoCom ServeurpersoCom ServeurpersoCom approved these changes

Assignees

No one assigned

Labels

devops improvements to build systems and github actions server

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@danbev @CISC @ggerganov @ServeurpersoCom