Skip to content

CI: fix SGLang router port collision in sanity test #1995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dpressle merged 1 commit into from
Jul 27, 2026
Merged

CI: fix SGLang router port collision in sanity test #1995

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .gitlab/test_vllm_sglang_sanity.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,14 +153,20 @@ elif [ "$FRAMEWORK" = "sglang" ]; then
wait_for "http://localhost:${PREFILL_PORT}/health" "$SERVER_TIMEOUT"
wait_for "http://localhost:${DECODE_PORT}/health" "$SERVER_TIMEOUT"

# Re-pick PROXY_PORT now that the SGLang servers are fully up. SGLang binds internal
# ports derived from its base port (gRPC, ZMQ, etc.) that aren't visible to
# get_next_tcp_port() until the servers are running; re-allocating here ensures the
# router gets a port that doesn't collide with any of those.
PROXY_PORT="$(get_next_tcp_port)"
Comment thread
coderabbitai[bot] marked this conversation as resolved.

# sglang-router fronts the prefill/decode pair (replaces the removed in-core mini_lb).
# Installed into the sglang weights-base image (contrib/Dockerfile.sglang-base).
setsid python3 -m sglang_router.launch_router --pd-disaggregation \
--prefill "http://localhost:${PREFILL_PORT}" \
--decode "http://localhost:${DECODE_PORT}" \
--host 0.0.0.0 --port "$PROXY_PORT" &
pids+=($!)
wait_for "http://localhost:${PROXY_PORT}/health" "$PROXY_TIMEOUT" || true
wait_for "http://localhost:${PROXY_PORT}/health" "$PROXY_TIMEOUT"
ENDPOINT="http://localhost:${PROXY_PORT}/v1/completions"

else
Expand Down
Loading