Skip to content

Conversation

@GaoYusong
Copy link
Contributor

@GaoYusong GaoYusong commented Apr 13, 2025

Motivation

The log [broadcast_pyobj] rank=2, device=cuda introduced in #5322 yesterday causes excessive log flooding, which blocks the initialization of the sglang server.

[2025-04-13 14:53:45 TP1] Capture cuda graph end. Time elapsed: 7.81 s. avail mem=6.60 GB. mem usage=1.45 GB.
[2025-04-13 14:53:45 TP1] max_total_num_tokens=6079237, chunked_prefill_size=8192, max_prefill_tokens=16384, max_running_requests=4097, context_len=32768
[2025-04-13 14:53:45 TP0] max_total_num_tokens=6079237, chunked_prefill_size=8192, max_prefill_tokens=16384, max_running_requests=4097, context_len=32768
[2025-04-13 14:53:45 TP2] max_total_num_tokens=6079237, chunked_prefill_size=8192, max_prefill_tokens=16384, max_running_requests=4097, context_len=32768
[2025-04-13 14:53:45 TP3] max_total_num_tokens=6079237, chunked_prefill_size=8192, max_prefill_tokens=16384, max_running_requests=4097, context_len=32768
[2025-04-13 14:53:46] INFO:     Started server process [216190]
[2025-04-13 14:53:46] INFO:     Waiting for application startup.
[2025-04-13 14:53:46] INFO:     Application startup complete.
[2025-04-13 14:53:46] INFO:     Uvicorn running on http://10.13.3.170:30000 (Press CTRL+C to quit)
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
[broadcast_pyobj] rank=2, device=cuda
...

Modifications

delete the print

Checklist

@GaoYusong
Copy link
Contributor Author

fixed in #5359

@GaoYusong GaoYusong closed this Apr 14, 2025
@zhyncs
Copy link
Member

zhyncs commented Apr 14, 2025

Ah I haven't noticed this PR before, thanks for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants