Repository navigation
Conversation
Signed-off-by: haowu1234 <13258260125@163.com>
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Rust server
/server_infonow returns the samekv_eventsdiscovery field that Python/server_infoalready returns (#25844).Before this PR:
kv_eventsfrom/server_infoAfter this PR:
/server_infoincludes a structuredkv_eventsdescriptor when the configured publisher is safe to advertise/server_inforeturns"kv_events": nullwhen the config is disabled, malformed, non-TCP, has an invalid port, or lacks a positive page size/server_infobehavior: no rawserver_args, no rawkv_events_config, and no secret fields are exposedThis is introspection-only. It does not implement KV-event publishing or move the scheduler-side publisher.
Code changes
rust/sglang-server/src/runtime/config.rsServerArgsfields forkv_events_config,page_size, anddp_sizeServerArgs::describe_kv_events_publisher()endpoint_hostandendpoint_port_baserust/sglang-server/src/api_server/common.rskv_eventsin the shaped Rust/server_inforesponseExample
Given these Rust server args:
{ "kv_events_config": "{\"publisher\":\"zmq\",\"endpoint\":\"tcp://*:5557\",\"topic\":\"kv\"}", "page_size": 64, "dp_size": 2 }Rust
/server_infonow includes:{ "kv_events": { "publisher": "zmq", "endpoint_host": "*", "endpoint_port_base": 5557, "topic": "kv", "block_size": 64, "dp_size": 2 } }Validation
Local macOS:
cargo fmt --manifest-path rust/Cargo.toml --package sglang-server --checkcargo test --manifest-path rust/Cargo.toml -p sglang-server --lib api_server::common::tests::shape_server_info4 passedRemote Linux/PyO3 validation on
root@134.199.199.149, with heavy build paths on/mnt/lmcache-build/sglang-rust-server-kv-events:cargo fmt --manifest-path rust/Cargo.toml --package sglang-server --checkcargo test --manifest-path rust/Cargo.toml -p sglang-server --lib api_server::common::tests::shape_server_info4 passedcargo test --manifest-path rust/Cargo.toml -p sglang-server1702c0aff:135 passed; 3 failedfd28242b68:132 passed; 3 failedapi_server::frame::tests::cumulative_frame_json_matches_serdeapi_server::frame::tests::cumulative_frame_json_matches_serde_with_logprobsmessage::egress::tests::chunk_event_frame_stays_smallLive Rust HTTP E2E:
sglang.srt.server._core.Serverwithskip_tokenizer_init=trueGET /server_inforequest to the Rust Axum listenerGetInternalStateReqand replied throughpush_result, simulating the scheduler control responsekv_eventsdescriptor exactly matches the expected ZMQ host/port/topic/block-size/dp-size payloadapi_key,admin_api_key, rawkv_events_config, or a nested test passwordResult:
Fixes #33055.
Part of #23206.
Related to #29799.
Follow-up to #25844.
CI States
Latest PR Test (Base): ✅ Run #30622023649
Latest PR Test (Extra): ❌ Run #30622023378