Skip to content

[Rust Server] Return kv_events from /server_info like Python - #33056

Open
haowu1234 wants to merge 1 commit into
sgl-project:mainfrom
haowu1234:codex/rust-server-kv-events-server-info
Open

haowu1234 wants to merge 1 commit into
sgl-project:mainfrom
haowu1234:codex/rust-server-kv-events-server-info

Conversation

@haowu1234

@haowu1234 haowu1234 commented Jul 31, 2026 •

Copy link
Copy Markdown

What changed

Rust server /server_info now returns the same kv_events discovery field that Python /server_info already returns (#25844).

Before this PR:

  • Python workers exposed kv_events from /server_info
  • Rust frontend workers did not expose that field
  • KV-aware routers could not use the same discovery path for Rust workers

After this PR:

  • Rust /server_info includes a structured kv_events descriptor when the configured publisher is safe to advertise
  • Rust /server_info returns "kv_events": null when the config is disabled, malformed, non-TCP, has an invalid port, or lacks a positive page size
  • Rust keeps its existing safe /server_info behavior: no raw server_args, no raw kv_events_config, and no secret fields are exposed

This is introspection-only. It does not implement KV-event publishing or move the scheduler-side publisher.

Code changes

  • rust/sglang-server/src/runtime/config.rs

    • add typed ServerArgs fields for kv_events_config, page_size, and dp_size
    • add ServerArgs::describe_kv_events_publisher()
    • parse ZMQ-style TCP endpoints into endpoint_host and endpoint_port_base
    • validate publisher state, TCP scheme, port range, and positive page size
  • rust/sglang-server/src/api_server/common.rs

    • include kv_events in the shaped Rust /server_info response
    • keep the response allowlisted instead of dumping raw server args
    • add tests for valid descriptors, invalid configs, and secret redaction

Example

Given these Rust server args:

{
  "kv_events_config": "{\"publisher\":\"zmq\",\"endpoint\":\"tcp://*:5557\",\"topic\":\"kv\"}",
  "page_size": 64,
  "dp_size": 2
}

Rust /server_info now includes:

{
  "kv_events": {
    "publisher": "zmq",
    "endpoint_host": "*",
    "endpoint_port_base": 5557,
    "topic": "kv",
    "block_size": 64,
    "dp_size": 2
  }
}

Validation

Local macOS:

  • cargo fmt --manifest-path rust/Cargo.toml --package sglang-server --check
  • cargo test --manifest-path rust/Cargo.toml -p sglang-server --lib api_server::common::tests::shape_server_info
    • 4 passed

Remote Linux/PyO3 validation on root@134.199.199.149, with heavy build paths on /mnt/lmcache-build/sglang-rust-server-kv-events:

  • cargo fmt --manifest-path rust/Cargo.toml --package sglang-server --check
  • cargo test --manifest-path rust/Cargo.toml -p sglang-server --lib api_server::common::tests::shape_server_info
    • 4 passed
  • cargo test --manifest-path rust/Cargo.toml -p sglang-server
    • branch 1702c0aff: 135 passed; 3 failed
    • base fd28242b68: 132 passed; 3 failed
    • same pre-existing failures on both branch and base:
      • api_server::frame::tests::cumulative_frame_json_matches_serde
      • api_server::frame::tests::cumulative_frame_json_matches_serde_with_logprobs
      • message::egress::tests::chunk_event_frame_stays_small

Live Rust HTTP E2E:

  • rebuilt the Rust server PyO3 extension from this branch
  • started a real sglang.srt.server._core.Server with skip_tokenizer_init=true
  • sent a real HTTP GET /server_info request to the Rust Axum listener
  • decoded the ingress GetInternalStateReq and replied through push_result, simulating the scheduler control response
  • verified HTTP 200
  • verified the returned kv_events descriptor exactly matches the expected ZMQ host/port/topic/block-size/dp-size payload
  • verified the serialized response does not contain api_key, admin_api_key, raw kv_events_config, or a nested test password

Result:

LIVE_E2E_PASS
commit 1702c0aff
kv_events {"block_size":64,"dp_size":2,"endpoint_host":"*","endpoint_port_base":5557,"publisher":"zmq","topic":"kv"}

Fixes #33055.
Part of #23206.
Related to #29799.
Follow-up to #25844.


CI States

Latest PR Test (Base): ✅ Run #30622023649
Latest PR Test (Extra): ❌ Run #30622023378

Signed-off-by: haowu1234 <13258260125@163.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@haowu1234 haowu1234 changed the title [Rust Server] Add /server_info kv_events parity [Rust Server] Return kv_events from /server_info like Python Jul 31, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Rust Server] Return kv_events from /server_info like Python

1 participant