Convert IPC dataclasses to msgspec.Struct with opt-in msgpack transport

[rainj-me]

Motivation

Prepare SGLang IPC payloads for typed msgpack serialization without changing the default runtime transport yet.

This PR converts the IPC request/output classes from dataclasses to msgspec.Struct, but keeps pickle IPC enabled by default for compatibility. Developers can opt into the msgspec/msgpack transport by setting:

SGLANG_USE_PICKLE_IPC=0

For auditing fallback behavior, set:

SGLANG_LOG_PICKLE_IPC_OBJECTS=1

Changes

• Convert IPC base/request/output classes in io_struct.py from dataclasses to msgspec.Struct.
• Add pydantic schema support for msgspec-backed request structs.
• Add msgpack encode/decode helpers and socket send/recv switching behind SGLANG_USE_PICKLE_IPC.
• Keep pickle as the default IPC transport with SGLANG_USE_PICKLE_IPC=True.
• Add typed encode/decode hooks for common payloads such as array, torch.Tensor, numpy.ndarray, and numpy floating scalars.
• Add transitional PickleWrapper support for fields and structs that still carry opaque Python objects or broad Any/Dict[str, Any] payloads.
• Wrap/unwrap known opaque IPC payloads such as time_stats, customized_info, multimodal inputs, and mooncake multimodal data at process boundaries.
• Update tokenizer, detokenizer, scheduler receiver, DP controller, and multi-tokenizer routing paths for the wrapper protocol.
• Add E2E coverage for customized_info through the normal scheduler -> detokenizer -> tokenizer-manager path.
• Update unit test helpers to introspect msgspec.Struct fields instead of dataclass fields.

Notes

PickleWrapper is an intentional workaround during the migration. It lets the new typed structs travel through msgpack while fields that are not yet msgpack-safe continue to round-trip through pickle explicitly. This keeps the migration incremental and makes remaining pickle usage easier to audit.

The default behavior is still pickle IPC. The msgpack/msgspec path is opt-in with SGLANG_USE_PICKLE_IPC=0 until the remaining opaque fields are narrowed or wrapped safely.

Test Plan

PATH=.venv/bin:$PATH .venv/bin/python test/registered/unit/managers/test_tokenizer_manager_rid_cleanup.py
PATH=.venv/bin:$PATH .venv/bin/python test/registered/core/test_customized_info_e2e.py

---

CI States

Latest PR Test (Base): ✅ Run #28248758478
Latest PR Test (Extra): 🚫 Run #28248758407

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[merrymercy]

note for myself:

• try to further simplify the type annotations
• strictly review the auto-gen fastapi specs

[merrymercy]

/tag-and-rerun-ci

[merrymercy]

good ci status

---

CI States

Latest PR Test (Base): ❌ Run #28207193093
Latest PR Test (Extra): ✅ Run #28207192971

[merrymercy]

/tag-and-rerun-ci