feat(workflow): surface max_running_requests on SGLang HTTP /server_info

[slin1237]

Summary

• SMG's curated ServerInfo struct in discover_metadata.rs deserializes a subset of SGLang's /server_info response (which has ~800 fields). max_running_requests — the per-instance batch-capacity cap from --max-running-requests — was being silently dropped.
• Adds pub max_running_requests: Option<usize> to the struct. flat_labels is serde-based and picks it up automatically — no other changes needed.
• The SGLang gRPC label pipeline (SGLANG_GRPC_KEYS in routers/grpc/client.rs) already extracts this field; this closes the HTTP-only path so capacity-aware consumers see the same label regardless of transport.

Test plan

• cargo test --package smg --lib workflow::steps::local::discover_metadata::tests::test_sglang_server_info — 2 new unit tests pass (present-and-absent cases).
• Existing #[ignore] integration tests against a live SGLang server still pass.
• No regression in the rest of the workflow test suite.

Notes

• Independent of feat(worker): add max_running_requests accessor on Worker trait #1526. Can merge in any order.
• vLLM's HTTP server doesn't expose an equivalent endpoint and vLLM's vllm_engine.proto GetServerInfoResponse has no batch-capacity field, so this fix is SGLang-HTTP-specific. TRT-LLM uses max_batch_size (different name); that's a separate, larger change.

Summary by CodeRabbit

• New Features

  • System now surfaces per-instance concurrency limits so users can see how many requests an instance will run concurrently.

• Tests

  • Added unit tests to verify correct detection and handling when the concurrency-limit information is present or omitted.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: cf3bf8d2-4cc3-4176-bb11-de2715bf6dc6

📥 Commits

Reviewing files that changed from the base of the PR and between 02421cb and bf42012.

📒 Files selected for processing (1)

• model_gateway/src/workflow/steps/local/discover_metadata.rs

---

📝 Walkthrough

Walkthrough

ServerInfo gains an optional max_running_requests field to expose SGLang's per-instance request concurrency cap. Unit tests confirm deserialization when present and omission from flattened labels when absent.

Changes

Server Metadata Enhancement

Layer / File(s)	Summary
ServerInfo concurrency field model_gateway/src/workflow/steps/local/discover_metadata.rs	ServerInfo struct adds optional max_running_requests: Option<usize> field to capture SGLang's per-instance concurrency cap.
Field validation tests model_gateway/src/workflow/steps/local/discover_metadata.rs	Unit tests validate that the field deserializes when present in /server_info and is absent from flattened labels when not provided.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• lightseekorg/smg#817: Both PRs extend GetServerInfo/server metadata with new “max-*” concurrency/limit fields (max_running_requests vs max_total_num_tokens) sourced from server info payloads.

Suggested reviewers

• CatherineSue

Poem

🐰 A field hops in, small and bright,
Counting requests through day and night,
Tests nibble gently, numbers clear,
Concurrency whispers, now we hear. ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(workflow): surface max_running_requests on SGLang HTTP /server_info' directly and accurately summarizes the main change: adding max_running_requests field to ServerInfo struct to surface SGLang's capacity information.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/sglang-server-info-max-running-requests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request adds the max_running_requests field to the ServerInfo struct in discover_metadata.rs to surface the per-instance concurrency cap for SGLang. It also includes unit tests to verify the deserialization and label extraction of this new field, ensuring it handles both present and missing values correctly. I have no feedback to provide.

[claude]

Clean, well-scoped change. The new max_running_requests field on ServerInfo is correctly typed as Option<usize>, matches the SGLang JSON field name (no rename needed), and is auto-surfaced by the existing generic flat_labels pipeline. Tests cover both the present and absent cases. No issues found.

[claude]

Clean, minimal change. The new max_running_requests: Option<usize> field correctly closes the HTTP-only gap (gRPC path already had it at SGLANG_GRPC_KEYS). flat_labels picks it up automatically via serde — no manual wiring needed. Tests cover both present and absent cases. No issues found.