feat: rust sglang server openai apis

[mrain]

Motivation

Implemented the full OpenAI APIs for rust sglang server.

Modifications

Implemented APIs:

• v1/models
• v1/completions
• v1/chat/completions

Some design decisions

• n fanout: Fan-out in HTTP layer — submits n separate GenerateRequests, each with n=1. Matches external API behavior, avoids native n > 1 rejection
• Auth: --api-key Bearer token gating on all OpenAI endpoints (default: open for no key configured)
• Deprecated function_call/functions: Explicitly rejected with migration message (Python silently ignores)
• text_offset: SGLang emits [-1] sentinels; post-processed in completion_response_value since Dynamo types it as u32
• system_fingerprint: Removed from responses (Python doesn't include it)
• reasoning_content: null: Serialized into every SSE delta chunk (Python compat)
• 400 for unknown model: Consistently 400 on generation endpoints (Python convention)

Accuracy Tests

• Python integration tests: Existing TestOpenAIServer, TestOpenAIFunctionCalling, TestOpenAIResponses classes reused verbatim behind is_rust_server_built() guards via class attribute aliasing
• Rust unit tests: 26 tests covering deserialization, unary fold ordering, streaming deltas, tool-call parsing, marker buffering, split-marker handling, response lifecycle events, logprob shapes

Speed Tests and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ⏳ Run #30788528554
Latest PR Test (Extra): ❌ Run #30788528388

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 06511cc44c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[rainj-me]

In which case, there is token-id sequence?

[mrain]

It's only needed for /v1/completions when the input prompt is a token-id sequence and echo is true.

[hnyls2002]

/rerun-test test/registered/openai_server/basic/test_openai_completion_rust.py test/registered/openai_server/basic/test_openai_server.py test/registered/openai_server/function_call/test_openai_function_calling.py

[github-actions]

Results for /rerun-test test/registered/openai_server/basic/test_openai_completion_rust.py test/registered/openai_server/basic/test_openai_server.py test/registered/openai_server/function_call/test_openai_function_calling.py:

🚀 1-gpu-5090 (2 tests): ✅ View workflow run

cd test/ && python3 registered/openai_server/basic/test_openai_completion_rust.py
cd test/ && python3 registered/openai_server/basic/test_openai_server.py

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/openai_server/function_call/test_openai_function_calling.py