[Router][Feat][Bugfix] Add Swagger UI (OpenAPI) support with Pydantic request models + fix rewrite Content-Length + test

[JaredforReal]

[Router][Feat][Bugfix] Add Swagger UI (OpenAPI) support with Pydantic request models + fix rewrite Content-Length + dev smoke tooling

Fixes #667

Summary

Introduce OpenAPI/Swagger UI documentation and typed (Pydantic) request models for the three OpenAI‑style endpoints:

• /v1/chat/completions
• /v1/completions
• /v1/embeddings
  Add a mock backend + smoke test tooling to simplify local and CI verification.
  Fix a routing bug where a request body rewrite did not refresh Content-Length, causing truncated JSON and backend 400 responses.
  Improve non‑stream responses to return application/json instead of always text/event-stream.

Key Changes

1. New protocols.py with minimal OpenAI-compatible Pydantic models (ChatCompletionRequest, CompletionRequest, EmbeddingRequest).
2. main_router.py: endpoints now accept typed models; preserve raw Request for semantic cache, callbacks, and rewriting.
3. Bugfix in route_general_request:
   • Refresh Content-Length after rewrite.
   • Dynamic media type: text/event-stream only when stream=true, else application/json.
4. tests:
   • test_swagger_integration.py unit test for swagger ui
   • main.py mock engine (chat/completions/embeddings).
   • _swagger_smoke_core.py shared smoke logic.
   • swagger_smoke.py standalone CLI smoke test.

Bug Fix Details

Before: Request rewrite produced new JSON body but stale Content-Length, backend read partial body → JSONDecodeError → 400.
After: Always call update_content_length() post rewrite; valid 200 responses confirmed.

Testing

Layer	What	Status
Unit	test_swagger_integration.py (Pydantic validation, schema)	Pass
Smoke	swagger_smoke.py (8 checks)	Pass
Manual	Browser docs, “Try it out”	Verified
Curl	Chat/completions/embeddings 200 + 422 invalid	Verified
Regression	Non-stream media type now JSON	Verified

How to Reproduce Locally

# Terminal 1
python examples/mock_backend/main.py --port 8000

# Terminal 2
python -m vllm_router.app \
--service-discovery static \
--static-backends http://localhost:8000,http://localhost:8000 \
--static-models gpt-3.5-turbo,text-embedding-ada-002 \
--routing-logic roundrobin \
--host 0.0.0.0 --port 8080 --log-level debug

# Terminal 3 (smoke)
python scripts/swagger_smoke.py
# or just try it out in http://localhost:8080/doc

Observability / Metrics

No change to metrics emission. Mock backend intentionally omits /metrics (router logs 404—benign).

Backward Compatibility

• No change to existing response shapes.
• Extra request fields still accepted (extra='allow')—only logged as warnings.
• Non‑OpenAPI endpoints untouched (e.g. /tokenize, /score).
• Streaming semantics unchanged except for correct content type when not streaming.

Limitations / Deferred

• messages elements are plain dict; future enhancement: strict Message model + role Enum.
• Response models still untyped (can add response_model= later).
• No streaming chunk schema; SSE contract unchanged.
• Semantic cache specific fields currently allowed as extra (not declared).

Review Notes

Focus areas:

• Ensure rewrite path + Content-Length fix is safe.
• Confirm no unintended change to routing selection.
• Validate OpenAPI schema suffices for downstream tooling.

Risk Assessment

Low runtime risk: new logic isolated to three endpoints + a small header update after rewrite.
Fallback: disabling Pydantic would require reverting router endpoint signatures (not included; no flag yet).

---

This is my first PR, just point out what I have done wrong, I will fix it ASAP.

[YuhanLiu11]

@JaredforReal Can you fix the CI test errors?

[JaredforReal]

@YuhanLiu11 Yeah! There is an import and a requirement error; I'm working on it.

[JaredforReal]

@YuhanLiu11 Thanks for your time! I think this version should pass the CI test. (or I will work on it till it succeeds orz

[YuhanLiu11]

Why do we need to change this function to add this swagger UI mock requests?

[JaredforReal]

When using Pydantic models, the request body is pre-parsed and serialized. Passing request_body directly avoids re-reading await request.body() (which could fail or duplicate work), ensuring efficiency and correctness in mock tests.
For non-Pydantic endpoints (e.g., /tokenize), the parameter defaults to None, and the function falls back to reading the raw body, maintaining backward compatibility.

[YuhanLiu11]

Similarly, I don't get why do we need to change this function to add this swagger UI mock requests?

[JaredforReal]

FastAPI uses the Pydantic model to generate the OpenAPI schema for /v1/chat/completions. This enables Swagger UI to display editable request fields and validate inputs at the API level.
Without this, the endpoint would lack schema details, making mock testing impossible (no "Try it out" functionality or 422 error simulation, mentioned in issue #667 ).

[JaredforReal]

@YuhanLiu11 Thanks for your time! I try to make a minimal change to achieve this feature, learning from the VLLM implementation. I am considering making a proposal to refactor route_general_request() without changing the routing logic to take a Pydantic model originally for type safety, if this PR is accepted.
I will try to fix the pre-commit error. Feel free to point out any mistakes I have made, love to learn from the community :)

[davidgao7]

Suggested change

	"httpx==0.28.1"
	"httpx==0.28.1"

Hi, starting from this MR: #589 httpx has been replaced by aiohttp, the reason is huge performance boost, and aiohttp has been part of the package requirements