[hope1262946533]

Motivation

With --enable-metrics and --tokenizer-worker-num > 1, the HTTP request
metrics are silently missing from /metrics:

• sglang:http_requests_total
• sglang:http_responses_total
• sglang:http_requests_active
• sglang:routing_keys_active

Per-endpoint request rates, in-flight counts and 5xx ratios therefore cannot be
monitored on a multi-tokenizer deployment, even though /metrics itself is
served fine.

Modifications

The track-response middleware was registered from _setup_and_run_http_server,
which runs in the parent process. With --tokenizer-worker-num > 1 the server
is started with uvicorn.run("...:app", workers=N): uvicorn spawns the HTTP
workers and each one re-imports http_server, so the app they actually serve is
a fresh object that never received the middleware.

• Register the middleware while the module is imported (next to the existing
  CORSMiddleware / RequestDecompressionMiddleware registrations) so that
  every tokenizer worker picks it up, and drop the parent-process registration.
• Create the metrics themselves in a separate step, init_http_request_metrics,
  called from the lifespan once the process has published its config.
  prometheus_client fixes its value backend when it is first imported, and
  SGLang only sets PROMETHEUS_MULTIPROC_DIR in _set_envs_and_config, which
  runs after http_server has been imported. Creating the metrics at import
  time would therefore leave them in the in-memory registry, where the
  MultiProcessCollector behind /metrics never sees them -- including in
  single-tokenizer mode.
• The middleware now runs inside each worker, so also expose that worker's
  in-flight request count as sglang:http_worker_active_requests{worker_id}.
  This is the number to watch when a shared listening socket concentrates
  connections on a few workers.

A unit test in test/registered/unit/observability/test_http_request_metrics.py
covers all three: that the middleware is on the module-level app, that importing
http_server does not pull prometheus_client in early, and that a worker
process records the counters and labels the per-worker gauge.

Accuracy Tests

N/A - metrics only.

Speed Tests and Profiling

N/A - no change to the inference path. With metrics enabled the per-request work
is unchanged; with metrics disabled the middleware returns after one config
lookup.

Checklist

• Format your code according to the Format code with
  pre-commit.
• Add unit tests according to the Run and add unit
  tests.
• Update documentation according to Write
  documentations.
  Not applicable: docs/docs/references/production_metrics.mdx does not
  document the HTTP request metrics at all, so adding only the new one would
  be inconsistent.
• Provide accuracy and speed benchmark results according to Test the
  accuracy
  and Benchmark the
  speed.
  N/A, see above.
• Follow the SGLang code style
  guidance.

---

CI States

Latest PR Test (Base): ⏳ Run #39325198175
Latest PR Test (Extra): ❌ Run #39325198198
Latest PR Test (AMD ROCm 10): ⏳ Run #39325198377

[hnyls2002]

/rerun-test test_http_request_metrics.py registered/observability/test_metrics.py registered/tokenizer/test_multi_tokenizer.py test_diffusion_metrics.py registered/openai_server/basic/test_http2_server.py registered/e2e/entrypoints/test_engine_multi_tokenizer.py

[hnyls2002]

/tag-and-rerun-ci

[github-actions]

🚀 ubuntu-latest (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/observability/test_http_request_metrics.py

🚀 1-gpu-5090 (2 tests): ✅ View workflow run

cd test/ && python3 registered/observability/test_metrics.py
cd test/ && python3 registered/openai_server/basic/test_http2_server.py

🚀 1-gpu-h100 (2 tests): ✅ View workflow run

cd test/ && python3 registered/tokenizer/test_multi_tokenizer.py
cd test/ && python3 registered/e2e/entrypoints/test_engine_multi_tokenizer.py

🚀 1-gpu-h100 (1 test): ✅ View workflow run

python3 -m pytest python/sglang/multimodal_gen/test/unit/test_diffusion_metrics.py -x