[Aphoh]

Summary

Native SGLang /generate now records the same frontend observability as /v1/completions. Closes #16640.

Builds on #16504 (merged: workers send typed output tokens, finish reason and usage). It also includes #16489's commit (agent context from headers on native generate requests) with that PR's review feedback applied; that commit drops out of this diff once #16489 merges. Co-authored with @ShaneGZhu.

Frontend only, reusing existing code:

• Metrics: /generate records dynamo_frontend_* TTFT, ITL, output tokens and length, ISL, cached tokens and the worker_last_* gauges, in both streaming and unary.
  • It uses the collector vLLM /inference/v1/generate already had, moved into a neutral engine_output_metrics.rs.
  • The request completed log line now carries token counts, TTFT, ITL and worker IDs.
• Request-end tracing: /generate writes one DYN_REQUEST_TRACE record per request, including on cancellation.
  • Each record carries token counts, timing, worker IDs and replay hashes. With session headers it also carries agent context, x-request-id and the backend finish reason.
  • The new wrap_engine_output_request_end_stream observes typed LLMEngineOutput and reuses the existing request-end emit path. When tracing is off it returns the stream unchanged.
  • Tool-call metadata is not recorded on /generate.
• fix(frontend): carry agent context on native generate requests #16489 review feedback: agent headers are parsed once, and the context's parsed value is shared with the request (get_optional, so no allocation when the headers are absent).
• Small changes:
  • The trace builders take any &Context<T>, so the request is no longer moved out of its context and back.
  • The metric model is looked up once per request.
  • The /generate rejection counter now uses the metric model label, as vLLM does.
  • sglang_generate.rs tests move to sglang_generate_tests.rs.
• Docs: /generate coverage is noted on the request-trace and agent-tracing pages.

Validation

• cargo clippy -p dynamo-llm --lib --tests -- -D warnings
• cargo test -p dynamo-llm --lib -- sglang request_trace generate preprocessed_context
• cargo test -p dynamo-llm --lib --no-default-features request_trace
• pre-commit on the changed files

GPU (B200), aggregated serving. Stock SGLang 0.5.21 in the container, Qwen/Qwen3-0.6B; the sidecar and this branch's frontend run on the host.

/generate request	Frontend metrics	Request-end trace
stream, 32 tokens	TTFT 1, ITL 31, output 32, ISL 141, cached 128, success 1	1 record: in 141, out 32, cached 128, timing, worker IDs, replay length 141
unary, 32 tokens	same as stream	same as stream
with session headers and x-request-id	same as stream	same, plus agent_context (session and parent), x_request_id, finish length
client cancels after 8 tokens	output 8, ITL 7, counted as cancelled	1 record with output 8

Client responses are unchanged (text, output_ids, meta_info).

Generated with Devin

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-01a14444-13a5-73ad-bb19-f1ff7a3858fa.docs.buildwithfern.com/dynamo/dev

[devin-ai-integration]

✅ Dynamo PR CI passed — run 38887255376 (attempt 1) on 0d37273371

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

_{Posted automatically by Devin for run 38887255376. Updated on every full-CI run of this PR.}

[Aphoh]

Closing: #16983 unifies the native generate handlers and takes over the frontend metrics part. The request-end tracing part will be redone as a small follow-up on the unified handler, owned separately.