Skip to content

feat(metrics): report abort reasons and terminal outcomes - #34132

Open
azuruce wants to merge 1 commit into
sgl-project:mainfrom
azuruce:observability/abort-reason-metrics
Open

azuruce wants to merge 1 commit into
sgl-project:mainfrom
azuruce:observability/abort-reason-metrics

Conversation

@azuruce

@azuruce azuruce commented Aug 8, 2026 •

Copy link
Copy Markdown

Motivation

sglang:num_aborted_requests_total counts abort dispatches rather than mutually exclusive request failures. It also has no reason label, so operators cannot distinguish client disconnects from explicit cancellation, admission-control rejection, scheduler timeout, KV exhaustion, or request-processing errors. abort_all and completion/abort races make the legacy counter especially unsuitable as an abort-rate numerator.

Modifications

  • Add a bounded AbortReason enum and propagate an explicit reason through every built-in SGLang abort path.
  • Add sglang:num_abort_signals_total{reason=...} for cancellation signals while preserving sglang:num_aborted_requests_total for compatibility.
  • Add sglang:request_outcomes_total{is_streaming,outcome,reason}. Each metrics-enabled request is counted once when its terminal state is observed, after completion/abort races resolve.
  • Count local request-cleanup outcomes before their request state is removed.
  • Document the reason categories, signal/outcome distinction, PromQL breakdown, and terminal abort-rate query.

Accuracy Tests

Not applicable. This changes request lifecycle metadata and Prometheus counters, not model execution or outputs.

Speed Tests and Profiling

Not benchmarked. The new work is limited to bounded-label counter increments on abort dispatch and terminal request handling; it is not in the token-generation loop.

Validation

  • SGLang pinned Black 26.1.0, isort 7.0.0, and Ruff 0.15.1 checks pass on all changed Python files.
  • Python compilation passes for all changed Python files.
  • Codespell and git diff --check pass.
  • AST audit confirms every built-in FINISH_ABORT, prepare_abort, and AbortReq construction supplies an explicit reason.
  • Unit tests were not added or run per the local repository instruction.

Checklist


CI States

Latest PR Test (Base): ❌ Run #31274727588
Latest PR Test (Extra): ❌ Run #31274727494

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant