Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/getting-started/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ spec:
|--------|------|-------------|
| `smg_http_requests_total` | Counter | Requests by method, path |
| `smg_http_request_duration_seconds` | Histogram | Request latency |
| `smg_http_responses_total` | Counter | Responses by status_code, error_code |
| `smg_http_responses_total` | Counter | Responses by path, status_code, error_code |
| `smg_http_connections_active` | Gauge | Active connections |
| `smg_http_rate_limit_total` | Counter | Rate limit decisions |

Expand Down Expand Up @@ -184,6 +184,13 @@ sum(rate(smg_http_responses_total{status_code=~"5.."}[5m]))
/ sum(rate(smg_http_responses_total[5m]))
```

**`/v1/responses` Success Rate**

```promql
sum(rate(smg_http_responses_total{path="/v1/responses",status_code=~"2.."}[5m]))
/ sum(rate(smg_http_responses_total{path="/v1/responses"}[5m]))
```

Comment thread
coderabbitai[bot] marked this conversation as resolved.
**Time to First Token (TTFT)**
```promql
histogram_quantile(0.5, rate(smg_router_ttft_seconds_bucket[5m]))
Expand Down
9 changes: 7 additions & 2 deletions docs/reference/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,18 +69,23 @@ rate(smg_http_request_duration_seconds_sum[5m]) / rate(smg_http_request_duration

### `smg_http_responses_total`

HTTP responses by status and error code.
HTTP responses by path, status, and error code.

| Type | Labels |
|------|--------|
| Counter | `status_code`, `error_code` |
| Counter | `path`, `status_code`, `error_code` |

```promql
# Error rate (5xx responses)
sum(rate(smg_http_responses_total{status_code=~"5.."}[5m])) / sum(rate(smg_http_responses_total[5m]))

# Success rate
sum(rate(smg_http_responses_total{status_code="200"}[5m])) / sum(rate(smg_http_responses_total[5m]))

# Success rate for /v1/responses
sum(rate(smg_http_responses_total{path="/v1/responses",status_code=~"2.."}[5m]))
/
sum(rate(smg_http_responses_total{path="/v1/responses"}[5m]))
```

---
Expand Down
14 changes: 3 additions & 11 deletions model_gateway/src/middleware/logging.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,9 @@ use tracing::{error, field::Empty, info, info_span, warn, Span};
use tracing_opentelemetry::OpenTelemetrySpanExt;

use super::{metrics::normalize_path_for_metrics, request_id::RequestId};
use crate::{
observability::{
metrics::{method_to_static_str, Metrics},
otel_trace::extract_trace_context_http,
},
routers::error::extract_error_code_from_response,
use crate::observability::{
metrics::{method_to_static_str, Metrics},
otel_trace::extract_trace_context_http,
};

/// Custom span maker that includes request ID
Expand Down Expand Up @@ -85,11 +82,6 @@ impl<B> OnResponse<B> for ResponseLogger {
let status = response.status();
let status_code = status.as_u16();

let error_code = extract_error_code_from_response(response);

// Layer 1: HTTP metrics
Metrics::record_http_response(status_code, error_code);

// Record these in the span for structured logging/observability tools
span.record("status_code", status_code);
// Use microseconds as integer to avoid format! string allocation
Expand Down
14 changes: 11 additions & 3 deletions model_gateway/src/middleware/metrics.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,12 @@ use std::{
use axum::{extract::Request, response::Response};
use tower::{Layer, Service};

use crate::observability::{
inflight_tracker::InFlightRequestTracker,
metrics::{method_to_static_str, Metrics},
use crate::{
observability::{
inflight_tracker::InFlightRequestTracker,
metrics::{method_to_static_str, Metrics},
},
routers::error::extract_error_code_from_response,
};

/// Tower Layer for HTTP metrics collection (SMG Layer 1 metrics)
Expand Down Expand Up @@ -86,6 +89,11 @@ where
let response = result?;

let duration = start.elapsed();
Metrics::record_http_response(
&path,
response.status().as_u16(),
extract_error_code_from_response(&response),
);
Metrics::record_http_duration(method, &path, duration);

Ok(response)
Expand Down
6 changes: 4 additions & 2 deletions model_gateway/src/observability/metrics.rs
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ pub(crate) fn init_metrics() {
);
describe_counter!(
"smg_http_responses_total",
"Total HTTP responses by status_code and error_code"
"Total HTTP responses by path, status_code and error_code"
);
describe_gauge!(
"smg_http_connections_active",
Expand Down Expand Up @@ -516,11 +516,13 @@ impl Metrics {
}

/// Record HTTP response.
pub fn record_http_response(status_code: u16, error_code: &str) {
pub fn record_http_response(path: &str, status_code: u16, error_code: &str) {
let path_interned = intern_string(path);
let status_str: Cow<'static, str> = status_code_to_cow(status_code);
let error_interned = intern_string(error_code);
counter!(
"smg_http_responses_total",
"path" => path_interned,
"status_code" => status_str,
"error_code" => error_interned
)
Expand Down
Loading