feat: add optional digest mode for Slack alert types

[dkindlund]

Summary

Adds an optional digest mode for Slack alert types that aggregates duplicate alerts within a configurable time window and emits a single summary message instead of individual messages for each event.

This dramatically reduces Slack noise for high-frequency alert types like llm_requests_hanging and llm_too_slow, which can fire hundreds of times per day with identical content.

Motivation

In production deployments, certain alert types (especially llm_requests_hanging) generate a high volume of duplicate Slack messages. For example, if a model endpoint is slow or unresponsive, every request to that model triggers a separate alert. Over a 24-hour period, this can produce hundreds of identical messages like:

Alert type: llm_requests_hanging
Level: Medium
Timestamp: 03:27:39

Message: Requests are hanging - 600s+ request time
Request Model: gemini-2.5-flash-search
API Base: None

This makes the Slack channel unusable for DevOps monitoring, as real issues get buried in noise.

Configuration

New alert_type_config setting in general_settings:

general_settings:
  alerting: ["slack"]
  alert_type_config:
    llm_requests_hanging:
      digest: true
      digest_interval: 86400  # 24 hours in seconds (default)
    llm_too_slow:
      digest: true
      digest_interval: 3600   # 1 hour
    llm_exceptions:
      digest: true
      # uses default interval (86400s)

Parameter	Type	Default	Description
digest	bool	false	Enable digest mode for this alert type
digest_interval	int	86400 (24h)	Time window in seconds for aggregating alerts

• Available for all 21 alert types — user opts in per type
• Fully backward compatible — existing behavior is unchanged when alert_type_config is not set

How It Works

1. When an alert fires for a digest-enabled type, it is grouped by (alert_type, request_model, api_base) instead of being sent immediately
2. A counter tracks how many times the alert fires within the fixed interval window
3. When the interval expires, a single summary message is sent:

Alert type: `llm_requests_hanging` (Digest)
Level: `Medium`
Start: `2026-02-19 03:27:39`
End: `2026-02-20 03:27:39`
Count: `847`

Message: `Requests are hanging - 600s+ request time`
Request Model: `gemini-2.5-flash`
API Base: `None`

Design Decisions

• Digest key: (alert_type, request_model, api_base) — groups alerts for the same model/endpoint together
• Structured params: send_alert() accepts optional request_model and api_base parameters directly from callers, avoiding fragile message parsing
• Fixed interval window: Resets after each digest emission for predictable timing
• Thread safety: All digest bucket operations are protected by asyncio.Lock
• Flush integration: Digest buckets are checked during the existing periodic_flush() cycle

Known Limitations

• Per-instance state: Digest buckets are held in memory per proxy instance. In multi-instance deployments (e.g., Cloud Run with autoscaling), each instance maintains its own digest and emits its own summary
• Not durable: If an instance is terminated before the digest interval expires, the aggregated alerts for that instance are lost

Both limitations are acceptable trade-offs — the primary goal is noise reduction, and even with multiple instances, receiving N digest messages per interval (where N = number of instances) is dramatically better than hundreds of individual alerts.

Files Changed

File	Change
litellm/types/integrations/slack_alerting.py	New AlertTypeConfig model, DigestEntry TypedDict
litellm/integrations/SlackAlerting/slack_alerting.py	Core digest logic in send_alert(), _flush_digest_buckets(), periodic_flush() override
litellm/integrations/SlackAlerting/hanging_request_check.py	Pass request_model and api_base to send_alert()
litellm/proxy/utils.py	Pass alert_type_config through ProxyLogging.update_values()
litellm/proxy/proxy_server.py	Load alert_type_config from general_settings
docs/my-website/docs/proxy/alerting.md	Documentation for digest mode configuration and usage
tests/.../test_slack_alerting_digest.py	11 unit tests covering all digest behaviors

Test plan

• Digest mode suppresses duplicate alerts within the interval (count accumulates)
• Different (model, api_base) combinations get separate digest entries
• Non-digest alert types are completely unaffected (go straight to queue)
• Digest summary is emitted after the interval expires with correct count and timestamps
• Digest is NOT emitted before the interval expires
• Digest message format includes (Digest) tag, Start/End timestamps, and Count
• Alerts without request_model/api_base group by alert type alone
• AlertTypeConfig defaults and custom values work correctly
• Config can be set at init time and updated dynamically via update_values()
• Existing Slack alerting tests pass with no regressions (32/32)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 20, 2026 4:24pm

[greptile-apps]

Greptile Summary

This PR adds an optional digest mode for Slack alert types to reduce noise from high-frequency alerts. When enabled via alert_type_config, duplicate alerts are aggregated in-memory by (alert_type, request_model, api_base) and emitted as a single summary message after a configurable interval (default 24 hours).

Key changes:

• Added AlertTypeConfig model and DigestEntry TypedDict to define configuration structure
• Implemented thread-safe digest buckets with asyncio.Lock protection
• Integrated _flush_digest_buckets() into existing periodic_flush() cycle
• Modified hanging_request_check.py to pass structured request_model and api_base parameters
• Configuration flows through proxy_server.py → utils.py → SlackAlerting

Design strengths:

• Fully backward compatible - existing behavior unchanged when config not set
• Uses structured parameters instead of fragile message parsing
• Comprehensive test coverage (11 tests) with no network calls
• Clear documentation with examples and limitations

Trade-offs acknowledged:

• Per-instance in-memory state (not shared across replicas)
• Non-durable (lost if instance terminates before flush)
• These are acceptable for the noise reduction use case

Confidence Score: 4/5

• Safe to merge with minor considerations around in-memory state management
• Well-designed feature with comprehensive testing and clear documentation. The implementation is thread-safe, backward compatible, and follows existing patterns. Score is 4 (not 5) due to in-memory state that is per-instance and non-durable, though these limitations are documented and acceptable for the use case.
• No files require special attention - all changes are well-implemented

Important Files Changed

Filename	Overview
litellm/types/integrations/slack_alerting.py	Added AlertTypeConfig model and DigestEntry TypedDict for digest mode configuration - well-structured type definitions with proper defaults
litellm/integrations/SlackAlerting/slack_alerting.py	Core digest implementation with thread-safe digest buckets, flush logic, and proper integration into existing alert flow - well-designed but adds in-memory state
litellm/integrations/SlackAlerting/hanging_request_check.py	Minimal change to pass request_model and api_base parameters to send_alert() for digest grouping - clean implementation
tests/test_litellm/integrations/SlackAlerting/test_slack_alerting_digest.py	Comprehensive test suite with 11 tests covering digest suppression, flushing, message format, and configuration - excellent coverage with no network calls

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Alert fires: send_alert called] --> B{Digest mode enabled<br/>for this alert type?}
    B -->|No| C[Format standard message]
    C --> D[Add to log queue]
    D --> E[Send to Slack immediately]
    
    B -->|Yes| F[Create digest key from<br/>alert type, model, api base]
    F --> G{Digest key exists<br/>in digest buckets?}
    G -->|Yes| H[Increment count<br/>Update last time]
    G -->|No| I[Create new DigestEntry<br/>count equals 1, start time now]
    H --> J[Return - no immediate send]
    I --> J
    
    K[periodic flush runs<br/>every flush interval] --> L[flush digest buckets]
    L --> M{For each bucket:<br/>elapsed exceeds digest interval?}
    M -->|No| N[Keep in bucket]
    M -->|Yes| O[Format digest summary<br/>with Start, End, Count]
    O --> P[Add to log queue]
    P --> Q[Remove from digest buckets]
    Q --> R[Send digest to Slack]

Loading

_{Last reviewed commit: 8fe0ed5}

[greptile-apps]

_{7 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}