Skip to content

fix(slack): bound all per-message tracking structures with oldest-first eviction - #69484

Merged
teknium1 merged 5 commits into
mainfrom
slack/c16-cache-bounds
Jul 23, 2026
Merged

fix(slack): bound all per-message tracking structures with oldest-first eviction#69484
teknium1 merged 5 commits into
mainfrom
slack/c16-cache-bounds

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

Every per-message/per-user tracking structure in the Slack adapter is now bounded with oldest-first eviction — fixing both the memory leaks on busy workspaces and the #51019-class bug where arbitrary set eviction made the bot silently stop responding to active threads.

Fixes #51019.

Changes

Credits

Salvaged with authorship preserved: #51097 (@markoub), #23676 + #23375 (@AhmetArif0, commits authored under their renamed account EloquentBrush — same person per the frozen legacy map).

Validation

Check Result
tests/gateway/ -q -k slack 617 passed, 0 failed
New: 13 bounds tests (oldest-not-arbitrary per structure, caps under churn, active entries survive pressure) green
#51019 regression shape (active thread survives eviction) covered

Infographic

slack-cache-bounds

markoub and others added 5 commits July 22, 2026 08:30
_resolve_user_name() caches every resolved (team_id, user_id) → display
name but never evicts entries. On a long-running bot in a large
workspace every unique user the bot encounters adds a permanent entry.
Sibling structures _bot_message_ts (BOT_TS_MAX=5000) and
_assistant_threads (ASSISTANT_THREADS_MAX=5000) already have caps;
_user_name_cache had none.

Fix: add _USER_NAME_CACHE_MAX = 5000 and evict the oldest half on
overflow after each write, matching the existing sibling pattern.
Consolidates the two cache-write branches (success + API error) into a
single write so eviction runs once per resolution.

Reapplied from #23676 onto the current adapter (cache moved to
plugins/platforms/slack/adapter.py and is now keyed by
(team_id, user_id) tuples for multi-workspace safety).
_fetch_thread_context() caches per-thread Slack history under a
60-second TTL, but expired entries are never removed. On the current
adapter this is worse than when first reported: each entry now also
retains the raw conversations.replies payloads (messages) for
watermark re-formatting, so a busy multi-channel workspace accumulates
full message lists forever. _bot_message_ts, _mentioned_threads, and
_assistant_threads all enforce a MAX constant with active eviction in
the same __init__; _thread_context_cache had the TTL declared but no
matching eviction path.

Fix: add _THREAD_CACHE_MAX = 2500 and purge expired entries
(fetched_at older than _THREAD_CACHE_TTL) whenever a new write pushes
the cache past the limit. TTL-based eviction is used instead of LRU
because fresh entries are still needed; evicting them would
immediately re-trigger a rate-limited conversations.replies call.

Adds two unit tests: one verifying stale entries are purged on
overflow, one verifying fresh entries survive.

Reapplied from #23375 onto the current adapter (moved to
plugins/platforms/slack/adapter.py; cache entries now carry raw
message payloads and parent_user_id).
…with oldest-first eviction

Widening pass over the whole adapter following the cluster-C16 audit
(#51019, #51097, #23676, #23375): every in-memory structure that
accumulates per-message, per-user, or per-thread state is now bounded,
and every eviction is oldest-first — never arbitrary set-iteration
order, which is the #51019 failure mode (bot silently going quiet on
the most ACTIVE thread because set.pop-order eviction removed it).

Newly bounded:
- _approval_resolved / _clarify_resolved (caps 1000): unclicked
  approval/clarify prompts leaked their double-click-guard entries
  forever; oldest-insertion eviction via _trim_oldest_dict_entries.
- _reacting_message_ids (cap 5000): reaction lifecycle entries leaked
  when an exception fired between add and finalize; oldest-ts eviction.
- _active_status_threads (cap 1000): statuses abandoned by error paths
  accumulated; oldest-thread-ts eviction so the newest live status is
  never cleared.
- _channel_team (cap 10000): grew with every DM channel the bot ever
  saw (DM channel IDs are per-user). All four write sites now route
  through _remember_channel_team; eviction is safe because entries are
  re-learned from the next event and _get_client falls back to the
  primary client.
- _slash_command_contexts (cap 1000): TTL cleanup only ran on lookup,
  so contexts whose ephemeral replies never happened accumulated;
  overflow purges expired entries first, then oldest-stash-first.

Converted from arbitrary set-order eviction to oldest-first:
- _titled_assistant_threads: keys are (team, channel, thread_ts) —
  now evicts oldest thread first via _discard_oldest_by_thread_ts.
- _thread_rehydration_checked: keys are team:channel:thread_ts[:user] —
  arbitrary eviction here would re-run an ACTIVE thread's restart
  rehydration check and re-inject the missed-delta context; now evicts
  oldest thread first.
- _reacting_message_ids uses #51097's _discard_oldest_slack_timestamps.

Deliberately NOT bounded (naturally tiny, per-workspace):
_team_clients, _team_bot_user_ids, _team_bot_names (one entry per
installed workspace), _assistant_threads / _agent_view_contexts /
_bot_message_ts / _mentioned_threads / _user_name_cache /
_thread_context_cache (already bounded), _dedup (MessageDeduplicator
has max_size + TTL internally).

New helpers: _trim_oldest_dict_entries (dicts preserve insertion
order, so oldest-first is exact) and _discard_oldest_by_thread_ts
(chronological sort on the embedded Slack ts for keyed sets).

Tests: caps hold under churn, eviction removes OLDEST not arbitrary
entries, newest/active entries survive eviction pressure (regression
shape for #51019), plus end-to-end paths through _resolve_user_name
and _handle_slash_command.

Part of the C16 cache-bounds consolidation with #51097 (markoub),
#23676 and #23375 (EloquentBrush). Fixes #51019.
EloquentBrush's noreply email already maps to AhmetArif0 in the frozen
legacy AUTHOR_MAP (same account, renamed) — no new mapping needed.
@teknium1 teknium1 added the platform/slack Slack app adapter label Jul 22, 2026
@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

૮ >ﻌ< ა ci review

running on 29b692f

CI timings

CI timings · View job

Wall time 8m47s vs 9m59s (-12.0%). 9 job(s) slower, 8 faster, 3 unchanged.

  • Build&Test Docker image / build (amd64, ubuntu-latest, linux/amd64, type=gha,scope=docker-amd64, type=gha,mode=max,scope=do...: -127.0s
  • Python tests / Run tests slice 6/8: -16.0s
  • Python tests / Run tests slice 1/8: +15.0s
  • Build&Test Docker image / build (arm64, ubuntu-24.04-arm, linux/arm64, type=gha,scope=docker-arm64, type=gha,mode=max,scope...: +11.0s
  • Python tests / Run tests slice 7/8: -8.0s

@teknium1
teknium1 merged commit 90d6296 into main Jul 23, 2026
37 checks passed
@teknium1
teknium1 deleted the slack/c16-cache-bounds branch July 23, 2026 03:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

platform/slack Slack app adapter

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Slack gateway evicts arbitrary thread timestamps from _bot_message_ts / _mentioned_threads — bot silently stops responding to active threads

3 participants