Skip to content

Fix performance regression related to delayed events processing #18926

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Sep 23, 2025
Merged

Fix performance regression related to delayed events processing #18926

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
fadb7eb
Add a storage function to pull out senders for event _ids
anoadragon453 Sep 17, 2025
e3ad18c
Stop calling `get_event` and use new storage function
anoadragon453 Sep 17, 2025
9f1c7e3
Only schedule next delayed event once
anoadragon453 Sep 17, 2025
76b885b
Bail out early if there are no delayed events to send
anoadragon453 Sep 17, 2025
a0faac9
newsfile
anoadragon453 Sep 17, 2025
8d97548
Note 100 row limit in docstrings
anoadragon453 Sep 18, 2025
d1974a2
Note that order of fetching stream orderings is important
anoadragon453 Sep 18, 2025
4ff8184
link MSC4140 in newsfile
anoadragon453 Sep 18, 2025
68fb552
More error handling
anoadragon453 Sep 18, 2025
1eb25f1
Revert "Only schedule next delayed event once"
anoadragon453 Sep 18, 2025
ef38b2e
Prevent potential infinite loop if we bypass the max
anoadragon453 Sep 19, 2025
a67b036
logger.error for missing sender
anoadragon453 Sep 19, 2025
b4877a6
state reset TODO
anoadragon453 Sep 19, 2025
6eaf31e
Add a test for `get_senders_for_event_ids`
anoadragon453 Sep 22, 2025
3d44c99
Handle the case where a room has been purged
anoadragon453 Sep 22, 2025
af7962d
Update logging and TODO to discuss state resets
anoadragon453 Sep 22, 2025
17d2845
Move `Sentinel` to util module
anoadragon453 Sep 22, 2025
c732bba
Replace ad-hoc Sentinel usage with utility module
anoadragon453 Sep 22, 2025
8c427f2
Update logs and comments to better cover cases
anoadragon453 Sep 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions synapse/handlers/delayed_events.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
from twisted.internet.interfaces import IDelayedCall

from synapse.api.constants import EventTypes
from synapse.api.errors import ShadowBanError
from synapse.api.errors import ShadowBanError, SynapseError
from synapse.api.ratelimiting import Ratelimiter
from synapse.config.workers import MAIN_PROCESS_INSTANCE_NAME
from synapse.logging.opentracing import set_tag
Expand Down Expand Up @@ -251,7 +251,23 @@ async def _handle_state_deltas(self, deltas: List[StateDelta]) -> None:
"Handling: %r %r, %s", delta.event_type, delta.state_key, delta.event_id
)

sender = UserID.from_string(event_id_and_sender_dict[delta.event_id])
sender_str = event_id_and_sender_dict.get(delta.event_id, None)
if sender_str is None:
logger.debug(
"Not handling delta for event where sender is unknown: %s",
delta.event_id,
)
continue

try:
sender = UserID.from_string(sender_str)
except SynapseError as e:
logger.error(
"Skipping state delta with Matrix User ID '%s' that failed to parse: %s",
sender_str,
e,
)
continue

next_send_ts = await self._store.cancel_delayed_state_events(
room_id=delta.room_id,
Expand Down