Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Faster joins: create an un-partial-stated events stream for notifying workers that events have been un-partial-stated #14418

Closed
reivilibre opened this issue Nov 11, 2022 · 0 comments · Fixed by #14545
Assignees
Labels
A-Federated-Join joins over federation generally suck T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks.

Comments

@reivilibre
Copy link
Contributor

reivilibre commented Nov 11, 2022

Part of #12994, we would like a stream that contains un-partial-stated events.

Once we un-partial-state an event, we need to tell other workers about it, so that they can:

  • flush their cache for is_partial_state_event
  • flush their cache for _get_state_group_for_event
  • if the event changed rejection status, do _invalidate_local_get_event_cache
  • unblock any requests which are waiting for full state at that event

I think this probably "just" wants to be a new replication stream listing the event ids, and the old and new rejection status, of any un-partial-stated events. This can then be used to flush the caches and to call _state_storage_controller.notify_event_un_partial_stated.

I think most of the infrastructure for the stream is already present (in synapse.replication.tcp.streams). We'll need to define a new stream type, and add a new stream table to contain the data.

#12994 (comment)

It might be interesting to ponder #13456 — getting rid of stream tables — but also that's likely to be too much of a distraction here (and I've had a little ponder and can't convince myself of any clean alternatives, at the moment).

Bear in mind the stream may have multiple writers in the future.

@reivilibre reivilibre added A-Federated-Join joins over federation generally suck T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. labels Nov 11, 2022
@reivilibre reivilibre self-assigned this Nov 11, 2022
@reivilibre reivilibre changed the title Faster joins: create a un-partial-stated events stream for notifying workers that events have been un-partial-stated Faster joins: create an un-partial-stated events stream for notifying workers that events have been un-partial-stated Nov 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A-Federated-Join joins over federation generally suck T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks.
Projects
None yet
1 participant