Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Faster joins: Update user directory once state re-sync completes #12815

Closed
1 task
Tracked by #14030
squahtx opened this issue May 20, 2022 · 3 comments · Fixed by matrix-org/complement#587 or #14874
Closed
1 task
Tracked by #14030

Faster joins: Update user directory once state re-sync completes #12815

squahtx opened this issue May 20, 2022 · 3 comments · Fixed by matrix-org/complement#587 or #14874
Assignees
@H-Shay
Labels
A-Federated-Join joins over federation generally suck T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks.
Milestone
Q4 2022: faster j...

Comments

@squahtx
Copy link
Contributor

squahtx commented May 20, 2022 •
edited by reivilibre
Loading

Potentially useful / possible blockers:
@babolivier babolivier added the T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. label May 20, 2022
@MadLittleMods MadLittleMods added the A-Federated-Join joins over federation generally suck label Jun 3, 2022
@richvdh
Copy link
Member

richvdh commented Jun 9, 2022

Also: what does the directory say during re-sync, and is that ok?

@richvdh
Copy link
Member

richvdh commented Jun 9, 2022 •
edited by squahtx
Loading

Particular points of note:

  • synapse/synapse/handlers/federation.py

    Lines 1561 to 1570 in 7c6b220

    if success:
    logger.info("State resync complete for %s", room_id)
    self._storage_controllers.state.notify_room_un_partial_stated(
    room_id
    )
    # TODO(faster_joins) update room stats and user directory?
    # https://github.com/matrix-org/synapse/issues/12814
    # https://github.com/matrix-org/synapse/issues/12815
    return

  • synapse/synapse/storage/controllers/state.py

    Lines 454 to 461 in 7c6b220

    # FIXME(faster_joins): what do we do here?
    # https://github.com/matrix-org/synapse/issues/12814
    # https://github.com/matrix-org/synapse/issues/12815
    # https://github.com/matrix-org/synapse/issues/13008
    return await self.stores.main.get_partial_current_state_deltas(
    prev_stream_id, max_stream_id
    )

@kittykat kittykat mentioned this issue Oct 3, 2022
Closed
16 tasks
@H-Shay H-Shay mentioned this issue Jan 19, 2023
Merged
squahtx pushed a commit that referenced this issue Jan 19, 2023
Update room stats and the user directory on workers when finishing join
3452224 
When finishing a partial state join to a room, we update the current
state of the room without persisting additional events. Workers receive
notice of the current state update over replication, but neglect to wake
the room stats and user directory updaters, which then get incidentally
triggered the next time an event is persisted or an unrelated event
persister sends out a stream position update.

We wake the room stats and user directory updaters at the appropriate
time in this commit.

Part of #12814 and #12815.

Signed-off-by: Sean Quah <[email protected]>
squahtx pushed a commit that referenced this issue Jan 19, 2023
Faster joins: Update room stats and user directory on workers when done
ff3a7b6 
When finishing a partial state join to a room, we update the current
state of the room without persisting additional events. Workers receive
notice of the current state update over replication, but neglect to wake
the room stats and user directory updaters, which then get incidentally
triggered the next time an event is persisted or an unrelated event
persister sends out a stream position update.

We wake the room stats and user directory updaters at the appropriate
time in this commit.

Part of #12814 and #12815.

Signed-off-by: Sean Quah <[email protected]>
@squahtx squahtx mentioned this issue Jan 19, 2023
Merged
@squahtx squahtx linked a pull request Jan 19, 2023 that will close this issue
Add a test to verify that the user directory is properly updated after a lazy load join finishes syncing matrix-org/complement#587
Merged
squahtx pushed a commit that referenced this issue Jan 19, 2023
Faster joins: Update room stats and user directory on workers when done
e921504 
When finishing a partial state join to a room, we update the current
state of the room without persisting additional events. Workers receive
notice of the current state update over replication, but neglect to wake
the room stats and user directory updaters, which then get incidentally
triggered the next time an event is persisted or an unrelated event
persister sends out a stream position update.

We wake the room stats and user directory updaters at the appropriate
time in this commit.

Part of #12814 and #12815.

Signed-off-by: Sean Quah <[email protected]>
@squahtx squahtx linked a pull request Jan 19, 2023 that will close this issue
Faster joins: Update room stats and the user directory on workers when finishing join #14874
Merged
@squahtx
Copy link
Contributor Author

squahtx commented Jan 19, 2023

Also: what does the directory say during re-sync, and is that ok?

During re-sync, the user directory reflects the current state of the room and excludes all remote users except new joiners.

DMRobertson pushed a commit that referenced this issue Jan 23, 2023
@squahtx
Faster joins: Update room stats and the user directory on workers whe…
2ec9c58 
…n finishing join (#14874)

* Faster joins: Update room stats and user directory on workers when done

When finishing a partial state join to a room, we update the current
state of the room without persisting additional events. Workers receive
notice of the current state update over replication, but neglect to wake
the room stats and user directory updaters, which then get incidentally
triggered the next time an event is persisted or an unrelated event
persister sends out a stream position update.

We wake the room stats and user directory updaters at the appropriate
time in this commit.

Part of #12814 and #12815.

Signed-off-by: Sean Quah <[email protected]>

* fixup comment

Signed-off-by: Sean Quah <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@H-Shay H-Shay

Labels
A-Federated-Join joins over federation generally suck T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks.
Projects
None yet
Milestone
Q4 2022: faster joins: worker-mode an...
5 participants
@MadLittleMods @richvdh @babolivier @squahtx @H-Shay