Faster joins: Handle leaving a room while re-syncing state #12802

squahtx · 2022-05-19T17:01:18Z

Lines 1512 to 1515 in 7c6b220

    
           # TODO(faster_joins): what happens if we leave the room during a resync? if we 
        
           #   really leave, that might mean we have difficulty getting the room state over 
        
           #   federation. 
        
           #   https://github.com/matrix-org/synapse/issues/12802

squahtx · 2022-06-06T17:21:31Z

I suspect this will block in update_membership_locked, just like #12801. Blocking isn't wrong, just far from ideal.

squahtx · 2022-09-30T13:38:37Z

As part of this issue, we'll want to enable / fix a skipped device list tracking test from Faster room joins: Add abandoned join tests for device list tracking complement#476.

richvdh · 2022-10-05T10:53:04Z

We'll need to:

abort the resync process
attempt send out leave events to anyone who is listening so as not to make the device-list-synchronisation problems any worse (so this is probably blocked on Faster joins: support sending messages from clients during the half-joined state #12997)
figure out if we need to clear out our partially synced room (maybe we can just leave it in place, to be picked up next time? but maybe that means we'll be wasting a bunch of time resyncing ancient history?)

DMRobertson · 2022-12-05T16:40:52Z

Note: all of this applies to when the last local user leaves the room.

DMRobertson · 2022-12-13T19:25:55Z

First thing I'm trying to do is make a regular CSAPI /leave work. This blocks:

  synapse_main | 2022-12-13 19:16:27,840 - synapse.storage.util.partial_state_events_tracker - 82 - INFO - POST-9 - Awaiting un-partial-stating of events ['$ugeJbJLZ9Zbo2b5VUeMgLqy1kxH8klw71XV5Lnzxw3s']
  synapse_main | Stack (most recent call last):
  synapse_main |   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
  synapse_main |     return _run_code(code, main_globals, None,
  synapse_main |   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
  synapse_main |     exec(code, run_globals)
  synapse_main |   File "/editable-src/synapse/app/homeserver.py", line 386, in <module>
  synapse_main |     main()
  synapse_main |   File "/editable-src/synapse/app/homeserver.py", line 382, in main
  synapse_main |     run(hs)
  synapse_main |   File "/editable-src/synapse/app/homeserver.py", line 361, in run
  synapse_main |     _base.start_reactor(
  synapse_main |   File "/editable-src/synapse/app/_base.py", line 191, in start_reactor
  synapse_main |     run()
  synapse_main |   File "/editable-src/synapse/app/_base.py", line 173, in run
  synapse_main |     run_command()
  synapse_main |   File "/editable-src/synapse/app/_base.py", line 148, in <lambda>
  synapse_main |     run_command: Callable[[], None] = lambda: reactor.run(),
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/base.py", line 1318, in run
  synapse_main |     self.mainLoop()
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/base.py", line 1328, in mainLoop
  synapse_main |     reactorBaseSelf.runUntilCurrent()
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/base.py", line 967, in runUntilCurrent
  synapse_main |     f(*a, **kw)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 696, in callback
  synapse_main |     self._startRunCallbacks(result)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks
  synapse_main |     self._runCallbacks()
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks
  synapse_main |     current.result = callback(  # type: ignore[misc]
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1792, in gotResult
  synapse_main |     _inlineCallbacks(r, gen, status, context)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1782, in _inlineCallbacks
  synapse_main |     status.deferred.callback(callbackValue)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 696, in callback
  synapse_main |     self._startRunCallbacks(result)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks
  synapse_main |     self._runCallbacks()
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks
  synapse_main |     current.result = callback(  # type: ignore[misc]
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 696, in callback
  synapse_main |     self._startRunCallbacks(result)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks
  synapse_main |     self._runCallbacks()
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks
  synapse_main |     current.result = callback(  # type: ignore[misc]
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1792, in gotResult
  synapse_main |     _inlineCallbacks(r, gen, status, context)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1782, in _inlineCallbacks
  synapse_main |     status.deferred.callback(callbackValue)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 696, in callback
  synapse_main |     self._startRunCallbacks(result)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks
  synapse_main |     self._runCallbacks()
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks
  synapse_main |     current.result = callback(  # type: ignore[misc]
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1371, in _cbDeferred
  synapse_main |     self.callback(cast(_DeferredListResultListT, self.resultList))
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 696, in callback
  synapse_main |     self._startRunCallbacks(result)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py",   synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks
  synapse_main |     self._runCallbacks()
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks
  synapse_main |     current.result = callback(  # type: ignore[misc]
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1792, in gotResult
  synapse_main |     _inlineCallbacks(r, gen, status, context)
  synapse_main |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks
  synapse_main |     result = context.run(gen.send, result)
  synapse_main |   File "/editable-src/synapse/http/server.py", line 229, in wrapped_async_request_handler
  synapse_main |     await h(self, request)
  synapse_main |   File "/editable-src/synapse/http/server.py", line 307, in _async_render_wrapper
  synapse_main |     callback_return = await self._async_render(request)
  synapse_main |   File "/editable-src/synapse/http/server.py", line 513, in _async_render
  synapse_main |     callback_return = await raw_callback_return
  synapse_main |   File "/editable-src/synapse/rest/client/room.py", line 988, in on_POST
  synapse_main |     await self.room_member_handler.update_membership(
  synapse_main |   File "/editable-src/synapse/handlers/room_member.py", line 575, in update_membership
  synapse_main |     result = await self.update_membership_locked(
  synapse_main |   File "/editable-src/synapse/handlers/room_member.py", line 804, in update_membership_locked
  synapse_main |     state_before_join = await self.state_handler.compute_state_after_events(
  synapse_main |   File "/editable-src/synapse/state/__init__.py", line 213, in compute_state_after_events
  synapse_main |     ret = await self.resolve_state_groups_for_events(
  synapse_main |   File "/editable-src/synapse/util/metrics.py", line 113, in measured_func
  synapse_main |     r = await func(self, *args, **kwargs)
  synapse_main |   File "/editable-src/synapse/state/__init__.py", line 514, in resolve_state_groups_for_events
  synapse_main |     state_groups = await self._state_storage_controller.get_state_group_for_events(
  synapse_main |   File "/editable-src/synapse/logging/opentracing.py", line 896, in _wrapper
  synapse_main |     return await func(*args, **kwargs)  # type: ignore[misc]
  synapse_main |   File "/editable-src/synapse/logging/opentracing.py", line 896, in _wrapper
  synapse_main |     return await func(*args, **kwargs)  # type: ignore[misc]
  synapse_main |   File "/editable-src/synapse/storage/controllers/state.py", line 374, in get_state_group_for_events
  synapse_main |     await self._partial_state_events_tracker.await_full_state(event_ids)
  synapse_main |   File "/editable-src/synapse/logging/opentracing.py", line 896, in _wrapper
  synapse_main |     return await func(*args, **kwargs)  # type: ignore[misc]
  synapse_main |   File "/editable-src/synapse/storage/util/partial_state_events_tracker
  synapse_main |     logger.info(

DMRobertson · 2023-01-20T13:33:55Z

Per Non lazy loading sync not blocking during fast join #14831 (comment), we should write to a complement test which checks that eager syncs don't see anything when the room becomes un-partial-stated after we leave.

DMRobertson · 2023-03-07T20:19:18Z

#15081

squahtx added this to the Q2 2022 ─ Faster joins phase 2: correctness milestone May 19, 2022

babolivier added the T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. label May 19, 2022

MadLittleMods added the A-Federated-Join joins over federation generally suck label Jun 3, 2022

erikjohnston modified the milestone: Q2 2022 ─ Faster joins phase 2: correctness Jul 19, 2022

richvdh modified the milestones: Q2 2022 ─ Faster joins phase 2: correctness, Q3 2022: Faster joins: fix major known bugs for monoliths Jul 20, 2022

richvdh added T-Enhancement New features, changes in functionality, improvements in performance, or user-facing enhancements. and removed T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. labels Jul 22, 2022

squahtx mentioned this issue Sep 23, 2022

Faster room joins: Outgoing device list updates not sent if we leave the room before join completes #13894

Open

richvdh mentioned this issue Sep 29, 2022

Faster room joins: Add abandoned join tests for device list tracking matrix-org/complement#476

Merged

kittykat mentioned this issue Oct 3, 2022

Improve time to join a remote room #14030

Closed

16 tasks

richvdh modified the milestones: Q3 2022: Faster joins: fix major known bugs for monoliths, Q4 2022: faster joins: worker-mode and remaining work Oct 5, 2022

squahtx added the z-blocked (Deprecated Label) label Nov 10, 2022

squahtx removed the z-blocked (Deprecated Label) label Nov 30, 2022

DMRobertson self-assigned this Dec 2, 2022

This was referenced Jan 10, 2023

Baby steps for testing leaves during a faster join matrix-org/complement#581

Merged

Faster joins: don't stall when a user joins during a fast join #14606

Merged

erikjohnston modified the milestones: Q4 2022: faster joins: worker-mode and remaining work, Remaining faster remote join work Feb 2, 2023

This was referenced Feb 17, 2023

Check eager sync behaviour after leave then un-partial-stating #15090

Open

More tests for leaves during a partial state resync matrix-org/complement#617

Merged

DMRobertson closed this as completed in matrix-org/complement#617 Feb 28, 2023

matrixbot mentioned this issue Dec 21, 2023

Check eager sync behaviour after leave then un-partial-stating element-hq/synapse#15090

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Faster joins: Handle leaving a room while re-syncing state #12802

Faster joins: Handle leaving a room while re-syncing state #12802

squahtx commented May 19, 2022 •

edited by richvdh

Loading

squahtx commented Jun 6, 2022

squahtx commented Sep 30, 2022

richvdh commented Oct 5, 2022

DMRobertson commented Dec 5, 2022

DMRobertson commented Dec 13, 2022

DMRobertson commented Jan 20, 2023

DMRobertson commented Mar 7, 2023

Faster joins: Handle leaving a room while re-syncing state #12802

Faster joins: Handle leaving a room while re-syncing state #12802

Comments

squahtx commented May 19, 2022 • edited by richvdh Loading

squahtx commented Jun 6, 2022

squahtx commented Sep 30, 2022

richvdh commented Oct 5, 2022

DMRobertson commented Dec 5, 2022

DMRobertson commented Dec 13, 2022

DMRobertson commented Jan 20, 2023

DMRobertson commented Mar 7, 2023

squahtx commented May 19, 2022 •

edited by richvdh

Loading