Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Backfill remote event fetched by MSC3030 so we can paginate from it later #13205

Merged
merged 33 commits into from
Jul 22, 2022
Merged

Backfill remote event fetched by MSC3030 so we can paginate from it later #13205

merged 33 commits into from
Jul 22, 2022

Commits on Jul 6, 2022

  1. Backfill remote event fetched by MSC3030 so we can paginate from it l… 

    …ater
    @MadLittleMods
    MadLittleMods committed Jul 6, 2022
    Configuration menu
    Copy the full SHA
    93bbac7 View commit details
    Browse the repository at this point in the history

  2. Working logs

    @MadLittleMods
    MadLittleMods committed Jul 6, 2022
    Configuration menu
    Copy the full SHA
    f645864 View commit details
    Browse the repository at this point in the history

  3. Working Complement test by getting new raw event which isn't an outlier

    @MadLittleMods
    MadLittleMods committed Jul 6, 2022
    Configuration menu
    Copy the full SHA
    6c4b618 View commit details
    Browse the repository at this point in the history

  4. Add changelog

    @MadLittleMods
    MadLittleMods committed Jul 6, 2022
    Configuration menu
    Copy the full SHA
    2e01535 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2022

  1. Debug logs

    @MadLittleMods
    MadLittleMods committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    b335d44 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2022

  1. More debugging

    @MadLittleMods
    MadLittleMods committed Jul 9, 2022
    Configuration menu
    Copy the full SHA
    dab7ad9 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2022

  1. Working with get_pdu 

    More debugging

Make proper clone/copy of event so it's not an outlier when backfilled.
This way we don't mess with the referenes in the cachese and get wacky results

Fast complement by excluding postgres stuff
    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    f3072c9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into madlittlemods/msc3030-backfill-at-remote-… 

    …event-fetched-v2

Conflicts:
	poetry.lock
	synapse/handlers/room.py
    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    3db79a1 View commit details
    Browse the repository at this point in the history

  3. Install tabulate back and working

    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    3f0da1b View commit details
    Browse the repository at this point in the history

  4. Clean up debug logs 

    ```
--- PASS: TestJumpToDateEndpoint (22.36s)
    --- PASS: TestJumpToDateEndpoint/parallel (1.41s)
        --- PASS: TestJumpToDateEndpoint/parallel/federation (0.00s)
            --- PASS: TestJumpToDateEndpoint/parallel/federation/can_get_pagination_token_after_getting_remote_event_from_timestamp_to_event_endpoint (1.40s)
PASS
ok  	github.com/matrix-org/complement/tests	25.604s
```
    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    b9c936c View commit details
    Browse the repository at this point in the history

  5. More cleanup

    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    e76fbc0 View commit details
    Browse the repository at this point in the history

  6. Better order and fix lints

    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    7d9c20a View commit details
    Browse the repository at this point in the history

  7. Revert fast complement changes

    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    48ca870 View commit details
    Browse the repository at this point in the history

  8. Remove unused persist_events_store

    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    6543bd5 View commit details
    Browse the repository at this point in the history

  9. Better logging

    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    8b0dd8c View commit details
    Browse the repository at this point in the history

  10. Add comment why _process_pulled_events

    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    ba344b4 View commit details
    Browse the repository at this point in the history

  11. Add docstring

    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    caa0fce View commit details
    Browse the repository at this point in the history

  12. Fix logic error

    @MadLittleMods
    MadLittleMods committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    05dc230 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2022

  1. Fix wrong scope in log

    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    7a316a5 View commit details
    Browse the repository at this point in the history

  2. Remove whitespace changes

    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    682399f View commit details
    Browse the repository at this point in the history

  3. Use shorthand

    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    4df2f0c View commit details
    Browse the repository at this point in the history

  4. Merge branch 'develop' into madlittlemods/msc3030-backfill-at-remote-… 

    …event-fetched-v2
    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    ce447f0 View commit details
    Browse the repository at this point in the history

  5. Remove duplicate information from error 

    See #13205 (comment)
    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    2beeccd View commit details
    Browse the repository at this point in the history

  6. Log what the remote event is closer than 

    See #13205 (comment)
    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    7f866f4 View commit details
    Browse the repository at this point in the history

  7. Explain why no persisting outliers in _process_pulled_event 

    See #13205 (comment)
    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    337d8be View commit details
    Browse the repository at this point in the history

  8. get_pdu returns pristine EventBase 

    See

 - #13205 (comment)
 - #13205 (comment)
    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    efaf434 View commit details
    Browse the repository at this point in the history

  9. Fix lints

    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    344e63e View commit details
    Browse the repository at this point in the history

  10. Fix unused ex lint

    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    cf5a324 View commit details
    Browse the repository at this point in the history

  11. Fix lint

    @MadLittleMods
    MadLittleMods committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    b3743c2 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2022

  1. Use timestamp from the event we backfilled instead of trusting the re… 

    …mote server blindly

See #13205 (comment)
    @MadLittleMods
    MadLittleMods committed Jul 16, 2022
    Configuration menu
    Copy the full SHA
    b2be2bc View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2022

  1. Merge branch 'develop' into madlittlemods/msc3030-backfill-at-remote-… 

    …event-fetched-v2

Conflicts:
	synapse/federation/federation_client.py
	synapse/handlers/federation_event.py
    @MadLittleMods
    MadLittleMods committed Jul 20, 2022
    Configuration menu
    Copy the full SHA
    7dbc4f7 View commit details
    Browse the repository at this point in the history

  2. Restore whitespace

    @MadLittleMods
    MadLittleMods committed Jul 20, 2022
    Configuration menu
    Copy the full SHA
    2d1a84b View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2022

  1. Add ideas the comment 

    See #13205 (comment)
    @MadLittleMods
    MadLittleMods committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    bf4e5d6 View commit details
    Browse the repository at this point in the history