Skip to content
This repository has been archived by the owner on Apr 12, 2024. It is now read-only.

Merge Synapse release v1.18.0 into 'dinsic' #53

Merged
merged 306 commits into from
Aug 18, 2020

Conversation

anoadragon453
Copy link
Member

@anoadragon453 anoadragon453 commented Aug 4, 2020

Sytest PR: matrix-org/sytest#924

Important things to note:

  • This change removes the code that forced private rooms to be encrypted, and instead replaces it with the mainline implementation of Add option to enable encryption by default for new rooms synapse#7639. Thus, for DINUM deployments, the new encryption_enabled_by_default_for_room_type config option should be set to "invite", meaning all newly created private (DM) rooms on the homeserver are encrypted by default.

General notes:

  • There were some tricky bits to this as we had a few releases that jumped back and forth in version numbers, for instance v1.15.2 was between v1.16.0rc1 and v1.16.0rc2 in the git history. Changelog should be in the right order though.
  • flake8 was updated to v3.8.3 from v3.7.9 to match mainline (which had some things that required v3.8.3).
  • Custom DINSIC code for synapse/handlers/profile.py still returns Deferred rather than Awaitables. I think we should convert these in a separate PRs. Leaving them unconverted here still works though.
  • Similar story as the above for proxy_lookup_3pid and proxy_bulk_lookup_3pid in synapse/handlers/identity.py. Also still functional, but should be converted, possibly during the merge into mainline.

clokep and others added 30 commits June 16, 2020 12:01
The aim here is to make it easier to reason about when streams are limited and when they're not, by moving the logic into the database functions themselves. This should mean we can kill of `db_query_to_update_function` function.
This requires a new config option to specify which media repo should be
responsible for running background jobs to e.g. clear out expired URL
preview caches.
* Always return an unread_count in get_unread_event_push_actions_by_room_for_user
* Don't always expect unread_count to be there so we don't take out sync entirely if something goes wrong
Older versions of `parameterized` package have no `parameterized_class` decorator. This decorator is used in tests.

Signed-off-by: Oleg Girko <[email protected]>
HTTP requires the response to contain a Content-Length header unless chunked encoding is being used.
Prometheus metrics endpoint did not set this, causing software such as prometheus-proxy to not be able to scrape synapse for metrics.

Signed-off-by: Christian Svensson <[email protected]>
State res v2 across large data sets can be very CPU intensive, and if
all the relevant events are in the cache the algorithm will run from
start to finish within a single reactor tick. This can result in
blocking the reactor tick for several seconds, which can have major
repercussions on other requests.

To fix this we simply add the occaisonal `sleep(0)` during iterations to
yield execution until the next reactor tick. The aim is to only do this
for large data sets so that we don't impact otherwise quick resolutions.=
- Remove the requirement for a specific version of Python
- Move dep comment to a separate line, Tox 3.7.0 like trailing ones

Signed-off-by: Dagfinn Ilmari Mannsåker <[email protected]>
* commit 'a973bcb8a':
  Add some tiny type annotations (#7870)
  Remove obsolete comment.
  Ensure that calls to `json.dumps` are compatible with the standard library json. (#7836)
  Avoid brand new rooms in `delete_old_current_state_events` (#7854)
  Allow accounts to be re-activated from the admin APIs. (#7847)
  Fix tests
  Fix typo
  Newsfile
  Use get_users_in_room rather than state handler in typing for speed
  Fix client reader sharding tests (#7853)
  Convert E2E key and room key handlers to async/await. (#7851)
  Return the proper 403 Forbidden error during errors with JWT logins. (#7844)
  remove `retry_on_integrity_error` wrapper for persist_events (#7848)
* commit 'de119063f': (31 commits)
  Convert room list handler to async/await. (#7912)
  Element CSS and logo in email templates (#7919)
  Lint the contrib/ directory in CI and linting scripts, add synctl to linting script (#7914)
  Remove unused code from synapse.logging.utils. (#7897)
  Fix a typo in the sample config. (#7890)
  Fix deprecation warning: import ABC from collections.abc (#7892)
  Change sample config's postgres user to synapse_user (#7889)
  Fix deprecation warning due to invalid escape sequences (#7895)
  Remove Ubuntu Eoan that is now EOL (#7888)
  Fix the trace function for async functions. (#7872)
  Add help for creating a user via docker (#7885)
  Switch to Debian:Slim from Alpine for the docker image (#7839)
  Stop using 'device_max_stream_id' (#7882)
  Fix TypeError in synapse.notifier (#7880)
  Add a default limit (of 100) to get/sync operations. (#7858)
  Change "unknown room ver" logging to warning. (#7881)
  Convert device handler to async/await (#7871)
  Convert synapse.app to async/await. (#7868)
  Convert _base, profile, and _receipts handlers to async/await (#7860)
  Add admin endpoint to get members in a room. (#7842)
  ...
* commit '15997618e':
  Clean up PreserveLoggingContext (#7877)
  fix an incorrect comment
* commit '05060e022':
  Track command processing as a background process (#7879)
* commit '931b02684':
  Remove an unused prometheus metric (#7878)
* commit 'b74919c72':
  Add debugging to sync response generation (#7929)
* commit '923c99502':
  Skip serializing /sync response if client has disconnected (#7927)
* commit 'a4cf94a3c':
  Update the dates for ACME v1 EOL
* commit 'cc9bb3dc3':
  Convert the message handler to async/await. (#7884)
* commit '13d77464c':
  Follow-up to admin API to re-activate accounts (#7908)
* commit '68cd93582':
  Convert the federation agent and related code to async/await. (#7874)
* commit 'ff22672fd':
  Reorder database docs to promote postgresql. (#7933)
* commit '4876af06d':
  Abort federation requests if the client disconnects early (#7930)
* commit 'f88c48f3b':
  1.18.0rc1
  Fix error reporting when using `opentracing.trace` (#7961)
  Fix typing replication not being handled on master (#7959)
  Remove hacky error handling for inlineDeferreds. (#7950)
  Convert tests/rest/admin/test_room.py to unix file endings (#7953)
  Support oEmbed for media previews. (#7920)
  Convert state resolution to async/await (#7942)
  Fix up types and comments that refer to Deferreds. (#7945)
  Do not convert async functions to Deferreds in the interactive_auth_handler (#7944)
  Convert more of the media code to async/await (#7873)
  Return an empty body for OPTIONS requests. (#7886)
  Downgrade warning on client disconnect to INFO (#7928)
  Convert presence handler helpers to async/await. (#7939)
  Update the auth providers to be async. (#7935)
  Put a cache on `/state_ids` (#7931)
* commit '7000a215e':
  1.18.0rc2
  Typing worker needs to handle stream update requests (#7967)
  Handle replication commands synchronously where possible (#7876)
  update changelog
* commit 'a9631b7b4':
  1.18.0
  Update worker docs with recent enhancements  (#7969)
* commit '320ef9885':
  Fix formatting of changelog and upgrade notes
  Add deprecation warnings
linting after the v1.18.0 mainline merge
@babolivier
Copy link
Contributor

babolivier commented Aug 4, 2020

This change removes the code that forced private rooms to be encrypted, and instead replaces it with the mainline implementation of matrix-org/synapse#7639. Thus, for DINUM deployments, the new encryption_enabled_by_default_for_room_type config option should be set to "private", meaning all newly created private (DM) rooms on the homeserver are encrypted by default.

Looking at the related Synapse PR, I assume you meant "invite" here? If so, could you fix the initial comment of this PR please?

@babolivier
Copy link
Contributor

Also can you open issues on this repo (or on the main dinsic one, whichever works best) about your two last points on asyncifying?

Copy link
Contributor

@babolivier babolivier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise lgtm

@anoadragon453
Copy link
Member Author

This change removes the code that forced private rooms to be encrypted, and instead replaces it with the mainline implementation of matrix-org/synapse#7639. Thus, for DINUM deployments, the new encryption_enabled_by_default_for_room_type config option should be set to "private", meaning all newly created private (DM) rooms on the homeserver are encrypted by default.

Looking at the related Synapse PR, I assume you meant "invite" here? If so, could you fix the initial comment of this PR please?

Oh, yes indeed! Great catch, thanks!

@anoadragon453
Copy link
Member Author

Also can you open issues on this repo (or on the main dinsic one, whichever works best) about your two last points on asyncifying?

#54 and #55 🙂

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.