Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Commit

Permalink
Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work
Browse files Browse the repository at this point in the history
  • Loading branch information
richvdh committed Sep 24, 2019
2 parents 366dc7d + 5077626 commit ed8b92f
Show file tree
Hide file tree
Showing 51 changed files with 840 additions and 231 deletions.
105 changes: 78 additions & 27 deletions UPGRADE.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,52 +2,78 @@ Upgrading Synapse
=================

Before upgrading check if any special steps are required to upgrade from the
what you currently have installed to current version of synapse. The extra
what you currently have installed to current version of Synapse. The extra
instructions that may be required are listed later in this document.

1. If synapse was installed in a virtualenv then activate that virtualenv before
upgrading. If synapse is installed in a virtualenv in ``~/synapse/env`` then
run:
* If Synapse was installed using `prebuilt packages
<INSTALL.md#prebuilt-packages>`_, you will need to follow the normal process
for upgrading those packages.

.. code:: bash
* If Synapse was installed from source, then:

source ~/synapse/env/bin/activate
2. If synapse was installed using pip then upgrade to the latest version by
running:
1. Activate the virtualenv before upgrading. For example, if Synapse is
installed in a virtualenv in ``~/synapse/env`` then run:

.. code:: bash
.. code:: bash
pip install --upgrade matrix-synapse[all]
source ~/synapse/env/bin/activate
# restart synapse
synctl restart
2. If Synapse was installed using pip then upgrade to the latest version by
running:

.. code:: bash
If synapse was installed using git then upgrade to the latest version by
running:
pip install --upgrade matrix-synapse
.. code:: bash
If Synapse was installed using git then upgrade to the latest version by
running:

# Pull the latest version of the master branch.
.. code:: bash
git pull
pip install --upgrade .
# Update synapse and its python dependencies.
pip install --upgrade .[all]
3. Restart Synapse:

# restart synapse
./synctl restart
.. code:: bash
./synctl restart
To check whether your update was successful, you can check the Server header
returned by the Client-Server API:
To check whether your update was successful, you can check the running server
version with:

.. code:: bash
# replace <host.name> with the hostname of your synapse homeserver.
# You may need to specify a port (eg, :8448) if your server is not
# configured on port 443.
curl -kv https://<host.name>/_matrix/client/versions 2>&1 | grep "Server:"
# you may need to replace 'localhost:8008' if synapse is not configured
# to listen on port 8008.
curl http://localhost:8008/_synapse/admin/v1/server_version
Rolling back to older versions
------------------------------

Rolling back to previous releases can be difficult, due to database schema
changes between releases. Where we have been able to test the rollback process,
this will be noted below.

In general, you will need to undo any changes made during the upgrade process,
for example:

* pip:

.. code:: bash
source env/bin/activate
# replace `1.3.0` accordingly:
pip install matrix-synapse==1.3.0
* Debian:

.. code:: bash
# replace `1.3.0` and `stretch` accordingly:
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
Upgrading to v1.4.0
===================
Expand Down Expand Up @@ -99,6 +125,31 @@ Synapse will expect these files to exist inside the configured template director
default templates, see `synapse/res/templates
<https://github.com/matrix-org/synapse/tree/master/synapse/res/templates>`_.

Rolling back to v1.3.1
----------------------

If you encounter problems with v1.4.0, it should be possible to roll back to
v1.3.1, subject to the following:

* The 'room statistics' engine was heavily reworked in this release (see
`#5971 <https://github.com/matrix-org/synapse/pull/5971>`_), including
significant changes to the database schema, which are not easily
reverted. This will cause the room statistics engine to stop updating when
you downgrade.

The room statistics are essentially unused in v1.3.1 (in future versions of
Synapse, they will be used to populate the room directory), so there should
be no loss of functionality. However, the statistics engine will write errors
to the logs, which can be avoided by setting the following in `homeserver.yaml`:

.. code:: yaml
stats:
enabled: false
Don't forget to re-enable it when you upgrade again, in preparation for its
use in the room directory!

Upgrading to v1.2.0
===================

Expand Down
2 changes: 1 addition & 1 deletion changelog.d/5893.misc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Drop some unused tables.
Stop populating some unused tables.
1 change: 1 addition & 0 deletions changelog.d/5972.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add m.require_identity_server flag to /version's unstable_features.
1 change: 1 addition & 0 deletions changelog.d/5974.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add m.id_access_token to unstable_features in /versions as per MSC2264.
2 changes: 1 addition & 1 deletion changelog.d/5980.feature
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Add POST /_matrix/client/r0/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.
Add POST /_matrix/client/unstable/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.
1 change: 1 addition & 0 deletions changelog.d/5992.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Give appropriate exit codes when synctl fails.
1 change: 1 addition & 0 deletions changelog.d/6000.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Apply the federation blacklist to requests to identity servers.
1 change: 1 addition & 0 deletions changelog.d/6028.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Replace `trust_identity_server_for_password_resets` config option with `account_threepid_delegates`.
1 change: 1 addition & 0 deletions changelog.d/6042.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Allow homeserver to handle or delegate email validation when adding an email to a user's account.
1 change: 1 addition & 0 deletions changelog.d/6043.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Implement new Client Server API endpoints `/account/3pid/add` and `/account/3pid/bind` as per [MSC2290](https://github.com/matrix-org/matrix-doc/pull/2290).
1 change: 1 addition & 0 deletions changelog.d/6044.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add an unstable feature flag for separate add/bind 3pid APIs.
2 changes: 2 additions & 0 deletions changelog.d/6047.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Stop populating some unused tables.

1 change: 1 addition & 0 deletions changelog.d/6049.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add some notes on rolling back to v1.3.1.
1 change: 1 addition & 0 deletions changelog.d/6050.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Update the upgrade notes.
1 change: 1 addition & 0 deletions changelog.d/6056.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Remove POST method from password reset submit_token endpoint until we implement submit_url functionality.
1 change: 1 addition & 0 deletions changelog.d/6058.docker
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Provide SYNAPSE_WORKER envvar to specify python module.
1 change: 1 addition & 0 deletions changelog.d/6059.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix logcontext spam on non-Linux platforms.
1 change: 1 addition & 0 deletions changelog.d/6062.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add POST /_matrix/client/unstable/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.
1 change: 1 addition & 0 deletions changelog.d/6063.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Ensure query parameters in email validation links are URL-encoded.
1 change: 1 addition & 0 deletions changelog.d/6067.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Remove `bind` parameter from Client Server POST `/account` endpoint as per [MSC2290](https://github.com/matrix-org/matrix-doc/pull/2290/).
1 change: 1 addition & 0 deletions changelog.d/6072.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add a 'failure_ts' column to the 'destinations' database table.
1 change: 1 addition & 0 deletions changelog.d/6073.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Return a clearer error message when a timeout occurs when attempting to contact an identity server.
1 change: 1 addition & 0 deletions changelog.d/6074.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Prevent password reset's submit_token endpoint from accepting trailing slashes.
1 change: 1 addition & 0 deletions changelog.d/6075.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Change mailer logging to reflect Synapse doesn't just do chat notifications by email now.
1 change: 1 addition & 0 deletions changelog.d/6078.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add `POST /add_threepid/msisdn/submit_token` endpoint for proxying submitToken on an account_threepid_handler.
1 change: 1 addition & 0 deletions changelog.d/6079.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add `submit_url` response parameter to `*/msisdn/requestToken` endpoints.
1 change: 1 addition & 0 deletions changelog.d/6082.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Return 403 on `/register/available` if registration has been disabled.
2 changes: 2 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,8 @@ The following environment variables are supported in run mode:
`/data`.
* `SYNAPSE_CONFIG_PATH`: path to the config file. Defaults to
`<SYNAPSE_CONFIG_DIR>/homeserver.yaml`.
* `SYNAPSE_WORKER`: module to execute, used when running synapse with workers.
Defaults to `synapse.app.homeserver`, which is suitable for non-worker mode.
* `UID`, `GID`: the user and group id to run Synapse as. Defaults to `991`, `991`.
* `TZ`: the [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) the container will run with. Defaults to `UTC`.

Expand Down
3 changes: 2 additions & 1 deletion docker/start.py
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ def main(args, environ):
mode = args[1] if len(args) > 1 else None
desired_uid = int(environ.get("UID", "991"))
desired_gid = int(environ.get("GID", "991"))
synapse_worker = environ.get("SYNAPSE_WORKER", "synapse.app.homeserver")
if (desired_uid == os.getuid()) and (desired_gid == os.getgid()):
ownership = None
else:
Expand Down Expand Up @@ -245,7 +246,7 @@ def main(args, environ):

log("Starting synapse with config file " + config_path)

args = ["python", "-m", "synapse.app.homeserver", "--config-path", config_path]
args = ["python", "-m", synapse_worker, "--config-path", config_path]
if ownership is not None:
args = ["su-exec", ownership] + args
os.execv("/sbin/su-exec", args)
Expand Down
19 changes: 18 additions & 1 deletion docs/sample_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,9 @@ pid_file: DATADIR/homeserver.pid
# blacklist IP address CIDR ranges. If this option is not specified, or
# specified with an empty list, no ip range blacklist will be enforced.
#
# As of Synapse v1.4.0 this option also affects any outbound requests to identity
# servers provided by user input.
#
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
# listed here, since they correspond to unroutable addresses.)
#
Expand Down Expand Up @@ -937,8 +940,10 @@ uploads_path: "DATADIR/uploads"
# by the Matrix Identity Service API specification:
# https://matrix.org/docs/spec/identity_service/latest
#
# If a delegate is specified, the config option public_baseurl must also be filled out.
#
account_threepid_delegates:
#email: https://example.com # Delegate email sending to matrix.org
#email: https://example.com # Delegate email sending to example.org
#msisdn: http://localhost:8090 # Delegate SMS sending to this local process

# Users who register on this homeserver will automatically be joined
Expand Down Expand Up @@ -1295,6 +1300,12 @@ password_config:
# #registration_template_html: registration.html
# #registration_template_text: registration.txt
#
# # Templates for validation emails sent by the homeserver when adding an email to
# # your user account
# #
# #add_threepid_template_html: add_threepid.html
# #add_threepid_template_text: add_threepid.txt
#
# # Templates for password reset success and failure pages that a user
# # will see after attempting to reset their password
# #
Expand All @@ -1306,6 +1317,12 @@ password_config:
# #
# #registration_template_success_html: registration_success.html
# #registration_template_failure_html: registration_failure.html
#
# # Templates for success and failure pages that a user will see after attempting
# # to add an email or phone to their account
# #
# #add_threepid_success_html: add_threepid_success.html
# #add_threepid_failure_html: add_threepid_failure.html


#password_providers:
Expand Down
36 changes: 36 additions & 0 deletions synapse/config/emailconfig.py
Original file line number Diff line number Diff line change
Expand Up @@ -169,12 +169,22 @@ def read_config(self, config, **kwargs):
self.email_registration_template_text = email_config.get(
"registration_template_text", "registration.txt"
)
self.email_add_threepid_template_html = email_config.get(
"add_threepid_template_html", "add_threepid.html"
)
self.email_add_threepid_template_text = email_config.get(
"add_threepid_template_text", "add_threepid.txt"
)

self.email_password_reset_template_failure_html = email_config.get(
"password_reset_template_failure_html", "password_reset_failure.html"
)
self.email_registration_template_failure_html = email_config.get(
"registration_template_failure_html", "registration_failure.html"
)
self.email_add_threepid_template_failure_html = email_config.get(
"add_threepid_template_failure_html", "add_threepid_failure.html"
)

# These templates do not support any placeholder variables, so we
# will read them from disk once during setup
Expand All @@ -184,16 +194,24 @@ def read_config(self, config, **kwargs):
email_registration_template_success_html = email_config.get(
"registration_template_success_html", "registration_success.html"
)
email_add_threepid_template_success_html = email_config.get(
"add_threepid_template_success_html", "add_threepid_success.html"
)

# Check templates exist
for f in [
self.email_password_reset_template_html,
self.email_password_reset_template_text,
self.email_registration_template_html,
self.email_registration_template_text,
self.email_add_threepid_template_html,
self.email_add_threepid_template_text,
self.email_password_reset_template_failure_html,
self.email_registration_template_failure_html,
self.email_add_threepid_template_failure_html,
email_password_reset_template_success_html,
email_registration_template_success_html,
email_add_threepid_template_success_html,
]:
p = os.path.join(self.email_template_dir, f)
if not os.path.isfile(p):
Expand All @@ -212,6 +230,12 @@ def read_config(self, config, **kwargs):
self.email_registration_template_success_html_content = self.read_file(
filepath, "email.registration_template_success_html"
)
filepath = os.path.join(
self.email_template_dir, email_add_threepid_template_success_html
)
self.email_add_threepid_template_success_html_content = self.read_file(
filepath, "email.add_threepid_template_success_html"
)

if self.email_enable_notifs:
required = [
Expand Down Expand Up @@ -328,6 +352,12 @@ def generate_config_section(self, config_dir_path, server_name, **kwargs):
# #registration_template_html: registration.html
# #registration_template_text: registration.txt
#
# # Templates for validation emails sent by the homeserver when adding an email to
# # your user account
# #
# #add_threepid_template_html: add_threepid.html
# #add_threepid_template_text: add_threepid.txt
#
# # Templates for password reset success and failure pages that a user
# # will see after attempting to reset their password
# #
Expand All @@ -339,6 +369,12 @@ def generate_config_section(self, config_dir_path, server_name, **kwargs):
# #
# #registration_template_success_html: registration_success.html
# #registration_template_failure_html: registration_failure.html
#
# # Templates for success and failure pages that a user will see after attempting
# # to add an email or phone to their account
# #
# #add_threepid_success_html: add_threepid_success.html
# #add_threepid_failure_html: add_threepid_failure.html
"""


Expand Down
4 changes: 3 additions & 1 deletion synapse/config/registration.py
Original file line number Diff line number Diff line change
Expand Up @@ -293,8 +293,10 @@ def generate_config_section(self, generate_secrets=False, **kwargs):
# by the Matrix Identity Service API specification:
# https://matrix.org/docs/spec/identity_service/latest
#
# If a delegate is specified, the config option public_baseurl must also be filled out.
#
account_threepid_delegates:
#email: https://example.com # Delegate email sending to matrix.org
#email: https://example.com # Delegate email sending to example.org
#msisdn: http://localhost:8090 # Delegate SMS sending to this local process
# Users who register on this homeserver will automatically be joined
Expand Down
2 changes: 1 addition & 1 deletion synapse/config/repository.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
# Copyright 2014, 2015 matrix.org
# Copyright 2014, 2015 OpenMarket Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
3 changes: 3 additions & 0 deletions synapse/config/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -545,6 +545,9 @@ def generate_config_section(
# blacklist IP address CIDR ranges. If this option is not specified, or
# specified with an empty list, no ip range blacklist will be enforced.
#
# As of Synapse v1.4.0 this option also affects any outbound requests to identity
# servers provided by user input.
#
# (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
# listed here, since they correspond to unroutable addresses.)
#
Expand Down
Loading

0 comments on commit ed8b92f

Please sign in to comment.