Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Recommend poetry in docs #12475

Merged
merged 10 commits into from
Apr 20, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 17 additions & 14 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -293,39 +293,42 @@ directory of your choice::
git clone https://github.com/matrix-org/synapse.git
cd synapse

Synapse has a number of external dependencies, that are easiest
to install using pip and a virtualenv::
Synapse has a number of external dependencies. We maintain a fixed development
environment using [poetry](https://python-poetry.org/). First, install poetry. We recommend

python3 -m venv ./env
source ./env/bin/activate
pip install -e ".[all,dev]"
pip install --user pipx
pipx install poetry

This will run a process of downloading and installing all the needed
dependencies into a virtual env. If any dependencies fail to install,
try installing the failing modules individually::
as described `here <https://python-poetry.org/docs/#installing-with-pipx>`_.
(See `poetry's installation docs <https://python-poetry.org/docs/#installation>`
for other installation methods.) Then ask poetry to create a virtual environment
from the project and install Synapse's dependencies::

poetry install --extras "all test"
DMRobertson marked this conversation as resolved.
Show resolved Hide resolved

pip install -e "module-name"
This will run a process of downloading and installing all the needed
dependencies into a virtual env.

We recommend using the demo which starts 3 federated instances running on ports `8080` - `8082`

./demo/start.sh
poetry run ./demo/start.sh

(to stop, you can use `./demo/stop.sh`)
(to stop, you can use `poetry run ./demo/stop.sh`)

See the [demo documentation](https://matrix-org.github.io/synapse/develop/development/demo.html)
for more information.

If you just want to start a single instance of the app and run it directly::

# Create the homeserver.yaml config once
python -m synapse.app.homeserver \
poetry run synapse_homeserver \
--server-name my.domain.name \
--config-path homeserver.yaml \
--generate-config \
--report-stats=[yes|no]

# Start the app
python -m synapse.app.homeserver --config-path homeserver.yaml
poetry run synapse_homeserver --config-path homeserver.yaml


Running the unit tests
Expand All @@ -334,7 +337,7 @@ Running the unit tests
After getting up and running, you may wish to run Synapse's unit tests to
check that everything is installed correctly::

trial tests
poetry run trial tests

This should end with a 'PASSED' result (note that exact numbers will
differ)::
Expand Down
1 change: 1 addition & 0 deletions changelog.d/12475.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Strongly recommend `poetry` for development.
46 changes: 25 additions & 21 deletions docs/development/contributing_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,19 +48,28 @@ can find many good git tutorials on the web.

# 4. Install the dependencies

Once you have installed Python 3 and added the source, please open a terminal and
setup a *virtualenv*, as follows:
Synapse uses the [poetry](https://python-poetry.org/) project to manage its dependencies
and development environment. Once you have installed Python 3 and added the
source, you should install `poetry`.
Of their installation methods, we recommend
[installing `poetry` using `pipx`](https://python-poetry.org/docs/#installing-with-pipx),

```shell
pip install --user pipx
pipx install poetry
```

but see poetry's [installation instructions](https://python-poetry.org/docs/#installation)
for other installation methods.

Next, open a terminal and install dependencies as follows:

```sh
cd path/where/you/have/cloned/the/repository
python3 -m venv ./env
source ./env/bin/activate
pip install wheel
pip install -e ".[all,dev]"
pip install tox
poetry install --extras all
```

This will install the developer dependencies for the project.
This will install the runtime and developer dependencies for the project.


# 5. Get in touch.
Expand Down Expand Up @@ -117,11 +126,10 @@ The linters look at your code and do two things:
- ensure that your code follows the coding style adopted by the project;
- catch a number of errors in your code.

The linter takes no time at all to run as soon as you've [downloaded the dependencies into your python virtual environment](#4-install-the-dependencies).
The linter takes no time at all to run as soon as you've [downloaded the dependencies](#4-install-the-dependencies).

```sh
source ./env/bin/activate
./scripts-dev/lint.sh
poetry run ./scripts-dev/lint.sh
```

Note that this script *will modify your files* to fix styling errors.
Expand All @@ -131,15 +139,13 @@ If you wish to restrict the linters to only the files changed since the last com
(much faster!), you can instead run:

```sh
source ./env/bin/activate
./scripts-dev/lint.sh -d
poetry run ./scripts-dev/lint.sh -d
```

Or if you know exactly which files you wish to lint, you can instead run:

```sh
source ./env/bin/activate
./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder
poetry run ./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder
```

## Run the unit tests (Twisted trial).
Expand All @@ -148,16 +154,14 @@ The unit tests run parts of Synapse, including your changes, to see if anything
was broken. They are slower than the linters but will typically catch more errors.

```sh
source ./env/bin/activate
trial tests
poetry run trial tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does poetry not have a concept of activating the virtualenv?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The closest thing is poetry shell, but that spawns a new shell rather than modifying the environment in your existing shell.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I say "but" because it means it's not a 1:1 replacement)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you can still just manually activate the virtualenv in the same way, if you wanted to.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we include a cross-reference to the direnv bits somewhere? (Or mention poetry shell somewhere?)

```

If you wish to only run *some* unit tests, you may specify
another module instead of `tests` - or a test class or a method:

```sh
source ./env/bin/activate
trial tests.rest.admin.test_room tests.handlers.test_admin.ExfiltrateData.test_invite
poetry run trial tests.rest.admin.test_room tests.handlers.test_admin.ExfiltrateData.test_invite
```

If your tests fail, you may wish to look at the logs (the default log level is `ERROR`):
Expand All @@ -169,7 +173,7 @@ less _trial_temp/test.log
To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`:

```sh
SYNAPSE_TEST_LOG_LEVEL=DEBUG trial tests
SYNAPSE_TEST_LOG_LEVEL=DEBUG poetry run trial tests
```

By default, tests will use an in-memory SQLite database for test data. For additional
Expand All @@ -180,7 +184,7 @@ database state to be stored in a file named `test.db` under the trial process'
working directory. Typically, this ends up being `_trial_temp/test.db`. For example:

```sh
SYNAPSE_TEST_PERSIST_SQLITE_DB=1 trial tests
SYNAPSE_TEST_PERSIST_SQLITE_DB=1 poetry run trial tests
```

The database file can then be inspected with:
Expand Down
Loading