Matrix v2 #439

Josue-T · 2024-02-26T21:05:20Z

Matrix V2 Implementation

This is more a tracking page and discussion about Matrix v2 implementation than a pull request.

The target here is to implement all component of matrix V2 so:

Matrix sliding sync proxy: https://github.com/matrix-org/sliding-sync
Element call: https://github.com/vector-im/element-call
Matrix authentication service: https://github.com/matrix-org/matrix-authentication-service

The idea here is to provide for user how want a beta version of synapse which will implement theses new feature. Note that this branch will be updated in same time than the testing so user can easily install the branch and get the update as same as than for testing.

frequently asked questions

Why not merging this and making this available to more people ?

Because some component are still really experimental on matrix side. On sliding proxy projet side we have currently theses issues:

This project are built to run only with new libc version which is not available on debian bullseye. A chroot was made to bypass this issue but it's not a clean way to implement this. The best solution about this is to wait for debian bookworm on yunohost.
The build of this project are available on for arm64 and amd64 architecture so it's really restricted.

How can I use this branch and experiment theses feature ?

As same as for testing you can install the app with this command:

sudo yunohost app install https://github.com/YunoHost-Apps/synapse_ynh/tree/matrix_v2

And you can also update the app with this command:

sudo yunohost app upgrade synapse -u https://github.com/YunoHost-Apps/synapse_ynh/tree/matrix_v2

Can I install this without breaking my instance ?

Short answer: the stability of this branch will be same than testing

Long answer: As same as for testing we try to avoid any issue with these branch and try to keep it stable. But it still be a community project and we can't guarantee the stability of the code. Note also that the stability depends of the upstream code.

When will this be merged into master ?

It depends of the development on package side but also of the upstream on matrix/element side so for now it's difficult to say. In all case the idea is not to merge before all component are ready and tested on the package.

Why adding element-call into this package as there are already a package with this

The idea here is to provide a package with all main feature of matrix configured directly into synapse so there is no more setup to do.

Can I switch back to master or testing branch after doing an upgrade from this branch ?

Nothing was designed for this, so there are a huge risk to break something or that something don't work any-more if you do this. But note that this branch in long term will be merged into testing than master so you won't be blocked if you move to this branch.

TODO

Finish Sliding sync proxy Sliding proxy #440
Implement Element call
Implement Matrix authentication service
Drop Debian bullseye support and clean-up sliding sync proxy implementation (when Yunohost will support officially debian bookworm).

lapineige · 2024-03-08T11:31:21Z

I want to congratulate @Josue-T for the huge recent work on both these improvements and work on packaging v2 #426 and various fixes 🎉 👏
Thanks also to all people contributing to this :)

Thatoo · 2024-03-08T11:47:38Z

Thank you so much @Josue-T for your incredible amount of work these last days.

centralscrutinizer21 · 2024-03-08T14:18:08Z

Adding my thanks to @Josue-T as well. I am going to test this branch and face the wrath of the 3 friends I convinced to get on my matrix instance 😄

centralscrutinizer21 · 2024-03-08T18:31:19Z

Installation went fine, upgrading not so much.
I still can't get to use Element x maybe something related to this warning?
https://paste.yunohost.org/agadaqileg
Also, do I need to open ports 8008 8009 ?

Josue-T · 2024-03-08T19:09:11Z

Normally all port are already open. But well what happen wrong. The upgrade process or the service was not working after the upgrade ?

centralscrutinizer21 · 2024-03-08T19:11:37Z

The service is running fine. Of course the two bridges (whatsapp & telegram) are not working anymore.
As I said I tried to login with Element X . Ii recognise the server as valid, but then it cannot reach it.
From the logs it seems that there could be some syntax erro maybe?

Josue-T · 2024-03-08T20:37:36Z

Do you have some log of theses bridges ?

centralscrutinizer21 · 2024-03-09T06:59:54Z

Sorry, my bad. I had a second application on the same domain, and that screwed up the installation.
Now it's working. Where do I find the procedure to move synapse to a different domain keeping the same user handle?

Josue-T · 2024-03-09T09:15:21Z

I created an issue about this here: #443

centralscrutinizer21 · 2024-03-09T09:36:43Z

Yes, thank you I found it and succesfully moved synapse to a different domain.
I am just not very proficient with the syntax of /etc/ssowat/conf.json.persistent can you show me how it should look?

Josue-T · 2024-03-09T09:50:55Z

About change url and domain constraints issue please post your message on #443

nathanael-h · 2024-03-19T08:08:10Z

Hello, I successfully upgraded to this branch. The Sliding Sync proxy's service is enabled and running well. As far as I see Synapse and the slinding sync proxy are installed on the same domain. Unfortunately I can't make Element X on Android connect to my server.

My matrix server is running at matrix.domain.fr and my main domain is domain.fr the .well-known are advertised here :

curl https://domain.fr/.well-known/matrix/client 
{
        "m.homeserver": { "base_url": "https://matrix.domain.fr" },
        "org.matrix.msc3575.proxy": { "url": "https://matrix.domain.fr"},
        "im.vector.riot.jitsi": {"preferredDomain": "jitsi.riot.im"},
        "im.vector.riot.e2ee": {"default": true }
    }%

When I try to connect Element X to domain.fr I have a "Nous n'avons pas pu atteindre ce serveur d'accueil. Vérifiez que vous avez correctement saisi l'URL du serveur d'accueil. Si l'URL est correcte contactez votre administrateur... etc."
I see this entries in logs nignx :

/var/log/nginx/domain.fr-access.log:90.66.163.140        - - [18/Mar/2024:18:17:03 +0100] "GET /.well-known/matrix/client HTTP/2.0" 200 273 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [18/Mar/2024:18:17:03 +0100] "GET /_matrix/client/versions HTTP/2.0" 200 929 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [18/Mar/2024:18:17:03 +0100] "GET /_matrix/client/v3/login HTTP/2.0" 200 170 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"

I'd like to help you with this pull request and adding the sliding sync proxy, by the way thank you for all you've already done.

Josue-T · 2024-03-19T08:19:01Z

Hello, I think you should define the server as matrix.domain.fr and not domain.fr ? Normally this should works.

nathanael-h · 2024-03-19T09:15:10Z

I tried to login with matrix.domain.fr URL in Element X, but it says "Error this homeserver does not support sliding sync".

In nginx logs I see this :

root@domain:/etc/nginx/conf.d# grep "Element X" /var/log/nginx/*.log
/var/log/nginx/domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:05:55 +0100] "GET /yunohost/sso/?r=aHR0cHM6Ly9tYXRyaXguaXNpZG9ydXxxxxxxxxxxx4vbWF0cml4L2NsaWVudA== HTTP/2.0" 200 3077 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:05 +0100] "GET /yunohost/sso/?r=aHR0cHM6Ly9tYXRyaXguaXNpZG9ydXxxxxxxxxxxx4vbWF0cml4L2NsaWVudA== HTTP/2.0" 200 3077 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:08 +0100] "GET /yunohost/sso/?r=aHR0cHM6Ly9tYXRyaXguaXNpZG9ydXxxxxxxxxxxx4vbWF0cml4L2NsaWVudA== HTTP/2.0" 200 3077 "https://matrix.domain.fr/.well-known/matrix/client" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"


/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:05:55 +0100] "GET /.well-known/matrix/client HTTP/2.0" 302 138 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:05 +0100] "GET /.well-known/matrix/client HTTP/2.0" 302 138 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:05 +0100] "GET /.well-known/matrix/client HTTP/2.0" 302 138 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:07 +0100] "GET /.well-known/matrix/client HTTP/2.0" 302 138 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:08 +0100] "GET /_matrix/client/versions HTTP/2.0" 200 929 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:08 +0100] "GET /_matrix/client/versions HTTP/2.0" 200 929 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:08 +0100] "GET /_matrix/client/v3/login HTTP/2.0" 200 170 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
root@domain:/etc/nginx/conf.d#

And if I curl https://matrix.domain.fr/.well-known/matrix/client I have a 302 to SSO ; indeed the .well-know/matrix/client is on the main yunohost domain, i.e. domain.fr... It looks like the Element X app is first calling the .well-know/matrix/client path on the URL I set as the homeserver URL, then nginx says 302 go to SSO, then it fails.

I see no entry in sliding-sync service's journal.

Josue-T · 2024-03-19T22:36:35Z

Hello,

Did some test, the issue is linked that Element X don't support CAS and if password authentication is disabled it don't works. By d42f646 it should work now.

nathanael-h · 2024-03-20T11:20:40Z

I confirm it works: I could log in Element X, indeed with username and password!

…orrectly

yunohost-bot · 2024-09-05T06:29:28Z

Alrighty!

Josue-T · 2024-09-18T06:39:17Z

!testme

yunohost-bot · 2024-09-18T06:39:19Z

🐛

yunohost-bot · 2024-09-18T06:39:19Z

🌻

bnjbvr · 2024-09-30T21:14:01Z

Great work here! FYI, the sliding sync proxy is no longer required, as Synapse implements (most) of sliding sync. There are still a few missing features for sliding sync in synapse (e.g. it doesn't return pinned events, and a few other things), but most of it should work with an up-to-date ElementX.

Josue-T · 2024-10-02T06:14:05Z

!testme

yunohost-bot · 2024-10-02T06:14:09Z

Meow 🐈

Josue-T · 2024-10-02T06:15:06Z

Great work here! FYI, the sliding sync proxy is no longer required, as Synapse implements (most) of sliding sync. There are still a few missing features for sliding sync in synapse (e.g. it doesn't return pinned events, and a few other things), but most of it should work with an up-to-date ElementX.

Agreed, I planed to remove the proxy and migrate to the native implementation into synapse. But for now I didn't have time to do it.

Josue-T · 2024-10-16T06:14:37Z

!testme

yunohost-bot · 2024-10-16T06:14:44Z

✌️

yekibud · 2024-10-27T18:27:53Z

Just to confirm, is this still the feature that needs to be completed for full element x support? I just updated synapse to the latest version and I'm able to use element x, but it says no push distributors are configured, and I'm not receiving push notifications.

Josue-T · 2024-10-27T19:36:14Z

Well this branch is manly to add all features related to matrix v2 and yes ElementX is matrix v2 client. But about your notification issue I'm not sure that it's related to this branch.

yekibud · 2024-10-27T19:37:56Z

Oh, okay. Well if element x is supposed to be working with push notifications, should I create a separate issue tracker for this then?

Josue-T · 2024-10-28T17:31:42Z

Well, push notification seem to me not really related matrixv2, but is more general thing which is for now not implemented at all. So yes please create a separate issue about this.

yekibud · 2024-10-28T17:38:08Z

So, turns out this only affects Android users who have disabled Google Play Services (like me) and can be easily solved by installing NTFY, which is available in the YNH catalog. So, maybe worth a small README PR, if that makes sense. I'll go ahead and submit that, if so.

Matrix v2 merge

Josue-T · 2024-11-27T07:32:19Z

!testme

yunohost-bot · 2024-11-27T07:32:24Z

Living in the past, are we? 🐮👁️

yunohost-bot · 2024-11-27T07:32:24Z

Test it, book it, worm it, technologic. 🤖 🎶

This was linked to issues Feb 26, 2024

OpenID Connect #411

Open

sliding sync proxy? #395

Closed

Josue-T mentioned this pull request Mar 8, 2024

sliding sync proxy? #395

Closed

Josue-T added 3 commits March 8, 2024 12:09

First implementation of matrix sliding proxy

3a1d10d

Make it less warning on remove

61add5e

fix test

8b4d2e1

Josue-T force-pushed the matrix_v2 branch from 3069137 to 8b4d2e1 Compare March 8, 2024 11:10

Fix upgrade with old 3pids_email values

632ecc4

Josue-T added 2 commits March 9, 2024 11:20

Merge branch 'testing' into matrix_v2

1b522bd

Merge branch 'testing' into matrix_v2

1efee60

Josue-T and others added 2 commits March 22, 2024 12:13

Force enable password authentication as some client need it to work c…

507ac9d

…orrectly

Auto-update README

72b21bf

Josue-T force-pushed the matrix_v2 branch from b207ea7 to af846cd Compare March 22, 2024 11:15

Merge branch 'testing' into matrix_v2

24fb003

Merge branch 'testing' into matrix_v2

823bbcc

Josue-T force-pushed the matrix_v2 branch from 96bad38 to 823bbcc Compare September 18, 2024 06:41

Merge branch 'testing' into matrix_v2

607fff3

Merge branch 'testing' into matrix_v2

70b585b

Josue-T added 6 commits October 28, 2024 23:52

Merge branch 'testing' into matrix_v2

0c17daa

Remove sliding sync proxy and use native synapse sliding sync

5e7b559

Remove /client as it not anymore necessary

75d3384

Merge pull request #494 from YunoHost-Apps/matrix_v2_merge

6ae1fc9

Matrix v2 merge

Merge branch 'testing' into matrix_v2

b5e25d0

Merge branch 'testing' into matrix_v2

418aba8

Josue-T added 2 commits November 30, 2024 21:05

Merge branch 'testing' into matrix_v2

52ce772

Merge branch 'testing' into matrix_v2

6e09114

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Matrix v2 #439

Matrix v2 #439

Josue-T commented Feb 26, 2024 •

edited

Loading

lapineige commented Mar 8, 2024

Thatoo commented Mar 8, 2024

centralscrutinizer21 commented Mar 8, 2024

centralscrutinizer21 commented Mar 8, 2024

Josue-T commented Mar 8, 2024 •

edited

Loading

centralscrutinizer21 commented Mar 8, 2024 •

edited

Loading

Josue-T commented Mar 8, 2024 •

edited

Loading

centralscrutinizer21 commented Mar 9, 2024

Josue-T commented Mar 9, 2024

centralscrutinizer21 commented Mar 9, 2024

Josue-T commented Mar 9, 2024 •

edited

Loading

nathanael-h commented Mar 19, 2024

Josue-T commented Mar 19, 2024

nathanael-h commented Mar 19, 2024

Josue-T commented Mar 19, 2024

nathanael-h commented Mar 20, 2024

yunohost-bot commented Sep 5, 2024

Josue-T commented Sep 18, 2024

yunohost-bot commented Sep 18, 2024

yunohost-bot commented Sep 18, 2024

bnjbvr commented Sep 30, 2024

Josue-T commented Oct 2, 2024

yunohost-bot commented Oct 2, 2024

Josue-T commented Oct 2, 2024

Josue-T commented Oct 16, 2024

yunohost-bot commented Oct 16, 2024

yekibud commented Oct 27, 2024

Josue-T commented Oct 27, 2024

yekibud commented Oct 27, 2024

Josue-T commented Oct 28, 2024

yekibud commented Oct 28, 2024

Josue-T commented Nov 27, 2024

yunohost-bot commented Nov 27, 2024

yunohost-bot commented Nov 27, 2024

Matrix v2 #439

Are you sure you want to change the base?

Matrix v2 #439

Conversation

Josue-T commented Feb 26, 2024 • edited Loading

Matrix V2 Implementation

frequently asked questions

Why not merging this and making this available to more people ?

How can I use this branch and experiment theses feature ?

Can I install this without breaking my instance ?

When will this be merged into master ?

Why adding element-call into this package as there are already a package with this

Can I switch back to master or testing branch after doing an upgrade from this branch ?

TODO

lapineige commented Mar 8, 2024

Thatoo commented Mar 8, 2024

centralscrutinizer21 commented Mar 8, 2024

centralscrutinizer21 commented Mar 8, 2024

Josue-T commented Mar 8, 2024 • edited Loading

centralscrutinizer21 commented Mar 8, 2024 • edited Loading

Josue-T commented Mar 8, 2024 • edited Loading

centralscrutinizer21 commented Mar 9, 2024

Josue-T commented Mar 9, 2024

centralscrutinizer21 commented Mar 9, 2024

Josue-T commented Mar 9, 2024 • edited Loading

nathanael-h commented Mar 19, 2024

Josue-T commented Mar 19, 2024

nathanael-h commented Mar 19, 2024

Josue-T commented Mar 19, 2024

nathanael-h commented Mar 20, 2024

yunohost-bot commented Sep 5, 2024

Josue-T commented Sep 18, 2024

yunohost-bot commented Sep 18, 2024

yunohost-bot commented Sep 18, 2024

bnjbvr commented Sep 30, 2024

Josue-T commented Oct 2, 2024

yunohost-bot commented Oct 2, 2024

Josue-T commented Oct 2, 2024

Josue-T commented Oct 16, 2024

yunohost-bot commented Oct 16, 2024

yekibud commented Oct 27, 2024

Josue-T commented Oct 27, 2024

yekibud commented Oct 27, 2024

Josue-T commented Oct 28, 2024

yekibud commented Oct 28, 2024

Josue-T commented Nov 27, 2024

yunohost-bot commented Nov 27, 2024

yunohost-bot commented Nov 27, 2024

Josue-T commented Feb 26, 2024 •

edited

Loading

Josue-T commented Mar 8, 2024 •

edited

Loading

centralscrutinizer21 commented Mar 8, 2024 •

edited

Loading

Josue-T commented Mar 8, 2024 •

edited

Loading

Josue-T commented Mar 9, 2024 •

edited

Loading