Skip to content

Commit

Permalink
⚠️ MAJOR UPDATE: Feature: add matrix sliding sync and matrix authenti…
Browse files Browse the repository at this point in the history 
…cation service subcharts (#560)

* add sliding sync subchart

* update helm deps

* add helm dep for matrix-sliding-sync-chart to ci

* update .well-known/matrix/client json to include org.matrix.msc3575.proxy

* call the chart slidingSync

* update element configmap to always have sliding sync bit in it

* try using matrix base url

* add a few more useful options to the helm chart for security sake and set web client location if element is enabled

* fix go template trimming

* add more info in comment about synapse

* add docs on how to return the correct json for sliding-sync at .well-known/matrix/client endpoint

* add details about returning correct .well-known url

* update chart.lock

* update matrix sliding sync to 0.2.3

* udpate docs

* add new matrix authentication service helm sub chart

* disable integrations by default

* add more yq templating for matrix.experiemental_features.msc3861

* add admin token secret key to values, deployment, and docs

* add authentication-service chart to repos to update in the beginning

* fix typo in experimental word

* we now template out the experimental_features section of matrix

* fix yq replace for experimental features

* update sliding sync and authentication charts

* update example ingress annotation to include the matrix auth service

* bump element version and add more docs

* update helm docs

* update readme to explain new features

* only add the org.matrix.msc3575.proxy section if sliding is enabled

* try to fix element-x links

* clean up grammar

* comment out some values for oidc by default and explain docs a bit more

* change where we grab the msc secret from

* update matrix authentication service to 0.1.3

* switch back to matrix-authentication-service

* attempt to use matrix-authetincation-service aliased and update to 0.1.5

* try calling the subchart mas instead

* update matrix auth service chart to 0.2.0

* update matrix authentication service to 0.2.1

* properly alias slidingSync

* ignore tesitng values

* update to mas v0.3.0

* update to mas v0.4.0

* update to matrix-authentication-service-chart 0.5.0

* update to mas 0.5.1

* update mas to 0.5.2

* update to mas 0.5.3

* update sliding sync to 0.4.0

* disable postgres by default

* add ingress for sliding sync and realias slidingSync to syncv3

* update to mas to 0.5.4

* update to mas 0.5.5

* bump deps and chart version

* sliding sync to 0.4.1

* update to mas 0.5.7

* update to mas 0.6.0

* remove default public base for mas

* update to mas 0.6.1

* update matrix auth password field templating

* update where we put msc3861 secretKeys and existingSecret

* fix synapse yq for msc3861 sensitive value replacement

* don't enable brand or human name for mas by default

* update to mas 0.6.2

* use well known from matrix instead of element directly

* add password_config.enabled option

* fix duplicate home server key

* don't template passwrod_config by default

* change account management url to be non by default

* use a proper ingress path for MAS if it and synapse are both enabled and ingress is also enabled

* fix mas.enabled and calling subchart

* change synapse ingress to be prefix

* update how we do ingress for synapse

* update mas to 0.7.0

* update mas to 0.7.1 and attempt to fix synapse ingress

* fix new synapse ingress

* update synapse ingress templating

* fix tls for synapse ingress

* fix ingress for synapse to properly template out tls if using new method

* quote path for synapse ingress

* updating element config.json

* fix base url for element config

* add matrix.hostname back as a default value since we removed the synapse.ingress.host

* add matrix.extra_well_known_client_content section for values.yaml

* update all sorts of docs and clean up ingress a bit more

* more docs and default values

* template out more matrix home server values

* update remaining old synapse links and bump this a major release version

* add matrix.hostname value for ci tests

---------

Co-authored-by: cloudymax <[email protected]>
  • Loading branch information
@jessebot @cloudymax
jessebot and cloudymax authored May 28, 2024
1 parent 1703212 commit d1713d5
Show file tree
Hide file tree
Showing 16 changed files with 1,524 additions and 185 deletions.
6 changes: 5 additions & 1 deletion .github/workflows/ci-helm-lint-test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ jobs:
run: |
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add coturn https://small-hack.github.io/coturn-chart
helm repo add matrix-sliding-sync https://small-hack.github.io/matrix-sliding-sync-chart
helm repo add matrix-authentication-service https://small-hack.github.io/matrix-authentication-service-chart
- name: Set up chart-testing
uses: helm/[email protected]
Expand All @@ -50,4 +52,6 @@ jobs:
- name: Run chart-testing (install)
id: install
if: steps.list-changed.outputs.changed == 'true'
run: ct install --target-branch ${{ github.event.repository.default_branch }} --helm-extra-set-args '--set=postgresql.volumePermissions.enabled=false --set=postgresql.primary.networkPolicy.enabled=false'
run: |
ct install --target-branch ${{ github.event.repository.default_branch }} \
--helm-extra-set-args '--set=postgresql.volumePermissions.enabled=false --set=postgresql.primary.networkPolicy.enabled=false --set=matrix.hostname=matrix-test.com'
241 changes: 226 additions & 15 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,20 +22,33 @@ helm install my-release-name matrix/matrix --values values.yaml

## Current Features ✨

- Latest version of [Synapse](https://github.com/matrix-org/synapse) (the official homeserver edition of matrix)
- Ingress definitions for federated Synapse (Matrix homeserver) and Element (frontend and CMS for matrix)
- Use existing Kubernetes secrets and existing Persistent Volume Claims
- Latest version of [Synapse](https://github.com/element-hq/synapse) (the official matrix homeserver)
- Ingress definitions for federated Synapse (Matrix homeserver) and Element (client for matrix)

### Optional Features

- Use (existing) Kubernetes Secrets for confidential data, such as passwords
- Use OIDC configs for SSO (see synapse [docs](https://github.com/matrix-org/synapse/blob/747416e94cd8f137b9173c132f7c44ea1c59534d/docs/openid.md) for more info)
- Latest version of [Element](https://element.io/)
- Use OIDC configs for SSO either directly via synapse (see [docs](https://github.com/element-hq/synapse/blob/develop/docs/openid.md) for more info) or via MAS
- Use MAS ([matrix-org/matrix-authentication-service](https://github.com/matrix-org/matrix-authentication-service)) via [matrix-authentication-service-chart](https://github.com/small-hack/matrix-authentication-service-chart) as a sub chart for using [element-x] which recommends for OIDC auth
- Latest version of the [Element web app](https://element.io/) to provide a web interface for chat (you can disable this and still use element apps)
- [Coturn TURN server subchart](https://github.com/small-hack/coturn-chart) for VoIP calls
- Use [s3 to store stuff](https://github.com/matrix-org/synapse-s3-storage-provider/tree/main)
- Use an existing Kubernetes Secret for an external mail server for email notifications
- Use s3 to store media using [element-hq/synapse-s3-storage-provider](https://github.com/matrix-org/synapse-s3-storage-provider/tree/main)
- Use [matrix-sliding-sync-chart](https://github.com/small-hack/matrix-sliding-sync-chart) as a sub chart for using [element-x] which requires [matrix-org/sliding-sync](https://github.com/matrix-org/sliding-sync)
- Use existing Kubernetes secrets and existing Persistent Volume Claims

### ⚠️ Optional Features (Untested Since Fork)

These features still need to be tested, but are technically baked into the chart from the fork:

- Use of lightweight Exim relay
- [Half-Shot/matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord) Discord bridge
- [matrix-org/matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) IRC bridge
- [tulir/mautrix-whatsapp](https://github.com/tulir/mautrix-whatsapp) WhatsApp bridge


#### Databases
# Notes

## Databases

You must select one of the following options:

Expand All @@ -44,19 +57,217 @@ You must select one of the following options:

Note: you cannot enable both `externalDatabase` and `postgresql`. You must select _one_.

### ⚠️ Optional Features (Untested Since Fork)

These features still need to be tested, but are technically baked into the chart from the fork:
## Notes on using Matrix Sliding Sync

- Use of lightweight Exim relay
- [Half-Shot/matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord) Discord bridge
- [matrix-org/matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) IRC bridge
- [tulir/mautrix-whatsapp](https://github.com/tulir/mautrix-whatsapp) WhatsApp bridge
To use [sliding sync](https://github.com/matrix-org/sliding-sync), which is required for [element-x], you'll need to ensure that requests to `.well-known/matrix/client` return the [correct json](https://github.com/matrix-org/sliding-sync/blob/main/README.md). To do that, you'll want update your `matrix.extra_well_known_client_content` values and set `syncv3.enabled` to `true`. Example below:

```yaml
matrix:
extra_well_known_client_content:
"org.matrix.msc3575.proxy":
"url": "https://your-sliding-sync-hostname.com"


syncv3:
# this enables this subchart: https://github.com/small-hack/matrix-sliding-sync-chart
# which deploys this: https://github.com/matrix-org/sliding-sync
enabled: true
server: "https://my-synapse-hostname.com"
secret: "this.is.a.test.secret"
bindaddr: "127.0.0.1:8008"
# note: you'll still have to actually fill out parameters
# under slidingSync.postgresql, but it is truncated here for brevity
# check out values.yaml for all possible slidingSync.postgresql values
postgresql:
enabled: true
```
## Notes on using MAS (Matrix Authentication Service)
MAS is currently the only way to use OIDC with [element-x]. If you're using MAS (Matrix Authentication Service), you'll need to set `mas.enabled` to `true`. You'll also need to setup proper routes for synapse to redirect to MAS. See example below:

```yaml
matrix:
experimental_features:
msc3861:
# Likely needed if using OIDC on synapse and you want to allow usage of Element-X (the beta of element)
enabled: false
# -- Synapse will call `{issuer}/.well-known/openid-configuration` to get the OIDC configuration
issuer: http://my-mas-domain.com/
# -- Matches the `mas.mas.client_id` in the auth service config
client_id: 0000000000000000000SYNAPSE
# -- Matches the `mas.mas.client_auth_method` in the auth service config
client_auth_method: client_secret_basic
# -- Matches the `mas.mas.clients.client_secret` in the auth service config
client_secret: "SomeRandomSecret"
# -- Matches the `mas.mas.matrix.secret` in the auth service config
admin_token: "special-secret-for-msc3861"
# -- URL to advertise to clients where users can self-manage their account
account_management_url: "https://my-mas-domain.com/account"

synapse:
enabled: true
ingress:
enabled: true
className: "nginx"
annotations:
# you need for the routing to work properly
nginx.ingress.kubernetes.io/use-regex: "true"
# -- This annotation is required for the Nginx ingress provider. You can
# remove it if you use a different ingress provider
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_intercept_errors off;
# -- required for TLS certs issued by cert-manager
cert-manager.io/cluster-issuer: letsencrypt-staging
hosts:
- host: 'my-synapse-hostname.com'
paths:
- path: "/_matrix/client/(r0|v3)/(refresh|login|logout).*"
pathType: ImplementationSpecific
backend:
service:
# this assumes you passed in mas.fullnameOverride="mas"
name: mas
port:
name: http

- path: /
pathType: Prefix
tls:
- secretName: matrix-tls
hosts:
- 'my-synapse-hostname.com'

mas:
enabled: true
# sets all MAS resources to be called mas
fullnameOverride: "mas"
postgresql:
enabled: true

ingress:
enabled: true
className: "nginx"
annotations:
cert-manager.io/cluster-issuer: 'letsencrypt-prod'
hosts:
- host: 'my-mas-domain.com'
paths:
- path: /
pathType: Prefix
tls:
- secretName: matrix-authentication-service-tls
hosts:
- 'my-mas-domain.com'

# templates out the Matrix Authentication Service config file
mas:
database:
# if blank, this can be autogenerated from mas.postgres or mas.externalDatabase
# settings, or you set this to a valid postgres URI
# https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING-URIS
uri: ""

http:
# -- Public URL base used when building absolute public URLs
public_base: "https://my-mas-domain.com/"
# List of HTTP listeners, see below
listeners:
# The name of the listener, used in logs and metrics
- name: web
# List of resources to serve
resources:
- name: discovery
- name: human
- name: oauth
- name: compat
- name: graphql
- name: assets
binds:
- host: 0.0.0.0
port: 8080

policy:
client_registration:
# don't require URIs to be on the same host. default: false
allow_host_mismatch: true
# allow non-SSL and localhost URIs. default: false
allow_insecure_uris: true

# this is mostly ignored in favor of the above masClientSecret variable
clients:
- client_id: "0000000000000000000SYNAPSE"
client_auth_method: client_secret_basic
client_secret: "SomeRandomSecret"

matrix:
homeserver: "my-synapse-hostname.com"
endpoint: "https://my-synapse-hostname.com"
secret: "special-secret-for-msc3861"

upstream_oauth2:
existingSecret: "synapse-oidc"
secretKeys:
# -- key in secret with the issuer
issuer: "issuer"
# -- key in secret with the client_id
client_id: "client_id"
# -- key in secret with the client_secret
client_secret: "client_secret"

# this below example is compatible with zitadel
providers:
# -- A unique identifier (ULID) for the provider: https://www.ulidtools.com
# in the valid redirect uris, you want to use this id
- id: "01HYZ2G7QS9P2BHSDS94F3GR80"
issuer: https://example-zitadel-domain.com/
client_id: "idgenreatedbyyourupstreamoidcprovider"
client_secret: "secretgenreatedbyyourupstreamoidcprovider"

token_endpoint_auth_method: client_secret_basic
claims_imports:
subject:
template: "{{ user.sub }}"

localpart:
action: require
template: "{{ user.preferred_username }}"

displayname:
action: suggest
template: "{{ user.name }}"

email:
action: suggest
template: "{{ user.email }}"
set_email_verification: always
```
After synapse is up, you should be able to verify it's returning correctly by doing:
```console
$ curl https://matrix.example.com/.well-known/matrix/client | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 136 100 136 0 0 1818 0 --:--:-- --:--:-- --:--:-- 1837
{
"m.homeserver": {
"base_url": "https://matrix.example.com"
},
"org.matrix.msc3575.proxy": {
"url": "https://matrix.example.com"
}
}

```

## About and Status

This is a fork of [Arkaniad/matrix-chart](https://github.com/Arkaniad/matrix-chart), which is a fork of [typokign/matrix-chart](https://github.com/typokign/matrix-chart). We recently transferred this chart from [@jessebot](https://github.com/jessebot) to the small-hack org to help with maintanence longterm :) Working on full stability, but always happy to receive GitHub Issues or PRs 💙 Please star the repo if you like our work <3

Our goal is to provide regular updates using renovatebot and provide some level of basic security from a k8s perspective. We're also trying to standardize the chart more by following predictable values.yaml patterns.

Note: We may stop supporting this if a larger entity maintains a better quality matrix chart (e.g. Bitnami releases a matrix helm chart), as then we'll just write PRs directly to them. At that time we'll put in a note in this README before publicly archiving the repo. As of right now though, in October 2023, there are no other actively maintained matrix helm charts for matrix that meet all our needs or are regularly updated to justify creating PRs.
<!-- links -->
[element-x]: https://element.io/labs/element-x "element x link"
[sliding sync]: https://github.com/matrix-org/sliding-sync "matrix sliding sync"
1 change: 1 addition & 0 deletions charts/matrix/.gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
testvalues.yaml
10 changes: 8 additions & 2 deletions charts/matrix/Chart.lock
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,11 @@ dependencies:
- name: coturn
repository: https://small-hack.github.io/coturn-chart
version: 5.2.0
digest: sha256:8a175b51faccae0bd188df5fcc17d482c9c5bb00a31734dd0e428b0f94d8cfda
generated: "2024-05-07T08:07:44.24269523+02:00"
- name: matrix-sliding-sync
repository: https://small-hack.github.io/matrix-sliding-sync-chart
version: 0.4.1
- name: matrix-authentication-service
repository: https://small-hack.github.io/matrix-authentication-service-chart
version: 0.7.1
digest: sha256:8bc477a8af3b974220136c6815a326b02aab9a1a04962b18a0a3603ffb57d0a4
generated: "2024-05-27T09:41:10.130206425+02:00"
16 changes: 14 additions & 2 deletions charts/matrix/Chart.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ sources:

type: application

version: 8.2.0
version: 9.0.0

# renovate: image=matrixdotorg/synapse
appVersion: v1.107.0

maintainers:
- name: "jessebot"
url: "https://github.com/jessebot/"
url: "https://github.com/jessebot"

dependencies:
- name: postgresql
Expand All @@ -27,3 +27,15 @@ dependencies:
version: 5.2.0
repository: https://small-hack.github.io/coturn-chart
condition: coturn.enabled

- name: matrix-sliding-sync
version: 0.4.1
repository: https://small-hack.github.io/matrix-sliding-sync-chart
alias: syncv3
condition: syncv3.enabled

- name: matrix-authentication-service
alias: mas
version: 0.7.1
repository: https://small-hack.github.io/matrix-authentication-service-chart
condition: mas.enabled
Loading

0 comments on commit d1713d5

Please sign in to comment.