Moira list synchronization

[celskeggs]

• Automatically creating accounts
• Automatically updating team membership
• (eventually) automatically updating private channel membership

[gabrc52]

IS&T tasks

- [x] request permission from IS&T to use the SOAP API and asking get a cert for Uplink
- [ ] hear back from IS&T
- [ ] yay we have official consent to make Moira queries

IS&T (Olu) said no, that it is a private API only for usage in WebMoira. So new plan:

• Make an entirely new Moira API abstraction: https://github.com/gabrc52/moira-rest-api/
• Get daemon keytab for matrix.mit.edu
• Allow authenticating with a keytab and hide it behind a firewall (or auth token/API key) so only we can use it (currently it is only for WebAthena usage)
• Get Moira access for said daemon keytab (seemed to resolve automatically or after using the keytab in the dialups - maybe it triggered some provisioning to use it directly in an actual Athena system idk)

List things (priority 1)

• allow users from other homeservers to communicate with the bot and link their account to their kerb (this was a suggestion from the privacy discussion--we shouldn't force people to have their account on our homeserver to access our rooms/infrastructure)
• get consent from users to add them to Moira list groups (see Rebranding/Customizing #7 (comment))
• room creation based on Moira lists
• automatically creating rooms (cron/systemd unit every so often)
• command/function to sync my mailing lists (add me to all my classes and mailing lists that have opted in)
• doing that when someone finishes creating an account on our homeserver (also Rebranding/Customizing #7 (comment))
• automatically doing that/automatically adding members to already created rooms (2 possible ways: iterate for each user and add them to their mailing lists, or iterate for each list and add the respective users)
• inviting external email accounts to the room too (Figure out a way to do something about external email addresses in Moira lists #17)
• make sure invites to users who have never logged in work - test manually. If they don't, only invite users who have logged in at least once.
• syncing permissions from the Moira lists (see below) to Matrix power levels
• public lists should be searchable and joinable, private lists should not - see https://spec.matrix.org/v1.5/client-server-api/#mroomjoin_rules
• Uplink should let people (list admins? everyone?) create a room linked to a mailing list, and they should be allowed to choose whether they want it to be a space (room with subrooms - analogous to a Slack workspace or Discord server but with differences) or not. Possible implementations: interacting with a bot (easiest), interacting with a widget on a given channel, or interacting with a dedicated web portal for that
• Canvas mailing lists should be automatically opted into the service
• make sure this is only done for students and not TAs or teachers
• users should not be mass-invited to hidden lists to protect the privacy of list members. They should only be invited when they create an account and accept a prompt
• bot should work with mailing lists (stretch goal) that use weird access control rules instead of a simple owner user or list (such as SIPB ones)
• show warning when attempting to create a room for a hidden mailing list
• persist user preferences when opting in a list (if user decides not to sync moira permissions, then um, then they should not be synced afterward)
• sync changes in moira properties (not just memberships)

List things (priority 2)

• Web client for administering all of this (assigned to Liam)
• Send emails to users who are invited to a room but have never logged in (don't be that aggressive, be careful, like you don't want to send them an email for every class that they're in or something). An implementation would be to opt them into receiving the normal Synapse emails even if they have never logged in.
• custom email invites from them? have the service automatically email them "you have a chat on Uplink - please join" or something
• think about what the best permissions would be: this should be customizable enough, and having list admins only (exec of a particular student group, for instance) may add some friction, so think about what the most user friendly permissions would be that don't let people abuse (like kicking other people) (honestly they seem good enough but idk - also i think this isn't too restrictive for admins)

List things (stretch goals)

• automatically removing members who were removed from the mailing list (?)

This is in stretch goals because it might interfere with adding people. When the Moira integration works very smoothly, it could be synced 1:1 to the Moira list and adding people from Matrix could be removed or when bidirectional sync works this could be a better experience. Otherwise we might need a way to persist state (if someone was intentionally added by an admin they shouldn't be removed, but if someone was removed from the original moira list, they should be removed). Just for now, give a warning to people that they should remove people from both the list and the room.

• customize email sent to external users when invited to a chat (this is currently delegated to matrix.org identity server)
• a modified client/fork of Element could make everything seamless without requiring a separate website
• bidirectional sync - if a user joins the room corresponding to a public mailing list, they should also be added to the mailing list. (probably not for kick or leave events)
• if we manage to get perms for that, we'd be all set to make an improved version of WebMoira with the missing features and an actually mobile-friendly UI
• hidden lists should ideally have membership hidden (but this requires a change in Matrix itself (Ability to hide users below a given power level in the membership list matrix-org/matrix-spec#367, Generic pubsub over Matrix - room participant privacy? matrix-org/matrix-spec#249)

NOTE: There's 2 types of perms in Moira

So apart from regular members, there is a membership access control list/user/etc, and an owner list/user/etc.

User things

• consider automatic user creation or an identity server (has pros and cons - users who have never used our homeserver should be labled as such)
• based on above implementation, hide or remove deactivated users (Remove deactivated athena accounts #11)
• make sure DMing users who have never logged in works

[gabrc52]

Should Canvas mailing lists be public on Matrix? No is best, but there is the use case when someone may be waiting for their advisor to approve their add/drop form, or something like that. In that case, it could be added to the list of possible requests that people can email about. If this adds too much friction, yes, make them public, and make the bot yell somewhere when someone joins.

[gabrc52]

Some questions from a comment I'm removing to clean up code

# how do users choose if they want a space or normal room?
# how do we determine which mailing lists have opted in?
# who can create a channel for a given mailing list?
# who is allowed to turn a mailing list into a channel?
#   anyone if they are in a public and nonhidden list and admins otherwise?
#   something else?
# Also for classes, do we want them to be public so people can add themselves?

[gabrc52]

For the consent part, with terms and conditions, this is how it looks like:

SSO:

User/password registration: