add relay support

[alphatownsman]

• initialize local Relay actor in post-migration
• subscribe to relay server
• fanout public posts to active relay server
• accept inbound relayed post (no work needed)
• unsubscribe from relay server
• UI to subscribe/unsubscribe relay

Protocol

from users/models/relay_actor.py:

    Relay services typically supports two modes: Mastodon and LitePub.

    in Mastodon mode:
    instance handshake with relay by sending Follow[1] to relay's inbox, relay won't follow back
    instance publishes its activities by sending Create/Update/Delete to relay's inbox
    relay publishes remote activities by sending Announce to instance's inbox

    in LitePub mode (also known as Pleroma mode):
    instance queries relay server's actor uri, sends Follow to relay's inbox and gets Follow back
    instance publishes its activities by sending Announce[2] to relay's inbox
    relay publishes remote activities by sending Announce to instance's inbox

    Our implementation do initial handshake in LitePub mode, then send activities in Mastodon mode;
    Most if not all modern relay implementations seem fine with this mixed mode

The mixed mode is chosen to minimize necessary hacks to the current code and data model

Test

The PR has been tested with instances running the following relay implementations

• https://git.pleroma.social/pleroma/relay/
• https://github.com/yukimochi/Activity-Relay
• https://git.asonix.dog/asonix/relay
• https://github.com/noellabo/pub-relay/ - not working, blocked by a strict check I don't feel very comfortable to accommodate

Usage

Relay can be added in admin UI (screen shot below), or run the following in python manage.py shell :

from users.models import RelayActor
RelayActor.subscribe('https://relay.infosec.exchange/actor')
RelayActor.unsubscribe('https://relay.infosec.exchange/actor')

be aware: large relay may take minutes or longer to respond your subscribing request, and once it does, large volume of activities will be posted to your server at once.

[andrewgodwin]

So, can you explain why you think a separate Relay actor is required, that's not SystemActor nor a standard Identity? From scanning through, it doesn't seem like it does anything that either of those couldn't be extended to do.

[andrewgodwin]

I'll do this on main, it shouldn't need to be in here.

[alphatownsman]

sure, reverted.

[alphatownsman]

@andrewgodwin I had the same idea as yours and was using system actor when started working on this. However, after digging thru a few relay implementations, I realize, in order to be compatible with them, the relay actor has to:

1. has actor uri end with /relay
2. has inbox uri and shared inbox url pointing to actual shared inbox

Changing these on system actor especially actor uri will cause problem for existing deployments, plus there might be other relay specific hacks as I might not cover all those relay implementation tricks. Hence I think the best approach is having them on a dedicated actor, not reuse system who has other duties.

[andrewgodwin]

The shared inbox thing we can do with system actor, but what software requires that the relay actor ends in '/relay'? That seems... strange.

[alphatownsman]

@andrewgodwin
Check ending with /relay :
https://github.com/yukimochi/Activity-Relay/blob/d8a385eb37fb0e0cef2088977a1a92cbb07e8526/api/resolver.go#L186

even more strange one:
https://github.com/noellabo/pub-relay/blob/f40750e2b061b12d9f791a3ed03371c97e43ce9e/src/web_server/http_signature.cr#L175
I haven't convince myself to accommodate for this (or maybe I should given the actor uri is already hardcoded?)...

I guess most of these were probably built when there were only Mastodon and Pleroma.

An alternative is, instead of LitePub (Pleroma) mode, we do handshake in Mastodon mode. That's doable and it was where I started, but I gave up bc that requires even more ugly hack in Takahe follow and inbox code flow.

[andrewgodwin]

What sort of hack is required for "mastodon mode"? I'm curious, as it would seem less fragile than mixing the modes as is done currently, and reading your short description of it makes it sound easier (just send a follow at the start and then sling posts at the remote inbox as we get them)

[alphatownsman]

@andrewgodwin mastodon mode is an incomplete Follow flow, so instead of reuse the current Follow code, extra code has to be added to

• create a model for relays inbox uri;
• construct and send a special Follow
• intercept Accept and Reject.

[andrewgodwin]

I might say that, instead of doing special things with Identities, we should just have a new Relay model that does this instead? It can have a "unsent"/"sent" state to handle this follow, and it would made the admin code cleaner as well.

[alphatownsman]

I can't say I'm very convinced. Conceptually, a relay is not much different from a real Person identity boosting public posts to their followers. Reusing existing Domain/Identity/Follow models and states are very natural choice.

Yes, there are some hacks but that's quite minimum, only to initialize the local relay identity with a predetermined actor_uri and username; the admin view is just to manage follower of an identity, if we can find existing view/template to reuse, it's not even necessary; or if the concern is mainly the state display, we can make it a property of Follow?

btw I just removed some unnecessary code accidentally left over from my previous approaches so the total code change is even less now.

On the other hand, a RelayState + Relay model will add/duplicate more code, and not much additional value, which feels a bit over kill to me,

[andrewgodwin]

Well, Takahē will accept anything posted to its inbox URLs without even needing a follow object for it - internally, it treats every inbound message the same (shared inbox or personal inbox) and works out its own fanout, so we really don't need to even know about the relay for inbound content.

Thus, that means the only thing we'd be using Identity for is the follow accept/reject logic; the message acceptance doesn't need an identity (it can use either the shared inbox URI, or we can invent one with relay in the name that goes to the same place if we want), and sending every local message out to a relay has to be special code anyway that adds a fanout for every relay in the targets calculation code.

(I have been planning to refactor out some of the actual ActivityPub parsing/formatting code a little, too, which might close the gap in a situation like this; there's a bit too much of it in the models right now)

[alphatownsman]

the only thing we'd be using Identity for is the follow accept/reject logic;

That's not only for the logic, but also to store property(target relay's actor uri and inbox uri) and state, so we don't have to add an extra Relay model to store their property and states.

---

It might be just personal aesthetics but I really don't prefer the redundant code in the alternative approach of extra Relay model which provides some clarity but very limited IMHO.

However, I think it's important for Takahe to support the relay feature, so if you really still prefer the approach of extra Relay model after considering the discussion so far, I can put together a different PR.

[alphatownsman]

reimplemented in #686