Serverless Subscriptions with aws Lambda and API Gateway

[reconbot]

I've got an app that uses architect to manage http and websocket functions. We have a graphql api implemented with apollo-server-lambda and have custom websocket code. Our current real time system uses dynamodb to track subscriptions and sns to broadcast events. It works for what we need. However, I'd like to get us going in a direction where we use graphql subscriptions as it would simplify our codebase quite a bit.

Since my setup decomposes the traditional graphql subscription setup into parts backed by external services, and doesn't keep a single server process alive outside the confines of a request, the async iterator approach seems incompatible. Are there any similar setups that I could learn from?

[enisdenjo]

Hey hey! The thing about graphql-ws is that it is server agnostic. You can implement any server you want (short or long lived) by simply leveraging the makeServer and implementing your own GraphQL over WS compliant server.

Check out the use/ directory for examples of concrete server implementations shipped by this lib (ws, uWebSockets and fastify-websockets). Additionally, you can take a look at the Cloudflare Workers graphql-ws template for an implementation on a short lived worker.

About the statement of async iterator approach being incompatible - you can convert any pull/push event based emitter to async iterators (which is what graphql-subscriptions does when doing pubsub.asyncIterator("event")). Moreover, I can recommend @n1ru4l/push-pull-async-iterable-iterator for converting anything event based to an async iterator.

Furthermore, specifically to AWS, you can check @andyrichardson's subscriptionless which seemingly does exactly what you're trying to achieve and can help point you in the right direction. Maybe @andyrichardson can shine some more light for you here?

Heads up, the subprotocol ping/pongs necessary for andyrichardson/subscriptionless#3 is resolved as of [email protected]. You can either implement a subprotocol level pinger on the client side using the keepAlive client option; or on the server side loosely following the "Server usage with ws and subprotocol pings and pongs" recipe.

[enisdenjo]

You're right with how you destructured graphql-ws. The idea behind it is very monolithic. It is a tool that does everything you need, yet it tries being extensible enough to support most cases and servers.

The server is actually just a coercion tool. With makeServer, you get a boneless, GraphQL over WebSocket compliant, API and you use the hooks to implement it over a real server. Check the use/ folder + the cloudflare-worker-graphql-ws-template repo for all backed up implementations.

The client is more "down to business" in terms of supporting the two platforms that rarely change - the browser and the Node environment.

Now, the "problem" with AWS powered WS is that it is not a single server, per se, it is a bunch of spread out functions getting invoked when stuff happens. Assuming that your breakdown of subscriptionless is right, AWS's WS is basically a microservice with isolated units of work. This fact makes things a tad more complicated because it is a complete design opposite of what graphql-ws is offering (currently of course).

However, a Lego set server would be cool, thinking about it further. Not just for supporting AWS, but other niche cases like one server running the WS server and the other handling the subscriptions where the two are connected in whatever way one sees fit (gRPC or whatever other custom transport protocol). (We've already seen a need for this with #194)

An interesting idea, I like it. It would be awesome if you guys could provide a high level overview of the required puzzle parts that graphql-ws would need to offer in order for it to form a picture in the AWS/serverless/microservice world.

[reconbot]

Exactly! My mind was thinking about google functions, but I actually think scaling any server would eventually run into some of the same problems we need solved here.

[reconbot]

These examples have been enlightening. We're actually pretty close to having all the "legos". The major pain points look like;

• We need to serialize and deserialize the graphql-ws context object and reconnect it to the websocket object every time we call opened
• We can't support iterating an async iterator result from graphql's subscribe() or execute() so we need an alternative / compatible api that supports both receiving the call to subscribe and emitting events

I think any pub/sub system someone wants to bring to the table can work as long as we can associate a context and connectionId with the subscription.

A few comments on the existing interfaces.

Server

• opened perfect, we can call this for every websocket event

WebSocket

• protocol: what's this about?
• onMessage: Worries me (see below)

onMessage(cb => Promise<void>): All operations requested from the client will block the promise until completed, this means that the callback will not resolve until all subscription events have been emitted (or until the client has completed the stream), or until the query/mutation resolves.

This api would have to change as we can't wait for a subscription to end. This is more of a graphql subscribe() issue it seems.

Since we get a function execution for each message, we'd have to expose that callback so we can immediately call it with the incoming message, but it's not a huge deal. (server.handleMessage(socketId: string, msg: string) would be an easier api) I'm assuming this callback has some connection to the context that we need to maintain.

ServerOptions

• connectionInitWaitTimeout: There's some exploration around timers and step functions, but that's aws specific and I'm not sure there's a google equivalent. In general we can't support timers. We can however run a function on an interval, usually around 1 minute. More interestingly, this implies there is a state of a connection (eg, it has or hasn't sent connection_init) that might need to be persisted.
• context: Our state about a connection! We'll need to store the output of this externally.
• execute: This would be fine but I see it responds with a AsyncIterableIterator<ExecutionResult> which I assume it consumed until it ends and it's messages are sent via send
• subscribe: the root of our "problem" this is where our async iterable comes from I suppose.

Context

• subscriptions since the context would have to be serializable it couldn't support an async iterator. The docs do suggest that it could just be a reference to another operation in flight, perhaps an external subscription id?

Everything else in the docs seem serializable.

---

In subscriptionless we manually call the field's subscribe() method (not to be confused with graphql'ssubscribe()) and more or less save the output in it's internal pub/sub system as a topic. We then provide a publish(topic, event) method that finds all the pub/sub subscriptions this has connectionIds and the queries (with thier variables, etc,) for each subscription. Then we call graphql's execute() passing the event from publish() as the root value. We then take those results and writes them to the correct sockets with the connectionId.

I wonder if we might use ServerOptions.subscribe and ServerOptions.execute to emulate this behavior.

[enisdenjo]

I'll be honest, doesnt really feel like we're close... Let me start by clarifying a few things and decisions:

• protocol

Its the client requested WS subprotocol that gets validated. If invalid, the connection is denied with an appropriate close event.

• onMessage

The behaviour is very much intentional. Having the function block until the operation completes, however long, allows for super easy error handling. Just by surrounding that function with a try/catch block you're able to catch errors in:

• connection initialisation
• query/mutation operations
• subscription operations

and handle them however you see fit.

• connectionInitWaitTimeout

Security wise, this wait timeout is very necessary. You dont want connections clogging your resources without identifying themselves.

• context

and

We need to serialize and deserialize the graphql-ws context object and reconnect it to the websocket object every time we call opened

Cant be serialised easily. It often holds objects like the request, WS connection in addition to live subscription iterators.

• execute

Very much necessary for the usual JS plugin systems and implementations. You want your graphql to live alongside the server. Furthermore, supports AsyncIterableIterator because of live queries and streaming directives (like @defer or @stream).

• subscribe

Fairly same as execute.

Maybe breaking graphql-ws apart might not be the right way to go. I think a separate serverless focused GraphQL over WS compliant library would allow for a cleaner and easier to maintain implementation. Not only would this breakdown introduce a bunch of breaking changes and performance implications (due to serialisation and deserialisation), but it would increase the error surface area and also complicate the internals making them hard to understand.

Following your findings, I dont see how the two paradigms (monolith vs microservice) can coexist together here. Furthermore, I'd say it is not only about graphql-ws, it is about every other program - radical changes are necessary and you end up messing with the balance trying to support both worlds.

Maybe rolling out improvements to subscriptionless allowing for a broader support (google functions in addition to AWS) would be a way? Its focus is on serverless anyway, so it should not be that far away. What do you say @andyrichardson and @reconbot?

[andyrichardson]

For how the protocol is managed across stateful (traditional servers) and stateless (serverless) environments, I think different implementations is the way to go - given how little overlap there is outside of the protocol itself.

Maybe rolling out improvements to subscriptionless allowing for a broader support (google functions in addition to AWS) would be a way?

This sounds totally doable. Without hijacking this thread, the main concerns that need to be considered for subscriptionless to work in serverless infrastructure is:

• A managed WebSocket API (API Gateway)
• A persistence layer (DynamoDB)

The latter, I started looking into a while back. For now, this is a lesser concern until ping/pong support is added.

TL;DR: +1 for keeping the graphql-ws node library how it is