Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@
"federation/event-driven-federated-subscriptions",
"federation/event-driven-federated-subscriptions/nats",
"federation/event-driven-federated-subscriptions/kafka",
"federation/event-driven-federated-subscriptions/redis",
"federation/event-driven-federated-subscriptions/composition-rules"
]
}
Expand Down Expand Up @@ -541,7 +542,8 @@
"router/event-driven-federated-subscriptions-edfs/nats/stream-and-consumer-configuration"
]
},
"router/event-driven-federated-subscriptions-edfs/kafka"
"router/event-driven-federated-subscriptions-edfs/kafka",
"router/event-driven-federated-subscriptions-edfs/redis"
]
},
"router/compliance-and-data-management",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Composition Rules"
description: "The Event-Driven Graph (or EDG) is an \"abstract\" subgraph, so it must not define any resolvers. EDGs are also subject to special compositional rules."
description: "The Event-Driven Graph (or EDG) is an "abstract" subgraph, so it must not define any resolvers. EDGs are also subject to special compositional rules."
icon: "network-wired"
---

Expand All @@ -13,8 +13,10 @@ EDG Root fields must define their respective event directive and a valid respons
| Query | @edfs__natsRequest | A non-nullable entity object |
| Mutation | @edfs__natsPublish | `edfs__PublishResult!` |
| | @edfs__kafkaPublish | `edfs__PublishResult!` |
| | @edfs__redisPublish | `edfs__PublishResult!` |
| Subscription | @edfs__natsSubscribe | A non-nullable entity object |
| | @edfs__kafkaSubscribe | A non-nullable entity object |
| | @edfs__redisSubscribe | A non-nullable entity object |

Note that the `edfs__NatsStreamConfiguration` input object must _always_ be defined to satisfy the `@edfs__natsSubscribe` directive:

Expand Down
48 changes: 48 additions & 0 deletions docs/federation/event-driven-federated-subscriptions/redis.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
title: "Redis"
icon: "sitemap"
---

## Definitions

<Note>
The `providerId` argument, including the default value "default", _must_ correspond to an equivalent property in _events.providers.redis_ entry of the router config.yml.
</Note>

### @edfs_redisPublish

```js
directive @edfs__redisPublish(
channel: String!,
providerId: String! = "default"
) on FIELD_DEFINITION

type edfs__PublishResult {
success: Boolean!
}
```

| Argument name | Type | Value |
| ------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| channel | String\! | The event channel. |
| providerId | String\! | The provider ID, which identifies the connection in the router config.yaml. <br />If unsupplied, the provider ID "default" will be used. |

### @edfs__redisSubscribe

```js
directive @edfs__redisSubscribe(
channels: [String!]!,
providerId: String! = "default"
) on FIELD_DEFINITION
```

| Argument name | Type | Value |
| ------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| channels | [String\!]\! | The event channels. It is possible to subscribe to multiple topics.<br />Also, subscription is done using `PSUBSCRIBE `so some pattern matching can be used, as per redis documentation: https://redis.io/docs/latest/commands/psubscribe/. |
| providerId | String\! | The provider ID, which identifies the connection in the router config.yaml. <br />If unsupplied, the provider ID "default" will be used. |

<CardGroup cols={2}>
<Card title="Redis" icon="sitemap" horizontal href="/router/event-driven-federated-subscriptions-edfs/redis">

</Card>
</CardGroup>
Binary file added docs/image-4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/image.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
20 changes: 15 additions & 5 deletions docs/router/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1408,7 +1408,7 @@ cdn:

The Events section lets you define Event Sources for [Event-Driven Federated Subscriptions (EDFS)](/router/event-driven-federated-subscriptions-edfs).

We support NATS and Kafka as event bus provider.
We support NATS, Kafka and Redis as event bus provider.

```yaml config.yaml
version: "1"
Expand All @@ -1433,13 +1433,17 @@ events:
username: "username"
brokers:
- "localhost:9092"
redis:
- id: my-redis
urls:
- "localhost:9092"
```

### Provider

| Environment Variable | YAML | Required | Description | Default Value |
| -------------------- | --------- | --------------------------------------------- | ------------------- | ------------- |
| | providers | <Icon icon="square-check" iconType="solid" /> | one of: nats, kafka | |
| Environment Variable | YAML | Required | Description | Default Value |
| -------------------- | --------- | --------------------------------------------- | -------------------------- | ------------- |
| | providers | <Icon icon="square-check" iconType="solid" /> | one of: nats, kafka, redis | |

### NATS Provider

Expand Down Expand Up @@ -1470,7 +1474,12 @@ events:
| | tls | <Icon icon="square" /> | TLS configuration for the Kafka provider. If enabled, it uses SystemCertPool for RootCAs by default. | |
| | tls.enabled | <Icon icon="square" /> | Enable the TLS. | |

### Nats Provider
### Redis Provider

| Environment Variable | YAML | Required | Description | Default Value |
| :------------------- | :--- | :------- | :--------------------------------------------------------------------------------------- | :------------ |
| | id | | The ID of the provider. This have to match with the ID specified in the subgraph schema. | [] |
| | urls | | A list of redis instances URLS, e.g: `redis://localhost:6379/2` | |

## Router Engine Configuration

Expand Down Expand Up @@ -1505,6 +1514,7 @@ Configure the GraphQL Execution Engine of the Router.

### Example YAML config:


```yaml config.yaml
version: "1"

Expand Down
30 changes: 22 additions & 8 deletions docs/router/event-driven-federated-subscriptions-edfs.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Event-Driven Federated Subscriptions (EDFS)"
description: "EDFS combines the power of GraphQL Federation and Event-Driven Architecture (Kafka, NATS) to update a user GraphQL Subscription after an event occurs in your system."
description: "EDFS combines the power of GraphQL Federation and Event-Driven Architecture (Kafka, NATS, Redis) to update a user GraphQL Subscription after an event occurs in your system."
icon: "circle-info"
sidebarTitle: "Overview"
---
Expand Down Expand Up @@ -40,14 +40,17 @@ Furthermore, classic Subscriptions with Federation are quite expensive when it c

Enter Event-Driven Federated Subscriptions, a simple way to scale Federated Subscriptions in a resource-efficient manner.

EDFS supports two event providers:
EDFS supports three event providers:

<CardGroup cols="2">
<CardGroup cols={3}>
<Card title="Kafka" icon="sitemap" href="/router/event-driven-federated-subscriptions-edfs/kafka">

</Card>
<Card title="NATS" icon="puzzle-piece" href="/router/event-driven-federated-subscriptions-edfs/nats">

</Card>
<Card title="Redis" icon="memory" href="/router/event-driven-federated-subscriptions-edfs/redis">

</Card>
</CardGroup>

Expand Down Expand Up @@ -86,6 +89,17 @@ directive @edfs__kafkaSubscribe(
providerId: String! = "default"
) on FIELD_DEFINITION

# Redis integration
directive @edfs__redisPublish(
channel: String!,
providerId: String! = "default"
) on FIELD_DEFINITION

directive @edfs__redisSubscribe(
channels: [String!]!,
providerId: String! = "default"
) on FIELD_DEFINITION

# OpenFederation directive to filter subscription events
directive @openfed__subscriptionFilter(
condition: openfed__SubscriptionFilterCondition!
Expand All @@ -96,17 +110,17 @@ Let's explain each directive in detail:

The `@edfs__natsRequest` directive is a specific NATS directive to extend a Graph through an Event Source. It makes a request to a NATS subject and waits synchronously of the response. Under the hood it uses [Request/Reply ](https://docs.nats.io/nats-concepts/core-nats/reqreply)semantics from NATS.

The `@edfs__natsPublish` and `@edfs__kafkaPublish` directive allows you to publish an event through a Mutation.
The `@edfs__natsPublish`, `@edfs__kafkaPublish`, and `@edfs__redisPublish ` directive allows you to publish an event through a Mutation.

Using the `@edfs__natsSubscribe` and `@edfs__kafkaSubscribe` directives, you can create a Subscription to the corresponding message bus. By default, both provider implementations are stateless, meaning every client receives the same events in a broadcast fashion. This behavior can be adjusted. NATS allows you to create a [consumer group](https://docs.nats.io/nats-concepts/jetstream/consumers), resulting in multiple independent streams of the subject, where each client can consume events at their own pace.
Using the `@edfs__natsSubscribe`, `@edfs__kafkaSubscribe` and `@edfs__redisSubscribe` directives, you can create a Subscription to the corresponding message bus. By default, all the provider implementations are stateless, meaning every client receives the same events in a broadcast fashion. This behavior can be adjusted. NATS allows you to create a [consumer group](https://docs.nats.io/nats-concepts/jetstream/consumers), resulting in multiple independent streams of the subject, where each client can consume events at their own pace.

The `@openfed__subscriptionFilter` directive allows you to filter subscription messages based on specified conditions. For more information see [Subscription Filter](/router/event-driven-federated-subscriptions-edfs#subscription-filter).

An Event-Driven Subgraph does not need to be implemented. It is simply a Subgraph Schema that instructs the Router on how to connect specific root fields to the Event Source. Scroll down for an example.

## Prerequisites

To use EDFS, you need to have an Event Source running and connected to the Router. Currently, we support NATS and Kafka. For simplicity, NATS is used to explain the examples.
To use EDFS, you need to have an Event Source running and connected to the Router. Currently, we support NATS, Kafka, and Redis. For simplicity, NATS is used to explain the examples.

To get started, run a NATS instance and add the following configuration to your `config.yaml` Router Configuration:

Expand Down Expand Up @@ -188,7 +202,7 @@ type Employee @key(fields: "id", resolvable: false) {

#### The "subjects" Argument

The subjects/topics argument of all events Directives allows you to use templating Syntax to use an argument to render the topic.
The subjects/topics/channels argument of all events Directives allows you to use templating Syntax to use an argument to render the topic.

Given the following Schema:

Expand Down Expand Up @@ -233,7 +247,7 @@ Once the initial result is coming back from the "Event Subgraph", the Router is

### Publish

The `@edfs_natsPublish` and `@edfs_kafkaPublish` directive sends a JSON representation of all arguments, including arguments being used to render the topic, to the rendered topic. Fields using the `eventsPublish` directive MUST return the type `PublishEventResult` with one single field `success` of type `Boolean!`, indicating whether publishing the event was successful or not.
The `@edfs_natsPublish`, `@edfs_kafkaPublish` and `@edfs__redisPublish` directive sends a JSON representation of all arguments, including arguments being used to render the topic, to the rendered topic. Fields using the `eventsPublish` directive MUST return the type `PublishEventResult` with one single field `success` of type `Boolean!`, indicating whether publishing the event was successful or not.
Comment thread
BrendanBondurant marked this conversation as resolved.

Given that we send the following Mutation:

Expand Down
44 changes: 23 additions & 21 deletions docs/router/event-driven-federated-subscriptions-edfs/kafka.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ descripton: "Kafka event provider support for EDFS"

## Minimum requirements

|Package|Minimum version|
|---|---|
|controlplane|0.88.3|
|router|0.88.0|
|wgc|0.55.0|
| Package | Minimum version |
| ------------ | --------------- |
| controlplane | 0.88.3 |
| router | 0.88.0 |
| wgc | 0.55.0 |

## Full schema example

Expand Down Expand Up @@ -97,20 +97,22 @@ wgc subgraph publish edg --namespace default --schema eedg.graphqls
Based on the example above, you will need a compatible router configuration.

<CodeGroup>
```yaml config.yaml
events:
providers:
kafka:
- id: my-kafka # Needs to match with the providerID in the directive
tls:
enabled: true
authentication:
sasl_plain:
password: "password"
username: "username"
brokers:
- "localhost:9092"
```

```yaml config.yaml
events:
providers:
kafka:
- id: my-kafka # Needs to match with the providerID in the directive
tls:
enabled: true
authentication:
sasl_plain:
password: "password"
username: "username"
brokers:
- "localhost:9092"
```

</CodeGroup>

## Example Query
Expand All @@ -132,5 +134,5 @@ subscription {
## System diagram

<Frame>
<img src="/images/router/event-driven-federated-subscriptions-edfs/Kafka2.png" />
</Frame>
![](/images/router/event-driven-federated-subscriptions-edfs/kafka2.png)
</Frame>
Loading