Skip to content

docs(gateway): add helpful details for new users #2109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
7596ff merged 2 commits into from
Feb 5, 2023
Merged

docs(gateway): add helpful details for new users #2109

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ce2a8b0
docs(gateway): add helpful details for new users
vilgotf Feb 2, 2023
fd0fe1f
run ci
vilgotf Feb 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 14 additions & 11 deletions twilight-gateway/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
[![codecov badge][]][codecov link] [![discord badge][]][discord link] [![github badge][]][github link] [![license badge][]][license link] ![rust badge]

`twilight-gateway` is an implementation of Discord's sharding gateway sessions.
This is responsible for receiving stateful events in real-time from Discord
and sending *some* stateful information.
This is responsible for receiving stateful events in real-time from Discord and
sending *some* stateful information.

The primary type is the `Shard`, a stateful interface to maintain a Websocket
connection to Discord's gateway. Much of its functionality can be configured, and
it's used to receive deserialized gateway event payloads or raw Websocket
messages, useful for load balancing and microservices.
connection to Discord's gateway. Much of its functionality can be configured,
and it's used to receive gateway events or raw Websocket messages, useful for
load balancing and microservices.

Using the `stream` module, shards can be easily managed in groups.

Expand All @@ -19,7 +19,6 @@ Using the `stream` module, shards can be easily managed in groups.
events
* TLS (mutually exclusive)
* `native`: platform's native TLS implementation via [`native-tls`]
equivalents
* `rustls-native-roots` (*default*): [`rustls`] using native root certificates
* `rustls-webpki-roots`: [`rustls`] using [`webpki-roots`] for root
certificates, useful for `scratch` containers
Expand All @@ -30,7 +29,7 @@ Using the `stream` module, shards can be easily managed in groups.

## Examples

Start a shard and loop over guild and voice state events:
Create a shard and loop over guild and voice state events:

```rust,no_run
use std::env;
Expand Down Expand Up @@ -66,14 +65,17 @@ async fn main() -> anyhow::Result<()> {
},
};

// You'd normally want to spawn a new tokio task for each event and
// handle the event there to not block the shard.
tracing::debug!(?event, "received event");
}

Ok(())
}
```

Create the recommended number of shards and stream over their events:
Create the recommended number of shards and run them concurrently through
`ShardEventStream`:

```rust,no_run
use futures::StreamExt;
Expand Down Expand Up @@ -105,6 +107,7 @@ async fn main() -> anyhow::Result<()> {
.await?
.collect::<Vec<_>>();

// Create an infinite stream over the shards' events.
let mut stream = ShardEventStream::new(shards.iter_mut());

while let Some((shard, event)) = stream.next().await {
Expand All @@ -121,6 +124,8 @@ async fn main() -> anyhow::Result<()> {
}
};

// You'd normally want to spawn a new tokio task for each event and
// handle the event there to not block the shards.
tracing::debug!(?event, shard = ?shard.id(), "received event");
}

Expand All @@ -131,13 +136,11 @@ async fn main() -> anyhow::Result<()> {
There are a few additional examples located in the
[repository][github examples link].

[`flate2`]: https://crates.io/crates/flate2
[`native-tls`]: https://crates.io/crates/native-tls
[`rustls`]: https://crates.io/crates/rustls
[`rustls-native-certs`]: https://crates.io/crates/rustls-native-certs
[`serde_json`]: https://crates.io/crates/serde_json
[`simd-json`]: https://crates.io/crates/simd-json
[`tokio-tungstenite`]: https://crates.io/crates/tokio-tungstenite
[`twilight-http`]: https://twilight-rs.github.io/twilight/twilight_http/index.html
[`webpki-roots`]: https://crates.io/crates/webpki-roots
[`zlib-ng`]: https://github.com/zlib-ng/zlib-ng
[codecov badge]: https://img.shields.io/codecov/c/gh/twilight-rs/twilight?logo=codecov&style=for-the-badge&token=E9ERLJL0L2
Expand Down
6 changes: 6 additions & 0 deletions twilight-gateway/src/config.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ impl Debug for Token {
/// Configuration used by the shard to identify with the gateway and operate.
///
/// Use [`Config::builder`] to configure a shard's configuration.
///
/// May be reused by cloning, also reusing the hidden TLS context---reducing
/// memory usage. The TLS context may still be reused with an otherwise
/// different config by turning it into to a [`ConfigBuilder`] through the
/// [`ConfigBuilder::with_config`] function and then rebuilding it into a new
/// config.
#[derive(Clone, Debug)]
pub struct Config {
/// Event type flags.
Expand Down
4 changes: 1 addition & 3 deletions twilight-gateway/src/connection.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,8 @@ const WEBSOCKET_CONFIG: WebSocketConfig = WebSocketConfig {

/// [`tokio_tungstenite`] library Websocket connection.
///
/// Connections are used by [`Shard`]s when [initially connecting] and when
/// reconnecting.
/// Connections are used by [`Shard`]s when reconnecting.
///
/// [initially connecting]: crate::Shard::with_config
/// [`Shard`]: crate::Shard
pub type Connection = WebSocketStream<MaybeTlsStream<TcpStream>>;

Expand Down
30 changes: 19 additions & 11 deletions twilight-gateway/src/shard.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,20 +259,28 @@ struct MinimalReady {
/// calls [`next_message`].
///
/// Shards go through an [identify queue][`queue`] that ratelimits the amount of
/// concurrent identify events (across all shards) per 5 seconds. Note that
/// shards must be identified before they start receiving dispatch events and
/// are able to send most other events.
/// concurrent identifies (across all shards) per 5 seconds. Exceeding this
/// limit invalidates the shard's session and it is therefore very important to
/// reuse the same queue when running multiple shards. Note that shards must be
/// identified before they start receiving dispatch events and are able to send
/// [`Command`]s.
///
/// # Sharding
///
/// Bots in more than 2500 guilds must run multiple shards with different
/// [`ShardId`]s, which is easiest done by using items in the [`stream`] module.
/// A shard may not be connected to more than 2500 guilds, so large bots must
/// split themselves across multiple shards. See the
/// [Discord Docs/Sharding][docs:sharding], [`ShardId`], and [`stream`]
/// documentation for more info.
///
/// # Sending shard commands in different tasks
///
/// Because a shard itself can't be used in multiple tasks it's not possible to
/// directly send [gateway commands] over a shard. To solve this
/// [`Shard::sender`] can be used to receive an MPSC channel to send commands.
/// Because shards should not be used across multiple tasks it's not always easy
/// to directly send [gateway commands] over a shard. As a convenience method,
/// [`Shard::sender`] can be used to receive an MPSC channel sender which, in
/// addition to being cheaply cloned, also only sends queued up commands when
/// the shard is identified and not ratelimited. Multiple shards' senders can,
/// for example, be collected into an `Arc<Vec<MessageSender>>` and be shared
/// across all event handler tasks.
///
/// # Examples
///
Expand Down Expand Up @@ -323,7 +331,7 @@ struct MinimalReady {
/// # Ok(()) }
/// ```
///
/// [docs:shards]: https://discord.com/developers/docs/topics/gateway#sharding
/// [docs:sharding]: https://discord.com/developers/docs/topics/gateway#sharding
/// [gateway commands]: Shard::command
/// [`next_event`]: Shard::next_event
/// [`next_message`]: Shard::next_message
Expand Down Expand Up @@ -833,8 +841,8 @@ impl Shard {
/// until it returns the response close message or a
/// [`ReceiveMessageErrorType::Io`] error type.
///
/// You do not need to call this method upon receiving a close message to
/// respond do it, Twilight handles this for you.
/// You do not need to call this method upon receiving a close message,
/// Twilight automatically responds for you.
///
/// # Example
///
Expand Down
8 changes: 4 additions & 4 deletions twilight-gateway/src/stream.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@
//!
//! A multi-threaded executor is able to run tasks in parallel, but splitting
//! shards across tasks requires channels to communicate between them, for
//! example, to coordinate shutdowns or to collect all events to a single event
//! handler task. It is therefore **not** recommended unless required for
//! performance reasons (a single core should, on a reasonably performant CPU,
//! be capable of handling tens of thousands of Discord events per second).
//! example, to coordinate shutdowns. It is therefore **not** recommended unless
//! required for performance reasons (a single core should, on a reasonably
//! performant CPU, be capable of handling tens of thousands of Discord events
//! per second).
//!
//! See the [gateway-parallel] example for how to implement this.
//!
Expand Down