Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add typos ci job, fix typos across the repo #338

Merged
merged 3 commits into from
Mar 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: yusancky/setup-typst@v2
id: setup-typst
with:
Expand All @@ -26,3 +26,11 @@ jobs:
with:
name: main.pdf
path: main.pdf

typos:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout@v4
- name: typos-action
uses: crate-ci/[email protected]
2 changes: 1 addition & 1 deletion balancer-requirements.typ
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ See @glossary
caption: [Balancer System Requirements]
)

== Non-Funcional Requirements <Section::req-nonfunc>
== Non-Functional Requirements <Section::req-nonfunc>

#figure(
table(
Expand Down
2 changes: 1 addition & 1 deletion constraints-justification.typ
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= Constraints & Justification <Chapter::ConstraintsJustification>

== Browser Websocket API Contraints
== Browser Websocket API Constraints

The Websocket API allows for one to open a bidirectional communication session between a browser and a server. The connection session stays open until the browser or server terminates it.
This allows for the client and server to send information to the other simultaneously. The browser's API does not allow custom HTTP headers, which means that authorization has to be done after the connection request has been made #cite(<MDNWebSocket>) #cite(<HerokuWebSocket>).
Expand Down
2 changes: 1 addition & 1 deletion figures/deploy/deployment-current.puml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Deployment_Node(browser, "User's Computer", "Web Browser") {
}

Deployment_Node(ott, "OpenTogetherTube", "Docker") {
Container(monolith, "Monolith", "Typescript, Javascipt")
Container(monolith, "Monolith", "Typescript, Javascript")
SystemDb(postgres, "Postgres", "Persists Users, Rooms")
SystemDb(redis, "Redis", "Caching, Session Store")
}
Expand Down
2 changes: 1 addition & 1 deletion figures/deploy/deployment-current.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion figures/deploy/deployment-new.puml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Deployment_Node(browser, "User's Computer", "Web Browser") {

Deployment_Node(ott, "OpenTogetherTube", "Docker") {
Container(balancer, "Balancer", "Rust", "Distributes Load")
Container(monolith, "Monolith", "Typescript, Javascipt", "Manages Rooms")
Container(monolith, "Monolith", "Typescript, Javascript", "Manages Rooms")
SystemDb(postgres, "Postgres", "Persists Users, Rooms")
SystemDb(redis, "Redis", "Caching, Session Store")
}
Expand Down
2 changes: 1 addition & 1 deletion figures/deploy/deployment-new.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion figures/discovery/service-discoverers.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion figures/monolith/components-monolith.puml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Component(api, "REST API", "", "Handles HTTP requests")
Component(infoext, "Info Extractor", "", "Extracts Video Metadata")
Component(roomman, "Room Manager", "", "Manages Rooms")
Component(storage, "Storage", "", "Handles Data Persistance")
Component(storage, "Storage", "", "Handles Data Persistence")
Component(clientman, "Client Manager", "", "Manages Websockets")

BiRel(roomman, clientman, "Send messages")
Expand Down
2 changes: 1 addition & 1 deletion figures/monolith/components-monolith.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions harness-requirements.typ
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ In order for the test harness to be effective, it must be able to cover as much
[#req("Must be able to emulate the behavior of Monoliths and Clients.", mustHave)],
[#req("Should be able to test multiple instances of the balancer at once.", wouldBeNiceToHave)],
[#req("Should have the option to run tests using a real Monolith", wouldBeNiceToHave)],
[#req("Must be able to specify tests that emit traffic in a specific order. (Sequencial tests)", mustHave)],
[#req("Must be able to specify tests that emit traffic in a specific order. (Sequential tests)", mustHave)],
[#req("Should minimize the amount of code that needs to be written to create a test.", shouldHave)],
[#req("Must be runnable in a CI environment.", mustHave)],
[#req("Must be able to generate enough traffic to stress test the balancer.", shouldHave)],
Expand Down Expand Up @@ -65,7 +65,7 @@ Desired sequence:
+ Assert that Client Bob received the message from Alice
+ Assert that Carol did not receive the message from Alice

When multiple clients connect through the balancer to a monolith containing multiple rooms, they should all first be routed to the monolith, and then to the appropriate room. Clients should only recieve messages from users in the same room.
When multiple clients connect through the balancer to a monolith containing multiple rooms, they should all first be routed to the monolith, and then to the appropriate room. Clients should only receive messages from users in the same room.

=== Test: Balancer should handle losing a Monolith gracefully

Expand Down
2 changes: 1 addition & 1 deletion harness-tests.typ
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ In this section, tests used to test the harness will be outlined.

=== Test: Malformed WebSocket Connection Does Not Crash Balancer

Scenario: A client with an incorrectly formatted header connects through the load balancer to OTT. WebSocket protocol dictates this should crash whatever is currently running, but this cannot happen when maintaining a highly avalible service. (@Figure::malformed-websocket-test-sequence)
Scenario: A client with an incorrectly formatted header connects through the load balancer to OTT. WebSocket protocol dictates this should crash whatever is currently running, but this cannot happen when maintaining a highly available service. (@Figure::malformed-websocket-test-sequence)

#figure(
image("figures/malformed-websocket-test-sequence.svg", width: 70%),
Expand Down
4 changes: 2 additions & 2 deletions join-process.typ
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ If the client fails to provide an auth token, the Balancer must terminate the co

Balancers must be able to determine which Monolith nodes are hosting which rooms.

Monolith nodes must gossip to Balancer nodes to inform them of the rooms that they have loaded. This also implies that they must notify all Balancers of their existance on startup. The Balancer must maintain a hashmap of room names to Monolith nodes.
Monolith nodes must gossip to Balancer nodes to inform them of the rooms that they have loaded. This also implies that they must notify all Balancers of their existence on startup. The Balancer must maintain a hashmap of room names to Monolith nodes.

They must also maintain a hashmap of monolith nodes to a list of rooms that they are hosting to verify that only one monolith is hosting a room at a time. When the gossip is received, the Balancer must check to see if In the event that a Balancer finds that more than one Monolith is hosting a room, it must randomly select one of the Monoliths to be the authoritative node for that room, and inform the other Monoliths that they must unload the room. This method will not work as effectively if there is more than one Balancer, but it is a simple solution for the initial implementation.

Expand All @@ -50,7 +50,7 @@ To save on bandwidth, when a room is loaded or unloaded, only information about

The client will have three options: generating a temporary room with a uuid, creating a temporary room with inputs, and creating a permanent room that can be rejoined in the future.

The options follow similar sequence paths, as shown in @Figure::create-room-diag. The client starts a new Balancer connection through the websocket, then the Balancer is responsible for deciding which node is best fit to handle the additional room. Once the right one is picked, the room is instantiated on that Monolith and the client connects. Before proceding with the client connection, the Balancer must wait for the Monolith to confirm that the room has been loaded. If the room is not loaded within a certain amount of time, the Balancer must terminate the connection as "timed out".
The options follow similar sequence paths, as shown in @Figure::create-room-diag. The client starts a new Balancer connection through the websocket, then the Balancer is responsible for deciding which node is best fit to handle the additional room. Once the right one is picked, the room is instantiated on that Monolith and the client connects. Before proceeding with the client connection, the Balancer must wait for the Monolith to confirm that the room has been loaded. If the room is not loaded within a certain amount of time, the Balancer must terminate the connection as "timed out".

Room generation requires no inputs from the client, instead a new uuid is automatically used as the room name. On the other hand, room creation has the client submit a set of inputs for the name and settings of the room. Generation only provides temporary rooms that are discarded after the room is unloaded. Creation can provide either temporary or permanent rooms, depending on the client inputs during the initial process. The settings for permanent rooms are stored in postgres even after being unloaded so that they persist and can be called upon to be reloaded at any point in the future.

Expand Down
2 changes: 1 addition & 1 deletion protocol.typ
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ These messages conform to the the protocol defined by the #link("https://github.

== Messages sent during Monolith connection startup

When a Monolith starts up, and load balancing is enabled, it must listen on a seperate port for incoming balancer connections. Balancers will initiate connections based on their configured Monolith discovery method. Upon accepting a new connection, the Monolith must send an "init" message to the Balancer to inform it of the port that it is listening for normal HTTP requests on, an auth token to verify authenticity, and a MonolithID to identify specific Monolith instances.
When a Monolith starts up, and load balancing is enabled, it must listen on a separate port for incoming balancer connections. Balancers will initiate connections based on their configured Monolith discovery method. Upon accepting a new connection, the Monolith must send an "init" message to the Balancer to inform it of the port that it is listening for normal HTTP requests on, an auth token to verify authenticity, and a MonolithID to identify specific Monolith instances.

The MonolithID is generated as a UUID. While it's theoretically possible for UUIDs to be duplicated due to the finite space of possible values, the probability of such an event is so incredibly low that it's considered practically negligible.

Expand Down
2 changes: 1 addition & 1 deletion room-states.typ
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ As required by @Req::room-uniqueness, 2 Monoliths must never have the same room
In the case that this does happen, the system is in a bad state and it must be resolved. This can occur as a race condition within the context of any number of Balancers. The planned solution to this issue is to have the load balancer unload rooms that were not the
first instance of that particular room. This means that the duplicate instances would be unloaded and the clients could then rejoin the room in a healthy state.

In order to accomplish this, every room must be accociated with a "load epoch". The load epoch #index[load epoch] is a system global atomic counter that is incremented every time a room is loaded, maintained in redis.
In order to accomplish this, every room must be associated with a "load epoch". The load epoch #index[load epoch] is a system global atomic counter that is incremented every time a room is loaded, maintained in redis.
When a room is loaded, the load epoch is incremented and the room is associated with the current load epoch. There is no need to ever reset the load epoch to 0. If the value rolls over, then the system will still function correctly. However, it must remain in the range of an unsigned 32 bit integer, because javascript does not support 64 bit integers #cite(<mdn-js-int>).

Whenever the Balancer is notified of the room load and the room is already loaded, it checks to see if the load epoch of the new room is less than the load epoch of the existing room. If it is, then the old room is unloaded, clients are kicked, and the new room is treated as the source of truth.
Expand Down
2 changes: 1 addition & 1 deletion solution-overview.typ
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ It suffers from years of ad-hoc development, and is not designed to scale. It's

== A Smart Load Balancer

The solution to scaling OTT is to use a load balancer. The Balancer will be an optional, seperate component that would be deployed alongside the Monoliths.
The solution to scaling OTT is to use a load balancer. The Balancer will be an optional, separate component that would be deployed alongside the Monoliths.

=== What is a load balancer?

Expand Down
3 changes: 3 additions & 0 deletions typos.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@

[files]
extend-exclude = ["*.svg", "*.bib"]
12 changes: 6 additions & 6 deletions use-cases.typ
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@
"Maintainer views metrics, creates dashboards, or creates alerts",
),
(
"Maintainer configures on-premesis prometheus instance to collect metrics",
"Maintainer browses to on-premesis hosted grafana",
"Maintainer configures on-premises prometheus instance to collect metrics",
"Maintainer browses to on-premises hosted grafana",
"Maintainer views metrics, creates dashboards, or creates alerts",
),
)
Expand Down Expand Up @@ -142,7 +142,7 @@

#usecase(
[Interface with Visualization],
description: [In this use case, a vistor comes up to our booth on the day of the innovation expo and interacts with the visualization for the load balancer. The visualization must be a completely seperate system that the load balancer interacts with.],
description: [In this use case, a vistor comes up to our booth on the day of the innovation expo and interacts with the visualization for the load balancer. The visualization must be a completely separate system that the load balancer interacts with.],
basic_flow:
(
"Visitor approaches booth",
Expand All @@ -153,7 +153,7 @@
"Room appears on screen tethered to the appropriate monolith",
"Client appears on screen tethered to the appropriate room",
"Client disconnects",
"Client dissappears from visualization screen",
"Client disappears from visualization screen",
"Visitor walks away"
),
alt_flows:
Expand All @@ -174,7 +174,7 @@
"Client rejoins the room they created",
"Client reappears on visualization screen",
"Client disconnects",
"Client dissappears from visualization screen",
"Client disappears from visualization screen",
"Visitor walks away"
),
(
Expand All @@ -183,7 +183,7 @@
"Visitor joins an already existing room",
"Client appears on screen tethered to the appropriate room",
"Client disconnects",
"Client dissappears from visualization screen",
"Client disappears from visualization screen",
"Visitor walks away"
),
(
Expand Down
12 changes: 6 additions & 6 deletions visualization-design.typ
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ The reason for using this library is because of its embrace of web standards lik

== Data Format

The visualization will need to recieve an array of balancers:
The visualization will need to receive an array of balancers:

```json
balancers = []
Expand Down Expand Up @@ -112,25 +112,25 @@ room = {

== Data Gathering

The load balancer is one part of a distributed system, and many instances of the balancer can be active simultaneously. One instance of the visualization should have the capability of recieving information from multiple balancers, and should do so in real time. Additionally, the visualization should have the capability of recieving data from both the official deployment of OTT on fly.io and self-hosted instances.
The load balancer is one part of a distributed system, and many instances of the balancer can be active simultaneously. One instance of the visualization should have the capability of receiving information from multiple balancers, and should do so in real time. Additionally, the visualization should have the capability of receiving data from both the official deployment of OTT on fly.io and self-hosted instances.

Grafana supports querying Prometheus, and there is documentation linked below on a quick start for creating a new data source. Given the balancer has already integrated Prometheus for metrics, this is the preferred method for gathering data. There is no mention on compatibility with D3.js, but assuming there are no issues integrating D3.js into a Grafana panel, this should not be a problem. #cite(<grafana-prometheus-visualization>)

== Recieving Information From Load Balancers
== Receiving Information From Load Balancers

Grafana is a tool primarily meant for time series data, and no current data source plugins support receiving the type of information needed. To add support a custom data source must be created.

=== Balancer Discovery

While the visualization is running: Multiple instances of the balancer can be active simultaneously, new instances can become active, and instances can go offline. The addresses of these balancers are not known at runtime, so a discovery process similiar to @Chapter::ServiceDiscovery must run to collect data from the discovered balancers.
While the visualization is running: Multiple instances of the balancer can be active simultaneously, new instances can become active, and instances can go offline. The addresses of these balancers are not known at runtime, so a discovery process similar to @Chapter::ServiceDiscovery must run to collect data from the discovered balancers.

In order to achieve this, a new rust crate will be created to handle this discovery process. Implementation will likely be similar to @Chapter::ServiceDiscovery. A port will be opened to listen for active instances of the balancer. When a connection or connections are found, the balancer discoverer clones the balancer(s) and connects.

=== Querying Balancers

#figure(
image("figures/vis/visualization-balancer-datasource-sequence.svg"),
caption: "Sequence Diagram Explaining How Grafana Recieves Data From Load Balancers"
caption: "Sequence Diagram Explaining How Grafana Receives Data From Load Balancers"
) <Figure::visualization-balancer-datasource-sequence>

=== Collection and Aggregation
Expand Down Expand Up @@ -197,7 +197,7 @@ disable_total_stats = false
# basic_auth_password =
```

Setting a username and password is optional, only set if you want to require authorization to view the metrics endpoint. After the configuration file is edited as shown above and Grafana is restarted, the metrics should be accessable at http://localhost:3000/metrics.
Setting a username and password is optional, only set if you want to require authorization to view the metrics endpoint. After the configuration file is edited as shown above and Grafana is restarted, the metrics should be accessible at http://localhost:3000/metrics.

Next, add the job to your `prometheus.yml` file. Example:

Expand Down
2 changes: 1 addition & 1 deletion visualization-requirements.typ
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The purpose of the global visualization is to show functionality and overall sta
caption: "Figma Mockup of Global Balancer Visualization"
) <Figure::basic-graph-visualizer>

This figure represents the global case of one balancer being active, and no specific views/groupings (such as region) being active. As new monoliths are instantiated, they will appear on screen tethered to the balancer. As new rooms are instantiated, they will appear on-screen alongside the client that created them tethered to the appropriate monolith. Clients will not appear on-screen until they are connected to a room. When a monolith goes offline, the node representing it and any clients connected to that monolith should dissappear The room(s) connected to the (now disconnected) monolith should remain on-screen for some predetermined amount of time in the case a client connected to that room reconnects so it can be tethered to a new monolith.
This figure represents the global case of one balancer being active, and no specific views/groupings (such as region) being active. As new monoliths are instantiated, they will appear on screen tethered to the balancer. As new rooms are instantiated, they will appear on-screen alongside the client that created them tethered to the appropriate monolith. Clients will not appear on-screen until they are connected to a room. When a monolith goes offline, the node representing it and any clients connected to that monolith should disappear The room(s) connected to the (now disconnected) monolith should remain on-screen for some predetermined amount of time in the case a client connected to that room reconnects so it can be tethered to a new monolith.

=== Monolith View

Expand Down
Loading