Skip to content

feat(events): reliable integration during the load phase #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
glevco merged 3 commits into from
Apr 20, 2023
Merged

feat(events): reliable integration during the load phase #52

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ae3d16d
feat(events): update designs
glevco Apr 13, 2023
d85c033
feat(events): remove unused image
glevco Apr 13, 2023
3a6c667
feat(events): add notes on genesis and metadata storage
glevco Apr 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 1 addition & 13 deletions projects/reliable-integration/0001-high-level-design.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,18 +39,6 @@ These features will not be part of the first phase of this project:
- Give users the choice to receive only a subset of events, according to some criteria.
- The `--flush-events` flag. By default, all events are retained. In the future, the user could provide a `--flush-events` flag to enable the flushing of events after each event is sent to the client.

## Event generation during the full-node cycle

Considering this full-node cycle:

![Full-Node-Cycle drawio](./0001-images/full_node_cycle.png)

Where:
- `Load` is the period right after the full-node is started, where the local database is read.
- `Sync` is the period where `Load` is finished and the full-node continuously receive/send txs to other peers until full node is stopped.

By default, the events generated during load will not be emitted. If the user wants to enable them, one must provide the `--emit-load-events` flag.

## Flow

![Event Flow drawio](./0001-images/event_flow.png)
Expand Down Expand Up @@ -217,7 +205,7 @@ It will be triggered when the full-node is initializing and is reading locally f

### LOAD_FINISHED

It will be triggered when the full-node is ready to establish new connections, sync, and exchange transactions, at the same that when the manager state changes to `READY` [here](https://github.com/HathorNetwork/hathor-core/blob/85206cb631b609a5680e276e4db8cffbb418eb88/hathor/manager.py#L652). Other events will be triggered ONLY after this one, if the `--emit-load-events` flag is not enabled. `EmptyData` is sent.
It will be triggered when the full-node is ready to establish new connections, sync, and exchange transactions, at the same that when the manager state changes to `READY` [here](https://github.com/HathorNetwork/hathor-core/blob/85206cb631b609a5680e276e4db8cffbb418eb88/hathor/manager.py#L652). `EmptyData` is sent.

### NEW_VERTEX_ACCEPTED

Expand Down
Binary file removed projects/reliable-integration/0001-images/full_node_cycle.png
View file
Open in desktop
Binary file not shown.
49 changes: 45 additions & 4 deletions projects/reliable-integration/0002-low-level-design.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,55 @@ This low-level design intends to go deeper on the technical aspects of the proje

## Full-node execution

Considering this full-node cycle:

![Full-Node-Cycle drawio](./0002-images/full_node_cycle.png)

### TODO: This section will be updated in a separate PR
At first, when the full node is initialized, it moves to the `LOAD` state, where data is loaded from the database, without consulting the network. Once all local data is read, the full-node continuously syncs with peers, changing to the `SYNC` state.

In order to know which events will be generated, we will store the states (`LOAD` and `SYNC`) as metadata. So, when the full node starts, we will check the metadata. If it is empty or `LOAD`, we do not know where it stopped and will regenerate all events during load. If it is `SYNC`, we will not generate any events until `LOAD_FINISHED` is emitted.

The state metadata will be stored in a specific RocksDB column family, used only for event-related metadata.

In the table below we describe all possible starting scenarios for the full node, and the respective desired outcome:

| Starting scenario | Vertices in the database | Events in the database | State metadata | Desired outcome |
|-----------------------------------|--------------------------|------------------------|----------------|-------------------------------------------------------------------------------------------------------------------------------------------|
| `Fresh start` | None | None | Empty | Generate all events, from `event_id = 0`. All events come during the `SYNC` phase. |
| `Snapshot start` | Some | None | Empty | Generate all events, from `event_id = 0`. Events come during both the `LOAD` and `SYNC` phases. |
| `Restart from aborted LOAD phase` | Some | Some | `LOAD` | All events in the database are discarded. Generate all events, from `event_id = 0`. Events come during both the `LOAD` and `SYNC` phases. |
| `Restart from aborted SYNC phase` | Some | Some | `SYNC` | Generate events from `event_id = last_event_id + 1`. All events come during the `SYNC` phase. |

There's no case in which there are some events in the database, but no vertices.

For the `Fresh start` scenario, if we consider there's a `LOAD` phase with 0 events, then the desired outcome becomes the same for the first three scenarios. This makes it possible to simplify the table into only two scenarios, only depending on `State metadata`, as described in the beginning of this section:

| Starting scenario | State metadata | Desired outcome |
|------------------------------------------------------------------------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------|
| `Fresh start` or `Snapshot start` or `Restart from aborted LOAD phase` | Empty or `LOAD` | All events in the database are discarded. Generate all events, from `event_id = 0`. Events come during both the `LOAD` and `SYNC` phases. |
| `Restart from aborted SYNC phase` | `SYNC` | Generate events from `event_id = last_event_id + 1`. All events come during the `SYNC` phase. |

Now, we can describe a step-by-step representing these rules:

1. The full node begins the initialization process
2. The `EventManager` starts
1. It retrieves the previous node state from the database and stores it
2. If its stored previous state is empty or `LOAD`, it discards all events currently in the database
3. The full node emits the `MANAGER_ON_START` PubSub event
1. The `EventManager` sets `LOAD` as the current node state in the database
4. The full node loads all vertices from the database
5. If the `EventManager`'s stored previous state is empty or `LOAD`, a new vertex event is generated for all vertices in the database, in topological order
6. The full node finishes loading and emits the `LOAD_FINISHED` PubSub event
1. The `EventManager` sets `SYNC` as the current node state in the database
7. The full node starts syncing new vertices from the network

### Genesis

Events for the genesis block and two transactions are always emitted when events are generated from `event_id = 0`.

~~At first, when the full node is initiated, it moves to the `LOAD` state, where data is load from the database, without consulting the network. Once all local data is read, the full-node continuously sync with peers, changing to `SYNC` state.~~
### Snapshots

~~In order to know which events will be triggered, we will store the states (`LOAD` and `SYNC`) as metadata. So, when the full node starts, we will check the metadata. If it is empty or `LOAD`, we do not know where it stopped and will emit all events. If it is `SYNC`, we will skip all event triggered during full node initialization until `LOAD_FINISHED`~~
When generating snapshots, events should not be included, as they will be discarded and regenerated anyway.

## Flags

Expand All @@ -21,7 +63,6 @@ The following flags will be provided:
| Flag | Description |
|----------------------|-------------------------------------------------|
| `--enable-event-queue` | Enable the message broker mechanism |
| `--emit-load-events` | Enable emission of events during the load phase |

## Event

Expand Down