op-node: Switch L1 Traversal to a pull based model#3583
Conversation
|
|
This PR changes implementation code, but doesn't include a changeset. Did you forget to add one? |
trianglesphere
force-pushed
the
jg/l1traversal
branch
from
738e637 to
7c741f8
Compare
|
Edit: fixed now |
This makes the L1 Retrieval a purely pull based stage. This commit required large modifications to the channel bank stage in order for the channel bank to maintain it's own progress.
The L1 Retrieval stage is now responsible for pulling data from the L1 Traversal stage. In addition, the pipeline is responsible for advancing the state of the L1 Traversal stage. The L1 Traversal stage only provides access to the current L1 block once - it pretends to be a queue that is consumed from.
trianglesphere
force-pushed
the
jg/l1traversal
branch
from
7c741f8 to
fe78c14
Compare
protolambda
left a comment
protolambda
left a comment
There was a problem hiding this comment.
LGTM, nice work. After all stage refactor PRs land we can revisit the pipeline function itself to get rid of the separate types of resets (old resets and new pull based resets).
This again requires a fair amount of changes to channel_in_reader.go for the channel in reader to maintain its progress state.
Like the rest of the changes, this also required modifications to the next stage - the batch queue in order for it to manage the progress API. This commit required even more changes than usual. I changed the pipeline to be reset to a common starting point and now use the L2SafeHead block to filter out adding batches & L1 blocks to the batch queue.
The attributes queue actually had pretty few modifications to work with the progress API. The logic of switching the batch queue over was a bit more complex because the batch queue is very stateful, but still not the worst.
The progress API is very nearly removed from the engine queue stage.
op-node: Switch batch queue to be pull based
op-node: Switch channel bank and channel in reader to a pull based stage
op-node: Switch L1 Retrieval to pull based
3 participants
Description
The L1 Retrieval stage is now responsible for pulling data from the L1 Traversal stage. In addition, the pipeline is responsible for advancing the state of the L1 Traversal stage.
The L1 Traversal stage only provides access to the current L1 block once - it pretends to be a queue that is consumed from.
Tests
I had to modify some of the unit tests, but they are passing.
Metadata