feat: add Persistence skeleton for db writes#8818
Conversation
Rjected
force-pushed
the
dan/database-thread
branch
from
ef337a1 to
eb51dad
Compare
Cargo.toml
Outdated
There was a problem hiding this comment.
can we roll this back
I'd like to send this directly to main
There was a problem hiding this comment.
I think we need to support unwind as well
There was a problem hiding this comment.
we expect that this will be spawned, right?
and internally this spawns regular std::thread to execute the actions?
There was a problem hiding this comment.
this should probably happen on a std thread
There was a problem hiding this comment.
do we expect this to be a spawned service that dispatches the actions?
if not then this perhaps doesn't need the receiver and instead controls only the spawned threads that do the IO
There was a problem hiding this comment.
yeah, this would be spawned, are you suggesting just passing the receiver to the spawned threads then?
There was a problem hiding this comment.
I guess that doesn't make a ton of sense, not sure what you mean then about not needing a receiver of some sort
There was a problem hiding this comment.
yeah, this would be spawned,
then this makes sense, but we also need to spawn the actual IO work to a dedicated thread
I wonder if we then even need to spawn this type at all because doesn't make to much sense to send a notification to another task just so this one can spawn the IO stuff
Rjected
force-pushed
the
dan/database-thread
branch
from
eb51dad to
2f15aca
Compare
There was a problem hiding this comment.
we expect that this will only do IO so this should be spawned on a regular endless thread, like we do with the mdbx db
There was a problem hiding this comment.
makes sense, switched to sync loop
There was a problem hiding this comment.
I'd also like a PersistenceHandle type that wraps the corresponding sender
There was a problem hiding this comment.
if we do std::thread then we need the std::mpsc variant
Rjected
force-pushed
the
dan/database-thread
branch
from
a5251e9 to
22563f0
Compare
| pub async fn save_blocks(&self, blocks: Vec<ExecutedBlock>) -> B256 { | ||
| let (tx, rx) = oneshot::channel(); | ||
| self.sender | ||
| .send(PersistenceAction::SaveFinalizedBlocks((blocks, tx))) | ||
| .expect("should be able to send"); | ||
| rx.await.expect("todo: err handling") | ||
| } | ||
|
|
||
| /// Tells the persistence task to remove blocks above a certain block number. The removed blocks | ||
| /// are returned by the task. | ||
| pub async fn remove_blocks_above(&self, block_num: u64) -> Vec<ExecutedBlock> { | ||
| let (tx, rx) = oneshot::channel(); | ||
| self.sender | ||
| .send(PersistenceAction::RemoveBlocksAbove((block_num, tx))) | ||
| .expect("should be able to send"); | ||
| rx.await.expect("todo: err handling") |
There was a problem hiding this comment.
thinking about how we will integrate this (poll) then we likely want a function that accepts the action directly, but these should be useful for testing etc.
2 participants
This will eventually be what writes to disk during live sync, the idea being it will receive requests to persist things, and eventually persist them all at once. The idea is that some higher level of chain handling should be the one which knows where the tip is, ie, how much of recent history will be in memory. Then that component will send a request for flushing in memory state to disk