implement committed entries pagination #356
Conversation
Fullstop000
force-pushed
the
committed_entries_pagination
branch
2 times, most recently
from
bfc4ea9
to
60d7771
Compare
Fullstop000
force-pushed
the
committed_entries_pagination
branch
from
c0a0766
to
75bc583
Compare
| let mut ents = Vec::with_capacity(ents_count); | ||
| let mut size = 0u64; | ||
| for i in 0..ents_count { | ||
| let e = new_entry(1, i as u64 + 1, Some("a")); |
There was a problem hiding this comment.
ents_count is a usize, though I could use 0..ents_count as u64
| .unwrap() | ||
| .get_index(); | ||
| raw_node.advance(rd); | ||
| let mut m = new_message(1, 1, MessageType::MsgHeartbeat, 0); |
There was a problem hiding this comment.
What's the purpose of the heartbeat?
There was a problem hiding this comment.
Let the node learn the current commit_index is 11. Altered to use commit_to instead.
| // - the next Ready asks the app to apply index 11 (omitting index 10), losing a | ||
| // write. | ||
| #[test] | ||
| fn test_commit_pagination_after_restart() { |
There was a problem hiding this comment.
The test case doesn't seem to be the same as upstream.
There was a problem hiding this comment.
It is silently a little different from the one in etcd due to the diff of MemStorage
Signed-off-by: Fullstop000 <[email protected]>
Signed-off-by: Fullstop000 <[email protected]>
Signed-off-by: Fullstop000 <[email protected]>
Signed-off-by: Fullstop000 <[email protected]>
Signed-off-by: Fullstop000 <[email protected]>
Signed-off-by: Fullstop000 <[email protected]>
Signed-off-by: Fullstop000 <[email protected]>
Signed-off-by: Fullstop000 <[email protected]>
Fullstop000
force-pushed
the
committed_entries_pagination
branch
from
5233d9f
to
8fbd9a2
Compare
| // Set a max_size_per_msg that would suggest to Raft that the last committed entry should | ||
| // not be included in the initial rd.committed_entries. However, our storage will ignore | ||
| // this and *will* return it (which is how the Commit index ended up being 10 initially). | ||
| cfg.max_size_per_msg = size - 1; |
There was a problem hiding this comment.
I think it should be size - uint64(s.ents[len(s.ents)-1].Size()) - 1. The purpose is to let raft return 9 entries instead of 10, so that entry at 10 will get missed.
There was a problem hiding this comment.
Actually, the etcd codes of this test case are kind of buggy because the firstIndex here is 2 because the codes in this test replace the ents with start index 1 and the firstIndex of MemoryStorage is always s.ents[0].Index + 1 due to the dummy entry (the index 0). Therefore, len(rd.CommittedEntries) in the first loop is 8 in etcd version.
And since MemStorage in raft-rs has no such a dummy entry approach, we can correctly use size - 1 here to return 9 committed entries in the first loop.
Though I found I should use cfg.max_committed_size_per_ready here 🤣
Signed-off-by: Fullstop000 <[email protected]>
Finish the remaining work in #161
Signed-off-by: Fullstop000 [email protected]