Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvserver: partial implementation of ReplicasStorage #91579

Closed
wants to merge 3 commits into from

Conversation

tbg
Copy link
Member

@tbg tbg commented Nov 9, 2022

WIP

sumeerbhola and others added 2 commits November 4, 2022 15:25
The significant missing piece is part of the Init
implementation to handle a RecoveryInconsistentReplica that
requires applying committed raft log entries to the state
machine. This missing piece will need to wait until cockroachdb#75729
is fixed.

There are multiple TODOs, including related to concurrency,
but the implementation is complete enough for the datadriven
test to exercise many state transitions. Additionally, the
test exercises loss of unsynced state, and fixup of that
state in ReplicasStorage.Init, by using vfs.NewStrictMem.

Informs cockroachdb#16624

Release note: None
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @tbg)


pkg/kv/kvserver/replicas_storage.go line 693 at r2 (raw file):

	// contain the term. RangeStorage must simply remove all entries > Hi;
	// additionally it's too late to SingleDelete any existing entries since
	// that write would now apply to the new entries, not the old ones.

This is used for purging the sideloaded files, which include the term in the key, similar to what happens in

if len(rd.Entries) > 0 {
// We may have just overwritten parts of the log which contain
// sideloaded SSTables from a previous term (and perhaps discarded some
// entries that we didn't overwrite). Remove any such leftover on-disk
// payloads (we can do that now because we've committed the deletion
// just above).
firstPurge := rd.Entries[0].Index // first new entry written
purgeTerm := rd.Entries[0].Term - 1
lastPurge := prevLastIndex // old end of the log, include in deletion
purgedSize, err := maybePurgeSideloaded(ctx, r.raftMu.sideloaded, firstPurge, lastPurge, purgeTerm)

Maybe it would be sufficient for this to be the term of Lo?

@tbg tbg closed this Dec 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants