raft: use 3x replication in confchange_v2_add_single_implicit

This commit updates the confchange_v2_add_single_implicit test to start
with 2 nodes so that the addition of a third voter does not regress the
lead support and trip up the automatic transition out of the joint
configuration.

Epic: None
Release note: None

pkg/raft/testdata/async_storage_writes_append_aba_race.txt

@@ -497,7 +497,7 @@ stabilize 1
   AppendThread->1 MsgStorageAppendResp Term:0 Log:2/12
   INFO mark (term,index)=(2,12) mismatched the last accepted term 3 in unstable log; ignoring 
   AppendThread->1 MsgStorageAppendResp Term:0 Log:3/12
-        
+
 raft-log 1
 ----
 1/11 EntryNormal ""

pkg/raft/testdata/confchange_v2_add_single_implicit.txt

@@ -4,137 +4,171 @@
 # TODO(tbg): also verify that if the leader changes while in the joint state, the
 # new leader will auto-transition out of the joint state just the same.
 
-# Bootstrap n1.
-add-nodes 1 voters=(1) index=2
+# Bootstrap n1 and n2. We start with 2 nodes in this test so that the addition
+# of a third voter does not regress the lead support and trip up the automatic
+# transition out of the joint configuration.
+add-nodes 2 voters=(1, 2) index=2
 ----
-INFO 1 switched to configuration voters=(1)
+INFO 1 switched to configuration voters=(1 2)
 INFO 1 became follower at term 0
-INFO newRaft 1 [peers: [1], term: 0, commit: 2, applied: 2, lastindex: 2, lastterm: 1]
+INFO newRaft 1 [peers: [1,2], term: 0, commit: 2, applied: 2, lastindex: 2, lastterm: 1]
+INFO 2 switched to configuration voters=(1 2)
+INFO 2 became follower at term 0
+INFO newRaft 2 [peers: [1,2], term: 0, commit: 2, applied: 2, lastindex: 2, lastterm: 1]
 
 campaign 1
 ----
 INFO 1 is starting a new election at term 0
 INFO 1 became candidate at term 1
+INFO 1 [logterm: 1, index: 2] sent MsgVote request to 2 at term 1
 
-process-ready 1
+stabilize log-level=none
 ----
-Ready MustSync=true:
-State:StateCandidate
-HardState Term:1 Vote:1 Commit:2 Lead:0 LeadEpoch:0
-INFO 1 received MsgVoteResp from 1 at term 1
-INFO 1 has received 1 MsgVoteResp votes and 0 vote rejections
-INFO 1 became leader at term 1
+ok
 
+# Add v3 (with a joint configuration and an automatic transition out of joint).
 propose-conf-change 1 transition=implicit
-v2
+v3
 ----
 ok
 
-# Add n2.
+# Add n3.
 add-nodes 1
 ----
-INFO 2 switched to configuration voters=()
-INFO 2 became follower at term 0
-INFO newRaft 2 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
+INFO 3 switched to configuration voters=()
+INFO 3 became follower at term 0
+INFO newRaft 3 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
 
 # n1 commits the conf change using itself as commit quorum, then starts catching up n2.
 # When that's done, it starts auto-transitioning out. Note that the snapshots propagating
 # the joint config have the AutoLeave flag set in their config.
-stabilize 1 2
+stabilize
 ----
 > 1 handling Ready
   Ready MustSync=true:
-  State:StateLeader
-  HardState Term:1 Vote:1 Commit:2 Lead:1 LeadEpoch:1
   Entries:
-  1/3 EntryNormal ""
-  1/4 EntryConfChangeV2 v2
-> 1 handling Ready
-  Ready MustSync=true:
-  HardState Term:1 Vote:1 Commit:4 Lead:1 LeadEpoch:1
-  CommittedEntries:
-  1/3 EntryNormal ""
-  1/4 EntryConfChangeV2 v2
-  INFO 1 switched to configuration voters=(1 2)&&(1) autoleave
-  INFO initiating automatic transition out of joint configuration voters=(1 2)&&(1) autoleave
-> 1 handling Ready
-  Ready MustSync=true:
-  Entries:
-  1/5 EntryConfChangeV2
+  1/4 EntryConfChangeV2 v3
   Messages:
-  1->2 MsgFortifyLeader Term:1 Log:0/0
-  1->2 MsgApp Term:1 Log:1/3 Commit:4 Entries:[1/4 EntryConfChangeV2 v2]
+  1->2 MsgApp Term:1 Log:1/3 Commit:3 Entries:[1/4 EntryConfChangeV2 v3]
 > 2 receiving messages
-  1->2 MsgFortifyLeader Term:1 Log:0/0
-  INFO 2 [term: 0] received a MsgFortifyLeader message with higher term from 1 [term: 1]
-  INFO 2 became follower at term 1
-  1->2 MsgApp Term:1 Log:1/3 Commit:4 Entries:[1/4 EntryConfChangeV2 v2]
-  DEBUG 2 [logterm: 0, index: 3] rejected MsgApp [logterm: 1, index: 3] from 1
+  1->2 MsgApp Term:1 Log:1/3 Commit:3 Entries:[1/4 EntryConfChangeV2 v3]
 > 2 handling Ready
   Ready MustSync=true:
-  HardState Term:1 Commit:0 Lead:1 LeadEpoch:1
+  Entries:
+  1/4 EntryConfChangeV2 v3
   Messages:
-  2->1 MsgFortifyLeaderResp Term:1 Log:0/0 LeadEpoch:1
-  2->1 MsgAppResp Term:1 Log:0/3 Rejected (Hint: 0)
+  2->1 MsgAppResp Term:1 Log:0/4 Commit:3
 > 1 receiving messages
-  2->1 MsgFortifyLeaderResp Term:1 Log:0/0 LeadEpoch:1
-  2->1 MsgAppResp Term:1 Log:0/3 Rejected (Hint: 0)
-  DEBUG 1 received MsgAppResp(rejected, hint: (index 0, term 0)) from 2 for index 3
-  DEBUG 1 decreased progress of 2 to [StateProbe match=0 next=1 sentCommit=0 matchCommit=0]
-  DEBUG 1 [firstindex: 3, commit: 4] sent snapshot[index: 4, term: 1] to 2 [StateProbe match=0 next=1 sentCommit=0 matchCommit=0]
-  DEBUG 1 paused sending replication messages to 2 [StateSnapshot match=0 next=5 sentCommit=4 matchCommit=0 paused pendingSnap=4]
+  2->1 MsgAppResp Term:1 Log:0/4 Commit:3
 > 1 handling Ready
-  Ready MustSync=false:
+  Ready MustSync=true:
+  HardState Term:1 Vote:1 Commit:4 Lead:1 LeadEpoch:1
+  CommittedEntries:
+  1/4 EntryConfChangeV2 v3
   Messages:
-  1->2 MsgSnap Term:1 Log:0/0
-    Snapshot: Index:4 Term:1 ConfState:Voters:[1 2] VotersOutgoing:[1] Learners:[] LearnersNext:[] AutoLeave:true
+  1->2 MsgApp Term:1 Log:1/4 Commit:4
+  INFO 1 switched to configuration voters=(1 2 3)&&(1 2) autoleave
+  INFO initiating automatic transition out of joint configuration voters=(1 2 3)&&(1 2) autoleave
 > 2 receiving messages
-  1->2 MsgSnap Term:1 Log:0/0
-    Snapshot: Index:4 Term:1 ConfState:Voters:[1 2] VotersOutgoing:[1] Learners:[] LearnersNext:[] AutoLeave:true
-  INFO log [committed=0, applied=0, applying=0, unstable.offset=1, unstable.offsetInProgress=1, len(unstable.Entries)=0] starts to restore snapshot [index: 4, term: 1]
-  INFO 2 switched to configuration voters=(1 2)&&(1) autoleave
-  INFO 2 [commit: 4, lastindex: 4, lastterm: 1] restored snapshot [index: 4, term: 1]
-  INFO 2 [commit: 4] restored snapshot [index: 4, term: 1]
+  1->2 MsgApp Term:1 Log:1/4 Commit:4
+> 1 handling Ready
+  Ready MustSync=true:
+  Entries:
+  1/5 EntryConfChangeV2
+  Messages:
+  1->3 MsgFortifyLeader Term:1 Log:0/0
+  1->3 MsgApp Term:1 Log:1/3 Commit:4 Entries:[1/4 EntryConfChangeV2 v3]
+  1->2 MsgApp Term:1 Log:1/4 Commit:4 Entries:[1/5 EntryConfChangeV2]
 > 2 handling Ready
   Ready MustSync=true:
-  HardState Term:1 Commit:4 Lead:1 LeadEpoch:1
-  Snapshot Index:4 Term:1 ConfState:Voters:[1 2] VotersOutgoing:[1] Learners:[] LearnersNext:[] AutoLeave:true
+  HardState Term:1 Vote:1 Commit:4 Lead:1 LeadEpoch:1
+  CommittedEntries:
+  1/4 EntryConfChangeV2 v3
   Messages:
   2->1 MsgAppResp Term:1 Log:0/4 Commit:4
+  INFO 2 switched to configuration voters=(1 2 3)&&(1 2) autoleave
 > 1 receiving messages
   2->1 MsgAppResp Term:1 Log:0/4 Commit:4
-  DEBUG 1 recovered from needing snapshot, resumed sending replication messages to 2 [StateSnapshot match=4 next=5 sentCommit=4 matchCommit=4 paused pendingSnap=4]
-> 1 handling Ready
-  Ready MustSync=false:
-  Messages:
-  1->2 MsgApp Term:1 Log:1/4 Commit:4 Entries:[1/5 EntryConfChangeV2]
 > 2 receiving messages
   1->2 MsgApp Term:1 Log:1/4 Commit:4 Entries:[1/5 EntryConfChangeV2]
+> 3 receiving messages
+  1->3 MsgFortifyLeader Term:1 Log:0/0
+  INFO 3 [term: 0] received a MsgFortifyLeader message with higher term from 1 [term: 1]
+  INFO 3 became follower at term 1
+  1->3 MsgApp Term:1 Log:1/3 Commit:4 Entries:[1/4 EntryConfChangeV2 v3]
+  DEBUG 3 [logterm: 0, index: 3] rejected MsgApp [logterm: 1, index: 3] from 1
 > 2 handling Ready
   Ready MustSync=true:
   Entries:
   1/5 EntryConfChangeV2
   Messages:
   2->1 MsgAppResp Term:1 Log:0/5 Commit:4
+> 3 handling Ready
+  Ready MustSync=true:
+  HardState Term:1 Commit:0 Lead:1 LeadEpoch:1
+  Messages:
+  3->1 MsgFortifyLeaderResp Term:1 Log:0/0 LeadEpoch:1
+  3->1 MsgAppResp Term:1 Log:0/3 Rejected (Hint: 0)
 > 1 receiving messages
   2->1 MsgAppResp Term:1 Log:0/5 Commit:4
+  3->1 MsgFortifyLeaderResp Term:1 Log:0/0 LeadEpoch:1
+  3->1 MsgAppResp Term:1 Log:0/3 Rejected (Hint: 0)
+  DEBUG 1 received MsgAppResp(rejected, hint: (index 0, term 0)) from 3 for index 3
+  DEBUG 1 decreased progress of 3 to [StateProbe match=0 next=1 sentCommit=0 matchCommit=0]
+  DEBUG 1 [firstindex: 3, commit: 5] sent snapshot[index: 4, term: 1] to 3 [StateProbe match=0 next=1 sentCommit=0 matchCommit=0]
+  DEBUG 1 paused sending replication messages to 3 [StateSnapshot match=0 next=5 sentCommit=4 matchCommit=0 paused pendingSnap=4]
 > 1 handling Ready
   Ready MustSync=true:
   HardState Term:1 Vote:1 Commit:5 Lead:1 LeadEpoch:1
   CommittedEntries:
   1/5 EntryConfChangeV2
   Messages:
   1->2 MsgApp Term:1 Log:1/5 Commit:5
-  INFO 1 switched to configuration voters=(1 2)
+  1->3 MsgSnap Term:1 Log:0/0
+    Snapshot: Index:4 Term:1 ConfState:Voters:[1 2 3] VotersOutgoing:[1 2] Learners:[] LearnersNext:[] AutoLeave:true
+  INFO 1 switched to configuration voters=(1 2 3)
 > 2 receiving messages
   1->2 MsgApp Term:1 Log:1/5 Commit:5
+> 3 receiving messages
+  1->3 MsgSnap Term:1 Log:0/0
+    Snapshot: Index:4 Term:1 ConfState:Voters:[1 2 3] VotersOutgoing:[1 2] Learners:[] LearnersNext:[] AutoLeave:true
+  INFO log [committed=0, applied=0, applying=0, unstable.offset=1, unstable.offsetInProgress=1, len(unstable.Entries)=0] starts to restore snapshot [index: 4, term: 1]
+  INFO 3 switched to configuration voters=(1 2 3)&&(1 2) autoleave
+  INFO 3 [commit: 4, lastindex: 4, lastterm: 1] restored snapshot [index: 4, term: 1]
+  INFO 3 [commit: 4] restored snapshot [index: 4, term: 1]
 > 2 handling Ready
   Ready MustSync=true:
-  HardState Term:1 Commit:5 Lead:1 LeadEpoch:1
+  HardState Term:1 Vote:1 Commit:5 Lead:1 LeadEpoch:1
   CommittedEntries:
   1/5 EntryConfChangeV2
   Messages:
   2->1 MsgAppResp Term:1 Log:0/5 Commit:5
-  INFO 2 switched to configuration voters=(1 2)
+  INFO 2 switched to configuration voters=(1 2 3)
+> 3 handling Ready
+  Ready MustSync=true:
+  HardState Term:1 Commit:4 Lead:1 LeadEpoch:1
+  Snapshot Index:4 Term:1 ConfState:Voters:[1 2 3] VotersOutgoing:[1 2] Learners:[] LearnersNext:[] AutoLeave:true
+  Messages:
+  3->1 MsgAppResp Term:1 Log:0/4 Commit:4
 > 1 receiving messages
   2->1 MsgAppResp Term:1 Log:0/5 Commit:5
+  3->1 MsgAppResp Term:1 Log:0/4 Commit:4
+  DEBUG 1 recovered from needing snapshot, resumed sending replication messages to 3 [StateSnapshot match=4 next=5 sentCommit=4 matchCommit=4 paused pendingSnap=4]
+> 1 handling Ready
+  Ready MustSync=false:
+  Messages:
+  1->3 MsgApp Term:1 Log:1/4 Commit:5 Entries:[1/5 EntryConfChangeV2]
+> 3 receiving messages
+  1->3 MsgApp Term:1 Log:1/4 Commit:5 Entries:[1/5 EntryConfChangeV2]
+> 3 handling Ready
+  Ready MustSync=true:
+  HardState Term:1 Commit:5 Lead:1 LeadEpoch:1
+  Entries:
+  1/5 EntryConfChangeV2
+  CommittedEntries:
+  1/5 EntryConfChangeV2
+  Messages:
+  3->1 MsgAppResp Term:1 Log:0/5 Commit:5
+  INFO 3 switched to configuration voters=(1 2 3)
+> 1 receiving messages
+  3->1 MsgAppResp Term:1 Log:0/5 Commit:5

pkg/raft/testdata/fortification_checkquorum.txt

@@ -86,4 +86,3 @@ DEBUG 1 has not received messages from a quorum of peers in the last election ti
 DEBUG 1 does not have store liveness support from a quorum of peers
 WARN 1 stepped down to follower since quorum is not active
 INFO 1 became follower at term 1
-

pkg/raft/testdata/snapshot_succeed_via_app_resp.txt

@@ -98,7 +98,7 @@ Ready MustSync=false:
 Messages:
 1->3 MsgSnap Term:1 Log:0/0
   Snapshot: Index:11 Term:1 ConfState:Voters:[1 2 3] VotersOutgoing:[] Learners:[] LearnersNext:[] AutoLeave:false
-  
+
 status 1
 ----
 1: StateReplicate match=11 next=12 sentCommit=10 matchCommit=10