Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: v2store deprecation: Prepare to use publishV3 instead of publish V2. #12804

Merged
merged 3 commits into from
Apr 8, 2021

Conversation

ptabor
Copy link
Contributor

@ptabor ptabor commented Mar 26, 2021

Part of: #12800

Had to fix:

  • etcdctl backup command such that it scrubs additionally 'cluster_member_attr_set' commands from the WAL log (apart of other membership-driven-commands):

Without the fix, the process used to fail with:

./build.sh && (cd tests && EXPECT_DEBUG=true 'env' 'go' 'test' '-timeout=30m' '--count=1' 'go.etcd.io/etcd/tests/v3/e2e' --run TestCtlV2BackupV3$ -v)
{"level":"panic","ts":"2021-03-26T11:11:08.794+0100","caller":"membership/cluster.go:445","msg":"failed to update; member unknown","cluster-id":"786e04252702","local-member-id":"786e04252701","unknown-remote-peer-id":"ca50e9357181d758","stacktrace":"go.etcd.io/etcd/server/v3/etcdserver/api/membership.(*RaftCluster).UpdateAttributes\n\t/home/ptab/corp/etcd/server/etcdserver/api/membership/cluster.go:445\ngo.etcd.io/etcd/server/v3/etcdserver.(*applierV3backend).ClusterMemberAttrSet\n\t/home/ptab/corp/etcd/server/etcdserver/apply.go:911\ngo.etcd.io/etcd/server/v3/etcdserver.(*applierV3backend).Apply\n\t/home/ptab/corp/etcd/server/etcdserver/apply.go:229\ngo.etcd.io/etcd/server/v3/etcdserver.(*authApplierV3).Apply\n\t/home/ptab/corp/etcd/server/etcdserver/apply_auth.go:60\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).applyEntryNormal\n\t/home/ptab/corp/etcd/server/etcdserver/server.go:2111\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).apply\n\t/home/ptab/corp/etcd/server/etcdserver/server.go:2024\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).applyEntries\n\t/home/ptab/corp/etcd/server/etcdserver/server.go:1279\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).applyAll\n\t/home/ptab/corp/etcd/server/etcdserver/server.go:1105\ngo.etcd.io/etcd/server/v3/etcdserver.(*EtcdServer).run.func8\n\t/home/ptab/corp/etcd/server/etcdserver/server.go:1059\ngo.etcd.io/etcd/pkg/v3/schedule.(*fifo).run\n\t/home/ptab/corp/etcd/pkg/schedule/schedule.go:157"}
        panic: failed to update; member unknown
        
        goroutine 162 [running]:
        go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0003ee300, 0xc0003ee540, 0x3, 0x3)
        	/home/ptab/private/golang/pkg/mod/go.uber.org/[email protected]/zapcore/entry.go:234 +0x58d
        go.uber.org/zap.(*Logger).Panic(0xc00007c300, 0x118827e, 0x20, 0xc0003ee540, 0x3, 0x3)
        	/home/ptab/private/golang/pkg/mod/go.uber.org/[email protected]/logger.go:226 +0x85
        go.etcd.io/etcd/server/v3/etcdserver/api/membership.(*RaftCluster).UpdateAttributes(0xc0004d0000, 0xca50e9357181d758, 0xc0003c2258, 0x18, 0xc000230080, 0x1, 0x1)

@ptabor ptabor marked this pull request as draft March 26, 2021 21:04
@ptabor ptabor marked this pull request as ready for review March 26, 2021 21:42
@ptabor ptabor force-pushed the 20210326-v3-publish branch 2 times, most recently from c6e88d1 to b6a21ff Compare March 27, 2021 21:31
@ptabor ptabor marked this pull request as draft March 28, 2021 08:40
@ptabor ptabor changed the title server: v2store deprecation: Use publishV3 instead of publish V2. server: v2store deprecation: Prepare to use publishV3 instead of publish V2. Mar 28, 2021
@ptabor ptabor marked this pull request as ready for review March 28, 2021 19:36
@ptabor ptabor requested a review from hexfusion March 30, 2021 10:52
@ptabor ptabor added this to the etcd-v3.5 milestone Mar 30, 2021
@ptabor ptabor mentioned this pull request Apr 2, 2021
5 tasks
Copy link
Contributor

@jingyih jingyih left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -171,6 +171,12 @@ func loadWAL(srcWAL string, walsnap walpb.Snapshot, v3 bool) (etcdserverpb.Metad
continue
}

if raftReq.ClusterMemberAttrSet != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This was overlooked.

@codecov-io
Copy link

Codecov Report

Merging #12804 (cc0f812) into master (e24e72c) will decrease coverage by 19.56%.
The diff coverage is 12.50%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master   #12804       +/-   ##
===========================================
- Coverage   65.14%   45.58%   -19.57%     
===========================================
  Files         421      401       -20     
  Lines       33024    33106       +82     
===========================================
- Hits        21514    15090     -6424     
- Misses       9415    16027     +6612     
+ Partials     2095     1989      -106     
Flag Coverage Δ
all 45.58% <12.50%> (-19.57%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
client/v2/discover.go 0.00% <ø> (ø)
client/v2/keys.go 44.31% <ø> (-20.46%) ⬇️
client/v2/members.go 16.98% <ø> (-26.42%) ⬇️
client/v3/cluster.go 20.00% <ø> (-70.00%) ⬇️
client/v3/logger.go 38.09% <ø> (ø)
client/v3/snapshot/v3_snapshot.go 0.00% <ø> (-51.22%) ⬇️
etcdctl/ctlv2/command/backup_command.go 64.13% <0.00%> (-1.82%) ⬇️
etcdctl/ctlv2/command/role_commands.go 56.49% <ø> (ø)
etcdctl/ctlv2/command/util.go 42.76% <ø> (+2.51%) ⬆️
etcdctl/ctlv3/command/global.go 54.50% <ø> (ø)
... and 311 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e24e72c...cc0f812. Read the comment docs.

@ptabor ptabor merged commit 63c25bf into etcd-io:master Apr 8, 2021
@ptabor ptabor deleted the 20210326-v3-publish branch April 8, 2021 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants