[3.5] Add online & offline commands to check whether the v2store contains custom content

[ahrtr]

What would you like to be added?

Part of #12913

etcd 3.6 will panic if the v2store contains custom content. Before users upgrade from 3.5 to 3.6, they must ensure there is no any custom content in v2store; and both the online and offline commands need to check it. For the offline command, it also needs to check the WAL records after the latest v2 snapshot file.

The proposed commands:

etcdctl check v2store
etcdutl check v2store

EDIT: We only need to add the offline tool. For online tool, Users should be able to use the existing etcdctl v2 command ETCDCTL_API=2 ./bin/etcdctl ls directly, refer to #19114 (comment)

Also --enable-v2 and --experimental-enable-v2v3 have already been removed i 3.6, so users must ensure that they do not configure these two flags before upgrading to 3.6. Only the online command checks it.

Why is this needed?

To ensure smooth upgrade from 3.5 to 3.6.

[ahrtr]

For the command-line flags, let's take care of it in #18999. This issue focuses on checking the v2store.

[serathius]

Would it be better to integrate it into v3.6? Before starting etcd v3.6 we can scan for v2state and abort startup. Would prefer to minimize the number of steps needed to upgrade etcd. Upgrade should be safe by default.

[ahrtr]

Would it be better to integrate it into v3.6?

This tool is only targeted for 3.5. Users need to check the v2store before upgrading to 3.6 instead of after. But users don't have to run this tool if they have already removed or disabled the --enable-v2 for a long time.

In 3.6, etcd already has the function to check the v2store and will fail the start if v2store has custom content.

etcd/server/etcdserver/bootstrap.go

Lines 407 to 409 in fbac2a0

	if err = serverstorage.AssertNoV2StoreContent(cfg.Logger, st, cfg.V2Deprecation); err != nil {
	cfg.Logger.Error("illegal v2store content", zap.Error(err))
	return nil, be, err

[ahrtr]

@fuweid any progress on this feature?

[fuweid]

any progress on this feature?

@ahrtr I'm still working on it. Will file pull request once it's ready.