Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix memberList inconsistent output #11812

Merged
merged 4 commits into from
May 11, 2020

Conversation

tangcong
Copy link
Contributor

We also encountered issue #9975 #9535, this pr supports to print memberlist in hex format json and does not break backward compatibility.
It seems to be a simple problem, but it is not so elegant to solve. I prefer to implement Marshal MemberListResponse manually, rather than redefining a redundant structure to implement the member id Marshal method. @jingyih @gyuho

@tangcong
Copy link
Contributor Author

etcdctl member list

8211f1d0f64f3269, started, infra1, http://127.0.0.1:12380, http://127.0.0.1:2379, false
91bc3c398fb3c146, started, infra2, http://127.0.0.1:22380, http://127.0.0.1:22379, false
fd422379fda50e48, started, infra3, http://127.0.0.1:32380, http://127.0.0.1:32379, false

etcdctl member list -w=json --hex | python -m json.tool

{
    "header": {
        "cluster_id": "ef37ad9dc622a7c4",
        "member_id": "8211f1d0f64f3269",
        "raft_term": 7
    },
    "members": [
        {
            "ID": "8211f1d0f64f3269",
            "clientURLS": [
                "http://127.0.0.1:2379"
            ],
            "name": "infra1",
            "peerURLs": [
                "http://127.0.0.1:12380"
            ]
        },
        {
            "ID": "91bc3c398fb3c146",
            "clientURLS": [
                "http://127.0.0.1:22379"
            ],
            "name": "infra2",
            "peerURLs": [
                "http://127.0.0.1:22380"
            ]
        },
        {
            "ID": "fd422379fda50e48",
            "clientURLS": [
                "http://127.0.0.1:32379"
            ],
            "name": "infra3",
            "peerURLs": [
                "http://127.0.0.1:32380"
            ]
        }
    ]
}

var buffer bytes.Buffer
var b []byte
buffer.WriteString("{\"header\":{\"cluster_id\":\"")
b = strconv.AppendUint(nil, r.Header.ClusterId, 16)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tangcong

can you just redefine ClusterID and other IDs as a JSONHexID type? And define a json marshaller on it? So we do not need to reconstruct a full json encoder for the entire memberlistrepons.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@tangcong tangcong May 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks.i tried this implementation but memberlist response is a pb srtuct and we have to redefine a repeat struct, it looks not good. do you prefer this implementation?

@xiang90
Copy link
Contributor

xiang90 commented May 11, 2020

@tangcong The idea looks fine. But I suggest a better implementation.

@tangcong
Copy link
Contributor Author

if we redefine ClusterID and other IDs as a JSONHexID type, we have to redefine following pb generated structs. @xiang90

type MemberListResponse struct {
	Header *ResponseHeader `protobuf:"bytes,1,opt,name=header" json:"header,omitempty"`
	// members is a list of all members associated with the cluster.
	Members []*Member `protobuf:"bytes,2,rep,name=members" json:"members,omitempty"`
}

type ResponseHeader struct {
	// cluster_id is the ID of the cluster which sent the response.
	ClusterId uint64 `protobuf:"varint,1,opt,name=cluster_id,json=clusterId,proto3" json:"cluster_id,omitempty"`
	// member_id is the ID of the member which sent the response.
	MemberId uint64 `protobuf:"varint,2,opt,name=member_id,json=memberId,proto3" json:"member_id,omitempty"`
	// revision is the key-value store revision when the request was applied.
	// For watch progress responses, the header.revision indicates progress. All future events
	// recieved in this stream are guaranteed to have a higher revision number than the
	// header.revision number.
	Revision int64 `protobuf:"varint,3,opt,name=revision,proto3" json:"revision,omitempty"`
	// raft_term is the raft term when the request was applied.
	RaftTerm uint64 `protobuf:"varint,4,opt,name=raft_term,json=raftTerm,proto3" json:"raft_term,omitempty"`
}

type Member struct {
	// ID is the member ID for this member.
	ID uint64 `protobuf:"varint,1,opt,name=ID,proto3" json:"ID,omitempty"`
	// name is the human-readable name of the member. If the member is not started, the name will be an empty string.
	Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
	// peerURLs is the list of URLs the member exposes to the cluster for communication.
	PeerURLs []string `protobuf:"bytes,3,rep,name=peerURLs" json:"peerURLs,omitempty"`
	// clientURLs is the list of URLs the member exposes to clients for communication. If the member is not started, clientURLs will be empty.
	ClientURLs []string `protobuf:"bytes,4,rep,name=clientURLs" json:"clientURLs,omitempty"`
	// isLearner indicates if the member is raft learner.
	IsLearner bool `protobuf:"varint,5,opt,name=isLearner,proto3" json:"isLearner,omitempty"`
}

@xiang90
Copy link
Contributor

xiang90 commented May 11, 2020

@tangcong

yes. but i feel redefining the type is better than writing our own encoder, which is error-prone. alternatively, we can write tests to ensure the hex encoder is always correct.

@@ -110,6 +112,52 @@ func getMemberList(cx ctlCtx) (etcdserverpb.MemberListResponse, error) {
return resp, nil
}

func memberListWithHexTest(cx ctlCtx) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes,i added a test case to verify its correctness. @xiang90

@xiang90
Copy link
Contributor

xiang90 commented May 11, 2020

lgtm

@xiang90 xiang90 merged commit c667c14 into etcd-io:master May 11, 2020
@tangcong tangcong deleted the fix-inconsistent-output branch February 26, 2021 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants