Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

delete shadow copy proto-struct #4962

Closed
wants to merge 1 commit into from

Conversation

pfi79
Copy link
Contributor

@pfi79 pfi79 commented Aug 23, 2024

The new fabric-protos-go-apiv2 has copy protection in the structures.
This code is adapted for the transition to the new protos-go-apiv2.

@pfi79 pfi79 requested a review from a team as a code owner August 23, 2024 18:51
@pfi79 pfi79 force-pushed the del-shallow-copy branch 2 times, most recently from dd9b758 to 099cd85 Compare August 24, 2024 10:48
@pfi79 pfi79 marked this pull request as draft August 24, 2024 11:37
@pfi79 pfi79 force-pushed the del-shallow-copy branch from 099cd85 to 1874966 Compare August 25, 2024 19:46
@pfi79 pfi79 marked this pull request as ready for review August 25, 2024 20:14
@denyeart
Copy link
Contributor

There is an open issue discussing potential update to protos-go-apiv2. Due to some concerns that were raised in that issue, my latest thinking was to not proceed with the update. Let's continue the discussion in that issue prior to moving forward with the PRs to update to protos-go-apiv2.

@pfi79
Copy link
Contributor Author

pfi79 commented Aug 26, 2024

There is an open issue discussing potential update to protos-go-apiv2. Due to some concerns that were raised in that issue, my latest thinking was to not proceed with the update. Let's continue the discussion in that issue prior to moving forward with the PRs to update to protos-go-apiv2.

Yes, it's a move towards protos-go-apiv2. But independent of it. No new protos, no new protobuf.

I specifically highlighted this code, which is independent of protos-go-apiv2.

I would suggest that it should be considered independent.

@denyeart
Copy link
Contributor

There is an open issue discussing potential update to protos-go-apiv2. Due to some concerns that were raised in that issue, my latest thinking was to not proceed with the update. Let's continue the discussion in that issue prior to moving forward with the PRs to update to protos-go-apiv2.

Yes, it's a move towards protos-go-apiv2. But independent of it. No new protos, no new protobuf.

I specifically highlighted this code, which is independent of protos-go-apiv2.

I would suggest that it should be considered independent.

Yes, it can be considered independent. But in that case the PR description should mention the rationale for the various changes. For example adding proto.Clone() in various places seems to add a dependency where none was required previously. Can you describe the benefits? And some of the other changes seem not related, ideally they would be done in a separate pull request or at least mention the rationale for the changes in the PR description.

@pfi79
Copy link
Contributor Author

pfi79 commented Aug 26, 2024

Yes, it can be considered independent. But in that case the PR description should mention the rationale for the various changes. For example adding proto.Clone() in various places seems to add a dependency where none was required previously. Can you describe the benefits? And some of the other changes seem not related, ideally they would be done in a separate pull request or at least mention the rationale for the changes in the PR description.

I think the developers of the protobuf package added shadow copy protection to the new structures for a reason.
It means that they had good reasons for it.
And if we can't switch to a new version of protobuf yet, we can at least remove shadow copying in them. And in those places where copying is required make it more explicit for further code maintenance.

But if you don't consider this an important reason, let's wait until I prepare a version with all the changes and present it in the corresponding issue.

@bestbeforetoday
Copy link
Member

See this comment describing why it is not safe to make shallow copies of Go protobuf messages, which explains why they are referred to in the protobuf API only as pointers (proto.Message interface):

golang/protobuf#1225 (comment)

@@ -2550,7 +2550,7 @@ var _ = Describe("Handler", func() {
})

It("sends an execute message to the chaincode with the correct proposal", func() {
expectedMessage := *incomingMessage
expectedMessage := proto.Clone(incomingMessage).(*pb.ChaincodeMessage)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the answer here is just to reference the protobuf message as a pointer rather than dereferencing it and causing a copy-by-value. I'm not sure there is any need to clone the message

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix

@@ -2559,7 +2559,7 @@ var _ = Describe("Handler", func() {
Eventually(fakeChatStream.SendCallCount).Should(Equal(1))
Consistently(fakeChatStream.SendCallCount).Should(Equal(1))
msg := fakeChatStream.SendArgsForCall(0)
Expect(msg).To(Equal(&expectedMessage))
Expect(msg).To(Equal(expectedMessage))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The standard Gomega Equal matcher is not safe for use on protobuf messages at all. It simply does a deep equal comparison, and this is not valid for protobuf messages since they can be logically equal while also having some implementation fields with different values.

Some other parts of the codebase use proto.Equal for protobuf message comparison, and then use Gomega to Expect a True return value. I think a better approach is to implement a matcher specifically for protobuf messages, for example:

func ProtoEqual(expected proto.Message) types.GomegaMatcher {
	return &protoEqualMatcher{
		expected: expected,
	}
}

type protoEqualMatcher struct {
	expected proto.Message
}

func (m *protoEqualMatcher) Match(actual interface{}) (bool, error) {
	actualMessage, ok := actual.(proto.Message)
	if !ok {
		return false, fmt.Errorf("ProtoEqual expects a proto.Message, got a %T", actual)
	}

	return proto.Equal(m.expected, actualMessage), nil
}

func (m *protoEqualMatcher) FailureMessage(actual interface{}) string {
	if message, ok := actual.(proto.Message); ok {
		return fmt.Sprintf("Expected\n\t%s\nto equal\n\t%s", message, m.expected)
	}

	return fmt.Sprintf("Expected\n\t%#v\nto equal\n\t%s", actual, m.expected)
}

func (m *protoEqualMatcher) NegatedFailureMessage(actual interface{}) string {
	if message, ok := actual.(proto.Message); ok {
		return fmt.Sprintf("Expected\n\t%s\nnot to equal\n\t%s", message, m.expected)
	}

	return fmt.Sprintf("Expected\n\t%#v\nnot to equal\n\t%s", actual, m.expected)
}

This is then used as follows:

Expect(actualMsg).To(ProtoEqual(expectedMsg))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix

Signed-off-by: Fedor Partanskiy <[email protected]>
@pfi79 pfi79 force-pushed the del-shallow-copy branch from 1874966 to 6e053d1 Compare August 27, 2024 14:19
@pfi79 pfi79 closed this Sep 13, 2024
@pfi79 pfi79 deleted the del-shallow-copy branch September 14, 2024 04:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants