-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
protoc-gen-go: fully generate marshaling and unmarshaling code #280
Comments
I really think the amount of speed gained is significant enough to warrant the extra code generation for marshal and unmarshal. Here is an independant benchmark
Also once you have those methods, the extra generated code required to not use the proto package is very little. |
fyi /cc @randall77 @cherrymui @dsnet |
This surely cannot be news? But if it is, this is gogoprotobuf, it does code generation (and other gadgets) for protobuffers for go, since 2013. It has users, including Kubernetes |
One of the newer things you can do with gogoprotobuf.
Maybe this can give you an idea of what generating code in golang/protobuf would be like. |
Bump, this discussion should be revisited, as it still has no satisfying conclusion. Recent discussion implies this still hasn't been closed as a possibility. What are the current thoughts on this? |
Wanted update this thread. @neild and I have talked on/off about this in the past. One of things we'd like to explore first before addressing this is to figure out whether to provide a streaming or scatter/gather API (see #609). One significant detriment with generated code is that the implementation easily grows stale unless users regenerate the code (which rarely occurs as has been observed). Though it's not a hard ordering requirement, we would presumably want to know if we're going to go anywhere with streaming or scatter/gather APIs before doing generated serialization as the former would likely affect the internal APIs and implementation of the latter. |
planetscale/vtprotobuf Can be used to generate these methods. I think it's best if it is optional and don't mind it living in a separate repo (just like grpc). Because a majority will not significantly benefit from the extra speed and most likely will grow to need this rather than need it from the on set. One important requirement for tools like vtprotobuf to work more seamlessly is to support custom marshal/unmarshal interfaces similar to Eg: type Marshaler interface {
MarshalPROTO() ([]byte, error)
}
func Marshal(v Message) ([]byte, error) {
if m, ok := v.(Marshaler); ok {
return m.MarshalPROTO()
}
// Current implementation
} Similarly for unmarshal. |
I've just finished optimising an application as much as I feasibly can for the time being, and I'm at a point now where just over half of the CPU time when under a load is dedicated solely to unmarshalling ProtoBuf messages. Historically, we did use gogoproto, but of course it's now deprecated, so we now use the most up-to-date official method. I'm really keen to see this at least become an option. The difference, as has been illustrated in the first comments here, are huge. In my current use-case I'd see a pretty significant speed boost! |
I am not sure why they are hesitating to implement this. This clearly has benefits and there is unnecessary fragmentation in Go protobuf ecosystem because of this. |
The Conversation started in #276
The text was updated successfully, but these errors were encountered: