protoc-gen-go: support generating standard Go types in place of well-known-types

[jhump]

This is quite similar in spirit to protocolbuffers/protobuf#2055 (which is for the Java protobuf codegen and runtime).

Protos would be easier to integrate with Go libraries that rely on struct reflection (serialization, ORM, etc) if struct fields used standard Go types in place of some of the well-known types:

• google.protobuf.Timestamp -> time.Time
• google.protobuf.Duration -> time.Duration
• google.protobuf.Int32Value -> *int32
• google.protobuf.Int64Value -> *int64
• google.protobuf.UInt32Value -> *uint32
• google.protobuf.UInt64Value -> *uint64
• google.protobuf.FloatValue -> *float32
• google.protobuf.DoubleValue -> *float64
• google.protobuf.BoolValue -> *bool
• google.protobuf.StringValue -> *string
• google.protobuf.BytesValue -> []byte

(For the lattermost one, we could distinguish a missing/default value from a BytesValue message that contained an empty Value field by assuming nil means absent and otherwise empty slice is a message with an empty value.)

Other solutions we've had to use to work around this (mostly the timestamp and duration ones):

• Creating a parallel struct that mirrors the generated proto, but with the above type swaps. Implement adapter methods to translate between the proto and the parallel type. While this can be done via codegen, we find ourselves doing it by hand more often than not.
• If the library allows customized representation (via a Marshaler or Saver sort of interface), write a reflection-based library that will recursively apply the transformations to a proto when generating the marshaled/saved form.
• Use JSON as a go-between. The JSON format for well-known-types matches the type expectations pretty well, so marshalling the proto to JSON and then to map[string]interface{} often does the trick. Timestamps are not converted correctly, but for libraries that are applying the data to a schema, they can typically handle parsing the string, without any extra effort, for fields where a timestamp value is expected.

Since it would impact compatibility, it could be an opt-in flag that changes the output of codegen. The runtime library could probably be updated to accommodate without any compatibility issues (e.g. could just examine the type of a field to make decisions or, if it's more efficient, could encode more bits into the struct tag so that the library knows what to do without extra reflection steps).

[dsnet]

Just wanted to note that it would have to be *time.Duration in order to preserve the fact that proto has a distinction between the zero duration and an unset duration.

[sanjaypsmc]

Hi, is there any update on this? Are there third-party options? If I wanted to implement it myself, what would the level of effort look like and would that necessitate a fork? Since this has been outstanding for nearly a year, it'd be great to get a comment from a maintainer on how much of a priority it is and whether there's a timeline in place. Thanks!

[dsnet]

This proposal is on hold until reflection work is finished (see #364) as generating native Go types will need to inter-operate with proto reflection in a reasonable way.

[johanbrandhorst]

@sanjaypsmc gogo/protobuf has supported timestamp and duration for a while, but using gogo over golang/protobuf is not a choice without other consequences. At this point you might just want to wait until the work in #364 has been completed.

[nwparker]

Hey, now that work on #364 is complete, is it reasonable to see work started back on this item? Or are there additional blockers?

[tgulacsi]

This would be very nice!
Till that, can we add database/sql.Scanner, database/sql/driver.Valuer, proper MarshalJSON, UnmarshalJSON, MarshalText, UnmarshalText to these WKTs?
It's only a handful of lines (github.com/UNO-SOFT/knownpb/timestamppb for example), and would alleviate the grudge induced by sed-ing the generated files' imports (https://github.com/UNO-SOFT/knownpb/blob/main/replace_timestamp.sh)...

Shall I open a new issue for this?

[baryluk]

Is there any recent work on this? Most interested in Duration and Time

[james-lawrence]

this is a very much needed improvement. the way golang protobufs handles these types is incredibly painful for interop.

[hariso]

+1 from me as well. I'm interested in time.Time. Are there any plans to support slices and maps as well?

[mesmerx]

+1 to not let it die, time.Time will be a world savior for all my work, with it i can set all database schema with protos