APIv2: generated messages are incompatible with current v1 package

[neild]

The APIv2 protoc-gen-go is generating code that doesn't work with the current APIv1 proto package, failing with a panic at runtime when passed to a v1 function:

panic: protobuf tag not enough fields in Message.state: 

We will have an updated APIv1 package that's compatible with these messages, but that doesn't help the case when someone is using the older package. It's okay for new generated code to require newer package versions, but compatibility failures need to be detected at compile time (or, worst case, init time), not runtime.

[puellanivis]

😢 I’m pretty sure that this is going to be a sticky issue indefinitely. Some project will always end up with a stale version of the APIv1 package, and end up complaining that we broke compatibility or something like that.

[dsnet]

https://go-review.googlesource.com/c/protobuf/+/218939 adds a compile-time assertion on a future v1.4 of github.com/golang/protobuf.

We'll remove this assertion after some period of time, but we think it's probably necessary for the initial release.

[stevvooe]

We are seeing a similar problem when using protobuild against the new package:

goroutine 1 [running]:
github.com/gogo/protobuf/proto.(*unmarshalInfo).computeUnmarshalInfo(0xc000090820)
        /go/src/github.com/gogo/protobuf/proto/table_unmarshal.go:341 +0x17fb
github.com/gogo/protobuf/proto.(*unmarshalInfo).unmarshal(0xc000090820, 0xc00008a5c0, 0xc000320000, 0x4752, 0x4952, 0x40c338, 0x20)
        /go/src/github.com/gogo/protobuf/proto/table_unmarshal.go:138 +0xe13
github.com/gogo/protobuf/proto.(*InternalMessageInfo).Unmarshal(0xc000077940, 0x7b0780, 0xc00008a5c0, 0xc000320000, 0x4752, 0x4952, 0xc00008a501, 0x0)
        /go/src/github.com/gogo/protobuf/proto/table_unmarshal.go:63 +0x66
github.com/gogo/protobuf/proto.(*Buffer).Unmarshal(0xc00010b440, 0x7b0780, 0xc00008a5c0, 0x0, 0x0)
        /go/src/github.com/gogo/protobuf/proto/decode.go:424 +0x1ec
github.com/gogo/protobuf/proto.Unmarshal(0xc000320000, 0x4752, 0x4952, 0x7b0780, 0xc00008a5c0, 0x0, 0x0)
        /go/src/github.com/gogo/protobuf/proto/decode.go:342 +0x170
main.readDesc(0xc000083300, 0x1d, 0x0, 0x0, 0x0)
        /go/src/github.com/stevvooe/protobuild/descriptors.go:96 +0x9e
main.main()
        /go/src/github.com/stevvooe/protobuild/main.go:218 +0x1819
make: *** [Makefile:143: protos] Error 2

What exactly is the remediation here?

[dsnet]

This is a problem to file against gogo/protobuf. They have a forked version of the proto runtime, so there is nothing we can do to remedy this.

[stevvooe]

Ok, thanks!

[stevvooe]

gogo/protobuf#678 for tracking.

[stevvooe]

@dsnet Actually, looking back at this issue, I think this is a compatibility issue with the new golang/protobuf changes. This is causing a major number of changes required across the container community. Basically, we're all stuck on v1.3 of golang/protobuf because any project using gogo is stuck.

I've noticed quite a few changes in this project that might be making it very hard for users to move forward on this. Is there anything we can do to fix this? I have tens of projects that have protobuf (both golang and gogo) as a dependency with no way forward due to these breaks. On the other side of this issue, the upstream changes aren't really addressing any major problems we've had in the past that would warrant this level of work.