-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
protobuf: remove gogoproto #5342
Conversation
e57b510
to
665e7c3
Compare
bf597a3
to
49259d7
Compare
6b9629f
to
4eccb90
Compare
Separating the vtproto component of this into a separate PR. |
e611d94
to
2e2ff21
Compare
We did some investigation of memory usage when transferring a large context and found some interesting stuff. The big thing we found is that a certain optimization utilized here does nothing and doesn't help with memory usage. This is because of the default grpc codec and is seen across different implementations. The default codec will always call This behavior is consistent with the standard go implementation which does the same thing, but it seems the standard implementation benefits a little bit from utilizing a v2 version of the codec that incorporates a memory pool. Still, it allocates a lot and that dominates the performance. vtproto also does the same thing if you do the same thing, but vtproto can generate its own reset function that's useful for memory pools and we can also customize the codec to remove the reset if that makes sense. Either way, I haven't seen a regression in the performance so far but I think we can like do much more with vtproto if we dive into things more thoroughly. |
2e2ff21
to
ee0d372
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great. Finally! Looks good to me change-wise. You still need to address the test failures though.
I'll close my PR.
go.mod
Outdated
gopkg.in/yaml.v3 v3.0.1 // indirect | ||
kernel.org/pub/linux/libs/security/libcap/psx v1.2.70 // indirect | ||
) | ||
|
||
replace github.com/tonistiigi/fsutil => github.com/jsternberg/fsutil v0.0.0-20240925215903-a3400686ae50 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this replace can be removed and just need to update github.com/tonistiigi/fsutil
now that tonistiigi/fsutil#206 is merged?
Thanks for laying all the groundwork for this @kzys -- happy it was finally addressed. |
ee0d372
to
ac79d25
Compare
abeb9a9
to
2142c51
Compare
2142c51
to
55e77e2
Compare
The benchmarks that are added are related to certain areas that are considered hotspots for performance where many messages are marshaled. Signed-off-by: Jonathan A. Sternberg <[email protected]>
Remove gogoproto in favor of the standard protobuf compiler. This removes any nonstandard extensions that were part of gogoproto such as the custom types. Signed-off-by: Jonathan A. Sternberg <[email protected]>
55e77e2
to
1a3fc0a
Compare
Fixes #3119
Remove gogoproto in favor of the standard protobuf compiler. This
removes any nonstandard extensions that were part of gogoproto such as
the custom types.
Re-spin of #4422