Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gRPC Collector Does Not Support Configurable Max Message Size #3210

Closed
js8080 opened this issue Aug 16, 2021 · 2 comments · Fixed by #3214
Closed

gRPC Collector Does Not Support Configurable Max Message Size #3210

js8080 opened this issue Aug 16, 2021 · 2 comments · Fixed by #3214
Labels
enhancement help wanted Features that maintainers are willing to accept but do not have cycles to implement

Comments

@js8080
Copy link
Contributor

js8080 commented Aug 16, 2021

Describe the bug
The Jaeger gRPC Collector Service has a default max receive message size of 4mb and does not support overriding this max size even though gRPC allows for it.

Source from cmd/collector/app/server/server.go:

var defaultServerOptions = serverOptions{
	maxReceiveMessageSize: defaultServerMaxReceiveMessageSize,
	maxSendMessageSize:    defaultServerMaxSendMessageSize,
	connectionTimeout:     120 * time.Second,
	writeBufferSize:       defaultWriteBufSize,
	readBufferSize:        defaultReadBufSize,
}

Where defaultServerMaxReceiveMessageSize is defined as:

const (
	defaultServerMaxReceiveMessageSize = 1024 * 1024 * 4

There is a method available to set the max receive message size also but it is not called and has no configuration options associated with it:

// MaxRecvMsgSize returns a ServerOption to set the max message size in bytes the server can receive.
// If this is not set, gRPC uses the default 4MB.
func MaxRecvMsgSize(m int) ServerOption {
	return newFuncServerOption(func(o *serverOptions) {
		o.maxReceiveMessageSize = m
	})
}

To Reproduce
Steps to reproduce the behavior:
See my initial question on this in the Discussion section: gRPC Collector Configurable Max Message Size #3200

Expected behavior
Some users of Jaeger (such as myself) may want the ability to log larger Spans and since gRPC is flexible and allows a larger message size limit, we should expose a setting to allow overriding the default max message size.

For example, a new CLI option for the Collector:

collector.grpc-max-receive-message-length

To change this behavior, we basically just need to add a new Collector CLI option as mentioned and then add the following code to cmd/collector/app/server/grpc.go -> StartGRPCServer:

// StartGRPCServer based on the given parameters
func StartGRPCServer(params *GRPCServerParams) (*grpc.Server, error) {
	var server *grpc.Server
	var grpcOpts []grpc.ServerOption

        // New - set max receive message size via new params property populated via CLI option
	grpcOpts = append(grpcOpts, grpc.MaxRecvMsgSize(params.MaxReceiveMessageLength))

	if params.TLSConfig.Enabled {
		// user requested a server with TLS, setup creds
		tlsCfg, err := params.TLSConfig.Config(params.Logger)
		if err != nil {
			return nil, err
		}

We'd also have to update the GRPCServerParams type and some other code as well but that's the most important part of it.

Version (please complete the following information):

  • OS: Linux
  • Jaeger version: Jaeger v1.21.0
  • Deployment: Jaeger deployed via Kubernetes but traced application is running on bare metal (Linux via WSL)

What troubleshooting steps did you try?
See my initial question on this in the Discussion section: gRPC Collector Configurable Max Message Size #3200

@js8080 js8080 added the bug label Aug 16, 2021
@yurishkuro yurishkuro added enhancement help wanted Features that maintainers are willing to accept but do not have cycles to implement and removed bug labels Aug 16, 2021
@js8080
Copy link
Contributor Author

js8080 commented Aug 16, 2021

@yurishkuro I tested a change to this with a local Jaeger build. Would you be interested in me contributing this change?

@yurishkuro
Copy link
Member

please do

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement help wanted Features that maintainers are willing to accept but do not have cycles to implement
Projects
None yet
2 participants