grpc: Add a pointer of server to ctx passed into stats handler #6750

zasweq · 2023-10-25T22:12:40Z

This PR adds a pointer to the grpc.Server into the context passed into the stats handler. It also adds an internal only helper on the server that returns whether a given method is registered or not. This will be used in OpenTelemetry instrumentation for security purposes with respect to the cardinality of the method attribute. The next step after these two PRs is to move the rest of the stats handler callouts to the gRPC layer, get registered methods plumbed client side, canoncalize Target() without breaking it and then the OpenTelemetry instrumentation itself :).

RELEASE NOTES: N/A

codecov · 2023-10-25T22:18:00Z

Codecov Report

Merging #6750 (646d55b) into master (c76d75f) will decrease coverage by 0.05%.
The diff coverage is 89.65%.

Additional details and impacted files

dfawley · 2023-10-25T22:34:38Z

internal/internal.go

@@ -73,6 +73,11 @@ var (
 	// xDS-enabled server invokes this method on a grpc.Server when a particular
 	// listener moves to "not-serving" mode.
 	DrainServerTransports any // func(*grpc.Server, string)
+	// IsRegisteredMethod returns whether the passed in method is registered as
+	// a method on the server.
+	IsRegisteredMethod any // func(*grpc.Server, string)


This returns a bool right?

Ah, right, whoops. Added.

dfawley · 2023-10-25T22:36:34Z

internal/stubstatshandler/stubstatshandler.go

+	// Guarantees we satisfy this interface; panics if unimplemented methods are
+	// called.
+	testgrpc.TestServiceServer


Why are you bundling a stats handler with a service implementation? That seems undesirable unless it's somehow important to keep the two things together. (And now that I've look at the rest of the code, it seems unused and unnecessary.)

Also, we should put this under testutils (either in that package or a subdirectory of it).

Ah, whoops, this is leftover from the stubserver copy paste. Changed to embedding a stats.Handler.

Moved file to within testutils as internal/testutils/stubstatshandler.go.

dfawley · 2023-10-25T22:39:30Z

server.go

+// isRegisteredMethod returns whether the passed in method is registered as a
+// method on the server. /service/method and service/method will match if the
+// service and method are registered on the server.
+func (s *Server) isRegisteredMethod(serviceMethod string) bool {


Isn't this very similar to the code that looks up the handler for a method? Can we reuse code somehow, or even just call it?

Yes, it is very similar, but I chose not to do it because this line is very specific: https://github.com/grpc/grpc-go/blob/master/server.go#L1731, and I thought it would be hard to generalize what's shared into a helper. If you feel strongly about this after reading the handleStream() code, I can try and pull out logic into helper.

Yeah please see what you can do. I think a method that returns (service string, method string, ok bool) would be good.

I tried writing it, and it seems distinct enough logically to not warrant a helper. Your function signature doesn't distinguish between a. malformed method name (important here for tracing and logging: https://github.com/grpc/grpc-go/blob/master/server.go#L1732) b. registered unary rpc c. registered streaming name (important here: https://github.com/grpc/grpc-go/blob/master/server.go#L1770, to chose to go through processUnary or processStreaming). This helper's scope is not to log anything in traces or channelz, but to parse a :method header received from the wire and determine whether it's registered or not registered. I think it's sufficiently different to not warrant a new helper.

dfawley · 2023-10-25T22:43:49Z

stats/stats_test.go

+		UnaryCallF: func(ctx context.Context, in *testpb.SimpleRequest) (*testpb.SimpleResponse, error) {
+			return &testpb.SimpleResponse{}, nil
+		},
+		FullDuplexCallF: func(stream testpb.TestService_FullDuplexCallServer) error {


You never call this; either delete it or test it please.

I do test this. See line 1493. However, you are right that I never initiate a bidirectional streaming RPC from the test. Would you like me to add that?

This closure being set or not doesn't change the test at line 1494. The method is registered because it's in the service descriptor. I'm fine with deleting this; I don't think there's any need to test both streaming and unary RPCs unless there are different code paths that lead to setting the server in the context, which there aren't.

Ah, right, these closures != service descriptor, it's all there anyway. Deleting.

dfawley · 2023-10-25T22:46:09Z

stats/stats_test.go

+			isRegisteredMethod := internal.IsRegisteredMethod.(func(*grpc.Server, string) bool)
+			// /s/m and s/m are valid.
+			if !isRegisteredMethod(server, "/grpc.testing.TestService/UnaryCall") {
+				errorCh.Send(errors.New("UnaryCall should be a registered method according to server"))


You don't actually need the error channel for this. You can just do t.Errorf here. Then all you need to do is ensure the code actually was called. You can do this by closing a done, channel or with a waitgroup, etc. One benefit of this is that you'll see all the test case failures in a single run if multiple of these checks don't work.

Oh, right, this was when it was a defined stats handler rather than a stub with methods declared in test. Thus, before I couldn't close on t *testing.T, and now I can :). Switched.

Chose a waitGroup.

dfawley · 2023-10-26T15:02:14Z

internal/testutils/stubstatshandler.go

 	// Guarantees we satisfy this interface; panics if unimplemented methods are
 	// called.
-	testgrpc.TestServiceServer
+	stats.Handler


If we're fully implementing it (which we must be given the earlier snapshot) then this is unnecessary; just delete it. If you want to have a paranoid check that we're implementing it, you can add a var _ stats.Handler = (*StubStatsHandler)(nil). But that's not strictly necessary.

Ah ok deleted.

…6750)

Rebase onto master

520ebcb

zasweq added the Type: Feature New features or improvements in behavior label Oct 25, 2023

zasweq added this to the 1.60 Release milestone Oct 25, 2023

zasweq requested a review from dfawley October 25, 2023 22:12

zasweq assigned dfawley Oct 25, 2023

zasweq mentioned this pull request Oct 25, 2023

grpc: Add a pointer of server to ctx passed into stats handler #6724

Closed

dfawley reviewed Oct 25, 2023

View reviewed changes

dfawley assigned zasweq and unassigned dfawley Oct 25, 2023

Responded to Doug's comments

b6254ef

zasweq assigned dfawley and unassigned zasweq Oct 25, 2023

Responded to Doug's comment

65047b6

dfawley approved these changes Oct 26, 2023

View reviewed changes

dfawley assigned zasweq and unassigned dfawley Oct 26, 2023

Responded to Doug's comment

646d55b

zasweq merged commit 8cb9846 into grpc:master Oct 26, 2023
1 check passed

arvindbr8 pushed a commit to arvindbr8/grpc-go that referenced this pull request Nov 7, 2023

grpc: Add a pointer of server to ctx passed into stats handler (grpc#…

9c42dca

…6750)

This was referenced Jan 17, 2024

Collector crashing with invalid memory address or nil pointer dereference on collector exposing a GRPC endpoint behind an AWS ALB open-telemetry/opentelemetry-collector#9296

Closed

stats.Handler's HandleRPC is called with invalid context #6928

Closed

github-actions bot locked as resolved and limited conversation to collaborators Apr 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grpc: Add a pointer of server to ctx passed into stats handler #6750

grpc: Add a pointer of server to ctx passed into stats handler #6750

zasweq commented Oct 25, 2023

codecov bot commented Oct 25, 2023 •

edited

Loading

dfawley Oct 25, 2023

zasweq Oct 25, 2023

dfawley Oct 25, 2023

zasweq Oct 25, 2023

zasweq Oct 25, 2023

dfawley Oct 25, 2023

zasweq Oct 25, 2023

dfawley Oct 26, 2023

zasweq Oct 26, 2023

dfawley Oct 25, 2023

zasweq Oct 25, 2023

dfawley Oct 26, 2023

zasweq Oct 26, 2023

dfawley Oct 25, 2023

zasweq Oct 25, 2023

zasweq Oct 25, 2023

dfawley Oct 26, 2023

zasweq Oct 26, 2023

grpc: Add a pointer of server to ctx passed into stats handler #6750

grpc: Add a pointer of server to ctx passed into stats handler #6750

Conversation

zasweq commented Oct 25, 2023

codecov bot commented Oct 25, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Oct 25, 2023 •

edited

Loading