Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test: AllMetricsOneFunction #6231

Closed
easwars opened this issue Apr 28, 2023 · 4 comments · Fixed by #6372
Closed

Flaky test: AllMetricsOneFunction #6231

easwars opened this issue Apr 28, 2023 · 4 comments · Fixed by #6372
Assignees

Comments

@easwars
Copy link
Contributor

easwars commented Apr 28, 2023

--- FAIL: Test (0.15s)
    --- FAIL: Test/AllMetricsOneFunction (0.07s)
        tlogger.go:116: INFO server.go:629 [core] [Server #19] Server created  (t=+1.130319ms)
        tlogger.go:116: INFO clientconn.go:177 [core] [Channel #20] Channel created  (t=+1.558427ms)
        tlogger.go:116: INFO clientconn.go:1600 [core] [Channel #20] original dial target is: "whatever:///127.0.0.1:45395"  (t=+1.668828ms)
        tlogger.go:116: INFO clientconn.go:1607 [core] [Channel #20] parsed dial target is: {Scheme:whatever Authority: URL:{Scheme:whatever Opaque: User: Host: Path:/127.0.0.1:45395 RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}  (t=+1.824031ms)
        tlogger.go:116: INFO clientconn.go:1707 [core] [Channel #20] Channel authority set to "127.0.0.1:45395"  (t=+1.928833ms)
        tlogger.go:116: INFO resolver_conn_wrapper.go:196 [core] [Channel #20] Resolver state updated: {
              "Addresses": [
                {
                  "Addr": "127.0.0.1:45395",
                  "ServerName": "",
                  "Attributes": null,
                  "BalancerAttributes": null,
                  "Type": 0,
                  "Metadata": null
                }
              ],
              "ServiceConfig": null,
              "Attributes": null
            } (resolver returned new addresses)  (t=+2.188437ms)
        tlogger.go:116: INFO balancer_conn_wrappers.go:276 [core] [Channel #20] Channel switches to new LB policy "pick_first"  (t=+2.37184ms)
        tlogger.go:116: INFO clientconn.go:759 [core] [Channel #20 SubChannel #21] Subchannel created  (t=+2.5[336](https://github.com/grpc/grpc-go/actions/runs/4824796293/jobs/8595006405?pr=6228#step:8:337)43ms)
        tlogger.go:116: INFO clientconn.go:451 [core] [Channel #20] Channel Connectivity change to CONNECTING  (t=+2.629444ms)
        tlogger.go:116: INFO server.go:817 [core] [Server #19 ListenSocket #22] ListenSocket created  (t=+2.788947ms)
        tlogger.go:116: INFO clientconn.go:1141 [core] [Channel #20 SubChannel #21] Subchannel Connectivity change to CONNECTING  (t=+3.064652ms)
        tlogger.go:116: INFO clientconn.go:1255 [core] [Channel #20 SubChannel #21] Subchannel picks a new address "127.0.0.1:45395" to connect  (t=+3.254755ms)
        tlogger.go:116: INFO clientconn.go:1141 [core] [Channel #20 SubChannel #21] Subchannel Connectivity change to READY  (t=+4.581777ms)
        tlogger.go:116: INFO clientconn.go:451 [core] [Channel #20] Channel Connectivity change to READY  (t=+4.880182ms)
        e2e_test.go:1019: got unexpected viewInformation for metric grpc.io/server/completed_rpcs, diff (-got, +want):   (*opencensus.viewInformation)(
            - 	&{
            - 		aggType:    s"Count",
            - 		aggBuckets: []float64{},
            - 		desc:       "Number of completed RPCs by method and status.",
            - 		tagKeys:    []tag.Key{{name: "grpc_server_method"}, {name: "grpc_server_status"}},
            - 		rows:       []*view.Row{s"{ { {grpc_server_method grpc.testing.TestService/FullDuplexCall}"...},
            - 	},
            + 	&{
            + 		aggType:    s"Count",
            + 		aggBuckets: []float64{},
            + 		desc:       "Number of completed RPCs by method and status.",
            + 		tagKeys:    []tag.Key{{name: "grpc_server_method"}, {name: "grpc_server_status"}},
            + 		rows: []*view.Row{
            + 			s"{ { {grpc_server_method grpc.testing.TestService/UnaryCall}{grpc"...,
            + 			s"{ { {grpc_server_method grpc.testing.TestService/FullDuplexCall}"...,
            + 		},
            + 	},
              )
        tlogger.go:116: INFO clientconn.go:451 [core] [Channel #20] Channel Connectivity change to SHUTDOWN  (t=+55.584632ms)
        tlogger.go:116: INFO clientconn.go:1141 [core] [Channel #20 SubChannel #21] Subchannel Connectivity change to SHUTDOWN  (t=+55.955638ms)
        tlogger.go:116: INFO clientconn.go:1468 [core] [Channel #20 SubChannel #21] Subchannel deleted  (t=+56.110[341](https://github.com/grpc/grpc-go/actions/runs/4824796293/jobs/8595006405?pr=6228#step:8:342)ms)
        tlogger.go:116: INFO clientconn.go:1095 [core] [Channel #20] Channel deleted  (t=+56.201042ms)
        tlogger.go:116: INFO server.go:768 [core] [Server #19 ListenSocket #22] ListenSocket deleted  (t=+57.087057ms)
FAIL
FAIL	google.golang.org/grpc/stats/opencensus	0.831s

https://github.com/grpc/grpc-go/actions/runs/4824796293/jobs/8595006405?pr=6228

@arvindbr8 arvindbr8 added the P2 label May 23, 2023
@dfawley dfawley added P1 and removed P2 labels May 26, 2023
@zasweq
Copy link
Contributor

zasweq commented Jun 9, 2023

@zasweq
Copy link
Contributor

zasweq commented Jun 14, 2023

Failed 18/10k times in Forge. This needs a sync point between server emitting this metric and the cmp.Diff call. Right now, it just waits on client receiving EOF, which server puts onto wire before metric recording with no sync point, thus not guaranteeing it works.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 18, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants