Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/registry/generic/registry TestStoreWatch] testing.go:699: race detected during execution of test #17974

Closed
simo5 opened this issue Jan 3, 2018 · 8 comments
Closed

[github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/registry/generic/registry TestStoreWatch] testing.go:699: race detected during execution of test #17974

simo5 opened this issue Jan 3, 2018 · 8 comments
Assignees
@smarterclayton
Labels
kind/test-flake Categorizes issue or PR as related to test flakes. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. priority/P2

Comments

@simo5
Copy link
Contributor

simo5 commented Jan 3, 2018 •
edited
Loading

Flaked:
https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/17640/test_pull_request_origin_unit/7499/

2018-01-03 18:25:10.306481 I | integration: launching 2895284791933743572 (unix://localhost:28952847919337435720)
2018-01-03 18:25:10.308754 I | etcdserver: name = 2895284791933743572
2018-01-03 18:25:10.308804 I | etcdserver: data dir = /tmp/etcd535097500
2018-01-03 18:25:10.308857 I | etcdserver: member dir = /tmp/etcd535097500/member
2018-01-03 18:25:10.308907 I | etcdserver: heartbeat = 10ms
2018-01-03 18:25:10.308951 I | etcdserver: election = 100ms
2018-01-03 18:25:10.308992 I | etcdserver: snapshot count = 0
2018-01-03 18:25:10.309037 I | etcdserver: advertise client URLs = unix://127.0.0.1:2105202041
2018-01-03 18:25:10.309098 I | etcdserver: initial advertise peer URLs = unix://127.0.0.1:2105102041
2018-01-03 18:25:10.309167 I | etcdserver: initial cluster = 2895284791933743572=unix://127.0.0.1:2105102041
2018-01-03 18:25:10.312380 I | etcdserver: starting member 3ca31e95adf38f75 in cluster 389b2a51aa37f3de
2018-01-03 18:25:10.312482 I | raft: 3ca31e95adf38f75 became follower at term 0
2018-01-03 18:25:10.312566 I | raft: newRaft 3ca31e95adf38f75 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2018-01-03 18:25:10.312637 I | raft: 3ca31e95adf38f75 became follower at term 1
2018-01-03 18:25:10.325285 W | auth: simple token is not cryptographically signed
2018-01-03 18:25:10.331220 I | etcdserver: set snapshot count to default 100000
2018-01-03 18:25:10.331286 I | etcdserver: starting server... [version: 3.2.8, cluster version: to_be_decided]
2018-01-03 18:25:10.333165 I | etcdserver/membership: added member 3ca31e95adf38f75 [unix://127.0.0.1:2105102041] to cluster 389b2a51aa37f3de
2018-01-03 18:25:10.334415 I | integration: launched 2895284791933743572 (unix://localhost:28952847919337435720)
2018-01-03 18:25:10.373187 I | raft: 3ca31e95adf38f75 is starting a new election at term 1
2018-01-03 18:25:10.373314 I | raft: 3ca31e95adf38f75 became candidate at term 2
2018-01-03 18:25:10.373389 I | raft: 3ca31e95adf38f75 received MsgVoteResp from 3ca31e95adf38f75 at term 2
2018-01-03 18:25:10.373478 I | raft: 3ca31e95adf38f75 became leader at term 2
2018-01-03 18:25:10.373547 I | raft: raft.node: 3ca31e95adf38f75 elected leader 3ca31e95adf38f75 at term 2
2018-01-03 18:25:10.374041 I | etcdserver: setting up the initial cluster version to 3.2
2018-01-03 18:25:10.375387 N | etcdserver/membership: set the initial cluster version to 3.2
2018-01-03 18:25:10.375764 I | etcdserver: published {Name:2895284791933743572 ClientURLs:[unix://127.0.0.1:2105202041]} to cluster 389b2a51aa37f3de
2018-01-03 18:25:10.392185 I | integration: terminating 2895284791933743572 (unix://localhost:28952847919337435720)
2018-01-03 18:25:10.396345 I | integration: terminated 2895284791933743572 (unix://localhost:28952847919337435720)
2018-01-03 18:25:10.408460 I | integration: launching 2490465766450078734 (unix://localhost:24904657664500787340)
2018-01-03 18:25:10.412281 I | etcdserver: name = 2490465766450078734
2018-01-03 18:25:10.412349 I | etcdserver: data dir = /tmp/etcd452181835
2018-01-03 18:25:10.412394 I | etcdserver: member dir = /tmp/etcd452181835/member
2018-01-03 18:25:10.412427 I | etcdserver: heartbeat = 10ms
2018-01-03 18:25:10.412459 I | etcdserver: election = 100ms
2018-01-03 18:25:10.412486 I | etcdserver: snapshot count = 0
2018-01-03 18:25:10.412518 I | etcdserver: advertise client URLs = unix://127.0.0.1:2105402041
2018-01-03 18:25:10.412552 I | etcdserver: initial advertise peer URLs = unix://127.0.0.1:2105302041
2018-01-03 18:25:10.412602 I | etcdserver: initial cluster = 2490465766450078734=unix://127.0.0.1:2105302041
2018-01-03 18:25:10.419235 I | etcdserver: starting member d7df156aa22608a5 in cluster fbdc81a5592ef710
2018-01-03 18:25:10.419343 I | raft: d7df156aa22608a5 became follower at term 0
2018-01-03 18:25:10.419416 I | raft: newRaft d7df156aa22608a5 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2018-01-03 18:25:10.419476 I | raft: d7df156aa22608a5 became follower at term 1
2018-01-03 18:25:10.427885 W | auth: simple token is not cryptographically signed
2018-01-03 18:25:10.431625 I | etcdserver: set snapshot count to default 100000
2018-01-03 18:25:10.431700 I | etcdserver: starting server... [version: 3.2.8, cluster version: to_be_decided]
2018-01-03 18:25:10.435592 I | etcdserver/membership: added member d7df156aa22608a5 [unix://127.0.0.1:2105302041] to cluster fbdc81a5592ef710
2018-01-03 18:25:10.438377 I | integration: launched 2490465766450078734 (unix://localhost:24904657664500787340)
2018-01-03 18:25:10.481135 I | raft: d7df156aa22608a5 is starting a new election at term 1
2018-01-03 18:25:10.481244 I | raft: d7df156aa22608a5 became candidate at term 2
2018-01-03 18:25:10.481308 I | raft: d7df156aa22608a5 received MsgVoteResp from d7df156aa22608a5 at term 2
2018-01-03 18:25:10.481397 I | raft: d7df156aa22608a5 became leader at term 2
2018-01-03 18:25:10.481461 I | raft: raft.node: d7df156aa22608a5 elected leader d7df156aa22608a5 at term 2
2018-01-03 18:25:10.482844 I | etcdserver: published {Name:2490465766450078734 ClientURLs:[unix://127.0.0.1:2105402041]} to cluster fbdc81a5592ef710
2018-01-03 18:25:10.482951 I | etcdserver: setting up the initial cluster version to 3.2
2018-01-03 18:25:10.483781 N | etcdserver/membership: set the initial cluster version to 3.2
2018-01-03 18:25:10.495272 I | integration: terminating 2490465766450078734 (unix://localhost:24904657664500787340)
2018-01-03 18:25:10.499067 I | integration: terminated 2490465766450078734 (unix://localhost:24904657664500787340)
2018-01-03 18:25:10.503455 I | integration: launching 2338629011849215602 (unix://localhost:23386290118492156020)
2018-01-03 18:25:10.510281 I | etcdserver: name = 2338629011849215602
2018-01-03 18:25:10.510357 I | etcdserver: data dir = /tmp/etcd108021294
2018-01-03 18:25:10.510422 I | etcdserver: member dir = /tmp/etcd108021294/member
2018-01-03 18:25:10.510478 I | etcdserver: heartbeat = 10ms
2018-01-03 18:25:10.510534 I | etcdserver: election = 100ms
2018-01-03 18:25:10.510587 I | etcdserver: snapshot count = 0
2018-01-03 18:25:10.510655 I | etcdserver: advertise client URLs = unix://127.0.0.1:2105602041
2018-01-03 18:25:10.510712 I | etcdserver: initial advertise peer URLs = unix://127.0.0.1:2105502041
2018-01-03 18:25:10.510785 I | etcdserver: initial cluster = 2338629011849215602=unix://127.0.0.1:2105502041
2018-01-03 18:25:10.514217 I | etcdserver: starting member 254c45b49102da64 in cluster fa9f0fd80af7cfd1
2018-01-03 18:25:10.514326 I | raft: 254c45b49102da64 became follower at term 0
2018-01-03 18:25:10.514409 I | raft: newRaft 254c45b49102da64 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2018-01-03 18:25:10.514471 I | raft: 254c45b49102da64 became follower at term 1
2018-01-03 18:25:10.522621 W | auth: simple token is not cryptographically signed
2018-01-03 18:25:10.529465 I | etcdserver: set snapshot count to default 100000
2018-01-03 18:25:10.529543 I | etcdserver: starting server... [version: 3.2.8, cluster version: to_be_decided]
2018-01-03 18:25:10.531977 I | etcdserver/membership: added member 254c45b49102da64 [unix://127.0.0.1:2105502041] to cluster fa9f0fd80af7cfd1
2018-01-03 18:25:10.533037 I | integration: launched 2338629011849215602 (unix://localhost:23386290118492156020)
2018-01-03 18:25:10.554995 I | raft: 254c45b49102da64 is starting a new election at term 1
2018-01-03 18:25:10.555122 I | raft: 254c45b49102da64 became candidate at term 2
2018-01-03 18:25:10.555195 I | raft: 254c45b49102da64 received MsgVoteResp from 254c45b49102da64 at term 2
2018-01-03 18:25:10.555285 I | raft: 254c45b49102da64 became leader at term 2
2018-01-03 18:25:10.555350 I | raft: raft.node: 254c45b49102da64 elected leader 254c45b49102da64 at term 2
2018-01-03 18:25:10.556326 I | etcdserver: published {Name:2338629011849215602 ClientURLs:[unix://127.0.0.1:2105602041]} to cluster fa9f0fd80af7cfd1
2018-01-03 18:25:10.556474 I | etcdserver: setting up the initial cluster version to 3.2
2018-01-03 18:25:10.557702 N | etcdserver/membership: set the initial cluster version to 3.2
==================
WARNING: DATA RACE
Write at 0x00c4202b66a1 by goroutine 23:
  github.com/openshift/origin/vendor/google.golang.org/grpc/transport.(*http2Client).operateHeaders()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/transport/http2_client.go:963 +0x93
  github.com/openshift/origin/vendor/google.golang.org/grpc/transport.(*http2Client).reader()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/transport/http2_client.go:1080 +0x832

Previous read at 0x00c4202b66a1 by goroutine 97:
  github.com/openshift/origin/vendor/google.golang.org/grpc/transport.(*Stream).BytesReceived()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/transport/transport.go:359 +0x8a
  github.com/openshift/origin/vendor/google.golang.org/grpc.(*clientStream).finish()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/stream.go:472 +0x8ac
  github.com/openshift/origin/vendor/google.golang.org/grpc.newClientStream.func3()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/stream.go:257 +0x2e4

Goroutine 23 (running) created at:
  github.com/openshift/origin/vendor/google.golang.org/grpc/transport.newHTTP2Client()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/transport/http2_client.go:250 +0x1268
  github.com/openshift/origin/vendor/google.golang.org/grpc/transport.NewClientTransport()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/transport/transport.go:439 +0xb1
  github.com/openshift/origin/vendor/google.golang.org/grpc.(*addrConn).resetTransport()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/clientconn.go:829 +0x43a
  github.com/openshift/origin/vendor/google.golang.org/grpc.(*ClientConn).resetAddrConn.func1()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/clientconn.go:612 +0x41

Goroutine 97 (running) created at:
  github.com/openshift/origin/vendor/google.golang.org/grpc.newClientStream()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/stream.go:240 +0x1401
  github.com/openshift/origin/vendor/google.golang.org/grpc.NewClientStream()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/google.golang.org/grpc/stream.go:105 +0x1fc
  github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver/etcdserverpb.(*watchClient).Watch()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver/etcdserverpb/rpc.pb.go:2449 +0xe9
  github.com/openshift/origin/vendor/github.com/coreos/etcd/clientv3.(*watchGrpcStream).openWatchClient()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/clientv3/watch.go:775 +0x173
  github.com/openshift/origin/vendor/github.com/coreos/etcd/clientv3.(*watchGrpcStream).newWatchClient()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/clientv3/watch.go:692 +0x632
  github.com/openshift/origin/vendor/github.com/coreos/etcd/clientv3.(*watchGrpcStream).run()
      /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/clientv3/watch.go:423 +0x126
==================
2018-01-03 18:25:10.578708 I | integration: terminating 2338629011849215602 (unix://localhost:23386290118492156020)
2018-01-03 18:25:10.582993 I | integration: terminated 2338629011849215602 (unix://localhost:23386290118492156020)
@simo5 simo5 added the kind/test-flake Categorizes issue or PR as related to test flakes. label Jan 3, 2018
@simo5 simo5 mentioned this issue Jan 3, 2018
Merged
@mfojtik
Copy link
Contributor

mfojtik commented Jan 4, 2018

@smarterclayton this looks suspiciously similar to the grpc problem we have in online... Maybe time to merge that grpc fix?

@simo5
Copy link
Contributor Author

simo5 commented Jan 5, 2018

/assign @smarterclayton

@ironcladlou ironcladlou mentioned this issue Jan 9, 2018
Merged
@sjenning
Copy link
Contributor

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/18165/test_pull_request_origin_unit/8206/

@sjenning sjenning mentioned this issue Jan 18, 2018
Merged
@sosiouxme sosiouxme mentioned this issue Jan 19, 2018
Merged
@sosiouxme
Copy link
Member

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/17773/test_pull_request_origin_unit/8295/

@gabemontero gabemontero mentioned this issue Jan 30, 2018
Merged
@gabemontero
Copy link
Contributor

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/18272/test_pull_request_origin_unit/8781/

@openshift-bot
Copy link
Contributor

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci-robot openshift-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 30, 2018
@openshift-bot
Copy link
Contributor

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

@openshift-ci-robot openshift-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels May 31, 2018
@openshift-bot
Copy link
Contributor

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@smarterclayton smarterclayton

Labels
kind/test-flake Categorizes issue or PR as related to test flakes. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. priority/P2
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@sosiouxme @mfojtik @sjenning @smarterclayton @openshift-bot @gabemontero @simo5 @openshift-ci-robot