Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[k8s.io] Probing container should *not* be restarted with a /healthz http liveness probe [Conformance] [Suite:openshift/conformance/parallel] [Suite:k8s] #18616

Closed
sosiouxme opened this issue Feb 14, 2018 · 2 comments
Closed

[k8s.io] Probing container should *not* be restarted with a /healthz http liveness probe [Conformance] [Suite:openshift/conformance/parallel] [Suite:k8s] #18616

sosiouxme opened this issue Feb 14, 2018 · 2 comments
Assignees
@ravisantoshgudimetla
Labels
component/kubernetes kind/test-flake Categorizes issue or PR as related to test flakes. priority/P1

Comments

@sosiouxme
Copy link
Member

It is not clear to me if this is an updated instance of #12072 or something new. Close if duplicate.

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/18556/test_pull_request_origin_extended_conformance_gce/16210/ against release-3.7

go run hack/e2e.go -v -test --test_args='--ginkgo.focus=Probing\scontainer\sshould\s\*not\*\sbe\srestarted\swith\sa\s\/healthz\shttp\sliveness\sprobe\s\[Conformance\]\s\[Suite\:openshift\/conformance\/parallel\]\s\[Suite\:k8s\]$'


/tmp/openshift/build-rpm-release/tito/rpmbuild-originhtvQY_/BUILD/origin-3.7.1/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/common/container_probe.go:236
Feb 13 22:57:08.636: pod e2e-tests-container-probe-gq6j4/liveness-http - expected number of restarts: 0, found restarts: 1
/tmp/openshift/build-rpm-release/tito/rpmbuild-originhtvQY_/BUILD/origin-3.7.1/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/common/container_probe.go:405
				
[BeforeEach] [Top Level]
  /tmp/openshift/build-rpm-release/tito/rpmbuild-originhtvQY_/BUILD/origin-3.7.1/_output/local/go/src/github.com/openshift/origin/test/extended/util/test.go:53
[BeforeEach] [k8s.io] Probing container
  /tmp/openshift/build-rpm-release/tito/rpmbuild-originhtvQY_/BUILD/origin-3.7.1/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:130
STEP: Creating a kubernetes client
Feb 13 22:52:39.308: INFO: >>> kubeConfig: /tmp/cluster-admin.kubeconfig
STEP: Building a namespace api object
Feb 13 22:52:40.065: INFO: About to run a Kube e2e test, ensuring namespace is privileged
STEP: Waiting for a default service account to be provisioned in namespace
[BeforeEach] [k8s.io] Probing container
  /tmp/openshift/build-rpm-release/tito/rpmbuild-originhtvQY_/BUILD/origin-3.7.1/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/common/container_probe.go:49
[It] should *not* be restarted with a /healthz http liveness probe [Conformance] [Suite:openshift/conformance/parallel] [Suite:k8s]
  /tmp/openshift/build-rpm-release/tito/rpmbuild-originhtvQY_/BUILD/origin-3.7.1/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/common/container_probe.go:236
STEP: Creating pod liveness-http in namespace e2e-tests-container-probe-gq6j4
Feb 13 22:53:07.818: INFO: Started pod liveness-http in namespace e2e-tests-container-probe-gq6j4
STEP: checking the pod's current state and verifying that restartCount is present
Feb 13 22:53:07.890: INFO: Initial restart count of pod liveness-http is 0
Feb 13 22:56:56.557: INFO: Restart count of pod e2e-tests-container-probe-gq6j4/liveness-http is now 1 (3m48.667361858s elapsed)
Feb 13 22:57:08.636: INFO: pod e2e-tests-container-probe-gq6j4/liveness-http - expected number of restarts: 0, found restarts: 1
STEP: deleting the pod
[AfterEach] [k8s.io] Probing container
  /tmp/openshift/build-rpm-release/tito/rpmbuild-originhtvQY_/BUILD/origin-3.7.1/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:131
STEP: Collecting events from namespace "e2e-tests-container-probe-gq6j4".
STEP: Found 7 events.
Feb 13 22:57:08.678: INFO: At 2018-02-13 22:52:41 +0000 UTC - event for liveness-http: {default-scheduler } Scheduled: Successfully assigned liveness-http to ci-prtest-5a37c28-16210-ig-n-r092
Feb 13 22:57:08.678: INFO: At 2018-02-13 22:52:42 +0000 UTC - event for liveness-http: {kubelet ci-prtest-5a37c28-16210-ig-n-r092} SuccessfulMountVolume: MountVolume.SetUp succeeded for volume "default-token-vgsrg" 
Feb 13 22:57:08.678: INFO: At 2018-02-13 22:52:50 +0000 UTC - event for liveness-http: {kubelet ci-prtest-5a37c28-16210-ig-n-r092} Pulled: Container image "gcr.io/google_containers/nginx-slim:0.7" already present on machine
Feb 13 22:57:08.678: INFO: At 2018-02-13 22:52:51 +0000 UTC - event for liveness-http: {kubelet ci-prtest-5a37c28-16210-ig-n-r092} Created: Created container
Feb 13 22:57:08.678: INFO: At 2018-02-13 22:52:52 +0000 UTC - event for liveness-http: {kubelet ci-prtest-5a37c28-16210-ig-n-r092} Started: Started container
Feb 13 22:57:08.678: INFO: At 2018-02-13 22:56:54 +0000 UTC - event for liveness-http: {kubelet ci-prtest-5a37c28-16210-ig-n-r092} Unhealthy: Liveness probe failed: Get http://172.16.2.162:80/: dial tcp 172.16.2.162:80: getsockopt: no route to host
Feb 13 22:57:08.678: INFO: At 2018-02-13 22:56:55 +0000 UTC - event for liveness-http: {kubelet ci-prtest-5a37c28-16210-ig-n-r092} Killing: Killing container with id docker://liveness:Container failed liveness probe.. Container will be killed and recreated.
Feb 13 22:57:08.724: INFO: POD                       NODE                               PHASE    GRACE  CONDITIONS
Feb 13 22:57:08.724: INFO: docker-registry-1-j96sv   ci-prtest-5a37c28-16210-ig-m-5lkp  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:44:40 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:44:50 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:44:40 +0000 UTC  }]
Feb 13 22:57:08.724: INFO: registry-console-1-9pmxs  ci-prtest-5a37c28-16210-ig-n-sc0d  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:44:41 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:45:01 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:44:41 +0000 UTC  }]
Feb 13 22:57:08.724: INFO: router-1-x5d4f            ci-prtest-5a37c28-16210-ig-m-5lkp  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:44:39 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:44:59 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:44:39 +0000 UTC  }]
Feb 13 22:57:08.724: INFO: liveness-exec             ci-prtest-5a37c28-16210-ig-n-kj01  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:54:51 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:54:54 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:54:51 +0000 UTC  }]
Feb 13 22:57:08.724: INFO: prometheus-0              ci-prtest-5a37c28-16210-ig-n-sc0d  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:47:13 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:48:04 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:47:13 +0000 UTC  }]
Feb 13 22:57:08.724: INFO: bootstrap-autoapprover-0  ci-prtest-5a37c28-16210-ig-m-5lkp  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:43:26 +0000 UTC  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:44:19 +0000 UTC  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:43:26 +0000 UTC  }]
Feb 13 22:57:08.724: INFO: 
Feb 13 22:57:08.742: INFO: 
Logging node info for node ci-prtest-5a37c28-16210-ig-m-5lkp
Feb 13 22:57:08.757: INFO: Node Info: &Node{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:ci-prtest-5a37c28-16210-ig-m-5lkp,GenerateName:,Namespace:,SelfLink:/api/v1/nodes/ci-prtest-5a37c28-16210-ig-m-5lkp,UID:3d85a517-110f-11e8-a4bd-42010a8e0005,ResourceVersion:34853,Generation:0,CreationTimestamp:2018-02-13 22:42:58 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{beta.kubernetes.io/arch: amd64,beta.kubernetes.io/instance-type: n1-standard-2,beta.kubernetes.io/os: linux,failure-domain.beta.kubernetes.io/region: us-east1,failure-domain.beta.kubernetes.io/zone: us-east1-c,kubernetes.io/hostname: ci-prtest-5a37c28-16210-ig-m-5lkp,node-role.kubernetes.io/infra: true,node-role.kubernetes.io/master: true,role: infra,},Annotations:map[string]string{volumes.kubernetes.io/controller-managed-attach-detach: true,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:NodeSpec{PodCIDR:,ExternalID:7914488602521687671,ProviderID:gce://openshift-gce-devel-ci/us-east1-c/ci-prtest-5a37c28-16210-ig-m-5lkp,Unschedulable:false,Taints:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},memory: {{7674253312 0} {<nil>} 7494388Ki BinarySI},pods: {{20 0} {<nil>} 20 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},memory: {{7569395712 0} {<nil>} 7391988Ki BinarySI},pods: {{20 0} {<nil>} 20 DecimalSI},},Phase:,Conditions:[{NetworkUnavailable False 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:42:58 +0000 UTC RouteCreated openshift-sdn cleared kubelet-set NoRouteCreated} {OutOfDisk False 2018-02-13 22:57:02 +0000 UTC 2018-02-13 22:42:58 +0000 UTC KubeletHasSufficientDisk kubelet has sufficient disk space available} {MemoryPressure False 2018-02-13 22:57:02 +0000 UTC 2018-02-13 22:42:58 +0000 UTC KubeletHasSufficientMemory kubelet has sufficient memory available} {DiskPressure False 2018-02-13 22:57:02 +0000 UTC 2018-02-13 22:42:58 +0000 UTC KubeletHasNoDiskPressure kubelet has no disk pressure} {Ready True 2018-02-13 22:57:02 +0000 UTC 2018-02-13 22:43:09 +0000 UTC KubeletReady kubelet is posting ready status}],Addresses:[{InternalIP 10.142.0.5} {ExternalIP 35.231.109.117} {Hostname ci-prtest-5a37c28-16210-ig-m-5lkp}],DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7f4f55608a86584781fbc6bd9ddb79c,SystemUUID:062DFA86-8711-2D5B-B5DA-D0E3E17D6B9A,BootID:4634bc2c-1a82-47bf-9911-319a78f67c90,KernelVersion:3.10.0-693.17.1.el7.x86_64,OSImage:Red Hat Enterprise Linux Server 7.4 (Maipo),ContainerRuntimeVersion:docker://1.12.6,KubeletVersion:v1.7.6+a08f5eeb62,KubeProxyVersion:v1.7.6+a08f5eeb62,OperatingSystem:linux,Architecture:amd64,},Images:[{[docker.io/openshift/node@sha256:53f60230a465d731b8946c81af569c93e6d7b5187d58b907e5dbd7a71094c84c docker.io/openshift/node:v3.7.0-rc.0] 1279914095} {[docker.io/openshift/origin-haproxy-router@sha256:a218e302a711a250db705e0fb0b27aabd0aef3229185bebb41053cf3fd751fdd docker.io/openshift/origin-haproxy-router:v3.7.1] 1101204513} {[docker.io/openshift/origin-deployer@sha256:db092c265f1a3095f7baa2fba8fc2e8d5ca5df9696130ee67d132fcf1838c104 docker.io/openshift/origin-deployer:v3.7.1] 1079561351} {[docker.io/openshift/origin-docker-registry@sha256:26a876a04bc50b3784bc4c98520b70c5e39858e438a2bb9a4b6554e2e79c3a91 docker.io/openshift/origin-docker-registry:v3.7.1] 468820107} {[docker.io/openshift/origin-pod@sha256:8fb61ce6a84faae404ae730bd829c7429464a3faedc787f3a082d1c53fbab635 docker.io/openshift/origin-pod:v3.7.1] 228494688} {[gcr.io/google_containers/nginx-slim@sha256:8b4501fe0fe221df663c22e16539f399e89594552f400408303c42f3dd8d0e52 gcr.io/google_containers/nginx-slim:0.8] 110487599} {[gcr.io/google_containers/nginx-slim@sha256:dd4efd4c13bec2c6f3fe855deeab9524efe434505568421d4f31820485b3a795 gcr.io/google_containers/nginx-slim:0.7] 86864428} {[gcr.io/google_containers/hostexec@sha256:cab8d4e2526f8f767c64febe4ce9e0f0e58cd35fdff81b3aadba4dd041ba9f00 gcr.io/google_containers/hostexec:1.2] 13209617} {[gcr.io/google_containers/netexec@sha256:56c53846f44ea214e4aa5df37c9c50331f0b09e64a32cc7cf17c7e1808d38eef gcr.io/google_containers/netexec:1.7] 8016035}],VolumesInUse:[],VolumesAttached:[],},}
Feb 13 22:57:08.757: INFO: 
Logging kubelet events for node ci-prtest-5a37c28-16210-ig-m-5lkp
Feb 13 22:57:08.773: INFO: 
Logging pods the kubelet thinks is on node ci-prtest-5a37c28-16210-ig-m-5lkp
Feb 13 22:57:08.807: INFO: bootstrap-autoapprover-0 started at 2018-02-13 22:43:26 +0000 UTC (0+1 container statuses recorded)
Feb 13 22:57:08.807: INFO: 	Container signer ready: true, restart count 0
Feb 13 22:57:08.807: INFO: router-1-x5d4f started at 2018-02-13 22:44:39 +0000 UTC (0+1 container statuses recorded)
Feb 13 22:57:08.807: INFO: 	Container router ready: true, restart count 0
Feb 13 22:57:08.807: INFO: docker-registry-1-j96sv started at 2018-02-13 22:44:40 +0000 UTC (0+1 container statuses recorded)
Feb 13 22:57:08.807: INFO: 	Container registry ready: true, restart count 0
Feb 13 22:57:08.879: INFO: 
Latency metrics for node ci-prtest-5a37c28-16210-ig-m-5lkp
Feb 13 22:57:08.879: INFO: {Operation:pull_image Method:docker_operations_latency_microseconds Quantile:0.99 Latency:11.340086s}
Feb 13 22:57:08.879: INFO: {Operation:pull_image Method:docker_operations_latency_microseconds Quantile:0.9 Latency:11.340086s}
Feb 13 22:57:08.879: INFO: {Operation:pull_image Method:docker_operations_latency_microseconds Quantile:0.5 Latency:11.340086s}
Feb 13 22:57:08.879: INFO: 
Logging node info for node ci-prtest-5a37c28-16210-ig-n-kj01
Feb 13 22:57:08.894: INFO: Node Info: &Node{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:ci-prtest-5a37c28-16210-ig-n-kj01,GenerateName:,Namespace:,SelfLink:/api/v1/nodes/ci-prtest-5a37c28-16210-ig-n-kj01,UID:412e4171-110f-11e8-a4bd-42010a8e0005,ResourceVersion:34858,Generation:0,CreationTimestamp:2018-02-13 22:43:05 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{beta.kubernetes.io/arch: amd64,beta.kubernetes.io/instance-type: n1-standard-2,beta.kubernetes.io/os: linux,failure-domain.beta.kubernetes.io/region: us-east1,failure-domain.beta.kubernetes.io/zone: us-east1-c,kubernetes.io/hostname: ci-prtest-5a37c28-16210-ig-n-kj01,node-role.kubernetes.io/compute: true,role: app,},Annotations:map[string]string{volumes.kubernetes.io/controller-managed-attach-detach: true,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:NodeSpec{PodCIDR:,ExternalID:6853072956704085623,ProviderID:gce://openshift-gce-devel-ci/us-east1-c/ci-prtest-5a37c28-16210-ig-n-kj01,Unschedulable:false,Taints:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},memory: {{7674253312 0} {<nil>} 7494388Ki BinarySI},pods: {{40 0} {<nil>} 40 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},memory: {{7569395712 0} {<nil>} 7391988Ki BinarySI},pods: {{40 0} {<nil>} 40 DecimalSI},},Phase:,Conditions:[{NetworkUnavailable False 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:43:05 +0000 UTC RouteCreated openshift-sdn cleared kubelet-set NoRouteCreated} {OutOfDisk False 2018-02-13 22:57:04 +0000 UTC 2018-02-13 22:43:05 +0000 UTC KubeletHasSufficientDisk kubelet has sufficient disk space available} {MemoryPressure False 2018-02-13 22:57:04 +0000 UTC 2018-02-13 22:43:05 +0000 UTC KubeletHasSufficientMemory kubelet has sufficient memory available} {DiskPressure False 2018-02-13 22:57:04 +0000 UTC 2018-02-13 22:43:05 +0000 UTC KubeletHasNoDiskPressure kubelet has no disk pressure} {Ready True 2018-02-13 22:57:04 +0000 UTC 2018-02-13 22:54:32 +0000 UTC KubeletReady kubelet is posting ready status}],Addresses:[{InternalIP 10.142.0.2} {ExternalIP 104.196.157.179} {Hostname ci-prtest-5a37c28-16210-ig-n-kj01}],DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7f4f55608a86584781fbc6bd9ddb79c,SystemUUID:7F478A81-75FE-E303-0E56-980EE73EC84A,BootID:6edf3f6c-4b3a-490d-b7be-67de366e0033,KernelVersion:3.10.0-693.17.1.el7.x86_64,OSImage:Red Hat Enterprise Linux Server 7.4 (Maipo),ContainerRuntimeVersion:docker://1.12.6,KubeletVersion:v1.7.6+a08f5eeb62,KubeProxyVersion:v1.7.6+a08f5eeb62,OperatingSystem:linux,Architecture:amd64,},Images:[{[docker.io/openshift/origin-haproxy-router@sha256:3290a06098c7eb2bb2f5ca35dc6c38c9a4d33405bd791d4e3926895426ce98dc docker.io/openshift/origin-haproxy-router:latest] 1281689482} {[docker.io/openshift/origin-docker-builder@sha256:435c885fd335014ad7d689bdf0d952aacd2a663a946047797dd5268af44f5ba6 docker.io/openshift/origin-docker-builder:v3.7.1] 1079561351} {[docker.io/openshift/origin-deployer@sha256:db092c265f1a3095f7baa2fba8fc2e8d5ca5df9696130ee67d132fcf1838c104 docker.io/openshift/origin-deployer:v3.7.1] 1079561351} {[docker.io/openshift/origin-sti-builder@sha256:13bd58ea406c849c836c34c21cc5b3b5914e5f0f14cc3ee199b881ae74752797 docker.io/openshift/origin-sti-builder:v3.7.1] 1079561351} {[docker-registry.default.svc:5000/extended-test-templates-gz76n-rt9gx/cakephp-mysql-example@sha256:29ae64dd37ddf836991f73b0902550b57803429c71f56f586d74d395297f96ef] 621170281} {[docker.io/centos/ruby-22-centos7@sha256:c5470425197e444078e0a62a5f01cd0be9dd39067330dd1a32b964405f9cbd79 docker.io/centos/ruby-22-centos7:latest] 549704762} {[docker-registry.default.svc:5000/extended-test-new-app-ng8xj-ts77q/a234567890123456789012345678901234567890123456789012345678@sha256:134c6c93baf0da30daa548739d755ac115704b5a7eebfb534d7cdf71a9c59031 docker-registry.default.svc:5000/extended-test-new-app-ng8xj-ts77q/a234567890123456789012345678901234567890123456789012345678:latest] 531162588} {[docker-registry.default.svc:5000/extended-test-s2i-build-root-9g96x-lz9b6/nodejspass@sha256:9f4d98f7136b6a70a981cf2680871a68b30fb4c803c896f92a2d29c0776dbbe0 docker-registry.default.svc:5000/extended-test-s2i-build-root-9g96x-lz9b6/nodejspass:latest] 531162453} {[docker.io/centos/ruby-24-centos7@sha256:11faa8e8abbb1eba349128f60e6d98ee7161bdb5c9f721d4b0984b84a8830d50] 530700401} {[docker.io/centos/nodejs-6-centos7@sha256:c51779bbc13bc9f3bb871d14cf549090a1686942c1cb77228b39af02139d3167] 515821008} {[docker-registry.default.svc:5000/extended-test-s2i-build-root-9g96x-lz9b6/nodejsroot@sha256:620c250d957e44b3704bd17832bebbeadd2d3fcf4d0e34e2d8ee5bf0d56882c7 docker-registry.default.svc:5000/extended-test-s2i-build-root-9g96x-lz9b6/nodejsroot:latest] 515821008} {[docker.io/centos/mysql-57-centos7@sha256:0889e0694fc88ab18b0ab704059d25f8617a4c9204041b14d982a3ecee32be7b] 448613200} {[docker.io/openshift/origin-pod@sha256:8fb61ce6a84faae404ae730bd829c7429464a3faedc787f3a082d1c53fbab635 docker.io/openshift/origin-pod:v3.7.1] 228494688} {[docker.io/centos@sha256:2671f7a3eea36ce43609e9fe7435ade83094291055f1c96d9d1d1d7c0b986a5d docker.io/centos:7 docker.io/centos:centos7] 207198847} {[gcr.io/google_containers/nginx-slim@sha256:8b4501fe0fe221df663c22e16539f399e89594552f400408303c42f3dd8d0e52 gcr.io/google_containers/nginx-slim:0.8] 110487599} {[gcr.io/google_containers/nginx-slim@sha256:dd4efd4c13bec2c6f3fe855deeab9524efe434505568421d4f31820485b3a795 gcr.io/google_containers/nginx-slim:0.7] 86864428} {[gcr.io/google_containers/nettest@sha256:8af3a0e8b8ab906b0648dd575e8785e04c19113531f8ffbaab9e149aa1a60763 gcr.io/google_containers/nettest:1.7] 24051275} {[gcr.io/google_containers/hostexec@sha256:cab8d4e2526f8f767c64febe4ce9e0f0e58cd35fdff81b3aadba4dd041ba9f00 gcr.io/google_containers/hostexec:1.2] 13209617} {[gcr.io/google_containers/netexec@sha256:56c53846f44ea214e4aa5df37c9c50331f0b09e64a32cc7cf17c7e1808d38eef gcr.io/google_containers/netexec:1.7] 8016035} {[docker.io/openshift/hello-openshift@sha256:93d608950b9a93f86bbdb7e6f067fa3f4d59afbd5a70d42aa14dcee30055256c docker.io/openshift/hello-openshift:latest] 6089990} {[gcr.io/k8s-testimages/redis@sha256:2c3f1112c32a23ad0f14a0a6ff8ac8006e23b5c27de89a4b5287467eb4844dad gcr.io/k8s-testimages/redis:e2e] 5861866} {[gcr.io/google_containers/update-demo@sha256:89ac104fa7c43880d2324f377b79be95b0b2b3fb32e4bd03b8d1e6d91a41f009 gcr.io/google_containers/update-demo:nautilus] 4555533} {[gcr.io/google_containers/update-demo@sha256:915a51866b3b77de77c833ac69c699b88d060312d41fb878fab7428b69ef382c gcr.io/google_containers/update-demo:kitten] 4549069} {[gcr.io/google_containers/liveness@sha256:90994881062c7de7bb1761f2f3d020fe9aa3d332a90e00ebd3ca9dcc1ed74f1c gcr.io/google_containers/liveness:e2e] 4387474} {[gcr.io/google_containers/eptest@sha256:bb088b26ed78613cce171420168db9a6c62a8dbea17d7be13077e7010bae162f gcr.io/google_containers/eptest:0.1] 2970692} {[gcr.io/google_containers/mounttest@sha256:bec3122ddcf8bd999e44e46e096659f31241d09f5236bc3dc212ea584ca06856 gcr.io/google_containers/mounttest:0.8] 1450761} {[gcr.io/google_containers/mounttest-user@sha256:5487c126b03abf4119a8f7950cd5f591f72dbe4ab15623f3387d3917e1268b4e gcr.io/google_containers/mounttest-user:0.5] 1450761} {[docker.io/busybox@sha256:4cee1979ba0bf7db9fc5d28fb7b798ca69ae95a47c5fecf46327720df4ff352d] 1146369} {[gcr.io/google_containers/busybox@sha256:4bdd623e848417d96127e16037743f0cd8b528c026e9175e22a84f639eca58ff gcr.io/google_containers/busybox:1.24] 1113554} {[gcr.io/google_containers/pause-amd64@sha256:163ac025575b775d1c0f9bf0bdd0f086883171eb475b5068e7defa4ca9e76516 gcr.io/google_containers/pause-amd64:3.0] 746888}],VolumesInUse:[],VolumesAttached:[],},}
Feb 13 22:57:08.895: INFO: 
Logging kubelet events for node ci-prtest-5a37c28-16210-ig-n-kj01
Feb 13 22:57:08.911: INFO: 
Logging pods the kubelet thinks is on node ci-prtest-5a37c28-16210-ig-n-kj01
Feb 13 22:57:08.947: INFO: liveness-exec started at 2018-02-13 22:54:51 +0000 UTC (0+1 container statuses recorded)
Feb 13 22:57:08.947: INFO: 	Container liveness ready: true, restart count 0
Feb 13 22:57:09.020: INFO: 
Latency metrics for node ci-prtest-5a37c28-16210-ig-n-kj01
Feb 13 22:57:09.020: INFO: {Operation:stop_container Method:docker_operations_latency_microseconds Quantile:0.99 Latency:10.687629s}
Feb 13 22:57:09.020: INFO: 
Logging node info for node ci-prtest-5a37c28-16210-ig-n-r092
Feb 13 22:57:09.035: INFO: Node Info: &Node{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:ci-prtest-5a37c28-16210-ig-n-r092,GenerateName:,Namespace:,SelfLink:/api/v1/nodes/ci-prtest-5a37c28-16210-ig-n-r092,UID:3db0109e-110f-11e8-a4bd-42010a8e0005,ResourceVersion:34852,Generation:0,CreationTimestamp:2018-02-13 22:42:59 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{beta.kubernetes.io/arch: amd64,beta.kubernetes.io/instance-type: n1-standard-2,beta.kubernetes.io/os: linux,failure-domain.beta.kubernetes.io/region: us-east1,failure-domain.beta.kubernetes.io/zone: us-east1-c,kubernetes.io/hostname: ci-prtest-5a37c28-16210-ig-n-r092,node-role.kubernetes.io/compute: true,role: app,},Annotations:map[string]string{volumes.kubernetes.io/controller-managed-attach-detach: true,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:NodeSpec{PodCIDR:,ExternalID:8621155037909800567,ProviderID:gce://openshift-gce-devel-ci/us-east1-c/ci-prtest-5a37c28-16210-ig-n-r092,Unschedulable:false,Taints:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},memory: {{7674253312 0} {<nil>} 7494388Ki BinarySI},pods: {{40 0} {<nil>} 40 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},memory: {{7569395712 0} {<nil>} 7391988Ki BinarySI},pods: {{40 0} {<nil>} 40 DecimalSI},},Phase:,Conditions:[{NetworkUnavailable False 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:42:59 +0000 UTC RouteCreated openshift-sdn cleared kubelet-set NoRouteCreated} {OutOfDisk False 2018-02-13 22:57:01 +0000 UTC 2018-02-13 22:42:59 +0000 UTC KubeletHasSufficientDisk kubelet has sufficient disk space available} {MemoryPressure False 2018-02-13 22:57:01 +0000 UTC 2018-02-13 22:42:59 +0000 UTC KubeletHasSufficientMemory kubelet has sufficient memory available} {DiskPressure False 2018-02-13 22:57:01 +0000 UTC 2018-02-13 22:42:59 +0000 UTC KubeletHasNoDiskPressure kubelet has no disk pressure} {Ready True 2018-02-13 22:57:01 +0000 UTC 2018-02-13 22:43:09 +0000 UTC KubeletReady kubelet is posting ready status}],Addresses:[{InternalIP 10.142.0.4} {ExternalIP 35.196.11.21} {Hostname ci-prtest-5a37c28-16210-ig-n-r092}],DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7f4f55608a86584781fbc6bd9ddb79c,SystemUUID:4BEF704A-3762-142B-75F7-88B2F02BC9FD,BootID:cfe52541-af03-4900-8e5c-94d993c9db85,KernelVersion:3.10.0-693.17.1.el7.x86_64,OSImage:Red Hat Enterprise Linux Server 7.4 (Maipo),ContainerRuntimeVersion:docker://1.12.6,KubeletVersion:v1.7.6+a08f5eeb62,KubeProxyVersion:v1.7.6+a08f5eeb62,OperatingSystem:linux,Architecture:amd64,},Images:[{[docker.io/openshift/origin-haproxy-router@sha256:3290a06098c7eb2bb2f5ca35dc6c38c9a4d33405bd791d4e3926895426ce98dc docker.io/openshift/origin-haproxy-router:latest] 1281689482} {[docker.io/openshift/origin-docker-builder@sha256:435c885fd335014ad7d689bdf0d952aacd2a663a946047797dd5268af44f5ba6 docker.io/openshift/origin-docker-builder:v3.7.1] 1079561351} {[docker.io/openshift/origin-deployer@sha256:db092c265f1a3095f7baa2fba8fc2e8d5ca5df9696130ee67d132fcf1838c104 docker.io/openshift/origin-deployer:v3.7.1] 1079561351} {[docker.io/openshift/origin-sti-builder@sha256:13bd58ea406c849c836c34c21cc5b3b5914e5f0f14cc3ee199b881ae74752797 docker.io/openshift/origin-sti-builder:v3.7.1] 1079561351} {[docker-registry.default.svc:5000/extended-test-templates-gz76n-rt9gx/cakephp-mysql-example@sha256:29ae64dd37ddf836991f73b0902550b57803429c71f56f586d74d395297f96ef docker-registry.default.svc:5000/extended-test-templates-gz76n-rt9gx/cakephp-mysql-example:latest] 621170281} {[docker.io/centos/php-70-centos7@sha256:ddfa75ed7d7e8ef0fb7917f6905de0f28589523d58d6ebc05e95dac9b9de6201] 600825838} {[docker-registry.default.svc:5000/extended-test-build-valuefrom-2b9sc-999r4/test@sha256:5a1909375ca915ea827538d09d838d96c11285bed52812dc4aad7b6e0f3ca101 docker-registry.default.svc:5000/extended-test-build-valuefrom-2b9sc-999r4/test:latest] 600817306} {[docker.io/centos/ruby-22-centos7@sha256:c5470425197e444078e0a62a5f01cd0be9dd39067330dd1a32b964405f9cbd79 docker.io/centos/ruby-22-centos7:latest] 549704762} {[docker-registry.default.svc:5000/extended-test-new-app-ng8xj-ts77q/a234567890123456789012345678901234567890123456789012345678@sha256:134c6c93baf0da30daa548739d755ac115704b5a7eebfb534d7cdf71a9c59031] 531162588} {[docker.io/centos/nodejs-6-centos7@sha256:c51779bbc13bc9f3bb871d14cf549090a1686942c1cb77228b39af02139d3167] 515821008} {[docker-registry.default.svc:5000/extended-test-s2i-build-root-vn9rg-cm7kc/nodejsroot@sha256:c919de36029e289600957b1e9adfe1e340aa0f54352b72a1d887bd7acc48f41b docker-registry.default.svc:5000/extended-test-s2i-build-root-vn9rg-cm7kc/nodejsroot:latest] 515821008} {[docker.io/centos/s2i-base-centos7@sha256:ce230a709c512b17d119bc07e89be52902ef718ce38abb1561b6ee3ee85edd95 docker.io/centos/s2i-base-centos7:latest] 484064854} {[docker.io/openshift/origin-base@sha256:107f74d8dd794f6f0b08f6a8ea77a3bce9bb64e46cf089d04275907132d53173 docker.io/openshift/origin-base:latest] 394637142} {[docker.io/openshift/origin-pod@sha256:8fb61ce6a84faae404ae730bd829c7429464a3faedc787f3a082d1c53fbab635 docker.io/openshift/origin-pod:v3.7.1] 228494688} {[docker-registry.default.svc:5000/extended-test-docker-build-pullsecret-496wh-74jf6/image1@sha256:fd98efb8e5f03534e940b8f724066341c4ab35230c6c731fa6920b3c6532e43d docker-registry.default.svc:5000/extended-test-docker-build-pullsecret-496wh-74jf6/image1:latest] 207198847} {[docker.io/centos@sha256:2671f7a3eea36ce43609e9fe7435ade83094291055f1c96d9d1d1d7c0b986a5d docker.io/centos:7 docker.io/centos:centos7] 207198847} {[gcr.io/google_containers/jessie-dnsutils@sha256:2460d596912244b5f8973573f7150e7264b570015f4becc2d0096f0bd1d17e36 gcr.io/google_containers/jessie-dnsutils:e2e] 190148402} {[gcr.io/google_containers/nginx-slim@sha256:8b4501fe0fe221df663c22e16539f399e89594552f400408303c42f3dd8d0e52 gcr.io/google_containers/nginx-slim:0.8] 110487599} {[gcr.io/google_containers/nginx-slim@sha256:dd4efd4c13bec2c6f3fe855deeab9524efe434505568421d4f31820485b3a795 gcr.io/google_containers/nginx-slim:0.7] 86864428} {[gcr.io/google_containers/hostexec@sha256:cab8d4e2526f8f767c64febe4ce9e0f0e58cd35fdff81b3aadba4dd041ba9f00 gcr.io/google_containers/hostexec:1.2] 13209617} {[gcr.io/google_containers/dnsutils@sha256:cd9182f6d74e616942db1cef6f25e1e54b49ba0330c2e19d3ec061f027666cc0 gcr.io/google_containers/dnsutils:e2e] 8897789} {[gcr.io/google_containers/netexec@sha256:56c53846f44ea214e4aa5df37c9c50331f0b09e64a32cc7cf17c7e1808d38eef gcr.io/google_containers/netexec:1.7] 8016035} {[gcr.io/google_containers/serve_hostname@sha256:a49737ee84a3b94f0b977f32e60c5daf11f0b5636f1f7503a2981524f351c57a gcr.io/google_containers/serve_hostname:v1.4] 6222101} {[docker.io/openshift/hello-openshift@sha256:93d608950b9a93f86bbdb7e6f067fa3f4d59afbd5a70d42aa14dcee30055256c docker.io/openshift/hello-openshift:latest] 6089990} {[gcr.io/k8s-testimages/redis@sha256:2c3f1112c32a23ad0f14a0a6ff8ac8006e23b5c27de89a4b5287467eb4844dad gcr.io/k8s-testimages/redis:e2e] 5861866} {[gcr.io/google_containers/fakegitserver@sha256:e974692bb4d422a4e9ea6ff9df85fa36f189010703400496fea44aac6589d0dd gcr.io/google_containers/fakegitserver:0.1] 5007469} {[gcr.io/google_containers/update-demo@sha256:89ac104fa7c43880d2324f377b79be95b0b2b3fb32e4bd03b8d1e6d91a41f009 gcr.io/google_containers/update-demo:nautilus] 4555533} {[gcr.io/google_containers/test-webserver@sha256:f804e8837490d1dfdb5002e073f715fd0a08115de74e5a4847ca952315739372 gcr.io/google_containers/test-webserver:e2e] 4534272} {[gcr.io/google_containers/eptest@sha256:bb088b26ed78613cce171420168db9a6c62a8dbea17d7be13077e7010bae162f gcr.io/google_containers/eptest:0.1] 2970692} {[gcr.io/google_containers/mounttest@sha256:c4dcedb26013ab4231a2b2aaa4eebd5c2a44d5c597fa0613c9ff8bde4fb9fe02 gcr.io/google_containers/mounttest:0.7] 2052704} {[gcr.io/google_containers/mounttest@sha256:bec3122ddcf8bd999e44e46e096659f31241d09f5236bc3dc212ea584ca06856 gcr.io/google_containers/mounttest:0.8] 1450761} {[gcr.io/google_containers/mounttest-user@sha256:5487c126b03abf4119a8f7950cd5f591f72dbe4ab15623f3387d3917e1268b4e gcr.io/google_containers/mounttest-user:0.5] 1450761} {[gcr.io/google_containers/busybox@sha256:4bdd623e848417d96127e16037743f0cd8b528c026e9175e22a84f639eca58ff gcr.io/google_containers/busybox:1.24] 1113554} {[docker.io/busybox@sha256:29f5d56d12684887bdfa50dcd29fc31eea4aaf4ad3bec43daf19026a7ce69912] 1093484} {[gcr.io/google_containers/pause-amd64@sha256:163ac025575b775d1c0f9bf0bdd0f086883171eb475b5068e7defa4ca9e76516 gcr.io/google_containers/pause-amd64:3.0] 746888}],VolumesInUse:[],VolumesAttached:[],},}
Feb 13 22:57:09.035: INFO: 
Logging kubelet events for node ci-prtest-5a37c28-16210-ig-n-r092
Feb 13 22:57:09.051: INFO: 
Logging pods the kubelet thinks is on node ci-prtest-5a37c28-16210-ig-n-r092
Feb 13 22:57:09.230: INFO: 
Latency metrics for node ci-prtest-5a37c28-16210-ig-n-r092
Feb 13 22:57:09.230: INFO: {Operation:sync Method:pod_worker_latency_microseconds Quantile:0.99 Latency:2m32.76893s}
Feb 13 22:57:09.230: INFO: {Operation:sync Method:pod_worker_latency_microseconds Quantile:0.9 Latency:2m21.567284s}
Feb 13 22:57:09.230: INFO: {Operation:create Method:pod_worker_latency_microseconds Quantile:0.99 Latency:2m3.421877s}
Feb 13 22:57:09.230: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.99 Latency:54.553303s}
Feb 13 22:57:09.230: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.9 Latency:40.456618s}
Feb 13 22:57:09.230: INFO: {Operation:create Method:pod_worker_latency_microseconds Quantile:0.9 Latency:39.474219s}
Feb 13 22:57:09.230: INFO: {Operation:stop_container Method:docker_operations_latency_microseconds Quantile:0.99 Latency:30.054981s}
Feb 13 22:57:09.230: INFO: {Operation:create Method:pod_worker_latency_microseconds Quantile:0.5 Latency:22.150082s}
Feb 13 22:57:09.230: INFO: {Operation: Method:pleg_relist_latency_microseconds Quantile:0.99 Latency:16.489374s}
Feb 13 22:57:09.230: INFO: {Operation:pull_image Method:docker_operations_latency_microseconds Quantile:0.99 Latency:12.412275s}
Feb 13 22:57:09.230: INFO: {Operation:pull_image Method:docker_operations_latency_microseconds Quantile:0.9 Latency:11.753754s}
Feb 13 22:57:09.230: INFO: {Operation:update Method:pod_worker_latency_microseconds Quantile:0.99 Latency:10.934753s}
Feb 13 22:57:09.230: INFO: {Operation:update Method:pod_worker_latency_microseconds Quantile:0.9 Latency:10.934753s}
Feb 13 22:57:09.230: INFO: 
Logging node info for node ci-prtest-5a37c28-16210-ig-n-sc0d
Feb 13 22:57:09.246: INFO: Node Info: &Node{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:ci-prtest-5a37c28-16210-ig-n-sc0d,GenerateName:,Namespace:,SelfLink:/api/v1/nodes/ci-prtest-5a37c28-16210-ig-n-sc0d,UID:416d5e9c-110f-11e8-a4bd-42010a8e0005,ResourceVersion:34849,Generation:0,CreationTimestamp:2018-02-13 22:43:05 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{beta.kubernetes.io/arch: amd64,beta.kubernetes.io/instance-type: n1-standard-2,beta.kubernetes.io/os: linux,failure-domain.beta.kubernetes.io/region: us-east1,failure-domain.beta.kubernetes.io/zone: us-east1-c,kubernetes.io/hostname: ci-prtest-5a37c28-16210-ig-n-sc0d,node-role.kubernetes.io/compute: true,role: app,},Annotations:map[string]string{volumes.kubernetes.io/controller-managed-attach-detach: true,},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:NodeSpec{PodCIDR:,ExternalID:8632426058591688311,ProviderID:gce://openshift-gce-devel-ci/us-east1-c/ci-prtest-5a37c28-16210-ig-n-sc0d,Unschedulable:false,Taints:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},memory: {{7674253312 0} {<nil>} 7494388Ki BinarySI},pods: {{40 0} {<nil>} 40 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},memory: {{7569395712 0} {<nil>} 7391988Ki BinarySI},pods: {{40 0} {<nil>} 40 DecimalSI},},Phase:,Conditions:[{NetworkUnavailable False 0001-01-01 00:00:00 +0000 UTC 2018-02-13 22:43:05 +0000 UTC RouteCreated openshift-sdn cleared kubelet-set NoRouteCreated} {OutOfDisk False 2018-02-13 22:57:00 +0000 UTC 2018-02-13 22:43:05 +0000 UTC KubeletHasSufficientDisk kubelet has sufficient disk space available} {MemoryPressure False 2018-02-13 22:57:00 +0000 UTC 2018-02-13 22:43:05 +0000 UTC KubeletHasSufficientMemory kubelet has sufficient memory available} {DiskPressure False 2018-02-13 22:57:00 +0000 UTC 2018-02-13 22:43:05 +0000 UTC KubeletHasNoDiskPressure kubelet has no disk pressure} {Ready True 2018-02-13 22:57:00 +0000 UTC 2018-02-13 22:43:15 +0000 UTC KubeletReady kubelet is posting ready status}],Addresses:[{InternalIP 10.142.0.3} {ExternalIP 35.227.68.241} {Hostname ci-prtest-5a37c28-16210-ig-n-sc0d}],DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7f4f55608a86584781fbc6bd9ddb79c,SystemUUID:879DBE5D-47FF-88D6-0807-213CB658AD30,BootID:bbd287c6-dcb4-4be9-b773-20f4d03d9bb2,KernelVersion:3.10.0-693.17.1.el7.x86_64,OSImage:Red Hat Enterprise Linux Server 7.4 (Maipo),ContainerRuntimeVersion:docker://1.12.6,KubeletVersion:v1.7.6+a08f5eeb62,KubeProxyVersion:v1.7.6+a08f5eeb62,OperatingSystem:linux,Architecture:amd64,},Images:[{[docker.io/openshift/origin-haproxy-router@sha256:3290a06098c7eb2bb2f5ca35dc6c38c9a4d33405bd791d4e3926895426ce98dc docker.io/openshift/origin-haproxy-router:latest] 1281689482} {[docker.io/openshift/origin-sti-builder@sha256:13bd58ea406c849c836c34c21cc5b3b5914e5f0f14cc3ee199b881ae74752797 docker.io/openshift/origin-sti-builder:v3.7.1] 1079561351} {[docker.io/openshift/origin-deployer@sha256:db092c265f1a3095f7baa2fba8fc2e8d5ca5df9696130ee67d132fcf1838c104 docker.io/openshift/origin-deployer:v3.7.1] 1079561351} {[docker-registry.default.svc:5000/extended-test-build-valuefrom-gjqzb-xg8hz/test@sha256:2762981e19bf925d58b3d608ba4f350d2db150ecb9f9c70f93f08de499116ec3 docker-registry.default.svc:5000/extended-test-build-valuefrom-gjqzb-xg8hz/test:latest] 621803861} {[docker.io/centos/mongodb-32-centos7@sha256:c884ae6c9c3330f351ddfcb2534ad9243e588d79c0bed38ba333c84e1d4066e7] 605285842} {[docker.io/centos/php-70-centos7@sha256:ddfa75ed7d7e8ef0fb7917f6905de0f28589523d58d6ebc05e95dac9b9de6201 docker.io/centos/php-70-centos7:latest] 600825838} {[docker.io/centos/ruby-22-centos7@sha256:c5470425197e444078e0a62a5f01cd0be9dd39067330dd1a32b964405f9cbd79 docker.io/centos/ruby-22-centos7:latest] 549704762} {[docker.io/centos/ruby-24-centos7@sha256:11faa8e8abbb1eba349128f60e6d98ee7161bdb5c9f721d4b0984b84a8830d50] 530700401} {[docker.io/centos/nodejs-6-centos7@sha256:c51779bbc13bc9f3bb871d14cf549090a1686942c1cb77228b39af02139d3167] 515821008} {[docker.io/cockpit/kubernetes@sha256:38024654f394ba8b4b57bcaf9f78143e20e4b611b989dd8000308689812d6d21 docker.io/cockpit/kubernetes:latest] 400805752} {[docker.io/openshift/origin-base@sha256:107f74d8dd794f6f0b08f6a8ea77a3bce9bb64e46cf089d04275907132d53173 docker.io/openshift/origin-base:latest] 394637142} {[docker.io/openshift/prometheus@sha256:61041872a61902c0dfd10ee5b3dc3fe0f465e98b6e0611f4fa5e26cb4feea4be docker.io/openshift/prometheus:v2.0.0-dev.3] 271096126} {[docker.io/openshift/origin-pod@sha256:ea19819b47721d947bc1f81bf62d62bc9b45ab2049ee44688c984cfdb5d5237c docker.io/openshift/origin-pod:latest] 228576418} {[docker.io/openshift/origin-pod@sha256:8fb61ce6a84faae404ae730bd829c7429464a3faedc787f3a082d1c53fbab635 docker.io/openshift/origin-pod:v3.7.1] 228494688} {[docker.io/openshift/oauth-proxy@sha256:4b73830ee6f7447d0921eedc3946de50016eb8f048d66ea3969abc4116f1e42a docker.io/openshift/oauth-proxy:v1.0.0] 228241928} {[docker.io/openshift/prometheus-alertmanager@sha256:9367a268442151beaab2938cfb8fd103d116c495d28c9228c86e425c0d86509f docker.io/openshift/prometheus-alertmanager:v0.9.1] 220895802} {[docker.io/centos@sha256:2671f7a3eea36ce43609e9fe7435ade83094291055f1c96d9d1d1d7c0b986a5d docker.io/centos:centos7] 207198847} {[docker.io/openshift/prometheus-alert-buffer@sha256:076f8dd576806f5c2dde7e536d020c31aa7d2ec7dcea52da6cbb944895def7ba docker.io/openshift/prometheus-alert-buffer:v0.0.2] 200521084} {[gcr.io/google_containers/jessie-dnsutils@sha256:2460d596912244b5f8973573f7150e7264b570015f4becc2d0096f0bd1d17e36 gcr.io/google_containers/jessie-dnsutils:e2e] 190148402} {[gcr.io/google_containers/nginx-slim@sha256:8b4501fe0fe221df663c22e16539f399e89594552f400408303c42f3dd8d0e52 gcr.io/google_containers/nginx-slim:0.8] 110487599} {[docker.io/nginx@sha256:285b49d42c703fdf257d1e2422765c4ba9d3e37768d6ea83d7fe2043dad6e63d docker.io/nginx:latest] 108492271} {[gcr.io/google_containers/nginx-slim@sha256:dd4efd4c13bec2c6f3fe855deeab9524efe434505568421d4f31820485b3a795 gcr.io/google_containers/nginx-slim:0.7] 86864428} {[gcr.io/google_containers/hostexec@sha256:cab8d4e2526f8f767c64febe4ce9e0f0e58cd35fdff81b3aadba4dd041ba9f00 gcr.io/google_containers/hostexec:1.2] 13209617} {[gcr.io/google_containers/dnsutils@sha256:cd9182f6d74e616942db1cef6f25e1e54b49ba0330c2e19d3ec061f027666cc0 gcr.io/google_containers/dnsutils:e2e] 8897789} {[gcr.io/google_containers/netexec@sha256:56c53846f44ea214e4aa5df37c9c50331f0b09e64a32cc7cf17c7e1808d38eef gcr.io/google_containers/netexec:1.7] 8016035} {[gcr.io/google_containers/serve_hostname@sha256:a49737ee84a3b94f0b977f32e60c5daf11f0b5636f1f7503a2981524f351c57a gcr.io/google_containers/serve_hostname:v1.4] 6222101} {[docker.io/openshift/hello-openshift@sha256:93d608950b9a93f86bbdb7e6f067fa3f4d59afbd5a70d42aa14dcee30055256c docker.io/openshift/hello-openshift:latest] 6089990} {[gcr.io/google_containers/update-demo@sha256:89ac104fa7c43880d2324f377b79be95b0b2b3fb32e4bd03b8d1e6d91a41f009 gcr.io/google_containers/update-demo:nautilus] 4555533} {[gcr.io/google_containers/update-demo@sha256:915a51866b3b77de77c833ac69c699b88d060312d41fb878fab7428b69ef382c gcr.io/google_containers/update-demo:kitten] 4549069} {[gcr.io/google_containers/test-webserver@sha256:f804e8837490d1dfdb5002e073f715fd0a08115de74e5a4847ca952315739372 gcr.io/google_containers/test-webserver:e2e] 4534272} {[gcr.io/google_containers/porter@sha256:e957fd700f36385f469a56bb668d2321a9e1c30f39b08473387cb93ac8b6d2ac gcr.io/google_containers/porter:4524579c0eb935c056c8e75563b4e1eda31587e0] 4413212} {[gcr.io/google_containers/eptest@sha256:bb088b26ed78613cce171420168db9a6c62a8dbea17d7be13077e7010bae162f gcr.io/google_containers/eptest:0.1] 2970692} {[gcr.io/google_containers/busybox@sha256:d8d3bc2c183ed2f9f10e7258f84971202325ee6011ba137112e01e30f206de67 gcr.io/google_containers/busybox:latest] 2433303} {[gcr.io/google_containers/mounttest@sha256:c4dcedb26013ab4231a2b2aaa4eebd5c2a44d5c597fa0613c9ff8bde4fb9fe02 gcr.io/google_containers/mounttest:0.7] 2052704} {[gcr.io/google_containers/mounttest@sha256:bec3122ddcf8bd999e44e46e096659f31241d09f5236bc3dc212ea584ca06856 gcr.io/google_containers/mounttest:0.8] 1450761} {[gcr.io/google_containers/mounttest-user@sha256:5487c126b03abf4119a8f7950cd5f591f72dbe4ab15623f3387d3917e1268b4e gcr.io/google_containers/mounttest-user:0.5] 1450761} {[docker.io/busybox@sha256:4cee1979ba0bf7db9fc5d28fb7b798ca69ae95a47c5fecf46327720df4ff352d] 1146369} {[gcr.io/google_containers/busybox@sha256:4bdd623e848417d96127e16037743f0cd8b528c026e9175e22a84f639eca58ff gcr.io/google_containers/busybox:1.24] 1113554} {[gcr.io/google_containers/pause-amd64@sha256:163ac025575b775d1c0f9bf0bdd0f086883171eb475b5068e7defa4ca9e76516 gcr.io/google_containers/pause-amd64:3.0] 746888} {[gcr.io/google-containers/pause@sha256:9ce5316f9752b8347484ab0f6778573af15524124d52b93230b9a0dcc987e73e gcr.io/google-containers/pause:2.0] 350164}],VolumesInUse:[],VolumesAttached:[],},}
Feb 13 22:57:09.246: INFO: 
Logging kubelet events for node ci-prtest-5a37c28-16210-ig-n-sc0d
Feb 13 22:57:09.262: INFO: 
Logging pods the kubelet thinks is on node ci-prtest-5a37c28-16210-ig-n-sc0d
Feb 13 22:57:09.297: INFO: prometheus-0 started at 2018-02-13 22:47:13 +0000 UTC (0+5 container statuses recorded)
Feb 13 22:57:09.297: INFO: 	Container alert-buffer ready: true, restart count 0
Feb 13 22:57:09.297: INFO: 	Container alertmanager ready: true, restart count 0
Feb 13 22:57:09.297: INFO: 	Container alerts-proxy ready: true, restart count 0
Feb 13 22:57:09.297: INFO: 	Container prom-proxy ready: true, restart count 0
Feb 13 22:57:09.297: INFO: 	Container prometheus ready: true, restart count 0
Feb 13 22:57:09.297: INFO: registry-console-1-9pmxs started at 2018-02-13 22:44:41 +0000 UTC (0+1 container statuses recorded)
Feb 13 22:57:09.297: INFO: 	Container registry-console ready: true, restart count 0
Feb 13 22:57:09.395: INFO: 
Latency metrics for node ci-prtest-5a37c28-16210-ig-n-sc0d
Feb 13 22:57:09.395: INFO: {Operation:sync Method:pod_worker_latency_microseconds Quantile:0.99 Latency:2m31.788994s}
Feb 13 22:57:09.395: INFO: {Operation:sync Method:pod_worker_latency_microseconds Quantile:0.9 Latency:2m22.382118s}
Feb 13 22:57:09.395: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.99 Latency:38.141521s}
Feb 13 22:57:09.395: INFO: {Operation:stop_container Method:docker_operations_latency_microseconds Quantile:0.99 Latency:30.082058s}
Feb 13 22:57:09.395: INFO: {Operation:create Method:pod_worker_latency_microseconds Quantile:0.99 Latency:29.135826s}
Feb 13 22:57:09.395: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.9 Latency:27.146263s}
Feb 13 22:57:09.395: INFO: {Operation:create Method:pod_worker_latency_microseconds Quantile:0.9 Latency:23.360424s}
Feb 13 22:57:09.395: INFO: {Operation:create Method:pod_worker_latency_microseconds Quantile:0.5 Latency:15.320695s}
STEP: Dumping a list of prepulled images on each node...
Feb 13 22:57:09.411: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "e2e-tests-container-probe-gq6j4" for this suite.
Feb 13 22:57:16.773: INFO: namespace: e2e-tests-container-probe-gq6j4, resource: bindings, ignored listing per whitelist
Feb 13 22:57:16.802: INFO: namespace e2e-tests-container-probe-gq6j4 deletion completed in 7.375155935s
@sosiouxme sosiouxme added the kind/test-flake Categorizes issue or PR as related to test flakes. label Feb 14, 2018
@sosiouxme sosiouxme mentioned this issue Feb 14, 2018
Closed
@jwforres
Copy link
Member

@ravisantoshgudimetla please double check that this is indeed a duplicate and close if so

@ravisantoshgudimetla
Copy link
Contributor

@jwforres @sosiouxme This is a dupe of #12072. So, closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ravisantoshgudimetla ravisantoshgudimetla

Labels
component/kubernetes kind/test-flake Categorizes issue or PR as related to test flakes. priority/P1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sosiouxme @jwforres @ravisantoshgudimetla