Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PVC stay "pending", storage provisioner is failing to list #8414

Closed
ctron opened this issue Jun 8, 2020 · 12 comments
Closed

PVC stay "pending", storage provisioner is failing to list #8414

ctron opened this issue Jun 8, 2020 · 12 comments
Labels
addon/storage-provisioner Issues relating to storage provisioner addon kind/support Categorizes issue or PR as a support question. triage/needs-information Indicates an issue needs more information in order to work on it.

Comments

@ctron
Copy link

ctron commented Jun 8, 2020

Steps to reproduce the issue:

  1. minikube start --cpus 4 --memory 8192
  2. Create a PVC
  3. Stays "pending"

Full output of failed command:

NAME                   STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
standard-authservice   Pending                                      standard       8m8s

Full output of minikube start command used, if not already included:

$ minikube start --cpus 4 --memory 8192
😄  minikube v1.11.0 on Fedora 32
✨  Using the kvm2 driver based on user configuration
👍  Starting control plane node minikube in cluster minikube
🔥  Creating kvm2 VM (CPUs=4, Memory=8192MB, Disk=20000MB) ...
🐳  Preparing Kubernetes v1.18.3 on Docker 19.03.8 ...
🔎  Verifying Kubernetes components...
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube"

Logs of the storage provisioner:

E0608 14:16:55.892092       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}

Optional: Full output of minikube logs command:

$  minikube logs
==> Docker <==
-- Logs begin at Mon 2020-06-08 14:13:35 UTC, end at Mon 2020-06-08 14:17:11 UTC. --
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084384384Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084412797Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084440278Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084466096Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084491913Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084518062Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084543569Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084571622Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084599764Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084638050Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084666233Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084691801Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084717598Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084813293Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084856459Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.084886476Z" level=info msg="containerd successfully booted in 0.003299s"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.089955261Z" level=info msg="parsed scheme: \"unix\"" module=grpc
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.089969042Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.089979956Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.089998919Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.090500871Z" level=info msg="parsed scheme: \"unix\"" module=grpc
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.090523161Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.090541953Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.090552046Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.788456402Z" level=warning msg="Your kernel does not support cgroup blkio weight"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.788492352Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.788501684Z" level=warning msg="Your kernel does not support cgroup blkio throttle.read_bps_device"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.788509802Z" level=warning msg="Your kernel does not support cgroup blkio throttle.write_bps_device"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.788517628Z" level=warning msg="Your kernel does not support cgroup blkio throttle.read_iops_device"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.788525266Z" level=warning msg="Your kernel does not support cgroup blkio throttle.write_iops_device"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.788707574Z" level=info msg="Loading containers: start."
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.846003934Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.871199018Z" level=info msg="Loading containers: done."
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.888423858Z" level=info msg="Docker daemon" commit=afacb8b7f0 graphdriver(s)=overlay2 version=19.03.8
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.888494336Z" level=info msg="Daemon has completed initialization"
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.902983257Z" level=info msg="API listen on /var/run/docker.sock"
Jun 08 14:13:47 minikube systemd[1]: Started Docker Application Container Engine.
Jun 08 14:13:47 minikube dockerd[2266]: time="2020-06-08T14:13:47.903413339Z" level=info msg="API listen on [::]:2376"
Jun 08 14:13:57 minikube dockerd[2266]: time="2020-06-08T14:13:57.688962778Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/5fd5ebc2acdeab57f839994966539f3a06679b03781b077e2b32dde7032c3e93/shim.sock" debug=false pid=3212
Jun 08 14:13:57 minikube dockerd[2266]: time="2020-06-08T14:13:57.730800002Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/6707362ed2ced666fb2a7ea684bf9333873852b4e41347e859990d4b6996f7f8/shim.sock" debug=false pid=3236
Jun 08 14:13:57 minikube dockerd[2266]: time="2020-06-08T14:13:57.740147011Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/06890b5bb97f5afa03e66ad7bff7767d17ffeb9665b1d40228c875f5529baf4b/shim.sock" debug=false pid=3257
Jun 08 14:13:57 minikube dockerd[2266]: time="2020-06-08T14:13:57.744826123Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/1e0ceb81030f33893c930b872c68743d7f47464d7261d16fa80cf0972edf38d2/shim.sock" debug=false pid=3273
Jun 08 14:13:57 minikube dockerd[2266]: time="2020-06-08T14:13:57.907415482Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/1849112b7e68fc556ae111e7cb5b045667ba8bd3c4705e7a7eefad618c18cd40/shim.sock" debug=false pid=3411
Jun 08 14:13:57 minikube dockerd[2266]: time="2020-06-08T14:13:57.940724717Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/18c6f9965f7135be476e841aa8b79478733ce7e1cd89ab928073306da4b02dd8/shim.sock" debug=false pid=3453
Jun 08 14:13:57 minikube dockerd[2266]: time="2020-06-08T14:13:57.955871813Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/23b3701b7964662a04c4f2b3f0e80f928b8cf01efdf325445951611636fab911/shim.sock" debug=false pid=3471
Jun 08 14:13:57 minikube dockerd[2266]: time="2020-06-08T14:13:57.971448374Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/ca6832737bb13accaf509153822592021d2465857628e796a7dbfa54f1103139/shim.sock" debug=false pid=3497
Jun 08 14:14:11 minikube dockerd[2266]: time="2020-06-08T14:14:11.692006190Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/d40b0c688033e58ec9712659f4434d79f95074f45d422042414a0f68f228190e/shim.sock" debug=false pid=4162
Jun 08 14:14:11 minikube dockerd[2266]: time="2020-06-08T14:14:11.873908501Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/fbf348e577f89e96e92e6ae64a52d84b9c828943c0e54add017ccda66fc06a74/shim.sock" debug=false pid=4204
Jun 08 14:14:13 minikube dockerd[2266]: time="2020-06-08T14:14:13.946171436Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/bd5d1780df3992f0f0a96c0abada21d1ebe7bfb2711a6fedc33e6d6ce1532657/shim.sock" debug=false pid=4347
Jun 08 14:14:13 minikube dockerd[2266]: time="2020-06-08T14:14:13.947816183Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/63eafbbf787e1e6b9ed9c301717bb47bdeceb5bd5bba42ea890092af534d3bb4/shim.sock" debug=false pid=4349
Jun 08 14:14:14 minikube dockerd[2266]: time="2020-06-08T14:14:14.260511722Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/2c52e693895a3e5a2c5f5b5fb1ad35f583cc378e67fb4170e2fdf293e7589f94/shim.sock" debug=false pid=4457
Jun 08 14:14:14 minikube dockerd[2266]: time="2020-06-08T14:14:14.269180609Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/e995371fe759f78aa36c8a97b8126b929ab1930a359eed851ea4180125377942/shim.sock" debug=false pid=4475
Jun 08 14:14:15 minikube dockerd[2266]: time="2020-06-08T14:14:15.732513327Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/0c6106243dc0cb7bd64bff599704b693609b60d5b12b6fa9b86d49cd66288aca/shim.sock" debug=false pid=4558
Jun 08 14:14:15 minikube dockerd[2266]: time="2020-06-08T14:14:15.922616771Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/80fe81b79446cc9bdb3c74d8762e3a7f98bbe0b10fe527a744e821b61b12d967/shim.sock" debug=false pid=4590
Jun 08 14:16:05 minikube dockerd[2266]: time="2020-06-08T14:16:05.021402093Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/2612dc2ef5a0288c94c3e5173bd6a5bee63aa72496861c6ef215cb2875356039/shim.sock" debug=false pid=5284
Jun 08 14:16:18 minikube dockerd[2266]: time="2020-06-08T14:16:18.208971682Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/6d39f5dd04c8838fbf2b154ef1c04a24d7928c29ef771e2d3aef9540d3448a6a/shim.sock" debug=false pid=5412
Jun 08 14:16:40 minikube dockerd[2266]: time="2020-06-08T14:16:40.442504756Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/bafbb776cb04c26667024b786f195f2dc38f46f976b1df4a74a94fd696d01a40/shim.sock" debug=false pid=5588
Jun 08 14:16:48 minikube dockerd[2266]: time="2020-06-08T14:16:48.069436650Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/97b64840b9ca643f97e248bab3874ac1cee621ee78d20cd49591c752b520c45b/shim.sock" debug=false pid=5811
Jun 08 14:17:01 minikube dockerd[2266]: time="2020-06-08T14:17:01.650719158Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/b9958422803dbd24fb6da668e0b9eac22ebfdb43aff11ddb12320d536e525988/shim.sock" debug=false pid=5967
Jun 08 14:17:04 minikube dockerd[2266]: time="2020-06-08T14:17:04.040112994Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/0ee77f8ba9304e24d1c0f2e8ebfe22afca9a16b4c4fbf7c54a3626dafefe9ed0/shim.sock" debug=false pid=6152

==> container status <==
CONTAINER           IMAGE                                                                                                                          CREATED             STATE               NAME                       ATTEMPT             POD ID
0ee77f8ba9304       quay.io/ctronenmassetesting/none-authservice@sha256:f8f3d6fbea77e025060b04d31dd8bf7a0426acfb544ce2d402b1fb60f85c62e5           8 seconds ago       Running             none-authservice           0                   97b64840b9ca6
b9958422803db       quay.io/ctronenmassetesting/address-space-controller@sha256:a3a3b880823e26741a3c0ff616ef7ba181ab227a6c108206ec6a833f8055293a   10 seconds ago      Running             address-space-controller   0                   bafbb776cb04c
6d39f5dd04c88       quay.io/ctronenmassetesting/controller-manager@sha256:c84c8e8ed1ece517e3220b580645454b82cfae6e5a76fb5a06046d2f0d8e1741         54 seconds ago      Running             controller                 0                   2612dc2ef5a02
80fe81b79446c       4689081edb103                                                                                                                  2 minutes ago       Running             storage-provisioner        0                   0c6106243dc0c
e995371fe759f       67da37a9a360e                                                                                                                  2 minutes ago       Running             coredns                    0                   63eafbbf787e1
2c52e693895a3       67da37a9a360e                                                                                                                  2 minutes ago       Running             coredns                    0                   bd5d1780df399
fbf348e577f89       3439b7546f29b                                                                                                                  3 minutes ago       Running             kube-proxy                 0                   d40b0c688033e
ca6832737bb13       303ce5db0e90d                                                                                                                  3 minutes ago       Running             etcd                       0                   1e0ceb81030f3
23b3701b79646       da26705ccb4b5                                                                                                                  3 minutes ago       Running             kube-controller-manager    0                   06890b5bb97f5
18c6f9965f713       76216c34ed0c7                                                                                                                  3 minutes ago       Running             kube-scheduler             0                   6707362ed2ced
1849112b7e68f       7e28efa976bd1                                                                                                                  3 minutes ago       Running             kube-apiserver             0                   5fd5ebc2acdea

==> coredns [2c52e693895a] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 4e235fcc3696966e76816bcd9034ebc7
CoreDNS-1.6.7
linux/amd64, go1.13.6, da7f65b

==> coredns [e995371fe759] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 4e235fcc3696966e76816bcd9034ebc7
CoreDNS-1.6.7
linux/amd64, go1.13.6, da7f65b

==> describe nodes <==
Name:               minikube
Roles:              master
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=minikube
                    kubernetes.io/os=linux
                    minikube.k8s.io/commit=57e2f55f47effe9ce396cea42a1e0eb4f611ebbd
                    minikube.k8s.io/name=minikube
                    minikube.k8s.io/updated_at=2020_06_08T16_14_04_0700
                    minikube.k8s.io/version=v1.11.0
                    node-role.kubernetes.io/master=
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
                    node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Mon, 08 Jun 2020 14:14:00 +0000
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  minikube
  AcquireTime:     <unset>
  RenewTime:       Mon, 08 Jun 2020 14:17:10 +0000
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Mon, 08 Jun 2020 14:17:10 +0000   Mon, 08 Jun 2020 14:13:58 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Mon, 08 Jun 2020 14:17:10 +0000   Mon, 08 Jun 2020 14:13:58 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Mon, 08 Jun 2020 14:17:10 +0000   Mon, 08 Jun 2020 14:13:58 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Mon, 08 Jun 2020 14:17:10 +0000   Mon, 08 Jun 2020 14:14:10 +0000   KubeletReady                 kubelet is posting ready status
Addresses:
  InternalIP:  192.168.39.150
  Hostname:    minikube
Capacity:
  cpu:                4
  ephemeral-storage:  16954224Ki
  hugepages-2Mi:      0
  memory:             7779552Ki
  pods:               110
Allocatable:
  cpu:                4
  ephemeral-storage:  16954224Ki
  hugepages-2Mi:      0
  memory:             7779552Ki
  pods:               110
System Info:
  Machine ID:                 3030806849aa411db383928972d0005f
  System UUID:                30308068-49aa-411d-b383-928972d0005f
  Boot ID:                    502d8f69-15dd-4c73-b4a2-4f580418ccf7
  Kernel Version:             4.19.107
  OS Image:                   Buildroot 2019.02.10
  Operating System:           linux
  Architecture:               amd64
  Container Runtime Version:  docker://19.3.8
  Kubelet Version:            v1.18.3
  Kube-Proxy Version:         v1.18.3
Non-terminated Pods:          (11 in total)
  Namespace                   Name                                         CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                   ----                                         ------------  ----------  ---------------  -------------  ---
  enmasse-infra               address-space-controller-66c7dcc457-qpcw2    0 (0%)        0 (0%)      0 (0%)           0 (0%)         32s
  enmasse-infra               enmasse-operator-7b9d8cd454-psn8n            0 (0%)        0 (0%)      0 (0%)           0 (0%)         68s
  enmasse-infra               none-authservice-6d5c494df4-gz9lj            0 (0%)        0 (0%)      0 (0%)           0 (0%)         25s
  kube-system                 coredns-66bff467f8-2fvjj                     100m (2%)     0 (0%)      70Mi (0%)        170Mi (2%)     3m2s
  kube-system                 coredns-66bff467f8-jvswj                     100m (2%)     0 (0%)      70Mi (0%)        170Mi (2%)     3m2s
  kube-system                 etcd-minikube                                0 (0%)        0 (0%)      0 (0%)           0 (0%)         3m2s
  kube-system                 kube-apiserver-minikube                      250m (6%)     0 (0%)      0 (0%)           0 (0%)         3m2s
  kube-system                 kube-controller-manager-minikube             200m (5%)     0 (0%)      0 (0%)           0 (0%)         3m2s
  kube-system                 kube-proxy-5rhjs                             0 (0%)        0 (0%)      0 (0%)           0 (0%)         3m2s
  kube-system                 kube-scheduler-minikube                      100m (2%)     0 (0%)      0 (0%)           0 (0%)         3m2s
  kube-system                 storage-provisioner                          0 (0%)        0 (0%)      0 (0%)           0 (0%)         3m8s
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests    Limits
  --------           --------    ------
  cpu                750m (18%)  0 (0%)
  memory             140Mi (1%)  340Mi (4%)
  ephemeral-storage  0 (0%)      0 (0%)
  hugepages-2Mi      0 (0%)      0 (0%)
Events:
  Type    Reason                   Age   From                  Message
  ----    ------                   ----  ----                  -------
  Normal  Starting                 3m2s  kubelet, minikube     Starting kubelet.
  Normal  NodeHasSufficientMemory  3m2s  kubelet, minikube     Node minikube status is now: NodeHasSufficientMemory
  Normal  NodeHasNoDiskPressure    3m2s  kubelet, minikube     Node minikube status is now: NodeHasNoDiskPressure
  Normal  NodeHasSufficientPID     3m2s  kubelet, minikube     Node minikube status is now: NodeHasSufficientPID
  Normal  NodeAllocatableEnforced  3m2s  kubelet, minikube     Updated Node Allocatable limit across pods
  Normal  NodeReady                3m2s  kubelet, minikube     Node minikube status is now: NodeReady
  Normal  Starting                 3m1s  kube-proxy, minikube  Starting kube-proxy.

==> dmesg <==
[Jun 8 14:13] You have booted with nomodeset. This means your GPU drivers are DISABLED
[  +0.000000] Any video related functionality will be severely degraded, and you may not even be able to suspend the system properly
[  +0.000000] Unless you actually understand what nomodeset does, you should reboot without enabling it
[  +0.031432] Decoding supported only on Scalable MCA processors.
[  +0.000351]  #2
[  +0.000971]  #3
[  +1.921033] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[  +0.380058] systemd[1]: Failed to bump fs.file-max, ignoring: Invalid argument
[  +0.002644] systemd-fstab-generator[1172]: Ignoring "noauto" for root device
[  +0.003140] systemd[1]: File /usr/lib/systemd/system/systemd-journald.service:12 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
[  +0.000001] systemd[1]: Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[  +0.580973] NFSD: the nfsdcld client tracking upcall will be removed in 3.10. Please transition to using nfsdcltrack.
[  +0.056294] vboxguest: loading out-of-tree module taints kernel.
[  +0.002378] vboxguest: PCI device not found, probably running on physical hardware.
[  +7.075468] systemd-fstab-generator[2049]: Ignoring "noauto" for root device
[  +0.051965] systemd-fstab-generator[2059]: Ignoring "noauto" for root device
[  +3.314768] systemd-fstab-generator[2255]: Ignoring "noauto" for root device
[  +1.933746] systemd-fstab-generator[2420]: Ignoring "noauto" for root device
[  +0.183058] systemd-fstab-generator[2495]: Ignoring "noauto" for root device
[  +0.811949] systemd-fstab-generator[2717]: Ignoring "noauto" for root device
[  +7.843596] kauditd_printk_skb: 182 callbacks suppressed
[Jun 8 14:14] systemd-fstab-generator[3864]: Ignoring "noauto" for root device
[  +8.222125] kauditd_printk_skb: 32 callbacks suppressed
[  +7.251879] kauditd_printk_skb: 50 callbacks suppressed
[Jun 8 14:15] NFSD: Unable to end grace period: -110
[Jun 8 14:16] kauditd_printk_skb: 2 callbacks suppressed
[ +40.630661] kauditd_printk_skb: 5 callbacks suppressed
[Jun 8 14:17] kauditd_printk_skb: 29 callbacks suppressed

==> etcd [ca6832737bb1] <==
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2020-06-08 14:13:58.081910 I | etcdmain: etcd Version: 3.4.3
2020-06-08 14:13:58.081932 I | etcdmain: Git SHA: 3cf2f69b5
2020-06-08 14:13:58.081934 I | etcdmain: Go Version: go1.12.12
2020-06-08 14:13:58.081936 I | etcdmain: Go OS/Arch: linux/amd64
2020-06-08 14:13:58.081939 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2020-06-08 14:13:58.081978 I | embed: peerTLS: cert = /var/lib/minikube/certs/etcd/peer.crt, key = /var/lib/minikube/certs/etcd/peer.key, trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = 
2020-06-08 14:13:58.082277 I | embed: name = minikube
2020-06-08 14:13:58.082281 I | embed: data dir = /var/lib/minikube/etcd
2020-06-08 14:13:58.082284 I | embed: member dir = /var/lib/minikube/etcd/member
2020-06-08 14:13:58.082285 I | embed: heartbeat = 100ms
2020-06-08 14:13:58.082287 I | embed: election = 1000ms
2020-06-08 14:13:58.082289 I | embed: snapshot count = 10000
2020-06-08 14:13:58.082292 I | embed: advertise client URLs = https://192.168.39.150:2379
2020-06-08 14:13:58.093872 I | etcdserver: starting member 2236e2deb63504cb in cluster d5d2d7cf60dc9e96
raft2020/06/08 14:13:58 INFO: 2236e2deb63504cb switched to configuration voters=()
raft2020/06/08 14:13:58 INFO: 2236e2deb63504cb became follower at term 0
raft2020/06/08 14:13:58 INFO: newRaft 2236e2deb63504cb [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
raft2020/06/08 14:13:58 INFO: 2236e2deb63504cb became follower at term 1
raft2020/06/08 14:13:58 INFO: 2236e2deb63504cb switched to configuration voters=(2465407292199470283)
2020-06-08 14:13:58.105988 W | auth: simple token is not cryptographically signed
2020-06-08 14:13:58.110563 I | etcdserver: starting server... [version: 3.4.3, cluster version: to_be_decided]
2020-06-08 14:13:58.113454 I | etcdserver: 2236e2deb63504cb as single-node; fast-forwarding 9 ticks (election ticks 10)
2020-06-08 14:13:58.114169 I | embed: ClientTLS: cert = /var/lib/minikube/certs/etcd/server.crt, key = /var/lib/minikube/certs/etcd/server.key, trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = 
2020-06-08 14:13:58.114281 I | embed: listening for metrics on http://127.0.0.1:2381
2020-06-08 14:13:58.114338 I | embed: listening for peers on 192.168.39.150:2380
raft2020/06/08 14:13:58 INFO: 2236e2deb63504cb switched to configuration voters=(2465407292199470283)
2020-06-08 14:13:58.115418 I | etcdserver/membership: added member 2236e2deb63504cb [https://192.168.39.150:2380] to cluster d5d2d7cf60dc9e96
raft2020/06/08 14:13:58 INFO: 2236e2deb63504cb is starting a new election at term 1
raft2020/06/08 14:13:58 INFO: 2236e2deb63504cb became candidate at term 2
raft2020/06/08 14:13:58 INFO: 2236e2deb63504cb received MsgVoteResp from 2236e2deb63504cb at term 2
raft2020/06/08 14:13:58 INFO: 2236e2deb63504cb became leader at term 2
raft2020/06/08 14:13:58 INFO: raft.node: 2236e2deb63504cb elected leader 2236e2deb63504cb at term 2
2020-06-08 14:13:58.898851 I | etcdserver: setting up the initial cluster version to 3.4
2020-06-08 14:13:58.921166 N | etcdserver/membership: set the initial cluster version to 3.4
2020-06-08 14:13:58.921262 I | embed: ready to serve client requests
2020-06-08 14:13:58.921353 I | embed: ready to serve client requests
2020-06-08 14:13:58.921433 I | etcdserver/api: enabled capabilities for version 3.4
2020-06-08 14:13:58.923260 I | embed: serving client requests on 127.0.0.1:2379
2020-06-08 14:13:58.923794 I | embed: serving client requests on 192.168.39.150:2379
2020-06-08 14:13:58.924420 I | etcdserver: published {Name:minikube ClientURLs:[https://192.168.39.150:2379]} to cluster d5d2d7cf60dc9e96
2020-06-08 14:17:01.595087 W | etcdserver: read-only range request "key:\"/registry/leases/kube-system/kube-scheduler\" " with result "range_response_count:1 size:480" took too long (303.164126ms) to execute
2020-06-08 14:17:01.595341 W | etcdserver: read-only range request "key:\"/registry/ingress\" range_end:\"/registry/ingrest\" count_only:true " with result "range_response_count:0 size:5" took too long (118.520218ms) to execute

==> kernel <==
 14:17:12 up 3 min,  0 users,  load average: 0.47, 0.23, 0.09
Linux minikube 4.19.107 #1 SMP Thu May 28 15:07:17 PDT 2020 x86_64 GNU/Linux
PRETTY_NAME="Buildroot 2019.02.10"

==> kube-apiserver [1849112b7e68] <==
I0608 14:14:01.851155       1 storage_scheduling.go:134] created PriorityClass system-node-critical with value 2000001000
I0608 14:14:01.873968       1 storage_scheduling.go:134] created PriorityClass system-cluster-critical with value 2000000000
I0608 14:14:01.874023       1 storage_scheduling.go:143] all system priority classes are created successfully or already exist.
I0608 14:14:02.230474       1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0608 14:14:02.265827       1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0608 14:14:02.357334       1 lease.go:224] Resetting endpoints for master service "kubernetes" to [192.168.39.150]
I0608 14:14:02.358088       1 controller.go:606] quota admission added evaluator for: endpoints
I0608 14:14:02.360287       1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0608 14:14:03.357256       1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0608 14:14:03.630347       1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0608 14:14:03.890249       1 controller.go:606] quota admission added evaluator for: deployments.apps
I0608 14:14:04.111101       1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0608 14:14:10.348811       1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0608 14:14:10.531492       1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0608 14:16:05.027456       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.027485       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.043453       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.043474       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.065078       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.065098       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.074852       1 controller.go:606] quota admission added evaluator for: addressplans.admin.enmasse.io
I0608 14:16:05.263344       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.263471       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.270702       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.270723       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.276736       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.276750       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.285483       1 controller.go:606] quota admission added evaluator for: addressspaceplans.admin.enmasse.io
E0608 14:16:05.377828       1 customresource_handler.go:652] error building openapi models for brokeredinfraconfigs.admin.enmasse.io: SchemaError(io.enmasse.admin.v1alpha1.BrokeredInfraConfig.spec.broker.podTemplate.spec.tolerations): array should have exactly one sub-item
I0608 14:16:05.378371       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.378386       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.383898       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.383916       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.392885       1 controller.go:606] quota admission added evaluator for: brokeredinfraconfigs.admin.enmasse.io
E0608 14:16:05.413991       1 customresource_handler.go:652] error building openapi models for standardinfraconfigs.admin.enmasse.io: SchemaError(io.enmasse.admin.v1alpha1.StandardInfraConfig.spec.broker.podTemplate.spec.tolerations): array should have exactly one sub-item
I0608 14:16:05.414520       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.414549       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.421609       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.421628       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.432752       1 controller.go:606] quota admission added evaluator for: standardinfraconfigs.admin.enmasse.io
I0608 14:16:05.849310       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:05.849392       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:05.859255       1 controller.go:606] quota admission added evaluator for: authenticationservices.admin.enmasse.io
I0608 14:16:12.250353       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:12.250374       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:12.268990       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:12.269011       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:12.289919       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:12.289940       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:12.311958       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:12.311980       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:12.335055       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:12.335074       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:12.360252       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:12.360270       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:12.901789       1 client.go:361] parsed scheme: "endpoint"
I0608 14:16:12.901809       1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379  <nil> 0 <nil>}]
I0608 14:16:40.035303       1 controller.go:606] quota admission added evaluator for: consoleservices.admin.enmasse.io
I0608 14:16:57.497054       1 trace.go:116] Trace[1375541189]: "Get" url:/api/v1/namespaces/kube-system/pods/storage-provisioner/log,user-agent:kubectl/v1.18.3 (linux/amd64) kubernetes/2e7996e,client:192.168.39.1 (started: 2020-06-08 14:16:56.910619152 +0000 UTC m=+178.934449651) (total time: 586.369761ms):
Trace[1375541189]: [586.367857ms] [585.060711ms] Transformed response object

==> kube-controller-manager [23b3701b7964] <==
I0608 14:14:10.429025       1 shared_informer.go:230] Caches are synced for endpoint 
W0608 14:14:10.429508       1 actual_state_of_world.go:506] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="minikube" does not exist
I0608 14:14:10.435082       1 event.go:278] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"coredns-66bff467f8", UID:"f02ae73c-3555-45cc-a1ae-b38e274de757", APIVersion:"apps/v1", ResourceVersion:"307", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: coredns-66bff467f8-2fvjj
I0608 14:14:10.440727       1 event.go:278] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"coredns-66bff467f8", UID:"f02ae73c-3555-45cc-a1ae-b38e274de757", APIVersion:"apps/v1", ResourceVersion:"307", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: coredns-66bff467f8-jvswj
I0608 14:14:10.527695       1 shared_informer.go:230] Caches are synced for GC 
I0608 14:14:10.528546       1 shared_informer.go:230] Caches are synced for TTL 
I0608 14:14:10.529152       1 shared_informer.go:230] Caches are synced for daemon sets 
I0608 14:14:10.538081       1 event.go:278] Event(v1.ObjectReference{Kind:"DaemonSet", Namespace:"kube-system", Name:"kube-proxy", UID:"9b219b47-ff73-4b66-883f-57ba88c18a5f", APIVersion:"apps/v1", ResourceVersion:"213", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: kube-proxy-5rhjs
I0608 14:14:10.547598       1 shared_informer.go:230] Caches are synced for job 
E0608 14:14:10.552191       1 daemon_controller.go:292] kube-system/kube-proxy failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kube-proxy", GenerateName:"", Namespace:"kube-system", SelfLink:"/apis/apps/v1/namespaces/kube-system/daemonsets/kube-proxy", UID:"9b219b47-ff73-4b66-883f-57ba88c18a5f", ResourceVersion:"348", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63727222444, loc:(*time.Location)(0x6d09200)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-proxy"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubeadm", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0xc001011040), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001011060)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0xc001011080), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0010110a0)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.LabelSelector)(0xc0010110c0), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-proxy"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"kube-proxy", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(nil), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(0xc000c5b780), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil)}}, v1.Volume{Name:"xtables-lock", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc0010110e0), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil)}}, v1.Volume{Name:"lib-modules", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc001011100), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"kube-proxy", Image:"k8s.gcr.io/kube-proxy:v1.18.3", Command:[]string{"/usr/local/bin/kube-proxy", "--config=/var/lib/kube-proxy/config.conf", "--hostname-override=$(NODE_NAME)"}, Args:[]string(nil), WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"NODE_NAME", Value:"", ValueFrom:(*v1.EnvVarSource)(0xc001011140)}}, Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList(nil)}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"kube-proxy", ReadOnly:false, MountPath:"/var/lib/kube-proxy", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"xtables-lock", ReadOnly:false, MountPath:"/run/xtables.lock", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"lib-modules", ReadOnly:true, MountPath:"/lib/modules", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0xc000f8dbd0), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0xc000783c58), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/os":"linux"}, ServiceAccountName:"kube-proxy", DeprecatedServiceAccount:"kube-proxy", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0xc000323340), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"CriticalAddonsOnly", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}}, HostAliases:[]v1.HostAlias(nil), PriorityClassName:"system-node-critical", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0xc0000c6520)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0xc000783ce8)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:1, NumberReady:0, ObservedGeneration:1, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:1, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "kube-proxy": the object has been modified; please apply your changes to the latest version and try again
I0608 14:14:10.586557       1 shared_informer.go:230] Caches are synced for taint 
I0608 14:14:10.586778       1 node_lifecycle_controller.go:1433] Initializing eviction metric for zone: 
I0608 14:14:10.586887       1 taint_manager.go:187] Starting NoExecuteTaintManager
I0608 14:14:10.587123       1 event.go:278] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"b418145e-7546-452d-9ec9-2fb0eb6bed99", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node minikube event: Registered Node minikube in Controller
W0608 14:14:10.587184       1 node_lifecycle_controller.go:1048] Missing timestamp for Node minikube. Assuming now as a timestamp.
I0608 14:14:10.587308       1 node_lifecycle_controller.go:1249] Controller detected that zone  is now in state Normal.
I0608 14:14:10.688760       1 shared_informer.go:230] Caches are synced for bootstrap_signer 
I0608 14:14:10.854237       1 shared_informer.go:230] Caches are synced for resource quota 
I0608 14:14:10.878472       1 shared_informer.go:230] Caches are synced for disruption 
I0608 14:14:10.878489       1 disruption.go:339] Sending events to api server.
I0608 14:14:10.879138       1 shared_informer.go:230] Caches are synced for stateful set 
I0608 14:14:10.883122       1 shared_informer.go:230] Caches are synced for namespace 
I0608 14:14:10.928209       1 shared_informer.go:230] Caches are synced for resource quota 
I0608 14:14:10.928502       1 shared_informer.go:230] Caches are synced for service account 
I0608 14:14:10.928902       1 shared_informer.go:223] Waiting for caches to sync for garbage collector
I0608 14:14:10.934153       1 shared_informer.go:230] Caches are synced for garbage collector 
I0608 14:14:10.934160       1 garbagecollector.go:142] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0608 14:14:10.973845       1 shared_informer.go:230] Caches are synced for persistent volume 
I0608 14:14:10.978173       1 shared_informer.go:230] Caches are synced for PV protection 
I0608 14:14:10.978620       1 shared_informer.go:230] Caches are synced for expand 
I0608 14:14:10.979228       1 shared_informer.go:230] Caches are synced for attach detach 
I0608 14:14:11.029078       1 shared_informer.go:230] Caches are synced for garbage collector 
I0608 14:16:04.575642       1 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"enmasse-infra", Name:"enmasse-operator", UID:"acf987eb-2562-4c36-84a8-52114359e8fa", APIVersion:"apps/v1", ResourceVersion:"754", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set enmasse-operator-7b9d8cd454 to 1
I0608 14:16:04.585447       1 event.go:278] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"enmasse-infra", Name:"enmasse-operator-7b9d8cd454", UID:"d1127b2d-878f-4d2a-a5d0-bbb09d0258aa", APIVersion:"apps/v1", ResourceVersion:"755", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: enmasse-operator-7b9d8cd454-psn8n
E0608 14:16:05.598222       1 clusterroleaggregation_controller.go:181] edit failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "edit": the object has been modified; please apply your changes to the latest version and try again
I0608 14:16:12.233204       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for addressspaceplans.admin.enmasse.io
I0608 14:16:12.233236       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for consoleservices.admin.enmasse.io
I0608 14:16:12.233255       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for standardinfraconfigs.admin.enmasse.io
I0608 14:16:12.233287       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for addressspaces.enmasse.io
I0608 14:16:12.233306       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for addressplans.admin.enmasse.io
I0608 14:16:12.233349       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for iotprojects.iot.enmasse.io
I0608 14:16:12.233371       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for addresses.enmasse.io
I0608 14:16:12.233390       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for brokeredinfraconfigs.admin.enmasse.io
I0608 14:16:12.233408       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for messagingusers.user.enmasse.io
I0608 14:16:12.233436       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for authenticationservices.admin.enmasse.io
I0608 14:16:12.233454       1 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for iotconfigs.iot.enmasse.io
I0608 14:16:12.233552       1 shared_informer.go:223] Waiting for caches to sync for resource quota
I0608 14:16:12.433667       1 shared_informer.go:230] Caches are synced for resource quota 
I0608 14:16:12.887151       1 shared_informer.go:223] Waiting for caches to sync for garbage collector
I0608 14:16:12.987373       1 shared_informer.go:230] Caches are synced for garbage collector 
I0608 14:16:40.007286       1 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"enmasse-infra", Name:"address-space-controller", UID:"98cd86b6-9d7d-4ad0-83fe-38ec79b95851", APIVersion:"apps/v1", ResourceVersion:"899", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set address-space-controller-66c7dcc457 to 1
I0608 14:16:40.015890       1 event.go:278] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"enmasse-infra", Name:"address-space-controller-66c7dcc457", UID:"0649f206-d573-4886-a55f-7650df56618e", APIVersion:"apps/v1", ResourceVersion:"900", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: address-space-controller-66c7dcc457-qpcw2
I0608 14:16:47.570043       1 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"enmasse-infra", Name:"none-authservice", UID:"13790299-a954-4e82-ae2e-a37fda6d3d4a", APIVersion:"apps/v1", ResourceVersion:"947", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set none-authservice-6d5c494df4 to 1
I0608 14:16:47.575481       1 event.go:278] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"enmasse-infra", Name:"none-authservice-6d5c494df4", UID:"2582a2ae-ba31-42e6-a400-529e8f9a6607", APIVersion:"apps/v1", ResourceVersion:"948", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: none-authservice-6d5c494df4-gz9lj
I0608 14:16:47.882097       1 event.go:278] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"enmasse-infra", Name:"standard-authservice", UID:"6dc09368-36ce-41b0-a7f8-321e0efda85b", APIVersion:"v1", ResourceVersion:"973", FieldPath:""}): type: 'Normal' reason: 'ExternalProvisioning' waiting for a volume to be created, either by external provisioner "k8s.io/minikube-hostpath" or manually created by system administrator
I0608 14:16:47.891756       1 event.go:278] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"enmasse-infra", Name:"standard-authservice", UID:"6dc09368-36ce-41b0-a7f8-321e0efda85b", APIVersion:"v1", ResourceVersion:"973", FieldPath:""}): type: 'Normal' reason: 'ExternalProvisioning' waiting for a volume to be created, either by external provisioner "k8s.io/minikube-hostpath" or manually created by system administrator
I0608 14:16:47.895826       1 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"enmasse-infra", Name:"standard-authservice", UID:"4bb0ffee-b607-4557-b339-8b0d208414cf", APIVersion:"apps/v1", ResourceVersion:"974", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set standard-authservice-7dfc8bd7bc to 1
I0608 14:16:47.915262       1 event.go:278] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"enmasse-infra", Name:"standard-authservice-7dfc8bd7bc", UID:"158d0e6e-2494-4d4a-991e-2186ac32e4c5", APIVersion:"apps/v1", ResourceVersion:"976", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: standard-authservice-7dfc8bd7bc-5m4zz
I0608 14:16:55.975462       1 event.go:278] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"enmasse-infra", Name:"standard-authservice", UID:"6dc09368-36ce-41b0-a7f8-321e0efda85b", APIVersion:"v1", ResourceVersion:"973", FieldPath:""}): type: 'Normal' reason: 'ExternalProvisioning' waiting for a volume to be created, either by external provisioner "k8s.io/minikube-hostpath" or manually created by system administrator
I0608 14:17:10.975614       1 event.go:278] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"enmasse-infra", Name:"standard-authservice", UID:"6dc09368-36ce-41b0-a7f8-321e0efda85b", APIVersion:"v1", ResourceVersion:"973", FieldPath:""}): type: 'Normal' reason: 'ExternalProvisioning' waiting for a volume to be created, either by external provisioner "k8s.io/minikube-hostpath" or manually created by system administrator

==> kube-proxy [fbf348e577f8] <==
W0608 14:14:11.981501       1 server_others.go:559] Unknown proxy mode "", assuming iptables proxy
I0608 14:14:11.986395       1 node.go:136] Successfully retrieved node IP: 192.168.39.150
I0608 14:14:11.986425       1 server_others.go:186] Using iptables Proxier.
W0608 14:14:11.986435       1 server_others.go:436] detect-local-mode set to ClusterCIDR, but no cluster CIDR defined
I0608 14:14:11.986442       1 server_others.go:447] detect-local-mode: ClusterCIDR , defaulting to no-op detect-local
I0608 14:14:11.986625       1 server.go:583] Version: v1.18.3
I0608 14:14:11.987036       1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_max' to 131072
I0608 14:14:11.987062       1 conntrack.go:52] Setting nf_conntrack_max to 131072
I0608 14:14:11.987119       1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
I0608 14:14:11.987155       1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
I0608 14:14:11.987744       1 config.go:315] Starting service config controller
I0608 14:14:11.987756       1 shared_informer.go:223] Waiting for caches to sync for service config
I0608 14:14:11.987775       1 config.go:133] Starting endpoints config controller
I0608 14:14:11.987788       1 shared_informer.go:223] Waiting for caches to sync for endpoints config
I0608 14:14:12.087932       1 shared_informer.go:230] Caches are synced for service config 
I0608 14:14:12.087936       1 shared_informer.go:230] Caches are synced for endpoints config 

==> kube-scheduler [18c6f9965f71] <==
I0608 14:13:58.053920       1 registry.go:150] Registering EvenPodsSpread predicate and priority function
I0608 14:13:58.053952       1 registry.go:150] Registering EvenPodsSpread predicate and priority function
I0608 14:13:58.517179       1 serving.go:313] Generated self-signed cert in-memory
W0608 14:14:00.880984       1 authentication.go:349] Unable to get configmap/extension-apiserver-authentication in kube-system.  Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0608 14:14:00.881000       1 authentication.go:297] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0608 14:14:00.881005       1 authentication.go:298] Continuing without authentication configuration. This may treat all requests as anonymous.
W0608 14:14:00.881009       1 authentication.go:299] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0608 14:14:00.890135       1 registry.go:150] Registering EvenPodsSpread predicate and priority function
I0608 14:14:00.890155       1 registry.go:150] Registering EvenPodsSpread predicate and priority function
W0608 14:14:00.891537       1 authorization.go:47] Authorization is disabled
W0608 14:14:00.891559       1 authentication.go:40] Authentication is disabled
I0608 14:14:00.891572       1 deprecated_insecure_serving.go:51] Serving healthz insecurely on [::]:10251
I0608 14:14:00.894067       1 secure_serving.go:178] Serving securely on 127.0.0.1:10259
I0608 14:14:00.894511       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0608 14:14:00.894525       1 shared_informer.go:223] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0608 14:14:00.894545       1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0608 14:14:00.908238       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0608 14:14:00.908250       1 reflector.go:178] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0608 14:14:00.908240       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0608 14:14:00.908435       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0608 14:14:00.908550       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0608 14:14:00.909128       1 reflector.go:178] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:233: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0608 14:14:00.909255       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0608 14:14:00.909304       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0608 14:14:00.909615       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0608 14:14:01.722022       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0608 14:14:01.957326       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
I0608 14:14:02.294651       1 shared_informer.go:230] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file 
I0608 14:14:04.294440       1 leaderelection.go:242] attempting to acquire leader lease  kube-system/kube-scheduler...
I0608 14:14:04.301845       1 leaderelection.go:252] successfully acquired lease kube-system/kube-scheduler
E0608 14:16:47.901518       1 scheduler.go:599] error selecting node for pod: running "VolumeBinding" filter plugin for pod "standard-authservice-7dfc8bd7bc-5m4zz": pod has unbound immediate PersistentVolumeClaims
E0608 14:16:47.901562       1 factory.go:478] Error scheduling enmasse-infra/standard-authservice-7dfc8bd7bc-5m4zz: running "VolumeBinding" filter plugin for pod "standard-authservice-7dfc8bd7bc-5m4zz": pod has unbound immediate PersistentVolumeClaims; retrying
E0608 14:16:47.914089       1 factory.go:503] pod: enmasse-infra/standard-authservice-7dfc8bd7bc-5m4zz is already present in the active queue
E0608 14:16:47.915527       1 scheduler.go:599] error selecting node for pod: running "VolumeBinding" filter plugin for pod "standard-authservice-7dfc8bd7bc-5m4zz": pod has unbound immediate PersistentVolumeClaims
E0608 14:16:47.915592       1 factory.go:478] Error scheduling enmasse-infra/standard-authservice-7dfc8bd7bc-5m4zz: running "VolumeBinding" filter plugin for pod "standard-authservice-7dfc8bd7bc-5m4zz": pod has unbound immediate PersistentVolumeClaims; retrying

==> kubelet <==
-- Logs begin at Mon 2020-06-08 14:13:35 UTC, end at Mon 2020-06-08 14:17:12 UTC. --
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.451309    3873 state_mem.go:88] [cpumanager] updated default cpuset: ""
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.451333    3873 state_mem.go:96] [cpumanager] updated cpuset assignments: "map[]"
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.451342    3873 policy_none.go:43] [cpumanager] none policy: Start
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.452687    3873 kubelet_node_status.go:70] Attempting to register node minikube
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.452905    3873 plugin_manager.go:114] Starting Kubelet Plugin Manager
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.470273    3873 kubelet_node_status.go:112] Node minikube was previously registered
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.470359    3873 kubelet_node_status.go:73] Successfully registered node minikube
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.643101    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.647731    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.654595    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.661512    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.666644    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.723934    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/6188fbbe64e28a0413e239e610f71669-ca-certs") pod "kube-controller-manager-minikube" (UID: "6188fbbe64e28a0413e239e610f71669")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824136    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "flexvolume-dir" (UniqueName: "kubernetes.io/host-path/6188fbbe64e28a0413e239e610f71669-flexvolume-dir") pod "kube-controller-manager-minikube" (UID: "6188fbbe64e28a0413e239e610f71669")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824180    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/a8caea92c80c24c844216eb1d68fe417-kubeconfig") pod "kube-scheduler-minikube" (UID: "a8caea92c80c24c844216eb1d68fe417")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824204    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/015ee3ca73683b78b91272c2b7d164be-usr-share-ca-certificates") pod "kube-apiserver-minikube" (UID: "015ee3ca73683b78b91272c2b7d164be")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824226    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "xtables-lock" (UniqueName: "kubernetes.io/host-path/61084b7c-b6d5-462a-9b04-9ecab5211976-xtables-lock") pod "kube-proxy-5rhjs" (UID: "61084b7c-b6d5-462a-9b04-9ecab5211976")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824268    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "lib-modules" (UniqueName: "kubernetes.io/host-path/61084b7c-b6d5-462a-9b04-9ecab5211976-lib-modules") pod "kube-proxy-5rhjs" (UID: "61084b7c-b6d5-462a-9b04-9ecab5211976")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824335    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/6188fbbe64e28a0413e239e610f71669-k8s-certs") pod "kube-controller-manager-minikube" (UID: "6188fbbe64e28a0413e239e610f71669")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824358    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/2d1927a8688115f59b09695d0df976ba-etcd-certs") pod "etcd-minikube" (UID: "2d1927a8688115f59b09695d0df976ba")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824402    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-data" (UniqueName: "kubernetes.io/host-path/2d1927a8688115f59b09695d0df976ba-etcd-data") pod "etcd-minikube" (UID: "2d1927a8688115f59b09695d0df976ba")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824438    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/015ee3ca73683b78b91272c2b7d164be-k8s-certs") pod "kube-apiserver-minikube" (UID: "015ee3ca73683b78b91272c2b7d164be")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824485    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "kube-proxy-token-v8zdc" (UniqueName: "kubernetes.io/secret/61084b7c-b6d5-462a-9b04-9ecab5211976-kube-proxy-token-v8zdc") pod "kube-proxy-5rhjs" (UID: "61084b7c-b6d5-462a-9b04-9ecab5211976")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824609    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/6188fbbe64e28a0413e239e610f71669-kubeconfig") pod "kube-controller-manager-minikube" (UID: "6188fbbe64e28a0413e239e610f71669")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824714    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/6188fbbe64e28a0413e239e610f71669-usr-share-ca-certificates") pod "kube-controller-manager-minikube" (UID: "6188fbbe64e28a0413e239e610f71669")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824742    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/015ee3ca73683b78b91272c2b7d164be-ca-certs") pod "kube-apiserver-minikube" (UID: "015ee3ca73683b78b91272c2b7d164be")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824763    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "kube-proxy" (UniqueName: "kubernetes.io/configmap/61084b7c-b6d5-462a-9b04-9ecab5211976-kube-proxy") pod "kube-proxy-5rhjs" (UID: "61084b7c-b6d5-462a-9b04-9ecab5211976")
Jun 08 14:14:10 minikube kubelet[3873]: I0608 14:14:10.824774    3873 reconciler.go:157] Reconciler: start to sync state
Jun 08 14:14:13 minikube kubelet[3873]: I0608 14:14:13.307258    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:14:13 minikube kubelet[3873]: I0608 14:14:13.343689    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:14:13 minikube kubelet[3873]: I0608 14:14:13.429193    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "config-volume" (UniqueName: "kubernetes.io/configmap/6351391e-8ec1-49bd-b0dc-a177b0031556-config-volume") pod "coredns-66bff467f8-jvswj" (UID: "6351391e-8ec1-49bd-b0dc-a177b0031556")
Jun 08 14:14:13 minikube kubelet[3873]: I0608 14:14:13.429243    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "coredns-token-bhpq5" (UniqueName: "kubernetes.io/secret/6351391e-8ec1-49bd-b0dc-a177b0031556-coredns-token-bhpq5") pod "coredns-66bff467f8-jvswj" (UID: "6351391e-8ec1-49bd-b0dc-a177b0031556")
Jun 08 14:14:13 minikube kubelet[3873]: I0608 14:14:13.529427    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "coredns-token-bhpq5" (UniqueName: "kubernetes.io/secret/cae89e7c-6ab4-4b54-9e80-db6a58c4937a-coredns-token-bhpq5") pod "coredns-66bff467f8-2fvjj" (UID: "cae89e7c-6ab4-4b54-9e80-db6a58c4937a")
Jun 08 14:14:13 minikube kubelet[3873]: I0608 14:14:13.529574    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "config-volume" (UniqueName: "kubernetes.io/configmap/cae89e7c-6ab4-4b54-9e80-db6a58c4937a-config-volume") pod "coredns-66bff467f8-2fvjj" (UID: "cae89e7c-6ab4-4b54-9e80-db6a58c4937a")
Jun 08 14:14:14 minikube kubelet[3873]: W0608 14:14:14.134998    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/coredns-66bff467f8-jvswj through plugin: invalid network status for
Jun 08 14:14:14 minikube kubelet[3873]: W0608 14:14:14.137671    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/coredns-66bff467f8-2fvjj through plugin: invalid network status for
Jun 08 14:14:14 minikube kubelet[3873]: W0608 14:14:14.386238    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/coredns-66bff467f8-jvswj through plugin: invalid network status for
Jun 08 14:14:14 minikube kubelet[3873]: W0608 14:14:14.389312    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/coredns-66bff467f8-2fvjj through plugin: invalid network status for
Jun 08 14:14:15 minikube kubelet[3873]: I0608 14:14:15.308497    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:14:15 minikube kubelet[3873]: I0608 14:14:15.432876    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "tmp" (UniqueName: "kubernetes.io/host-path/a7e08806-4308-47ab-8947-7075700c58be-tmp") pod "storage-provisioner" (UID: "a7e08806-4308-47ab-8947-7075700c58be")
Jun 08 14:14:15 minikube kubelet[3873]: I0608 14:14:15.432924    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "storage-provisioner-token-szwtr" (UniqueName: "kubernetes.io/secret/a7e08806-4308-47ab-8947-7075700c58be-storage-provisioner-token-szwtr") pod "storage-provisioner" (UID: "a7e08806-4308-47ab-8947-7075700c58be")
Jun 08 14:16:04 minikube kubelet[3873]: I0608 14:16:04.593819    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:16:04 minikube kubelet[3873]: I0608 14:16:04.736235    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "enmasse-operator-token-bnhhn" (UniqueName: "kubernetes.io/secret/142c7eec-ab29-4e22-8c6c-380894898fc2-enmasse-operator-token-bnhhn") pod "enmasse-operator-7b9d8cd454-psn8n" (UID: "142c7eec-ab29-4e22-8c6c-380894898fc2")
Jun 08 14:16:05 minikube kubelet[3873]: W0608 14:16:05.237514    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/enmasse-operator-7b9d8cd454-psn8n through plugin: invalid network status for
Jun 08 14:16:05 minikube kubelet[3873]: W0608 14:16:05.896219    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/enmasse-operator-7b9d8cd454-psn8n through plugin: invalid network status for
Jun 08 14:16:18 minikube kubelet[3873]: W0608 14:16:18.948699    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/enmasse-operator-7b9d8cd454-psn8n through plugin: invalid network status for
Jun 08 14:16:40 minikube kubelet[3873]: I0608 14:16:40.022121    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:16:40 minikube kubelet[3873]: I0608 14:16:40.195086    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "address-space-controller-token-thgg8" (UniqueName: "kubernetes.io/secret/dc018e6f-0599-4c5c-abbb-99e55a7c0278-address-space-controller-token-thgg8") pod "address-space-controller-66c7dcc457-qpcw2" (UID: "dc018e6f-0599-4c5c-abbb-99e55a7c0278")
Jun 08 14:16:40 minikube kubelet[3873]: W0608 14:16:40.597004    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/address-space-controller-66c7dcc457-qpcw2 through plugin: invalid network status for
Jun 08 14:16:41 minikube kubelet[3873]: W0608 14:16:41.057562    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/address-space-controller-66c7dcc457-qpcw2 through plugin: invalid network status for
Jun 08 14:16:47 minikube kubelet[3873]: I0608 14:16:47.586954    3873 topology_manager.go:233] [topologymanager] Topology Admit Handler
Jun 08 14:16:47 minikube kubelet[3873]: I0608 14:16:47.708063    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-jtb6l" (UniqueName: "kubernetes.io/secret/ec2e9a53-8a5b-46f0-82ea-d20876365a12-default-token-jtb6l") pod "none-authservice-6d5c494df4-gz9lj" (UID: "ec2e9a53-8a5b-46f0-82ea-d20876365a12")
Jun 08 14:16:47 minikube kubelet[3873]: I0608 14:16:47.708093    3873 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "none-authservice-cert" (UniqueName: "kubernetes.io/secret/ec2e9a53-8a5b-46f0-82ea-d20876365a12-none-authservice-cert") pod "none-authservice-6d5c494df4-gz9lj" (UID: "ec2e9a53-8a5b-46f0-82ea-d20876365a12")
Jun 08 14:16:48 minikube kubelet[3873]: W0608 14:16:48.230094    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/none-authservice-6d5c494df4-gz9lj through plugin: invalid network status for
Jun 08 14:16:48 minikube kubelet[3873]: W0608 14:16:48.231184    3873 pod_container_deletor.go:77] Container "97b64840b9ca643f97e248bab3874ac1cee621ee78d20cd49591c752b520c45b" not found in pod's containers
Jun 08 14:16:48 minikube kubelet[3873]: W0608 14:16:48.231529    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/none-authservice-6d5c494df4-gz9lj through plugin: invalid network status for
Jun 08 14:16:49 minikube kubelet[3873]: W0608 14:16:49.241571    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/none-authservice-6d5c494df4-gz9lj through plugin: invalid network status for
Jun 08 14:16:57 minikube kubelet[3873]: I0608 14:16:57.500455    3873 log.go:172] http: superfluous response.WriteHeader call from k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader (httplog.go:205)
Jun 08 14:17:02 minikube kubelet[3873]: W0608 14:17:02.301287    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/address-space-controller-66c7dcc457-qpcw2 through plugin: invalid network status for
Jun 08 14:17:04 minikube kubelet[3873]: W0608 14:17:04.318461    3873 docker_sandbox.go:400] failed to read pod IP from plugin/docker: Couldn't find network status for enmasse-infra/none-authservice-6d5c494df4-gz9lj through plugin: invalid network status for

==> storage-provisioner [80fe81b79446] <==
E0608 14:16:47.876298       1 streamwatcher.go:109] Unable to decode an event from the watch stream: v1.WatchEvent: Object: readObjectFieldAsBytes: expect : after object field, parsing 1043 ...:{},"k:{\"... at {"type":"ADDED","object":{"kind":"PersistentVolumeClaim","apiVersion":"v1","metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"970","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}}
E0608 14:16:47.881400       1 streamwatcher.go:109] Unable to decode an event from the watch stream: v1.WatchEvent: Object: readObjectFieldAsBytes: expect : after object field, parsing 1043 ...:{},"k:{\"... at {"type":"ADDED","object":{"kind":"PersistentVolumeClaim","apiVersion":"v1","metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"970","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}}
E0608 14:16:48.882924       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:49.884098       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:50.885425       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:51.886886       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:52.888407       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:53.889597       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:54.890813       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:55.892092       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:56.893961       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:57.895742       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:58.896981       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:16:59.898285       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:00.899433       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:01.900732       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:02.902457       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:03.903903       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:04.905398       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:05.908153       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:06.909557       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:07.911371       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:08.912790       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:09.914214       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:10.915549       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}
E0608 14:17:11.917531       1 reflector.go:205] k8s.io/minikube/vendor/github.com/r2d4/external-storage/lib/controller/controller.go:411: Failed to list *v1.PersistentVolumeClaim: v1.PersistentVolumeClaimList: Items: []v1.PersistentVolumeClaim: v1.PersistentVolumeClaim: ObjectMeta: v1.ObjectMeta: readObjectFieldAsBytes: expect : after object field, parsing 1204 ...:{},"k:{\"... at {"kind":"PersistentVolumeClaimList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/persistentvolumeclaims","resourceVersion":"973"},"items":[{"metadata":{"name":"standard-authservice","namespace":"enmasse-infra","selfLink":"/api/v1/namespaces/enmasse-infra/persistentvolumeclaims/standard-authservice","uid":"6dc09368-36ce-41b0-a7f8-321e0efda85b","resourceVersion":"973","creationTimestamp":"2020-06-08T14:16:47Z","labels":{"app":"enmasse","app.kubernetes.io/part-of":"standard-authservice","component":"standard-authservice","name":"standard-authservice"},"annotations":{"volume.beta.kubernetes.io/storage-provisioner":"k8s.io/minikube-hostpath"},"ownerReferences":[{"apiVersion":"admin.enmasse.io/v1beta1","kind":"AuthenticationService","name":"standard-authservice","uid":"86d21950-b07d-4517-b733-7c5551591b59","controller":true,"blockOwnerDeletion":true}],"finalizers":["kubernetes.io/pvc-protection"],"managedFields":[{"manager":"controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:app":{},"f:app.kubernetes.io/part-of":{},"f:component":{},"f:name":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"86d21950-b07d-4517-b733-7c5551591b59\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}},"f:status":{"f:phase":{}}}},{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2020-06-08T14:16:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:volume.beta.kubernetes.io/storage-provisioner":{}}}}}]},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"standard","volumeMode":"Filesystem"},"status":{"phase":"Pending"}}]}

@ctron ctron changed the title PVC stay "pending", storage provisioner is missing permissions PVC stay "pending", storage provisioner is failing to list Jun 8, 2020
@ctron
Copy link
Author

ctron commented Jun 8, 2020

Taking a closer look, this is some weird "fails to list" issue, rather than a permission issue.

@medyagh medyagh added the addon/storage-provisioner Issues relating to storage provisioner addon label Jun 8, 2020
@medyagh
Copy link
Member

medyagh commented Jun 8, 2020

@ctron I am curious, could you share the steps to create the PVC so I could replicate this issue myself?

if this is a permission issue, do you mind trying with

minikube start --alsologtostderr --wait=all

@medyagh medyagh added triage/needs-information Indicates an issue needs more information in order to work on it. kind/support Categorizes issue or PR as a support question. labels Jun 8, 2020
@bonifaido
Copy link

It looks like the storage-provisioner image (v1.8.1) that comes from the addon is quite old, and it doesn't work well with more recent versions of Kubernetes. The error message says the error is coming from the github.com/r2d4/external-storage package, which this repository doesn't contain anymore at v1.11.0.

If I change the image to gcr.io/k8s-minikube/storage-provisioner:latest (and apply some necessary RBAC rules, since the new image requires a broader scope of permissions) all PVCs are coming up correctly.

@bonifaido
Copy link

Related: #6523

@mostafa-eltaher
Copy link

It looks like the storage-provisioner image (v1.8.1) that comes from the addon is quite old, and it doesn't work well with more recent versions of Kubernetes. The error message says the error is coming from the github.com/r2d4/external-storage package, which this repository doesn't contain anymore at v1.11.0.

If I change the image to gcr.io/k8s-minikube/storage-provisioner:latest (and apply some necessary RBAC rules, since the new image requires a broader scope of permissions) all PVCs are coming up correctly.

That worked for me, in particular, the following commands fixed the issue on minikube 1.12.1

#minikube delete

docker pull gcr.io/k8s-minikube/storage-provisioner:latest
docker tag gcr.io/k8s-minikube/storage-provisioner:latest gcr.io/k8s-minikube/storage-provisioner:v1.8.1

minikube start

# Taken from: https://github.com/kubernetes/minikube/blob/e5cfa386b1d978096d3e8d917f5ed07b960a457c/deploy/addons/storage-provisioner/storage-provisioner.yaml.tmpl
cat <<EOF | kubectl apply -f -
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: system::leader-locking-storage-provisioner
  namespace: kube-system
  labels:
    addonmanager.kubernetes.io/mode: EnsureExists
rules:
- apiGroups:
  - ""
  resources:
  - endpoints
  verbs:
  - watch
- apiGroups:
  - ""
  resourceNames:
  - k8s.io-minikube-hostpath
  resources:
  - endpoints
  verbs:
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: leader-locking-storage-provisioner
  namespace: kube-system
  labels:
    addonmanager.kubernetes.io/mode: EnsureExists
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: system::leader-locking-storage-provisioner
subjects:
  - kind: ServiceAccount
    name: storage-provisioner
    namespace: kube-system
---
apiVersion: v1
kind: Endpoints
metadata:
  name: k8s.io-minikube-hostpath
  namespace: kube-system
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
EOF

@priyawadhwa
Copy link

Hey @ctron in the past few months we've upgraded the storage provisioner and fixed some bugs. Could you try:

  1. Upgrading to the latest version of minikube, 1.13.1
  2. Let us know if you still see this issue with the latest version?

Thanks.

@priyawadhwa priyawadhwa added the triage/needs-information Indicates an issue needs more information in order to work on it. label Sep 23, 2020
@ctron
Copy link
Author

ctron commented Sep 25, 2020

Hey @ctron in the past few months we've upgraded the storage provisioner and fixed some bugs. Could you try:

1. Upgrading to the latest version of minikube, 1.13.1

2. Let us know if you still see this issue with the latest version?

Thanks.

Will do.

@tstromberg
Copy link
Contributor

@ctron - any update? We're now on minikube v1.4.0 -- though I don't think the storage provisioner has changed since v1.13.1.

@medyagh
Copy link
Member

medyagh commented Nov 25, 2020

@ctron do you still have this issue with latest version of minikube?

@ctron
Copy link
Author

ctron commented Nov 26, 2020

I am now on minikube version: v1.14.0 … and it seems to work.

@ctron
Copy link
Author

ctron commented Nov 26, 2020

I just checked, looks like 0.15.1 is the most recent. I will upgrade and test in the next few days. But I think it is ok to close this issue.

@tstromberg
Copy link
Contributor

Thanks for the update!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
addon/storage-provisioner Issues relating to storage provisioner addon kind/support Categorizes issue or PR as a support question. triage/needs-information Indicates an issue needs more information in order to work on it.
Projects
None yet
Development

No branches or pull requests

6 participants