Add Connect function #11

jsafrane · 2019-01-18T14:07:57Z

Add a common function to connect to a CSI driver.

Expectation is that it will be used by every CSI sidecar container so they have the same handling ~~+ the same default timeout (1 minute)~~. Sidecar container should wait indefinitely until CSI driver exposes a socket.

~~Is 1 minute the right timeout?~~

@msau42 @saad-ali @pohly @sbezverk @lpabon @vladimirvivien @davidz627 @gnufied

There is no (unit) test because there is not much to test...

jsafrane · 2019-01-18T16:12:03Z

"Build Still Running" for 2 hours but it's marked as failed.

/retest

jsafrane · 2019-01-21T11:55:16Z

Reworked to wait forever as agreed on Friday's CSI meeting. As result, logging was added to print something every 10 seconds.

pohly

Besides the code changes themselves I think it also would be good to have unit tests. The various timing related aspects are easier to test here with unit tests than with E2E testing in the users of this code:

start connecting while server is not ready, connect quickly (how quickly?) once it is
once connected, detect when server dies and don't reconnect

pohly · 2019-01-21T13:02:31Z

connection/connection.go

+	for {
+		select {
+		case <-ticker.C:
+			glog.Warningf("Still connecting to %s", address)


glog has been replaced in Kubernetes. We should do the same in kubernetes-csi. Can you replace with k8s.io/klog here and in the rest of the new code?

We should move all repos to klog in one (short) burst. Is it the right time?

I think so:

klog became stable in Kubernetes 1.13

if a sidecar hasn't been updated to utility code from that release, it's time to update now

the master branch is open for more intrusive changes like this and we have time before the next release because there are no plans for one yet (in particular we don't need a major release for the upcoming Kubernetes 1.14)

I agree, but we should run it through CSI WG meeting. And add common logging initialization function to this library (as separate PR), so all the sidecar containers have the same cmdline arguments for logging.

connection/connection.go

pohly · 2019-01-21T13:04:52Z

connection/connection.go

+		grpc.WithBlock(),                      // Block until connection succeeds.
+		grpc.WithUnaryInterceptor(LogGRPC),    // Log all messages.
+	)
+	if strings.HasPrefix(address, "/") {


This doesn't handle an explicit unix:/// prefix.

Dial() handles unix:/// just fine without any special dialer.

Are you sure? grpc/grpc-go#1741 was asking for this and then the feature request moved to grpc/grpc-go#1911, which is still open.

I was under the impression that the custom dialer was used specifically to work around that shortcoming in gRPC. Perhaps I misunderstood. 🤷‍♂️

Looks like a good test case, don't you think? 😉

It does indeed work. Then I don't understand what those open gRPC issues are about. It looks like the custom dialer is needed because an absolute path without the unix:/// prefix will not be treated as a Unix domain socket.

I'm still a bit wary about not always using that dialer, though. It creates two different code paths in our Connect function which (theoretically) could have different behavior. In practice I've not found any relevant difference, but the slow tests in PR jsafrane#1 now only cover the case with an absolute path. I only tried the unix:/// variant of those by editing the test source code.

I added code that removes custom dialer and uses unix:// for all sockets. This changes behavior of existing sidecar containers that used custom dialer in old releases, IMO it should be fine.

pohly · 2019-01-21T13:09:30Z

connection/connection.go

+// Connect opens insecure gRPC connection to a CSI driver. Address must have either '<protocol>://' prefix, or be
+// a path to a socket file. The function tries to connect indefinitely every second until it connects.
+// The function automatically adds interceptor for gRPC message logging.
+func Connect(address string, dialOptions ...grpc.DialOption) (*grpc.ClientConn, error) {


Is it worthwhile to pass in a context? Then a caller can decide to abort after a certain time (context.WithTimeout) or run Connect in a goroutine and cancel the operation (context.WithCancel).

IMO, the point of this function is to have the same behavior in all sidecar containers. Adding optional parameters works against this unification.

We can have the same behavior in all sidecars by calling the function the same way, while still writing the underlying function in a more flexible way. But I get your point.

Let me add another reason for passing in a context: when discussing glog vs. klog, it was mentioned that it would be good to pass in a logger via the context instead of using the same global instance. If we had that now, a unit test could select a mock logger to test that the warning messages are printed.

But we don't have that now, so I'm fine with leaving out the context and adding it when really needed. We can deal with the API change in the sidecars on a case-by-case basis, should we do it later.

When we move to klog, a test can add its own logging target to check that the right things are logged. IMO, the less options we offer to callers the better its usage is going to be. I removed dialOptions completely as separate commit.

pohly · 2019-01-22T10:29:17Z

Gopkg.lock

+  branch = "master"
+  name = "github.com/golang/glog"
+  packages = ["."]
+  revision = "23def4e6c14b4da8ac2ed8007337bc5eb5007998"


I think we should all switch to dep >= v0.5.0. There have been some changes to the format of Gopkg.toml (adding a pruneopts = "NUT"). If we use inconsistent versions of dep, we'll end up going back and forth with the content of Gopkg.toml.

Ack, updated to 0.5

pohly · 2019-01-22T11:56:13Z

connection/connection.go

@@ -0,0 +1,71 @@
+package connection


I think we need the usual copyright and license boiler plate code at the beginning of this file.

pohly · 2019-01-25T18:41:44Z

In the January 25th CSI WG meeting we discussed klog and how to handle disconnects.

For klog, we agreed to make the switch, so we should start also here in this PR.

For disconnects I proposed the following:

add an optional callback to Connect that gets called by the dialer before reconnecting.
apps that don't do caching can ignore that and transparently get reconnected to the CSI driver
apps that do caching can inside their callback:
- exit to trigger a restart
- invalidate their cached information and continue running

pohly · 2019-01-25T18:42:35Z

@jsafrane let's chat on Monday to determine who does what. I'm fine with leaving it all to you or take on some of the work.

Some but not all CSI sidecar apps may want to cache information retrieved from a CSI driver. Those (and only) those apps needs a way to detect that the cached information became invalid due to a CSI driver restart and then react to that. The Connect function supports notifying the app via an optional callback which then can: - kill the app via os.Exit - invalidate the cached information and continue by reconnecting - prevent reconnecting and exit the application when gRPC method calls fail with status.Unavailable The default behavior is as before, i.e. the connection is getting restablished.

This verifies that both unix:/// and absolute paths work and that the timing behavior is as expected (wait for server, react promptly once it appears).

connection: add tests

Kubernetes 1.13 already transitioned and the CSI sidecar containers are in the process of following that (for example, in kubernetes-csi/external-provisioner#219).

connection: replace glog with klog

pohly · 2019-02-04T15:32:50Z

/lgtm

msau42 · 2019-02-07T01:32:37Z

/approve

k8s-ci-robot · 2019-02-07T01:32:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jsafrane, msau42

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jsafrane,msau42]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

verify shellcheck

a0f195c Merge pull request kubernetes-csi#106 from msau42/fix-canary 7100c12 Only set staging registry when running canary job b3c65f9 Merge pull request kubernetes-csi#99 from msau42/add-release-process e53f3e8 Merge pull request kubernetes-csi#103 from msau42/fix-canary d129462 Document new method for adding CI jobs are new K8s versions e73c2ce Use staging registry for canary tests 2c09846 Add cleanup instructions to release-notes generation 60e1cd3 Merge pull request kubernetes-csi#98 from pohly/kubernetes-1-19-fixes 0979c09 prow.sh: fix E2E suite for Kubernetes >= 1.18 3b4a2f1 prow.sh: fix installing Go for Kubernetes 1.19.0 1fbb636 Merge pull request kubernetes-csi#97 from pohly/go-1.15 82d108a switch to Go 1.15 d8a2530 Merge pull request kubernetes-csi#95 from msau42/add-release-process 843bddc Add steps on promoting release images 0345a83 Merge pull request kubernetes-csi#94 from linux-on-ibm-z/bump-timeout 1fdf2d5 cloud build: bump timeout in Prow job 41ec6d1 Merge pull request kubernetes-csi#93 from animeshk08/patch-1 5a54e67 filter-junit: Fix gofmt error 0676fcb Merge pull request kubernetes-csi#92 from animeshk08/patch-1 36ea4ff filter-junit: Fix golint error f5a4203 Merge pull request kubernetes-csi#91 from cyb70289/arm64 43e50d6 prow.sh: enable building arm64 image 0d5bd84 Merge pull request kubernetes-csi#90 from pohly/k8s-staging-sig-storage 3df86b7 cloud build: k8s-staging-sig-storage c5fd961 Merge pull request kubernetes-csi#89 from pohly/cloud-build-binfmt db0c2a7 cloud build: initialize support for running commands in Dockerfile be902f4 Merge pull request kubernetes-csi#88 from pohly/multiarch-windows-fix 340e082 build.make: optional inclusion of Windows in multiarch images 5231f05 build.make: properly declare push-multiarch 4569f27 build.make: fix push-multiarch ambiguity 17dde9e Merge pull request kubernetes-csi#87 from pohly/cloud-build bd41690 cloud build: initial set of shared files 9084fec Merge pull request kubernetes-csi#81 from msau42/add-release-process 6f2322e Update patch release notes generation command 0fcc3b1 Merge pull request kubernetes-csi#78 from ggriffiths/fix_csi_snapshotter_rbac_version_set d8c76fe Support local snapshot RBAC for pull jobs c1bdf5b Merge pull request kubernetes-csi#80 from msau42/add-release-process ea1f94a update release tools instructions 152396e Merge pull request kubernetes-csi#77 from ggriffiths/snapshotter201_update 7edc146 Update snapshotter to version 2.0.1 4cf843f Merge pull request kubernetes-csi#76 from pohly/build-targets 3863a0f build for multiple platforms only in CI, add s390x 8322a7d Merge pull request kubernetes-csi#72 from pohly/hostpath-update 7c5a89c prow.sh: use 1.3.0 hostpath driver for testing b8587b2 Merge pull request kubernetes-csi#71 from wozniakjan/test-vet fdb3218 Change 'make test-vet' to call 'go vet' d717c8c Merge pull request kubernetes-csi#69 from pohly/test-driver-config a1432bc Merge pull request kubernetes-csi#70 from pohly/kubelet-feature-gates 5f74333 prow.sh: also configure feature gates for kubelet 84f78b1 prow.sh: generic driver installation 3c34b4f Merge pull request kubernetes-csi#67 from windayski/fix-link fa90abd fix incorrect link ff3cc3f Merge pull request kubernetes-csi#54 from msau42/add-release-process ac8a021 Document the process for releasing a new sidecar 23be652 Merge pull request kubernetes-csi#65 from msau42/update-hostpath 6582f2f Update hostpath driver version to get fix for connection-timeout 4cc9174 Merge pull request kubernetes-csi#64 from ggriffiths/snapshotter_2_version_update 8191eab Update snapshotter to version v2.0.0 3c463fb Merge pull request kubernetes-csi#61 from msau42/enable-snapshots 8b0316c Fix overriding of junit results by using unique names for each e2e run 5f444b8 Merge pull request kubernetes-csi#60 from saad-ali/updateHostpathVersion af9549b Update prow hostpath driver version to 1.3.0-rc2 f6c74b3 Merge pull request kubernetes-csi#57 from ggriffiths/version_gt_kubernetes_fix fc80975 Fix version_gt to work with kubernetes prefix 9f1f3dd Merge pull request kubernetes-csi#56 from msau42/enable-snapshots b98b2ae Enable snapshot tests in 1.17 to be run in non-alpha jobs. 9ace020 Merge pull request kubernetes-csi#52 from msau42/update-readme 540599b Merge pull request kubernetes-csi#53 from msau42/fix-make a4e6299 fix syntax for ppc64le build 771ca6f Merge pull request kubernetes-csi#49 from ggriffiths/prowsh_improve_version_gt d7c69d2 Merge pull request kubernetes-csi#51 from msau42/enable-multinode 4ad6949 Improve snapshot pod running checks and improve version_gt 53888ae Improve README by adding an explicit Kubernetes dependency section 9a7a685 Create a kind cluster with two worker nodes so that the topology feature can be tested. Test cases that test accessing volumes from multiple nodes need to be skipped 4ff2f5f Merge pull request kubernetes-csi#50 from darkowlzz/kind-0.6.0 80bba1f Use kind v0.6.0 6d674a7 Merge pull request kubernetes-csi#47 from Pensu/multi-arch 8adde49 Merge pull request kubernetes-csi#45 from ggriffiths/snapshot_beta_crds 003c14b Add snapshotter CRDs after cluster setup a41f386 Merge pull request kubernetes-csi#46 from mucahitkurt/kind-cluster-cleanup 1eaaaa1 Delete kind cluster after tests run. 83a4ef1 Adding build for ppc64le 4fcafec Merge pull request kubernetes-csi#43 from pohly/system-pod-logging f41c135 prow.sh: also log output of system containers ee22a9c Merge pull request kubernetes-csi#42 from pohly/use-vendor-dir 8067845 travis.yml: also use vendor directory 23df4ae prow.sh: use vendor directory if available a53bd4c Merge pull request kubernetes-csi#41 from pohly/go-version c8a1c4a better handling of Go version 5e773d2 update CI to use Go 1.13.3 f419d74 Merge pull request kubernetes-csi#40 from msau42/add-1.16 e0fde8c Add new variables for 1.16 and remove 1.13 adf00fe Merge pull request kubernetes-csi#36 from msau42/full-clone f1697d2 Do full git clones in travis. Shallow clones are causing test-subtree errors when the depth is exactly 50. 2c81919 Merge pull request kubernetes-csi#34 from pohly/go-mod-tidy 518d6af Merge pull request kubernetes-csi#35 from ddebroy/winbld2 2d6b3ce Build Windows only for amd64 c1078a6 go-get-kubernetes.sh: automate Kubernetes dependency handling 194289a update Go mod support 0affdf9 Merge pull request kubernetes-csi#33 from gnufied/enable-hostpath-expansion 6208f6a Enable hostpath expansion 6ecaa76 Merge pull request kubernetes-csi#30 from msau42/fix-windows ea2f1b5 build windows binaries with .exe suffix 2d33550 Merge pull request kubernetes-csi#29 from mucahitkurt/create-2-node-kind-cluster a8ea8bc create 2-node kind cluster since topology support is added to hostpath driver df8530d Merge pull request kubernetes-csi#27 from pohly/dep-vendor-check 35ceaed prow.sh: install dep if needed f85ab5a Merge pull request kubernetes-csi#26 from ddebroy/windows1 9fba09b Add rule for building Windows binaries 0400867 Merge pull request kubernetes-csi#25 from msau42/fix-master-jobs dc0a5d8 Update kind to v0.5.0 aa85b82 Merge pull request kubernetes-csi#23 from msau42/fix-master-jobs f46191d Kubernetes master changed the way that releases are tagged, which needed changes to kind. There are 3 changes made to prow.sh: 1cac3af Merge pull request kubernetes-csi#22 from msau42/add-1.15-jobs 0c0dc30 prow.sh: tag master images with a large version number f4f73ce Merge pull request kubernetes-csi#21 from msau42/add-1.15-jobs 4e31f07 Change default hostpath driver name to hostpath.csi.k8s.io 4b6fa4a Update hostpath version for sidecar testing to v1.2.0-rc2 ecc7918 Update kind to v0.4.0. This requires overriding Kubernetes versions with specific patch versions that kind 0.4.0 supports. Also, feature gate setting is only supported on 1.15+ due to kind.sigs.k8s.io/v1alpha3 and kubeadm.k8s.io/v1beta2 dependencies. a6f21d4 Add variables for 1.15 db8abb6 Merge pull request kubernetes-csi#20 from pohly/test-driver-config b2f4e05 prow.sh: flexible test driver config 0399988 Merge pull request kubernetes-csi#19 from pohly/go-mod-vendor 066143d build.make: allow repos to use 'go mod' for vendoring 0bee749 Merge pull request kubernetes-csi#18 from pohly/go-version e157b6b update to Go 1.12.4 88dc9a4 Merge pull request kubernetes-csi#17 from pohly/prow 0fafc66 prow.sh: skip sanity testing if component doesn't support it bcac1c1 Merge pull request kubernetes-csi#16 from pohly/prow 0b10f6a prow.sh: update csi-driver-host-path 0c2677e Merge pull request kubernetes-csi#15 from pengzhisun/master ff9bce4 Replace 'return' to 'exit' to fix shellcheck error c60f382 Merge pull request kubernetes-csi#14 from pohly/prow 7aaac22 prow.sh: remove AllAlpha=all, part II 6617773 Merge pull request kubernetes-csi#13 from pohly/prow cda2fc5 prow.sh: avoid AllAlpha=true 546d550 prow.sh: debug failing KinD cluster creation 9b0d9cd build.make: skip shellcheck if Docker is not available aa45a1c prow.sh: more efficient execution of individual tests f3d1d2d prow.sh: fix hostpath driver version check 31dfaf3 prow.sh: fix running of just "alpha" tests f501443 prow.sh: AllAlpha=true for unknown Kubernetes versions 95ae9de Merge pull request kubernetes-csi#9 from pohly/prow d87eccb prow.sh: switch back to upstream csi-driver-host-path 6602d38 prow.sh: different E2E suite depending on Kubernetes version 741319b prow.sh: improve building Kubernetes from source 29545bb prow.sh: take Go version from Kubernetes source 429581c prow.sh: pull Go version from travis.yml 0a0fd49 prow.sh: comment clarification 2069a0a Merge pull request kubernetes-csi#11 from pohly/verify-shellcheck 55212ff initial Prow test job 6c7ba1b build.make: integrate shellcheck into "make test" b2d25d4 verify-shellcheck.sh: make it usable in csi-release-tools 3b6af7b Merge pull request kubernetes-csi#12 from pohly/local-e2e-suite 104a1ac build.make: avoid unit-testing E2E test suite 34010e7 Merge pull request kubernetes-csi#10 from pohly/vendor-check e6db50d check vendor directory fb13c51 verify-shellcheck.sh: import from Kubernetes 94fc1e3 build.make: avoid unit-testing E2E test suite 849db0a Merge pull request kubernetes-csi#8 from pohly/subtree-check-relax cc564f9 verify-subtree.sh: relax check and ignore old content 33d58fd Merge pull request kubernetes-csi#5 from pohly/test-enhancements be8a440 Merge pull request kubernetes-csi#4 from pohly/canary-fix b0336b5 build.make: more readable "make test" output 09436b9 build.make: fix pushing of "canary" image from master branch 147892c build.make: support suppressing checks 154e33d build.make: clarify usage of "make V=1" git-subtree-dir: release-tools git-subtree-split: a0f195c

22c0395 Merge pull request kubernetes-csi#108 from bnrjee/master b5b447b Add go ldflags using LDFLAGS at the time of compilation 16f4afb Merge pull request kubernetes-csi#107 from pohly/kind-update 7bcee13 prow.sh: update to kind 0.9, support Kubernetes 1.19 df518fb prow.sh: usage of Bazel optional c3afd42 Merge pull request kubernetes-csi#104 from xing-yang/snapshot dde93b2 Update to snapshot-controller v3.0.0 a0f195c Merge pull request kubernetes-csi#106 from msau42/fix-canary 7100c12 Only set staging registry when running canary job b3c65f9 Merge pull request kubernetes-csi#99 from msau42/add-release-process e53f3e8 Merge pull request kubernetes-csi#103 from msau42/fix-canary d129462 Document new method for adding CI jobs are new K8s versions e73c2ce Use staging registry for canary tests 2c09846 Add cleanup instructions to release-notes generation 60e1cd3 Merge pull request kubernetes-csi#98 from pohly/kubernetes-1-19-fixes 0979c09 prow.sh: fix E2E suite for Kubernetes >= 1.18 3b4a2f1 prow.sh: fix installing Go for Kubernetes 1.19.0 1fbb636 Merge pull request kubernetes-csi#97 from pohly/go-1.15 82d108a switch to Go 1.15 d8a2530 Merge pull request kubernetes-csi#95 from msau42/add-release-process 843bddc Add steps on promoting release images 0345a83 Merge pull request kubernetes-csi#94 from linux-on-ibm-z/bump-timeout 1fdf2d5 cloud build: bump timeout in Prow job 41ec6d1 Merge pull request kubernetes-csi#93 from animeshk08/patch-1 5a54e67 filter-junit: Fix gofmt error 0676fcb Merge pull request kubernetes-csi#92 from animeshk08/patch-1 36ea4ff filter-junit: Fix golint error f5a4203 Merge pull request kubernetes-csi#91 from cyb70289/arm64 43e50d6 prow.sh: enable building arm64 image 0d5bd84 Merge pull request kubernetes-csi#90 from pohly/k8s-staging-sig-storage 3df86b7 cloud build: k8s-staging-sig-storage c5fd961 Merge pull request kubernetes-csi#89 from pohly/cloud-build-binfmt db0c2a7 cloud build: initialize support for running commands in Dockerfile be902f4 Merge pull request kubernetes-csi#88 from pohly/multiarch-windows-fix 340e082 build.make: optional inclusion of Windows in multiarch images 5231f05 build.make: properly declare push-multiarch 4569f27 build.make: fix push-multiarch ambiguity 17dde9e Merge pull request kubernetes-csi#87 from pohly/cloud-build bd41690 cloud build: initial set of shared files 9084fec Merge pull request kubernetes-csi#81 from msau42/add-release-process 6f2322e Update patch release notes generation command 0fcc3b1 Merge pull request kubernetes-csi#78 from ggriffiths/fix_csi_snapshotter_rbac_version_set d8c76fe Support local snapshot RBAC for pull jobs c1bdf5b Merge pull request kubernetes-csi#80 from msau42/add-release-process ea1f94a update release tools instructions 152396e Merge pull request kubernetes-csi#77 from ggriffiths/snapshotter201_update 7edc146 Update snapshotter to version 2.0.1 4cf843f Merge pull request kubernetes-csi#76 from pohly/build-targets 3863a0f build for multiple platforms only in CI, add s390x 8322a7d Merge pull request kubernetes-csi#72 from pohly/hostpath-update 7c5a89c prow.sh: use 1.3.0 hostpath driver for testing b8587b2 Merge pull request kubernetes-csi#71 from wozniakjan/test-vet fdb3218 Change 'make test-vet' to call 'go vet' d717c8c Merge pull request kubernetes-csi#69 from pohly/test-driver-config a1432bc Merge pull request kubernetes-csi#70 from pohly/kubelet-feature-gates 5f74333 prow.sh: also configure feature gates for kubelet 84f78b1 prow.sh: generic driver installation 3c34b4f Merge pull request kubernetes-csi#67 from windayski/fix-link fa90abd fix incorrect link ff3cc3f Merge pull request kubernetes-csi#54 from msau42/add-release-process ac8a021 Document the process for releasing a new sidecar 23be652 Merge pull request kubernetes-csi#65 from msau42/update-hostpath 6582f2f Update hostpath driver version to get fix for connection-timeout 4cc9174 Merge pull request kubernetes-csi#64 from ggriffiths/snapshotter_2_version_update 8191eab Update snapshotter to version v2.0.0 3c463fb Merge pull request kubernetes-csi#61 from msau42/enable-snapshots 8b0316c Fix overriding of junit results by using unique names for each e2e run 5f444b8 Merge pull request kubernetes-csi#60 from saad-ali/updateHostpathVersion af9549b Update prow hostpath driver version to 1.3.0-rc2 f6c74b3 Merge pull request kubernetes-csi#57 from ggriffiths/version_gt_kubernetes_fix fc80975 Fix version_gt to work with kubernetes prefix 9f1f3dd Merge pull request kubernetes-csi#56 from msau42/enable-snapshots b98b2ae Enable snapshot tests in 1.17 to be run in non-alpha jobs. 9ace020 Merge pull request kubernetes-csi#52 from msau42/update-readme 540599b Merge pull request kubernetes-csi#53 from msau42/fix-make a4e6299 fix syntax for ppc64le build 771ca6f Merge pull request kubernetes-csi#49 from ggriffiths/prowsh_improve_version_gt d7c69d2 Merge pull request kubernetes-csi#51 from msau42/enable-multinode 4ad6949 Improve snapshot pod running checks and improve version_gt 53888ae Improve README by adding an explicit Kubernetes dependency section 9a7a685 Create a kind cluster with two worker nodes so that the topology feature can be tested. Test cases that test accessing volumes from multiple nodes need to be skipped 4ff2f5f Merge pull request kubernetes-csi#50 from darkowlzz/kind-0.6.0 80bba1f Use kind v0.6.0 6d674a7 Merge pull request kubernetes-csi#47 from Pensu/multi-arch 8adde49 Merge pull request kubernetes-csi#45 from ggriffiths/snapshot_beta_crds 003c14b Add snapshotter CRDs after cluster setup a41f386 Merge pull request kubernetes-csi#46 from mucahitkurt/kind-cluster-cleanup 1eaaaa1 Delete kind cluster after tests run. 83a4ef1 Adding build for ppc64le 4fcafec Merge pull request kubernetes-csi#43 from pohly/system-pod-logging f41c135 prow.sh: also log output of system containers ee22a9c Merge pull request kubernetes-csi#42 from pohly/use-vendor-dir 8067845 travis.yml: also use vendor directory 23df4ae prow.sh: use vendor directory if available a53bd4c Merge pull request kubernetes-csi#41 from pohly/go-version c8a1c4a better handling of Go version 5e773d2 update CI to use Go 1.13.3 f419d74 Merge pull request kubernetes-csi#40 from msau42/add-1.16 e0fde8c Add new variables for 1.16 and remove 1.13 adf00fe Merge pull request kubernetes-csi#36 from msau42/full-clone f1697d2 Do full git clones in travis. Shallow clones are causing test-subtree errors when the depth is exactly 50. 2c81919 Merge pull request kubernetes-csi#34 from pohly/go-mod-tidy 518d6af Merge pull request kubernetes-csi#35 from ddebroy/winbld2 2d6b3ce Build Windows only for amd64 c1078a6 go-get-kubernetes.sh: automate Kubernetes dependency handling 194289a update Go mod support 0affdf9 Merge pull request kubernetes-csi#33 from gnufied/enable-hostpath-expansion 6208f6a Enable hostpath expansion 6ecaa76 Merge pull request kubernetes-csi#30 from msau42/fix-windows ea2f1b5 build windows binaries with .exe suffix 2d33550 Merge pull request kubernetes-csi#29 from mucahitkurt/create-2-node-kind-cluster a8ea8bc create 2-node kind cluster since topology support is added to hostpath driver df8530d Merge pull request kubernetes-csi#27 from pohly/dep-vendor-check 35ceaed prow.sh: install dep if needed f85ab5a Merge pull request kubernetes-csi#26 from ddebroy/windows1 9fba09b Add rule for building Windows binaries 0400867 Merge pull request kubernetes-csi#25 from msau42/fix-master-jobs dc0a5d8 Update kind to v0.5.0 aa85b82 Merge pull request kubernetes-csi#23 from msau42/fix-master-jobs f46191d Kubernetes master changed the way that releases are tagged, which needed changes to kind. There are 3 changes made to prow.sh: 1cac3af Merge pull request kubernetes-csi#22 from msau42/add-1.15-jobs 0c0dc30 prow.sh: tag master images with a large version number f4f73ce Merge pull request kubernetes-csi#21 from msau42/add-1.15-jobs 4e31f07 Change default hostpath driver name to hostpath.csi.k8s.io 4b6fa4a Update hostpath version for sidecar testing to v1.2.0-rc2 ecc7918 Update kind to v0.4.0. This requires overriding Kubernetes versions with specific patch versions that kind 0.4.0 supports. Also, feature gate setting is only supported on 1.15+ due to kind.sigs.k8s.io/v1alpha3 and kubeadm.k8s.io/v1beta2 dependencies. a6f21d4 Add variables for 1.15 db8abb6 Merge pull request kubernetes-csi#20 from pohly/test-driver-config b2f4e05 prow.sh: flexible test driver config 0399988 Merge pull request kubernetes-csi#19 from pohly/go-mod-vendor 066143d build.make: allow repos to use 'go mod' for vendoring 0bee749 Merge pull request kubernetes-csi#18 from pohly/go-version e157b6b update to Go 1.12.4 88dc9a4 Merge pull request kubernetes-csi#17 from pohly/prow 0fafc66 prow.sh: skip sanity testing if component doesn't support it bcac1c1 Merge pull request kubernetes-csi#16 from pohly/prow 0b10f6a prow.sh: update csi-driver-host-path 0c2677e Merge pull request kubernetes-csi#15 from pengzhisun/master ff9bce4 Replace 'return' to 'exit' to fix shellcheck error c60f382 Merge pull request kubernetes-csi#14 from pohly/prow 7aaac22 prow.sh: remove AllAlpha=all, part II 6617773 Merge pull request kubernetes-csi#13 from pohly/prow cda2fc5 prow.sh: avoid AllAlpha=true 546d550 prow.sh: debug failing KinD cluster creation 9b0d9cd build.make: skip shellcheck if Docker is not available aa45a1c prow.sh: more efficient execution of individual tests f3d1d2d prow.sh: fix hostpath driver version check 31dfaf3 prow.sh: fix running of just "alpha" tests f501443 prow.sh: AllAlpha=true for unknown Kubernetes versions 95ae9de Merge pull request kubernetes-csi#9 from pohly/prow d87eccb prow.sh: switch back to upstream csi-driver-host-path 6602d38 prow.sh: different E2E suite depending on Kubernetes version 741319b prow.sh: improve building Kubernetes from source 29545bb prow.sh: take Go version from Kubernetes source 429581c prow.sh: pull Go version from travis.yml 0a0fd49 prow.sh: comment clarification 2069a0a Merge pull request kubernetes-csi#11 from pohly/verify-shellcheck 55212ff initial Prow test job 6c7ba1b build.make: integrate shellcheck into "make test" b2d25d4 verify-shellcheck.sh: make it usable in csi-release-tools 3b6af7b Merge pull request kubernetes-csi#12 from pohly/local-e2e-suite 104a1ac build.make: avoid unit-testing E2E test suite 34010e7 Merge pull request kubernetes-csi#10 from pohly/vendor-check e6db50d check vendor directory fb13c51 verify-shellcheck.sh: import from Kubernetes 94fc1e3 build.make: avoid unit-testing E2E test suite 849db0a Merge pull request kubernetes-csi#8 from pohly/subtree-check-relax cc564f9 verify-subtree.sh: relax check and ignore old content 33d58fd Merge pull request kubernetes-csi#5 from pohly/test-enhancements be8a440 Merge pull request kubernetes-csi#4 from pohly/canary-fix b0336b5 build.make: more readable "make test" output 09436b9 build.make: fix pushing of "canary" image from master branch 147892c build.make: support suppressing checks 154e33d build.make: clarify usage of "make V=1" git-subtree-dir: release-tools git-subtree-split: 22c0395c9cb16a65d47197b405a8cc3cc82fced9

4aff857 Merge pull request kubernetes-csi#109 from pohly/alpha-test-defaults 0427289 Merge pull request kubernetes-csi#110 from pohly/kind-0.9-bazel-build-workaround 9a370ab prow.sh: work around "kind build node-image" failure 522361e prow.sh: only run alpha tests for latest Kubernetes release 22c0395 Merge pull request kubernetes-csi#108 from bnrjee/master b5b447b Add go ldflags using LDFLAGS at the time of compilation 16f4afb Merge pull request kubernetes-csi#107 from pohly/kind-update 7bcee13 prow.sh: update to kind 0.9, support Kubernetes 1.19 df518fb prow.sh: usage of Bazel optional c3afd42 Merge pull request kubernetes-csi#104 from xing-yang/snapshot dde93b2 Update to snapshot-controller v3.0.0 a0f195c Merge pull request kubernetes-csi#106 from msau42/fix-canary 7100c12 Only set staging registry when running canary job b3c65f9 Merge pull request kubernetes-csi#99 from msau42/add-release-process e53f3e8 Merge pull request kubernetes-csi#103 from msau42/fix-canary d129462 Document new method for adding CI jobs are new K8s versions e73c2ce Use staging registry for canary tests 2c09846 Add cleanup instructions to release-notes generation 60e1cd3 Merge pull request kubernetes-csi#98 from pohly/kubernetes-1-19-fixes 0979c09 prow.sh: fix E2E suite for Kubernetes >= 1.18 3b4a2f1 prow.sh: fix installing Go for Kubernetes 1.19.0 1fbb636 Merge pull request kubernetes-csi#97 from pohly/go-1.15 82d108a switch to Go 1.15 d8a2530 Merge pull request kubernetes-csi#95 from msau42/add-release-process 843bddc Add steps on promoting release images 0345a83 Merge pull request kubernetes-csi#94 from linux-on-ibm-z/bump-timeout 1fdf2d5 cloud build: bump timeout in Prow job 41ec6d1 Merge pull request kubernetes-csi#93 from animeshk08/patch-1 5a54e67 filter-junit: Fix gofmt error 0676fcb Merge pull request kubernetes-csi#92 from animeshk08/patch-1 36ea4ff filter-junit: Fix golint error f5a4203 Merge pull request kubernetes-csi#91 from cyb70289/arm64 43e50d6 prow.sh: enable building arm64 image 0d5bd84 Merge pull request kubernetes-csi#90 from pohly/k8s-staging-sig-storage 3df86b7 cloud build: k8s-staging-sig-storage c5fd961 Merge pull request kubernetes-csi#89 from pohly/cloud-build-binfmt db0c2a7 cloud build: initialize support for running commands in Dockerfile be902f4 Merge pull request kubernetes-csi#88 from pohly/multiarch-windows-fix 340e082 build.make: optional inclusion of Windows in multiarch images 5231f05 build.make: properly declare push-multiarch 4569f27 build.make: fix push-multiarch ambiguity 17dde9e Merge pull request kubernetes-csi#87 from pohly/cloud-build bd41690 cloud build: initial set of shared files 9084fec Merge pull request kubernetes-csi#81 from msau42/add-release-process 6f2322e Update patch release notes generation command 0fcc3b1 Merge pull request kubernetes-csi#78 from ggriffiths/fix_csi_snapshotter_rbac_version_set d8c76fe Support local snapshot RBAC for pull jobs c1bdf5b Merge pull request kubernetes-csi#80 from msau42/add-release-process ea1f94a update release tools instructions 152396e Merge pull request kubernetes-csi#77 from ggriffiths/snapshotter201_update 7edc146 Update snapshotter to version 2.0.1 4cf843f Merge pull request kubernetes-csi#76 from pohly/build-targets 3863a0f build for multiple platforms only in CI, add s390x 8322a7d Merge pull request kubernetes-csi#72 from pohly/hostpath-update 7c5a89c prow.sh: use 1.3.0 hostpath driver for testing b8587b2 Merge pull request kubernetes-csi#71 from wozniakjan/test-vet fdb3218 Change 'make test-vet' to call 'go vet' d717c8c Merge pull request kubernetes-csi#69 from pohly/test-driver-config a1432bc Merge pull request kubernetes-csi#70 from pohly/kubelet-feature-gates 5f74333 prow.sh: also configure feature gates for kubelet 84f78b1 prow.sh: generic driver installation 3c34b4f Merge pull request kubernetes-csi#67 from windayski/fix-link fa90abd fix incorrect link ff3cc3f Merge pull request kubernetes-csi#54 from msau42/add-release-process ac8a021 Document the process for releasing a new sidecar 23be652 Merge pull request kubernetes-csi#65 from msau42/update-hostpath 6582f2f Update hostpath driver version to get fix for connection-timeout 4cc9174 Merge pull request kubernetes-csi#64 from ggriffiths/snapshotter_2_version_update 8191eab Update snapshotter to version v2.0.0 3c463fb Merge pull request kubernetes-csi#61 from msau42/enable-snapshots 8b0316c Fix overriding of junit results by using unique names for each e2e run 5f444b8 Merge pull request kubernetes-csi#60 from saad-ali/updateHostpathVersion af9549b Update prow hostpath driver version to 1.3.0-rc2 f6c74b3 Merge pull request kubernetes-csi#57 from ggriffiths/version_gt_kubernetes_fix fc80975 Fix version_gt to work with kubernetes prefix 9f1f3dd Merge pull request kubernetes-csi#56 from msau42/enable-snapshots b98b2ae Enable snapshot tests in 1.17 to be run in non-alpha jobs. 9ace020 Merge pull request kubernetes-csi#52 from msau42/update-readme 540599b Merge pull request kubernetes-csi#53 from msau42/fix-make a4e6299 fix syntax for ppc64le build 771ca6f Merge pull request kubernetes-csi#49 from ggriffiths/prowsh_improve_version_gt d7c69d2 Merge pull request kubernetes-csi#51 from msau42/enable-multinode 4ad6949 Improve snapshot pod running checks and improve version_gt 53888ae Improve README by adding an explicit Kubernetes dependency section 9a7a685 Create a kind cluster with two worker nodes so that the topology feature can be tested. Test cases that test accessing volumes from multiple nodes need to be skipped 4ff2f5f Merge pull request kubernetes-csi#50 from darkowlzz/kind-0.6.0 80bba1f Use kind v0.6.0 6d674a7 Merge pull request kubernetes-csi#47 from Pensu/multi-arch 8adde49 Merge pull request kubernetes-csi#45 from ggriffiths/snapshot_beta_crds 003c14b Add snapshotter CRDs after cluster setup a41f386 Merge pull request kubernetes-csi#46 from mucahitkurt/kind-cluster-cleanup 1eaaaa1 Delete kind cluster after tests run. 83a4ef1 Adding build for ppc64le 4fcafec Merge pull request kubernetes-csi#43 from pohly/system-pod-logging f41c135 prow.sh: also log output of system containers ee22a9c Merge pull request kubernetes-csi#42 from pohly/use-vendor-dir 8067845 travis.yml: also use vendor directory 23df4ae prow.sh: use vendor directory if available a53bd4c Merge pull request kubernetes-csi#41 from pohly/go-version c8a1c4a better handling of Go version 5e773d2 update CI to use Go 1.13.3 f419d74 Merge pull request kubernetes-csi#40 from msau42/add-1.16 e0fde8c Add new variables for 1.16 and remove 1.13 adf00fe Merge pull request kubernetes-csi#36 from msau42/full-clone f1697d2 Do full git clones in travis. Shallow clones are causing test-subtree errors when the depth is exactly 50. 2c81919 Merge pull request kubernetes-csi#34 from pohly/go-mod-tidy 518d6af Merge pull request kubernetes-csi#35 from ddebroy/winbld2 2d6b3ce Build Windows only for amd64 c1078a6 go-get-kubernetes.sh: automate Kubernetes dependency handling 194289a update Go mod support 0affdf9 Merge pull request kubernetes-csi#33 from gnufied/enable-hostpath-expansion 6208f6a Enable hostpath expansion 6ecaa76 Merge pull request kubernetes-csi#30 from msau42/fix-windows ea2f1b5 build windows binaries with .exe suffix 2d33550 Merge pull request kubernetes-csi#29 from mucahitkurt/create-2-node-kind-cluster a8ea8bc create 2-node kind cluster since topology support is added to hostpath driver df8530d Merge pull request kubernetes-csi#27 from pohly/dep-vendor-check 35ceaed prow.sh: install dep if needed f85ab5a Merge pull request kubernetes-csi#26 from ddebroy/windows1 9fba09b Add rule for building Windows binaries 0400867 Merge pull request kubernetes-csi#25 from msau42/fix-master-jobs dc0a5d8 Update kind to v0.5.0 aa85b82 Merge pull request kubernetes-csi#23 from msau42/fix-master-jobs f46191d Kubernetes master changed the way that releases are tagged, which needed changes to kind. There are 3 changes made to prow.sh: 1cac3af Merge pull request kubernetes-csi#22 from msau42/add-1.15-jobs 0c0dc30 prow.sh: tag master images with a large version number f4f73ce Merge pull request kubernetes-csi#21 from msau42/add-1.15-jobs 4e31f07 Change default hostpath driver name to hostpath.csi.k8s.io 4b6fa4a Update hostpath version for sidecar testing to v1.2.0-rc2 ecc7918 Update kind to v0.4.0. This requires overriding Kubernetes versions with specific patch versions that kind 0.4.0 supports. Also, feature gate setting is only supported on 1.15+ due to kind.sigs.k8s.io/v1alpha3 and kubeadm.k8s.io/v1beta2 dependencies. a6f21d4 Add variables for 1.15 db8abb6 Merge pull request kubernetes-csi#20 from pohly/test-driver-config b2f4e05 prow.sh: flexible test driver config 0399988 Merge pull request kubernetes-csi#19 from pohly/go-mod-vendor 066143d build.make: allow repos to use 'go mod' for vendoring 0bee749 Merge pull request kubernetes-csi#18 from pohly/go-version e157b6b update to Go 1.12.4 88dc9a4 Merge pull request kubernetes-csi#17 from pohly/prow 0fafc66 prow.sh: skip sanity testing if component doesn't support it bcac1c1 Merge pull request kubernetes-csi#16 from pohly/prow 0b10f6a prow.sh: update csi-driver-host-path 0c2677e Merge pull request kubernetes-csi#15 from pengzhisun/master ff9bce4 Replace 'return' to 'exit' to fix shellcheck error c60f382 Merge pull request kubernetes-csi#14 from pohly/prow 7aaac22 prow.sh: remove AllAlpha=all, part II 6617773 Merge pull request kubernetes-csi#13 from pohly/prow cda2fc5 prow.sh: avoid AllAlpha=true 546d550 prow.sh: debug failing KinD cluster creation 9b0d9cd build.make: skip shellcheck if Docker is not available aa45a1c prow.sh: more efficient execution of individual tests f3d1d2d prow.sh: fix hostpath driver version check 31dfaf3 prow.sh: fix running of just "alpha" tests f501443 prow.sh: AllAlpha=true for unknown Kubernetes versions 95ae9de Merge pull request kubernetes-csi#9 from pohly/prow d87eccb prow.sh: switch back to upstream csi-driver-host-path 6602d38 prow.sh: different E2E suite depending on Kubernetes version 741319b prow.sh: improve building Kubernetes from source 29545bb prow.sh: take Go version from Kubernetes source 429581c prow.sh: pull Go version from travis.yml 0a0fd49 prow.sh: comment clarification 2069a0a Merge pull request kubernetes-csi#11 from pohly/verify-shellcheck 55212ff initial Prow test job 6c7ba1b build.make: integrate shellcheck into "make test" b2d25d4 verify-shellcheck.sh: make it usable in csi-release-tools 3b6af7b Merge pull request kubernetes-csi#12 from pohly/local-e2e-suite 104a1ac build.make: avoid unit-testing E2E test suite 34010e7 Merge pull request kubernetes-csi#10 from pohly/vendor-check e6db50d check vendor directory fb13c51 verify-shellcheck.sh: import from Kubernetes 94fc1e3 build.make: avoid unit-testing E2E test suite 849db0a Merge pull request kubernetes-csi#8 from pohly/subtree-check-relax cc564f9 verify-subtree.sh: relax check and ignore old content 33d58fd Merge pull request kubernetes-csi#5 from pohly/test-enhancements be8a440 Merge pull request kubernetes-csi#4 from pohly/canary-fix b0336b5 build.make: more readable "make test" output 09436b9 build.make: fix pushing of "canary" image from master branch 147892c build.make: support suppressing checks 154e33d build.make: clarify usage of "make V=1" git-subtree-dir: release-tools git-subtree-split: 4aff857d88149e07951fcd1322f462f765401a86

Add Connect function

6ee8a5f

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 18, 2019

k8s-ci-robot requested a review from saad-ali January 18, 2019 14:08

k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jan 18, 2019

jsafrane mentioned this pull request Jan 18, 2019

Add connection package for rpc connection logic #8

Closed

jsafrane force-pushed the connection branch from 1ed39ab to aa1e7af Compare January 21, 2019 11:54

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 21, 2019

pohly requested changes Jan 21, 2019

View reviewed changes

jsafrane force-pushed the connection branch from aa1e7af to 6ee8a5f Compare January 21, 2019 15:00

pohly reviewed Jan 22, 2019

View reviewed changes

jsafrane added 2 commits January 23, 2019 10:29

Update to dep 0.5

e7386db

Review updates

635b0f9

jsafrane force-pushed the connection branch from f15b2dc to 635b0f9 Compare January 23, 2019 09:29

pohly and others added 5 commits January 31, 2019 14:27

connection: add tests

fad9258

This verifies that both unix:/// and absolute paths work and that the timing behavior is as expected (wait for server, react promptly once it appears).

Merge pull request #1 from pohly/connection-tests

23d286e

connection: add tests

connection: replace glog with klog

e2d2c14

Kubernetes 1.13 already transitioned and the CSI sidecar containers are in the process of following that (for example, in kubernetes-csi/external-provisioner#219).

Merge pull request #2 from pohly/connection-logging

155f487

connection: replace glog with klog

k8s-ci-robot assigned pohly Feb 4, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 4, 2019

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 7, 2019

k8s-ci-robot merged commit 763b95d into kubernetes-csi:master Feb 7, 2019

gnufied mentioned this pull request Feb 7, 2019

add implement for CSI plugin kubernetes-csi/external-resizer#7

Merged

This was referenced Feb 7, 2019

Improve driver rpc connection retries #10

Closed

Create common library for rpc connection logic #6

Closed

pohly mentioned this pull request Mar 1, 2019

bring up Kubernetes deployment without restarts intel/pmem-csi#182

Closed

pohly pushed a commit to pohly/csi-lib-utils that referenced this pull request Apr 8, 2019

Merge pull request kubernetes-csi#11 from pohly/verify-shellcheck

2069a0a

verify shellcheck

pohly mentioned this pull request Oct 12, 2020

master: update release-tools #61

Closed

pohly mentioned this pull request Oct 27, 2020

master: update release-tools #63

Closed

pohly mentioned this pull request Oct 27, 2020

master: update release-tools #64

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Connect function #11

Add Connect function #11

jsafrane commented Jan 18, 2019 •

edited

Loading

jsafrane commented Jan 18, 2019

jsafrane commented Jan 21, 2019

pohly left a comment

pohly Jan 21, 2019

jsafrane Jan 21, 2019

pohly Jan 21, 2019

jsafrane Jan 23, 2019

pohly Jan 21, 2019

jsafrane Jan 21, 2019

pohly Jan 21, 2019

pohly Jan 22, 2019

jsafrane Jan 23, 2019

pohly Jan 21, 2019

jsafrane Jan 21, 2019

pohly Jan 21, 2019

jsafrane Jan 23, 2019

pohly Jan 22, 2019

jsafrane Jan 23, 2019

pohly Jan 22, 2019

jsafrane Jan 23, 2019

pohly commented Jan 25, 2019

pohly commented Jan 25, 2019

pohly commented Feb 4, 2019

msau42 commented Feb 7, 2019

k8s-ci-robot commented Feb 7, 2019

Add Connect function #11

Add Connect function #11

Conversation

jsafrane commented Jan 18, 2019 • edited Loading

jsafrane commented Jan 18, 2019

jsafrane commented Jan 21, 2019

pohly left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pohly commented Jan 25, 2019

pohly commented Jan 25, 2019

pohly commented Feb 4, 2019

msau42 commented Feb 7, 2019

k8s-ci-robot commented Feb 7, 2019

jsafrane commented Jan 18, 2019 •

edited

Loading