CSI migration from in-tree driver to CSI driver go beta

[andyzhangx]

Is your feature request related to a problem? Please describe.

Describe the solution you'd like in detail

design doc: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-migration.md

depends on cs-attacher v3.0.0: #430

depends on csi-provisioner fix: kubernetes/kubernetes#94853

Describe alternatives you've considered

Additional context

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[andyzhangx]

now depends on this PR: kubernetes/kubernetes#90162, this PR would cherry-pick to 1.17, 1.18, and azure file and disk migration would go beta in 1.19

[rfranzke]

Sounds great. Happy to help with opening the cherry-pick PRs after kubernetes/kubernetes#90162 is merged.

[andyzhangx]

Azure disk & azure file migration would go beta in 1.19, here is the PR: kubernetes/kubernetes#90896
related KEP:
kubernetes/enhancements#1490

also there is a migration issue should be fixed in https://github.com/kubernetes-csi/external-provisioner (by vendoring fix kubernetes/kubernetes#90162 in k8s.io/csi-translation-lib v0.19.0)
working image: andyzhangx/csi-provisioner:v1.3.0-fix

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[andyzhangx]

waiting for https://github.com/kubernetes-csi/external-provisioner v2.0.0 release

[andyzhangx]

related work: #419

[andyzhangx]

waiting for

• v0.19.3 release on https://github.com/kubernetes/csi-translation-lib with PR: Automated cherry pick of #94853: fix azure file migration panic kubernetes/kubernetes#94904
• upgrade to csi-translation-lib v0.19.3 and release https://github.com/kubernetes-csi/external-provisioner v2.0.4
• upgrade to csi-provisioner v2.0.4 in this driver (current version is v2.0.2)
• enable migration CI test

[andyzhangx]

csi-provisioner has been upgraded to v2.0.4, now try to enable csi migration test: kubernetes/test-infra#19790

[andyzhangx]

update:
https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_azurefile-csi-driver/466/pull-azurefile-csi-driver-e2e-migration/1323155056895201280/build-log.txt

Summarizing 5 Failures:

[Fail] Dynamic Provisioning [It] should create a volume on demand with mount options [kubernetes.io/azure-file] [file.csi.azure.com] [Windows] 
/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:514

[Fail] Dynamic Provisioning [It] should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [kubernetes.io/azure-file] [file.csi.azure.com] [Windows] 
/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:519

[Fail] Dynamic Provisioning [It] should create a volume on demand and mount it as readOnly in a pod [kubernetes.io/azure-file] [file.csi.azure.com] [Windows] 
/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:548

[Fail] Dynamic Provisioning [It] should create a deployment object, write and read to it, delete the pod and write and read to it again [kubernetes.io/azure-file] [file.csi.azure.com] [Windows] 
/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:408

[Fail] Dynamic Provisioning [It] should create a volume on demand and resize it [kubernetes.io/azure-file] [file.csi.azure.com] [Windows] 
/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/test/e2e/testsuites/testsuites.go:514

Ran 6 of 25 Specs in 4414.212 seconds
FAIL! -- 1 Passed | 5 Failed | 0 Pending | 19 Skipped
--- FAIL: TestE2E (4414.22s)

[andyzhangx]

waiting for kubernetes/kubernetes#96099 merged into k8s v1.17.14, v1.18.11

[andyzhangx]

would be addressed by PR: #430
mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.0.2 has the fix already: kubernetes/kubernetes#94904

[andyzhangx]

found blocking issue: kubernetes/kubernetes#96508

[andyzhangx]

kubernetes/kubernetes#96508 would be fixed by kubernetes/kubernetes#97877, I think that's the last fix, migration test pass with PR: kubernetes/kubernetes#97877

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.