kubetest: add "EKS" provider

[gyuho]

https://github.com/aws/awstester implements

test-infra/kubetest/main.go

Lines 223 to 230 in 9a88e32

	type deployer interface {
	Up() error
	IsUp() error
	DumpClusterLogs(localPath, gcsPath string) error
	TestSetup() error
	Down() error
	GetClusterCreated(gcpProject string) (time.Time, error)
	}

We have been running internal Prow cluster to test AWS ALB Ingress Controller. Please see https://github.com/gyuho/aws-alb-ingress-controller/pull/3 for example output.

Action items:

• Create a PR to add EKS plugin to "kubetest" (done via kubetest: add EKS plugin #9811)
• Change DumpClusterLogs and Publish methods to use upstream Google Cloud Storage
• Merge EKS plugin to "kubernetes/test-infra" (pending kubetest: add EKS plugin #9811)
• Add Prow job for aws-ebs-csi-driver (Done via Add AWS EBS CSI driver to prow job #9872)
• Add aws-ebs-csi-driver tests to upstream test grid (Done via Add AWS EBS CSI driver to prow job #9872)
• Mount AWS test account credential to upstream Prow cluster
• Add ci-kubernetes-e2e-aws-eks Prow job (as a non-blocking post-submit job) (pending *: add EKS periodic jobs, upgrade "awstester" vendor #9940)
• Add aws-alb-ingress-controller tests to upstream test grid
  • adding prow configuration for aws-alb-ingress-controller #9976
• Add Prow job for aws-alb-ingress-controller
  • adding prow configuration for aws-alb-ingress-controller #9976
• Figure out how to use "dind" for building aws-alb-ingress-controller images
  • Just like containerd-cri-presubmit-jobs.yaml
• Add Prow job for cloud-provider-aws
• Add cloud-provider-aws tests to upstream test grid

/cc @d-nishi @BenTheElder @krzyzacy

[BenTheElder]

Add pull-kubernetes-e2e-aws-eks Prow job

What's the intention with this job? NB we've been moving providers out of tree and we've moved away from having GKE in the blocking path generally. We're even looking at not having any providers in k/k presubmit someday ...

cc @spiffxp @fejta

[gyuho]

we've been moving providers out of tree and we've moved away from having GKE in the blocking path generally.

Good to know. Yes, we are happy to follow the new pattern.

Do we have any issue to track this? Would each provider have its own repository to trigger Prow jobs?

[BenTheElder]

I'm not sure what the tracking issue is but yeah, each cloud provider has a repo and can have presubmits on that repo. We'll also need non-presubmits though to give more signal for Kubernetes releases. @dims might be a good person to talk to on that.

[d-nishi]

@BenTheElder this is not just for cloud provider code. this tool allows us to post CI signal for all the SIG AWS code that sits in k8s-sigs.

[BenTheElder]

Right I get that the tool does this, I was specifically asking about pull-kubernetes-e2e-aws-eks which sounds like something we're getting away from. VS say a presubmit on the k8s-sigs repo, or continuous integration testing against k8s (postsubmit / periodic, not presubmit).

[krzyzacy]

cc @justinsb

[d-nishi]

@BenTheElder pull-kubernetes-e2e-aws-eks is not blocking k/k.
We reused a naming convention that might be causing the confusion. This tool is only providing CI signal for k8s-sigs repos related to sig-aws subprojects until we merge this work into cluster-api framework. @gyuho can you confirm?

[gyuho]

@BenTheElder I was proposing EKS k/k e2e test suites equivalent to pull-kubernetes-e2e-gke (named pull-kubernetes-e2e-aws-eks), and agree that we should have provider-specific tests as a non-blocking presubmit job. We can start as a post-submit.

This tool will also be used for sub-projects like ALB Ingress Controller, which requires EKS cluster.

This tool is only providing CI signal for k8s-sigs repos related to sig-aws subprojects until we merge this work into cluster-api framework

Correct.

[krzyzacy]

pull-kubernetes-e2e-gke only runs if cluster/ changes because it's using underlying scripts.

I'll be able to make better suggestions after reading what your deployer is doing :-)

[BenTheElder]

SGTM, thanks!

On a related note the naming conventions are probably worth reconsidering at some point, would love suggestions :-)

Also +1 to what @krzyzacy said, adding that:

We should definitely have blocking vendor presubmits in k8s-sigs! I'd start by introducing it as non blocking there too until we work out any necessary fixes but quickly transition it to blocking there 😅

We might even want a release blocking job (periodic) eventually for k/k, how that will work with the k-sigs providers is still tbd but eventually when there are no in-tree providers we'll need to make some decision about this.

I'd love to get this in the conformance dashboard once we have something working. (Note: nothing CNCF official, just signal for us to keep those tests and kubernetes core functionality healthy, similar TBD discussing more about how we should continue or not to block releases on some or all of those..)

[BenTheElder]

BTW: have you see the prowjob pod presets support? It seems like a lot of those AWSTESTER_EKS* env could go in one or more of those to make it easier for people to create prowjobs using it.

[gyuho]

BTW: have you see the prowjob pod presets support.

Have not. Great idea to package them into a preset. Will look into it. Thanks!

[d-nishi]

Fantastic!

[spiffxp]

/milestone v1.14

[spiffxp]

/area kubetest

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[dims]

/close

we don't need this right @gyuho ?

[k8s-ci-robot]

@dims: Closing this issue.

In response to this:

/close

we don't need this right @gyuho ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.