feat: Add support for using ephemeral credentials for S3 artifacts. F…

…ixes argoproj#5446 (argoproj#5446)

Signed-off-by: Raymond Chow <[email protected]>

.spelling

@@ -190,6 +190,7 @@ namespace
 namespaces
 natively
 nix.conf
+non-IAM
 non-member
 p.m.
 parameterizing

docs/configure-artifact-repository.md

@@ -75,16 +75,20 @@ artifacts:
 
 ## Configuring AWS S3
 
-Create your bucket and access keys for the bucket. AWS access keys have the same
-permissions as the user they are associated with. In particular, you cannot
-create access keys with reduced scope. If you want to limit the permissions for
-an access key, you will need to create a user with just the permissions you want
-to associate with the access key. Otherwise, you can just create an access key
-using your existing user account.
+First, create your bucket:
 
 ```bash
-$ export mybucket=bucket249
-$ cat > policy.json <<EOF
+mybucket=my-bucket-name
+aws s3 mb s3://mybucket [--region xxx]
+```
+
+### AWS S3 IAM Access
+
+Next, create a policy file.
+You will attach this in one of the sections below based on your chosen authentication method.
+
+```bash
+cat > policy.json <<EOF
 {
    "Version":"2012-10-17",
    "Statement":[
@@ -107,43 +111,181 @@ $ cat > policy.json <<EOF
    ]
 }
 EOF
-$ aws s3 mb s3://$mybucket [--region xxx]
-$ aws iam create-user --user-name $mybucket-user
-$ aws iam put-user-policy --user-name $mybucket-user --policy-name $mybucket-policy --policy-document file://policy.json
-$ aws iam create-access-key --user-name $mybucket-user > access-key.json
 ```
 
 If you do not have Artifact Garbage Collection configured, you should remove `s3:DeleteObject` from the list of Actions above.
 
-NOTE: if you want argo to figure out which region your buckets belong in, you
-must additionally set the following statement policy. Otherwise, you must
-specify a bucket region in your workflow configuration.
+!!! Note "Discovering bucket regions"
+    If you want Argo Workflows to figure out which region your buckets belong in, you must additionally set the following statement policy.
+    Otherwise, you must specify a bucket region in your workflow configuration.
 
 ```json
       {
-         "Effect":"Allow",
-         "Action":[
+        "Effect":"Allow",
+        "Action":[
             "s3:GetBucketLocation"
-         ],
-         "Resource":"arn:aws:s3:::*"
+        ],
+        "Resource":"arn:aws:s3:::*"
       }
     ...
 ```
 
-### AWS S3 IRSA
+#### AWS S3 IRSA
+
+IAM Roles for Service Accounts (IRSA) is the recommended Kubernetes native mechanism to authenticate to S3.
+If you are using EKS, follow [the IRSA setup guide](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).
+If not, follow the [Pod Identity Webhook self-hosted setup guide](https://github.com/aws/amazon-eks-pod-identity-webhook/blob/master/SELF_HOSTED_SETUP.md).
 
-If you wish to use S3 IRSA instead of passing in an `accessKey` and `secretKey`, you need to annotate the service account of both the running workflow (in order to save logs/artifacts) and the argo-server pod (in order to retrieve the logs/artifacts).
+With the bucket and policy as described above, create an IAM role and add the policy:
+
+```bash
+aws iam create-role --role-name $mybucket-role
+aws iam put-role-policy --role-name $mybucket-user --policy-name $mybucket-policy --policy-document file://policy.json
+```
+
+To use S3 IRSA, you need to annotate the service account of both the running workflow (in order to save logs/artifacts) and the argo-server pod (in order to retrieve the logs/artifacts).
+You can attach only one IAM role to a service account, so your roles must have permission for all buckets you may wish to use.
 
 ```yaml
 apiVersion: v1
 kind: ServiceAccount
 metadata:
   annotations:
-    eks.amazonaws.com/role-arn: arn:aws:iam::012345678901:role/mybucket
+    eks.amazonaws.com/role-arn: arn:aws:iam::012345678901:role/mybucket-role
   name: myserviceaccount
   namespace: mynamespace
 ```
 
+Use the service account in a workflow:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Workflow
+spec:
+  serviceAccountName: myserviceaccount
+```
+
+#### AWS S3 with IAM Access Keys
+
+!!! Note "Least privilege user"
+    To reduce the privileges of an access key, create a user with only the necessary permissions and no more.
+
+With the bucket and policy described above, create an IAM user and add the policy:
+
+```bash
+aws iam create-user --user-name $mybucket-user
+aws iam put-user-policy --user-name $mybucket-user --policy-name $mybucket-policy --policy-document file://policy.json
+aws iam create-access-key --user-name $mybucket-user > access-key.json
+```
+
+Configure an artifact with the access keys:
+
+```yaml
+artifacts:
+  - name: my-output-artifact
+    path: /my-output-artifact
+    s3:
+      endpoint: s3.amazonaws.com
+      bucket: my-s3-bucket
+      key: path/in/bucket/my-output-artifact.tgz
+      # The following fields are secret selectors.
+      # They reference the k8s secret named 'my-s3-credentials'.
+      # This secret is expected to have the keys 'accessKey' and 'secretKey',
+      # containing the base64 encoded credentials to the bucket.
+      accessKeySecret:
+        name: my-s3-credentials
+        key: accessKey
+      secretKeySecret:
+        name: my-s3-credentials
+        key: secretKey
+```
+
+#### AWS S3 with IAM Assume Role
+
+> v3.6 and after
+
+You can use an [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) for temporary access.
+
+With the bucket and policy described above, create an IAM role and add the policy:
+
+```bash
+aws iam create-role --role-name $mybucket-role
+aws iam put-role-policy --role-name $mybucket-user --policy-name $mybucket-policy --policy-document file://policy.json
+```
+
+Retrieve the role credentials:
+
+```bash
+aws sts assume-role --role-arn arn:aws:iam::012345678901:role/$mybucket-role
+```
+
+Configure an artifact with the credentials:
+
+```yaml
+artifacts:
+  - name: my-output-artifact
+    path: /my-output-artifact
+    s3:
+      endpoint: s3.amazonaws.com
+      bucket: my-s3-bucket
+      key: path/in/bucket/my-output-artifact.tgz
+      # The following fields are secret selectors.
+      # They reference the k8s secret named 'my-s3-credentials'.
+      # This secret is expected to have the keys 'accessKey', 'secretKey', and 'sessionToken',
+      # containing the base64 encoded credentials to the bucket.
+      accessKeySecret:
+        name: my-s3-credentials
+        key: accessKey
+      secretKeySecret:
+        name: my-s3-credentials
+        key: secretKey
+      sessionTokenSecret:
+        name: my-s3-credentials
+        key: sessionToken
+```
+
+!!! Note "Temporary"
+    IAM role credentials are temporary, so you must refresh them periodically via an external mechanism.
+    Argo will not refresh them for you.
+
+### AWS S3 non-IAM Access
+
+#### AWS S3 with Access Grants
+
+> v3.6 and after
+
+You can use [S3 Access Grants](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-grants.html) for temporary, reduced scope access.
+Follow the [AWS guide](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-grants-get-started.html) to set this up in your AWS account and retrieve access grant credentials.
+
+Configure an artifact with the access grant credentials:
+
+```yaml
+artifacts:
+  - name: my-output-artifact
+    path: /my-output-artifact
+    s3:
+      endpoint: s3.amazonaws.com
+      bucket: my-s3-bucket
+      key: path/in/bucket/my-output-artifact.tgz
+      # The following fields are secret selectors.
+      # They reference the k8s secret named 'my-s3-credentials'.
+      # This secret is expected to have the keys 'accessKey', 'secretKey', and 'sessionToken',
+      # containing the base64 encoded credentials to the bucket.
+      accessKeySecret:
+        name: my-s3-credentials
+        key: accessKey
+      secretKeySecret:
+        name: my-s3-credentials
+        key: secretKey
+      sessionTokenSecret:
+        name: my-s3-credentials
+        key: sessionToken
+```
+
+!!! Note "Temporary"
+    S3 Access Grants are temporary, so you must refresh them periodically via an external mechanism.
+    Argo will not refresh them for you.
+
 ## Configuring GCS (Google Cloud Storage)
 
 Create a bucket from the GCP Console

docs/executor_swagger.md

@@ -3139,6 +3139,7 @@ cause implementors to also use a fixed point implementation.
 | region | string| `string` |  | | Region contains the optional bucket region |  |
 | roleARN | string| `string` |  | | RoleARN is the Amazon Resource Name (ARN) of the role to assume. |  |
 | secretKeySecret | [SecretKeySelector](#secret-key-selector)| `SecretKeySelector` |  | |  |  |
+| sessionTokenSecret | [SecretKeySelector](#secret-key-selector)| `SecretKeySelector` |  | |  |  |
 | useSDKCreds | boolean| `bool` |  | | UseSDKCreds tells the driver to figure out credentials based on sdk defaults. |  |
 
 

docs/fields.md

@@ -3526,6 +3526,7 @@ S3Artifact is the location of an S3 artifact
 |`region`|`string`|Region contains the optional bucket region|
 |`roleARN`|`string`|RoleARN is the Amazon Resource Name (ARN) of the role to assume.|
 |`secretKeySecret`|[`SecretKeySelector`](#secretkeyselector)|SecretKeySecret is the secret selector to the bucket's secret key|
+|`sessionTokenSecret`|[`SecretKeySelector`](#secretkeyselector)|SessionTokenSecret is used for credentials which are ephemeral (e.g. IAM assume role or S3 access grant)|
 |`useSDKCreds`|`boolean`|UseSDKCreds tells the driver to figure out credentials based on sdk defaults.|
 
 ## ValueFrom
@@ -4246,6 +4247,7 @@ S3ArtifactRepository defines the controller configuration for an S3 artifact rep
 |`region`|`string`|Region contains the optional bucket region|
 |`roleARN`|`string`|RoleARN is the Amazon Resource Name (ARN) of the role to assume.|
 |`secretKeySecret`|[`SecretKeySelector`](#secretkeyselector)|SecretKeySecret is the secret selector to the bucket's secret key|
+|`sessionTokenSecret`|[`SecretKeySelector`](#secretkeyselector)|SessionTokenSecret is used for credentials which are ephemeral (e.g. IAM assume role or S3 access grant)|
 |`useSDKCreds`|`boolean`|UseSDKCreds tells the driver to figure out credentials based on sdk defaults.|
 
 ## MutexHolding

go.mod

@@ -13,7 +13,7 @@ require (
 	github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible
 	github.com/aliyun/credentials-go v1.3.2
 	github.com/argoproj/argo-events v1.9.1
-	github.com/argoproj/pkg v0.13.7-0.20240208112602-3bb8fe9a0527
+	github.com/argoproj/pkg v0.13.7-0.20240704113442-a69fd34a8117
 	github.com/blushft/go-diagrams v0.0.0-20201006005127-c78c821223d9
 	github.com/colinmarc/hdfs/v2 v2.4.0
 	github.com/coreos/go-oidc/v3 v3.9.0
@@ -46,7 +46,7 @@ require (
 	github.com/sirupsen/logrus v1.9.3
 	github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966
 	github.com/soheilhy/cmux v0.1.5
-	github.com/spf13/cobra v1.8.0
+	github.com/spf13/cobra v1.8.1
 	github.com/spf13/pflag v1.0.5
 	github.com/spf13/viper v1.18.2
 	github.com/stretchr/testify v1.9.0
@@ -87,8 +87,8 @@ require (
 	dario.cat/mergo v1.0.0 // indirect
 	github.com/TylerBrock/colorjson v0.0.0-20200706003622-8a50f05110d2 // indirect
 	github.com/alibabacloud-go/debug v0.0.0-20190504072949-9472017b5c68 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.7 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.3 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.26.2 // indirect
 	github.com/cenkalti/backoff/v4 v4.2.1 // indirect
 	github.com/cloudflare/circl v1.3.7 // indirect
 	github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
@@ -165,25 +165,25 @@ require (
 	github.com/ajg/form v1.5.1 // indirect
 	github.com/andybalholm/brotli v1.0.4 // indirect
 	github.com/awalterschulze/gographviz v0.0.0-20200901124122-0eecad45bd71 // indirect
-	github.com/aws/aws-sdk-go-v2 v1.24.1 // indirect
-	github.com/aws/aws-sdk-go-v2/config v1.26.6 // indirect
-	github.com/aws/aws-sdk-go-v2/credentials v1.16.16 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.11 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.10 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.10 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/ini v1.7.3 // indirect
+	github.com/aws/aws-sdk-go-v2 v1.30.1 // indirect
+	github.com/aws/aws-sdk-go-v2/config v1.27.23 // indirect
+	github.com/aws/aws-sdk-go-v2/credentials v1.17.24 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.9 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.13 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.13 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 // indirect
 	github.com/aws/aws-sdk-go-v2/service/ecr v1.17.8 // indirect
 	github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.13.8 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.10 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.18.7 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.26.7 // indirect
-	github.com/aws/smithy-go v1.19.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.15 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.22.1 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.30.1 // indirect
+	github.com/aws/smithy-go v1.20.3 // indirect
 	github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20220706184558-ce46abcd012b // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/cespare/xxhash/v2 v2.2.0 // indirect
 	github.com/chai2010/gettext-go v1.0.2 // indirect
 	github.com/chrismellard/docker-credential-acr-env v0.0.0-20220327082430-c57b701bfc08 // indirect
-	github.com/cpuguy83/go-md2man/v2 v2.0.3 // indirect
+	github.com/cpuguy83/go-md2man/v2 v2.0.4 // indirect
 	github.com/creack/pty v1.1.21
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
 	github.com/daviddengcn/go-colortext v1.0.0 // indirect