Use head_bucket vs list_buckets to determine if s3 bucket exists (#357)

Use head_bucket vs list_buckets to determine if s3 bucket exists

SUMMARY
Some S3 endpoints, notably FIPS ones, only support Virtual Hosted-Style addressing which means operations like ListBuckets / Boto3's list_buckets aren't available.
head_bucket is a suitable alternative in this case, and also performs better than list_buckets, especially when many S3 buckets are present
I'm not sure which inconsistencies the developer(s) experienced that led to the comment at https://github.com/ansible-collections/amazon.aws/blob/main/plugins/modules/s3_bucket.py#L448
I didn't run into any issues when running this code modified to use list_buckets and it correctly works with both FIPS and non-FIPS S3 endpoints.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
amazon.aws.s3_bucket

Reviewed-by: Mark Chappell <None>
Reviewed-by: Alina Buzachis <None>
Reviewed-by: None <None>

changelogs/fragments/357-s3_bucket-use-head.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- s3_bucket - updated to use HeadBucket instead of ListBucket when testing for bucket existence (https://github.com/ansible-collections/amazon.aws/pull/357).

plugins/modules/s3_bucket.py

@@ -509,10 +509,12 @@ def create_or_update_bucket(s3_client, module, location):
 
 
 def bucket_exists(s3_client, bucket_name):
-    # head_bucket appeared to be really inconsistent, so we use list_buckets instead,
-    # and loop over all the buckets, even if we know it's less performant :(
-    all_buckets = s3_client.list_buckets(Bucket=bucket_name)['Buckets']
-    return any(bucket['Name'] == bucket_name for bucket in all_buckets)
+    try:
+        s3_client.head_bucket(Bucket=bucket_name)
+        bucket_exists = True
+    except is_boto3_error_code('404'):
+        bucket_exists = False
+    return bucket_exists
 
 
 @AWSRetry.exponential_backoff(max_delay=120)