Fix S3 directory detection based on ContentType header

[MiguelWeezardo]

It seems S3 is now returning a charset in the Content-Type header (e.g. application/x-directory; charset=UTF-8) for directories uploaded through S3 GUI.
This breaks directory detection.

[findepi]

Should this perhaps be case insensitive?
or perhaps parse this using a proper utility class?

MediaType.parse(metadata.getContentType()).is(MediaType.create("application", "x-directory"))

[findepi]

(edited)

[MiguelWeezardo]

I also noticed this code above my change, which seems dead, since the metadata object is present even for a directory path:

ObjectMetadata metadata = getS3ObjectMetadata(path);

if (metadata == null) {
    // check if this path is a directory
    Iterator<LocatedFileStatus> iterator = listPrefix(path, OptionalInt.of(1), ListingMode.SHALLOW_ALL);
    if (iterator.hasNext()) {
        return new FileStatus(0, true, 1, 0, 0, qualifiedPath(path));
    }
    throw new FileNotFoundException("File does not exist: " + path);
}

return new FileStatus(
        getObjectSize(path, metadata),
        // Some directories (e.g. uploaded through S3 GUI) return a charset in the Content-Type header
        MediaType.parse(metadata.getContentType()).is(DIRECTORY_MEDIA_TYPE),
        1,
        BLOCK_SIZE.toBytes(),
        lastModifiedTime(metadata),
        qualifiedPath(path));

Is it OK that directories on S3 report a size of 32MB (BLOCK_SIZE)?

[findepi]

cc @electrum