Image corruption when uploading to S3 using Boto3 in Lambda

[RyanFitzSimmonsAK]

Describe the bug

Original issue : aws/aws-sdk#799

I'm encountering an issue when uploading image files to S3 using Boto3 in a Lambda function. The uploaded image is corrupted.

Image format: JPG
S3 bucket settings: No Public Access, no specific ACLs or encryption.
Lambda environment: Python 3.12

I've tried different image formats and S3 configurations without success. Any help would be appreciated.

Expected Behavior

The image should be uploaded to S3 without corruption.

Current Behavior

The uploaded image is corrupted and cannot be viewed correctly.

Reproduction Steps

s3 = boto3.client('s3')

postMethod = 'POST'
uploadFile = '/media/file'

def lambda_handler(event, context):
    logger.info(event)
    httpMethod = event['httpMethod']
    path = event['path']
    
    if httpMethod == postMethod and path == uploadFile:
        file_data = event['body']
        response = s3_upload_file(file_data)
    else:
        response = {
            'statusCode': 404,
            'body': json.dumps('Resource not found')
        }
    return response
        
def s3_upload_file(data):
    try:
        file_name = str(uuid.uuid4())
        response = s3.put_object(
            Body=data,
            Bucket='imggallery-st',
            Key=file_name,
            ContentType='image/jpg'
        )
        return {
            'statusCode': 200,
            'body': json.dumps('File uploaded successfully')
        }
    except ClientError as e:
        return {
            'statusCode': 500,
            'body': json.dumps(str(e))
        }

Possible Solution

No response

Additional Information/Context

No response

SDK version used

Unknown

Environment details (OS name and version, etc.)

Python 3.12

[RyanFitzSimmonsAK]

Hey @xanderbilla, thanks for reaching out. Could you provide debug logs of uploading an image that exhibits this behavior? You can debug logs by adding boto3.set_stream_logger('') to the top of your script, and redacting any sensitive information. Could you also be more specific about what happens with these images when you try to open them? Thanks!

[xanderbilla]

Thank You for your response @RyanFitzSimmonsAK . Here is the log you requested and what happen to the image when I try to open them they show image not supported

[INFO]	2024-08-06T21:54:03.450Z	964aa95d-5012-4e1d-8841-88747d40248e	{'resource': '/media/file', 'path': '/media/file', 'httpMethod': 'POST', 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br', 'Cache-Control': 'no-cache', 'Content-Type': 'image/jpeg', 'Host': 'ubn7z2jcn6.execute-api.ap-south-1.amazonaws.com', 'Postman-Token': '5addb0b1-d69b-45ff-9b20-6dd0bc1cf247', 'User-Agent': 'PostmanRuntime/7.39.1', 'X-Amzn-Trace-Id': 'Root=1-66b29b7a-47be17322a532a572c368539', 'X-Forwarded-For': '2.5.191.210', 'X-Forwarded-Port': '443', 'X-Forwarded-Proto': 'https'}, 'multiValueHeaders': {'Accept': ['*/*'], 'Accept-Encoding': ['gzip, deflate, br'], 'Cache-Control': ['no-cache'], 'Content-Type': ['image/jpeg'], 'Host': ['7z2jcn6.execute-api.ap-south-1.amazonaws.com'], 'Postman-Token': ['5addb0b1-d69b-45ff-9b20-6dd0bc1cf247'], 'User-Agent': ['PostmanRuntime/7.39.1'], 'X-Amzn-Trace-Id': ['Root=1-66b29b7a-47be17322a532a572c368539'], 'X-Forwarded-For': ['2.5.191.210'], 'X-Forwarded-Port': ['443'], 'X-Forwarded-Proto': ['https']}, 'queryStringParameters': None, 'multiValueQueryStringParameters': None, 'pathParameters': None, 'stageVariables': None, 'requestContext': {'resourceId': 'fpalt8', 'resourcePath': '/media/file', 'httpMethod': 'POST', 'extendedRequestId': 'cG07OEdCBcwEJ5w=', 'requestTime': '06/Aug/2024:21:54:02 +0000', 'path': '/dev/media/file', 'accountId': '929910138721', 'protocol': 'HTTP/1.1', 'stage': 'dev', 'domainPrefix': 'ubn7z2jcn6', 'requestTimeEpoch': 1722981242791, 'requestId': '05de19f4-1303-4b0f-8c9d-e8c03268496d', 'identity': {'cognitoIdentityPoolId': None, 'accountId': None, 'cognitoIdentityId': None, 'caller': None, 'sourceIp': '152.59.191.210', 'principalOrgId': None, 'accessKey': None, 'cognitoAuthenticationType': None, 'cognitoAuthenticationProvider': None, 'userArn': None, 'userAgent': 'PostmanRuntime/7.39.1', 'user': None}, 'domainName': 'ubn7z2jcn6.execute-api.ap-south-1.amazonaws.com', 'deploymentId': '9kv73x', 'apiId': 'ubn7z2jcn6'}, 'body':  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX, 'isBase64Encoded': False}

2024-08-06 21:54:03,450 root [INFO] {'resource': '/media/file', 'path': '/media/file', 'httpMethod': 'POST', 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br', 'Cache-Control': 'no-cache', 'Content-Type': 'image/jpeg', 'Host': 'ubn7z2jcn6.execute-api.ap-south-1.amazonaws.com', 'Postman-Token': '5addb0b1-d69b-45ff-9b20-6dd0bc1cf247', 'User-Agent': 'PostmanRuntime/7.39.1', 'X-Amzn-Trace-Id': 'Root=1-66b29b7a-47be17322a532a572c368539', 'X-Forwarded-For': '2.5.191.210', 'X-Forwarded-Port': '443', 'X-Forwarded-Proto': 'https'}, 'multiValueHeaders': {'Accept': ['*/*'], 'Accept-Encoding': ['gzip, deflate, br'], 'Cache-Control': ['no-cache'], 'Content-Type': ['image/jpeg'], 'Host': ['7z2jcn6.execute-api.ap-south-1.amazonaws.com'], 'Postman-Token': ['5addb0b1-d69b-45ff-9b20-6dd0bc1cf247'], 'User-Agent': ['PostmanRuntime/7.39.1'], 'X-Amzn-Trace-Id': ['Root=1-66b29b7a-47be17322a532a572c368539'], 'X-Forwarded-For': ['2.5.191.210'], 'X-Forwarded-Port': ['443'], 'X-Forwarded-Proto': ['https']}, 'queryStringParameters': None, 'multiValueQueryStringParameters': None, 'pathParameters': None, 'stageVariables': None, 'requestContext': {'resourceId': 'fpalt8', 'resourcePath': '/media/file', 'httpMethod': 'POST', 'extendedRequestId': 'cG07OEdCBcwEJ5w=', 'requestTime': '06/Aug/2024:21:54:02 +0000', 'path': '/dev/media/file', 'accountId': '929910138721', 'protocol': 'HTTP/1.1', 'stage': 'dev', 'domainPrefix': 'ubn7z2jcn6', 'requestTimeEpoch': 1722981242791, 'requestId': '05de19f4-1303-4b0f-8c9d-e8c03268496d', 'identity': {'cognitoIdentityPoolId': None, 'accountId': None, 'cognitoIdentityId': None, 'caller': None, 'sourceIp': '152.59.191.210', 'principalOrgId': None, 'accessKey': None, 'cognitoAuthenticationType': None, 'cognitoAuthenticationProvider': None, 'userArn': None, 'userAgent': 'PostmanRuntime/7.39.1', 'user': None}, 'domainName': 'ubn7z2jcn6.execute-api.ap-south-1.amazonaws.com', 'deploymentId': '9kv73x', 'apiId': 'ubn7z2jcn6'}, 'body': '����\x00\x, 'isBase64Encoded': False}

[RyanFitzSimmonsAK]

Hey, thanks for providing those. I see that your user agent contains 'PostmanRuntime/7.39.1'. If you try to upload this image using exclusively Boto3 and Lambda, are you still getting this behavior? Full debug logs would preferable, showing client initialization, request, and response. Thanks!

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.