Task 5

[Lunarr0]

Title: Implement Import Service with S3 Integration and Lambda Functions

Description:

This pull request implements the Import Service as part of Task 5 requirements. The Import Service is responsible for handling file uploads to an S3 bucket and processing CSV files upon creation. Below are the implemented features:

Implemented Features

✅ Link to frontend app

✅ Link to frontend pr

✅ Task 5.1: Import Service and S3 Configuration

• Created a new service import-service at the same level as product-service.
• Configured AWS CDK Stack for Import Service.
• Created and configured a new S3 bucket with a folder named uploaded.

✅ Task 5.2: Import Products File Lambda Function

• Implemented a Lambda function importProductsFile.
• The function is triggered by an HTTP GET request to /import.
• It expects a query parameter name for the CSV file name.
• Generates a signed URL with the key uploaded/${fileName}.
• Configured AWS CDK Stack with necessary IAM policies for Lambda-S3 interaction.
• Updated the frontend application to integrate with the importProductsFile Lambda function.

✅ Task 5.3: Import File Parser Lambda Function

• Implemented a Lambda function importFileParser.
• The function is triggered by the s3:ObjectCreated:* event.
• It processes only new objects in the uploaded/ folder.
• Uses a readable stream to retrieve and parse the CSV file with csv-parser.
• Logs each parsed record to CloudWatch for monitoring.
• Configured AWS CDK Stack to define the event trigger and permission

s.

✅ Task 5.4: Git Workflow and Submission

• Committed all work to a separate branch task-5.
• Created this pull request to merge task-5 into master.
• Ready for code review and crosscheck submission.

Additional Implementations (Optional Tasks)

✅ Unit Tests:

• Added unit tests for importProductsFile using mocked AWS SDK methods.
• Added unit tests for importFileParser to ensure correct parsing behavior.

✅ File Processing Enhancement:

• Implemented functionality to move processed files from uploaded/ to parsed/.
• This involves copying the file to parsed/ and then deleting it from uploaded/.

How to Test:

1. Invoke importProductsFile Lambda function via the API Gateway URL with a valid name query parameter.
   example:

   
   
    TEST_URL=$(curl -X GET "https://5uj67empu0.execute-api.us-east-1.amazonaws.com/prod/import?name=test.csv")
   
   

   Then use the echo "$TEST_URL" command to get Response:

https://myimportservicebucket.s3.amazonaws.com/uploaded/test.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAYQNJSSB3ABIYGCZZ%2F20250306%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250306T173036Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEOr%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIQDAIOgZx7a3JeIRf0HaOquM4WXITp6CxMpgs0ARrYWOuQIgIfqJ8Axk%2F42CTfuVTQMTSJ7V1oxeFZTcAxaV5kr38t0qyQMIMxAAGgw1ODUwMDgwNTg0ODYiDOsgPGNYAFVtv%2FUk5CqmAzorXs3K3p1p0brVgeFJiC8D0uejz4md1ob9WCh%2Bh%2Ffv3qvQ2GFdoE9Wb1dmtMtf2eepAna1uSEIzTvt46LH0ss2x%2BGnypgYgHf1tx4StP6Se%2B9gSNrgjminzqz1pkjGd83JT3jyIamGypABBfihR1P64ppEZHA2CrOCrIDMr4sFQwTmvGScgwb82k8zBoY%2FGlJLcgt%2FqCMhknIKF9Bw0pv3mz8AsycQuXw4bP%2FnM%2BXBlHbQG4LmfBDDtFcVrz6cKDro631TzusUlSyjVRFQjJSr4riQDyUdOI8OCfXXlfeCLgs%2FOrSkc7EktUCxpMXM9n0xMD1cQn8t1H4XNSBiSqwB1qYDcezn5UTgOtDkY46e9ihFq9xmM18WGf7LQSPwXl1xm6GPtBJqSIRXTve0VRroGhp5dcwdns3iPLONZSOjRC1Eq46L%2FCXHktoIZ02XSVJV8y5uRu4NdCCiWTbICx2sEBUIxCrZp0j3b%2BwhGimtFZ2KgKt2CTrSaGePJBhKnxMUUYuTh7BJUMR2f1dB1%2BoqtRe1vx2ThLTWlkbCgBqcIpWu853EMLG1p74GOp4BCWUtzKc3QodydHRfzsbxto7VG8zI%2FeNwvRNqYGnB8OLdDcVQ9OHH%2FXsazMQAtUeDXdxXmMnVIYq5Iqjvw%2BMI19IcjiucXkCgEavw0pLko%2BEsReqCNBBCUjEy7RaTIGwXL16%2FPBWcd9XUpIwQnsGDMA%2B0HXoUo%2BdNo86hXy2cqokn9MjRc8BmF63FBgi76KJ4Sg4naDcgS89LfYhqZn0%3D&X-Amz-Signature=38cf48384626f89da7a97afa00155607091300275b07047fc2669d384a97e32a

2. Upload a CSV file using the signed URL returned by the Lambda function.

   example:

   curl -X PUT -T test.csv "$TEST_URL" -H "Content-Type: text/csv"
   
   

3. Monitor importFileParser function in CloudWatch to verify log outputs.
   

4. Check S3 bucket to confirm file movement from uploaded/ to parsed/.
   

Reviewers:

• Please review the implementation and test the provided URLs for functionality verification.