Skip to content

code for uploading large files to s3 by chunking them and uploading chunks in parallel to signed S3 url's

manuongithub/s3-large-file-upload

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

s3-large-file-upload

Step-by-Step Setup

1. Install Required Dependencies

Navigate to the project directory and install the required dependencies:

npm install

2. Configure AWS Credentials

Update the server.js file with your AWS credentials and bucket details. Replace the placeholders with your actual AWS credentials and bucket details:

  • region

  • accessKeyId

  • secretAccessKey

  • bucket

3. Run the Backend Server

Start the backend server:

node server.js

This will start the backend server on port 3000.

4. Set Up the Frontend Server

Ensure you have a simple HTTP server to serve the frontend files. You can use http-server for this:

Install http-server globally if you haven't already:

npm install -g http-server

Navigate to the project directory and start the HTTP server:

http-server

By default, http-server will start serving files on port 8080.

5. Test the Application

Open your browser and navigate to http://127.0.0.1:8080 (or http://localhost:8080). You should see the upload interface with a file input and an upload button.

  • Select a large video file using the file input.

  • Click the upload button.

The file will be split into chunks and uploaded to your S3 bucket using the signed URLs generated by the backend server.

Notes

  • Ensure your S3 bucket has the appropriate CORS configuration to allow the necessary operations from your frontend.

  • Monitor the console for any errors or progress updates during the upload process.

  • Verify the uploaded file in your S3 bucket to ensure it is properly uploaded and accessible.

By following these steps, you should be able to set up and run the large file upload system successfully.

About

code for uploading large files to s3 by chunking them and uploading chunks in parallel to signed S3 url's

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published