Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

parallel backend to s3 uploads #1155

Open
Tracked by #1142
kosarko opened this issue Jan 13, 2025 · 0 comments
Open
Tracked by #1142

parallel backend to s3 uploads #1155

kosarko opened this issue Jan 13, 2025 · 0 comments
Milestone
DTQ next

Comments

@kosarko
Copy link
Member

kosarko commented Jan 13, 2025 •
edited
Loading

  • we've discussed using TransferManager in the past
    • afaik it should also do some sort of checksum validation (check the docs/implementation)
  • when operating like this we have the whole file from the user, we should compute/display/store the checksum on that
@kosarko kosarko mentioned this issue Jan 13, 2025
Open
5 tasks
@kosarko kosarko changed the title parallel backend to s3 uploads (we've discussed using TransferManager in the past) parallel backend to s3 uploads Jan 13, 2025
@kosarko kosarko added this to the DTQ next milestone Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
DTQ next
Development

No branches or pull requests
1 participant
@kosarko