Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dataset upload for >2GB files #5889

Merged
merged 1 commit into from
Dec 6, 2021
Merged

Fix dataset upload for >2GB files #5889

merged 1 commit into from
Dec 6, 2021

Conversation

fm3
Copy link
Member

@fm3 fm3 commented Dec 6, 2021

Fixes seek offset integer overflow. All components fit comfortably into Int, but their product did not. We thought the product to be Long already, but there was a lossy conversion after all. Now we just make all the values in the upload service Long. Memory overhead should be negligible.

Steps to test:

  • Upload a dataset as single zip that is >2GB in size
  • Should upload correctly.

Issues:


(Please delete unneded items, merge only when none are left open)

@fm3 fm3 requested a review from normanrz December 6, 2021 11:00
@fm3 fm3 self-assigned this Dec 6, 2021
@fm3 fm3 merged commit ed0c278 into master Dec 6, 2021
@fm3 fm3 deleted the upload-seek-long branch December 6, 2021 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integer overflow when uploading dataset with large single file(s)
2 participants