Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PR #2108/48292178 backport][stable-8] s3_object - fix MemoryError when downloading large files #2109

Conversation

patchback[bot]
Copy link

@patchback patchback bot commented May 22, 2024

This is a backport of PR #2108 as merged into main (4829217).

SUMMARY

fixes: #2107

The refactor in #1139 is triggering a full download of the file into memory when downloading files, this downloaded content was then being thrown away.

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

s3_object

ADDITIONAL INFORMATION

s3_object - fix MemoryError when downloading large files

SUMMARY
fixes: #2107
The refactor in #1139 is triggering a full download of the file into memory when downloading files, this downloaded content was then being thrown away.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
s3_object
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
Reviewed-by: Bikouo Aubin
(cherry picked from commit 4829217)
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/b85b39c3badd4fec8f2d8c12b5d37806

✔️ ansible-galaxy-importer SUCCESS in 5m 14s
✔️ build-ansible-collection SUCCESS in 17m 01s
✔️ ansible-test-splitter SUCCESS in 7m 37s
✔️ integration-amazon.aws-1 SUCCESS in 16m 03s
✔️ integration-community.aws-1 SUCCESS in 26m 37s
✔️ integration-community.aws-2 SUCCESS in 55m 54s
integration-community.aws-3 TIMED_OUT in 1h 00m 48s
✔️ integration-community.aws-4 SUCCESS in 12m 50s
✔️ integration-community.aws-5 SUCCESS in 29m 15s
✔️ integration-community.aws-6 SUCCESS in 35m 27s
✔️ integration-community.aws-7 SUCCESS in 31m 03s
✔️ integration-community.aws-8 SUCCESS in 19m 27s
✔️ integration-community.aws-9 SUCCESS in 32m 51s
✔️ integration-community.aws-10 SUCCESS in 32m 02s
✔️ integration-community.aws-11 SUCCESS in 19m 18s
✔️ integration-community.aws-12 SUCCESS in 20m 35s
✔️ integration-community.aws-13 SUCCESS in 19m 16s
✔️ integration-community.aws-14 SUCCESS in 29m 41s
✔️ integration-community.aws-15 SUCCESS in 23m 23s
✔️ integration-community.aws-16 SUCCESS in 54m 13s
integration-community.aws-17 RETRY_LIMIT in 6m 47s
integration-community.aws-18 RETRY_LIMIT in 6m 48s
✔️ integration-community.aws-19 SUCCESS in 13m 36s
Skipped 24 jobs

@tremble tremble added the mergeit Merge the PR (SoftwareFactory) label May 22, 2024
Copy link
Contributor

Build failed (gate pipeline). For information on how to proceed, see
http://docs.openstack.org/infra/manual/developers.html#automated-testing

https://ansible.softwarefactory-project.io/zuul/buildset/88c5ead716254bd48e1be4fa19424bbd

✔️ ansible-galaxy-importer SUCCESS in 4m 44s
✔️ build-ansible-collection SUCCESS in 14m 51s
ansible-test-splitter FAILURE in 6m 53s
⚠️ integration-amazon.aws-1 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-2 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-3 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-4 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-5 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-6 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-7 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-8 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-9 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-10 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-11 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-12 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-13 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-14 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-15 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-16 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-17 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-18 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-19 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-20 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-21 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-amazon.aws-22 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-1 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-2 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-3 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-4 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-5 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-6 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-7 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-8 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-9 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-10 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-11 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-12 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-13 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-14 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-15 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-16 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-17 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-18 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-19 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-20 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-21 SKIPPED Skipped due to failed job ansible-test-splitter
⚠️ integration-community.aws-22 SKIPPED Skipped due to failed job ansible-test-splitter

@tremble
Copy link
Contributor

tremble commented May 22, 2024

regate

Copy link
Contributor

Build failed (gate pipeline). For information on how to proceed, see
http://docs.openstack.org/infra/manual/developers.html#automated-testing

https://ansible.softwarefactory-project.io/zuul/buildset/ff2cd1726e4b4d2a866bee78aa1582b6

✔️ ansible-galaxy-importer SUCCESS in 5m 12s
✔️ build-ansible-collection SUCCESS in 19m 36s
✔️ ansible-test-splitter SUCCESS in 9m 56s
✔️ integration-amazon.aws-1 SUCCESS in 13m 31s
✔️ integration-community.aws-1 SUCCESS in 24m 20s
✔️ integration-community.aws-2 SUCCESS in 42m 59s
✔️ integration-community.aws-3 SUCCESS in 53m 27s
✔️ integration-community.aws-4 SUCCESS in 7m 03s
✔️ integration-community.aws-5 SUCCESS in 26m 41s
✔️ integration-community.aws-6 SUCCESS in 24m 59s
✔️ integration-community.aws-7 SUCCESS in 27m 04s
✔️ integration-community.aws-8 SUCCESS in 15m 54s
integration-community.aws-9 RETRY_LIMIT in 1m 43s
✔️ integration-community.aws-10 SUCCESS in 24m 29s
✔️ integration-community.aws-11 SUCCESS in 18m 13s
✔️ integration-community.aws-12 SUCCESS in 18m 54s
✔️ integration-community.aws-13 SUCCESS in 20m 09s
✔️ integration-community.aws-14 SUCCESS in 28m 28s
✔️ integration-community.aws-15 SUCCESS in 19m 12s
✔️ integration-community.aws-16 SUCCESS in 47m 58s
✔️ integration-community.aws-17 SUCCESS in 14m 13s
✔️ integration-community.aws-18 SUCCESS in 19m 23s
✔️ integration-community.aws-19 SUCCESS in 14m 25s
Skipped 24 jobs

@alinabuzachis
Copy link
Contributor

regate

Copy link
Contributor

Build succeeded (gate pipeline).
https://ansible.softwarefactory-project.io/zuul/buildset/268cf59a5a424c31b53e0351e129ec6e

✔️ ansible-galaxy-importer SUCCESS in 4m 23s
✔️ build-ansible-collection SUCCESS in 17m 54s
✔️ ansible-test-splitter SUCCESS in 7m 26s
✔️ integration-amazon.aws-1 SUCCESS in 13m 36s
Skipped 43 jobs

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit cc45aaa into stable-8 May 29, 2024
36 of 37 checks passed
@softwarefactory-project-zuul softwarefactory-project-zuul bot deleted the patchback/backports/stable-8/482921780bb6a7c08b74931a496d4ab8e5dbc71f/pr-2108 branch May 29, 2024 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mergeit Merge the PR (SoftwareFactory)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants