Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Swapping out google.streaming for google-resumable-media in GCS downloads. #3323

Merged
merged 3 commits into from
Apr 24, 2017
Merged

Swapping out google.streaming for google-resumable-media in GCS downloads. #3323

merged 3 commits into from
Apr 24, 2017

Conversation

dhermes
Copy link
Contributor

@dhermes dhermes commented Apr 24, 2017

@dhermes dhermes added the api: storage Issues related to the Cloud Storage API. label Apr 24, 2017
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 24, 2017
theacodes
theacodes reviewed Apr 24, 2017
View reviewed changes
storage/setup.py Outdated
@@ -52,6 +52,9 @@

REQUIREMENTS = [
'google-cloud-core >= 0.24.0, < 0.25dev',
'google-auth',

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

storage/setup.py Outdated
@@ -52,6 +52,9 @@

REQUIREMENTS = [
'google-cloud-core >= 0.24.0, < 0.25dev',
'google-auth',
'google-resumable-media == 0.0.2',
'requests',

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

storage/google/cloud/storage/blob.py Outdated
@@ -30,6 +30,9 @@
import six
from six.moves.urllib.parse import quote

import google.auth.transport.requests as tr_requests

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

tseaver
tseaver reviewed Apr 24, 2017
View reviewed changes
storage/google/cloud/storage/blob.py Outdated
@@ -30,6 +30,9 @@
import six
from six.moves.urllib.parse import quote

import google.auth.transport.requests as tr_requests

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

storage/google/cloud/storage/blob.py
if self.chunk_size is None:
download = resumable_media.Download(download_url, headers=headers)
response = download.consume(transport)
file_obj.write(response.content)

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

tseaver
tseaver reviewed Apr 24, 2017
View reviewed changes
storage/google/cloud/storage/blob.py
download.chunksize = self.chunk_size

# Get the download URL.
download_url = self._get_download_url()

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@dhermes dhermes mentioned this pull request Apr 24, 2017
Closed
@dhermes
Copy link
Contributor Author

dhermes commented Apr 24, 2017

I filed googleapis/google-resumable-media-python#5 to carry on the discussion about streaming the contents in a more memory-efficient manner (when possible).

For now, I am going to merge this based on an LGTM from @jonparrott (given in chat)

@dhermes dhermes merged commit d0ef6a9 into googleapis:master Apr 24, 2017
@dhermes dhermes deleted the resumable-media-for-GCS-downloads branch April 24, 2017 22:36
This was referenced Apr 27, 2017
Merged
Closed
richkadel pushed a commit to richkadel/google-cloud-python that referenced this pull request May 6, 2017
@dhermes
Swapping out google.streaming for google-resumable-media in GCS downl…
153c4b9 
…oads. (googleapis#3323)

* Swapping out google.streaming for google-resumable-media in GCS downloads.

* Dropping tr_requests import alias in storage blob module.

* Putting lower bounds on new deps in storage package.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tseaver tseaver tseaver left review comments

@theacodes theacodes theacodes left review comments

Assignees

@lukesneeringer lukesneeringer

@tseaver tseaver

Labels
api: storage Issues related to the Cloud Storage API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@dhermes @tseaver @theacodes @lukesneeringer @googlebot