Missing dependency six: object has no attribute 'python_2_unicode_compatible'

[dojeda]

I am using a dataproc cluster where the workers have google-cloud-python installed. I recently deployed a new cluster, which pulled the latest version of this library (1.4.0). My workers when doing from google.cloud import storage now fail with the following error:

  File "/usr/local/lib/python3.4/dist-packages/google/cloud/storage/__init__.py", line 37, in <module>
    from google.cloud.storage.batch import Batch
  File "/usr/local/lib/python3.4/dist-packages/google/cloud/storage/batch.py", line 30, in <module>
    from google.cloud import exceptions
  File "/usr/local/lib/python3.4/dist-packages/google/cloud/exceptions.py", line 24, in <module>
    from google.api.core import exceptions
  File "/usr/local/lib/python3.4/dist-packages/google/api/core/exceptions.py", line 43, in <module>
    @six.python_2_unicode_compatible
AttributeError: 'module' object has no attribute 'python_2_unicode_compatible'

Which is probably because there is no six package declared in the dependencies of this project?

A workaround is to ensure that six==1.10.0 is installed (I don't know exactly which version introduced the attribute python2_unicode_compatible).

This is the information of the system where the bug occurs:

$ uname -a
Linux dojeda-test-job-cluster-w-1 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u2 (2017-06-26) x86_64 GNU/Linux
$ cat /etc/issue
Debian GNU/Linux 8 \n \l

$ pip freeze
backports.ssl-match-hostname==3.5.0.1
cachetools==2.0.1
certifi==2017.7.27.1
chardet==3.0.4
colorama==0.3.2
docker==2.5.1
docker-pycreds==0.2.1
google-auth==1.0.2
google-cloud-core==0.27.0
google-cloud-storage==1.4.0
google-resumable-media==0.2.3
googleapis-common-protos==1.5.2
html5lib==0.999
idna==2.6
protobuf==3.4.0
pyasn1==0.3.2
pyasn1-modules==0.0.11
pygobject==3.14.0
python-apt==0.9.3.12
requests==2.18.4
rsa==3.4.2
six==1.8.0
unattended-upgrades==0.1
urllib3==1.22
websocket-client==0.44.0

Here is a working pip freeze:

backports.ssl-match-hostname==3.5.0.1
cachetools==2.0.1
certifi==2017.7.27.1
chardet==3.0.4
colorama==0.3.2
docker==2.5.1
docker-pycreds==0.2.1
google-auth==1.0.2
google-cloud-core==0.27.0
google-cloud-storage==1.4.0
google-resumable-media==0.2.3
googleapis-common-protos==1.5.2
html5lib==0.999
idna==2.6
protobuf==3.4.0
pyasn1==0.3.2
pyasn1-modules==0.0.11
pygobject==3.14.0
python-apt==0.9.3.12
requests==2.18.4
rsa==3.4.2
six==1.10.0
unattended-upgrades==0.1
urllib3==1.22
websocket-client==0.44.0

[lukesneeringer]

Hi @dojeda,
Thank you for reporting.

It actually looks like six is installed in both cases, but it is 1.8.0 in the case where you are getting the error, and 1.10.0 when you were not. Our requirement listing has no version specification at all (six is actually a dependency of google-cloud-core). I will update this to require the newest version of six.

[dojeda]

@lukesneeringer yes, indeed: the 1.8.0 version was installed before by some other dependency. I think your PR should fix this.

Thank you for your quick reply!