utils.py python3 error with create: write() argument must be str, not bytes

[climatebrad]

Running on OSX 10.13.6.

I have in my profile:
export CLOUDSDK_PYTHON=python3

when I run the command
datalab --verbosity=debug create new-instance

I get

Traceback (most recent call last):
  File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/datalab-cli/commands/utils.py", line 179, in prompt_for_zone
    stdout=stdout, stderr=stderr)
  File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/datalab-cli/datalab.py", line 201, in gcloud_compute
    cmd, stdin=stdin, stdout=stdout, stderr=stderr)
  File "/usr/local/anaconda3/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gcloud', 'compute', '--verbosity=debug', 'zones', '--quiet', 'list', '--format=value(name)']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/datalab-cli/datalab.py", line 434, in run
    sdk_version=sdk_version, datalab_version=datalab_version)
  File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 1011, in run
    args.zone = utils.prompt_for_zone(args, gcloud_compute)
  File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/datalab-cli/commands/utils.py", line 184, in prompt_for_zone
    sys.stderr.write(stderr.read())
TypeError: write() argument must be str, not bytes
write() argument must be str, not bytes

[edmondop]

I also encounter this on Cloud Shell

student_00_6d48d5d15da5@cloudshell:~ (qwiklabs-gcp-00-739d963f4e36)$ datalab create mydatalab --zone us-east1
Creating the disk mydatalab-pd

write() argument must be str, not bytes

[cjmoram-acc]

I got the same error on Cloud Shell:

datalab create mydatalabvm --zone us-east1-b

Creating the network datalab-network
write() argument must be str, not bytes

[raphtlw]

I got the same error on Arch Linux

~
❯ datalab create writer
write() argument must be str, not bytes

[davidbigrasGC]

I got the same here on Cloud Shell:

datalab create mydatalabvm --zone us-east1-b

Creating the network datalab-network
write() argument must be str, not bytes

It worked before and suddenly it stopped working.

[bodegus]

I am encountering the same issue - command and stack trace below

datalab --verbosity=debug --zone us-west2-b create new-instance

Creating the network datalab-network

Traceback (most recent call last):
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 666, in ensure_network_exists
    args, gcloud_compute, get_cmd, report_errors=False)
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/commands/utils.py", line 142, in call_gcloud_quietly
    gcloud_surface(args, cmd, stdout=stdout, stderr=stderr)
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/datalab.py", line 201, in gcloud_compute
    cmd, stdin=stdin, stdout=stdout, stderr=stderr)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gcloud', 'compute', '--verbosity=debug', '--quiet', 'networks', 'describe', '--format', 'value(name)', 'datalab-network']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/commands/utils.py", line 142, in call_gcloud_quietly
    gcloud_surface(args, cmd, stdout=stdout, stderr=stderr)
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/datalab.py", line 201, in gcloud_compute
    cmd, stdin=stdin, stdout=stdout, stderr=stderr)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gcloud', 'compute', '--verbosity=debug', '--quiet', 'networks', 'create', 'datalab-network', '--description', 'Network for Google Cloud Datalab instances']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/datalab.py", line 434, in run
    sdk_version=sdk_version, datalab_version=datalab_version)
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 1012, in run
    disk_cfg = prepare(args, gcloud_compute, gcloud_repos)
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 960, in prepare
    ensure_network_exists(args, gcloud_compute, network_name)
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 668, in ensure_network_exists
    create_network(args, gcloud_compute, network_name)
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 649, in create_network
    utils.call_gcloud_quietly(args, gcloud_compute, create_cmd)
  File "/Users/ajohanss/Downloads/google-cloud-sdk/platform/datalab-cli/commands/utils.py", line 148, in call_gcloud_quietly
    sys.stderr.write(stderr.read())
TypeError: write() argument must be str, not bytes
write() argument must be str, not bytes

[bodegus]

Root cause found - this is a GCP project

I found my root cause - I had not enabled the google compute APIs on this project.

You can confirm by doing the "datalab list" operaiton - this has the correct error handling and will give you a prompt to eanble the api.

datalab list

API [compute.googleapis.com] not enabled on project [############].
Would you like to enable and retry (this will take a few minutes)?
(y/N)? y

Enabling service [compute.googleapis.com] on project [############]...

[Matthew-Curry]

I have this same issue, but have Compute Engine API enabled.

datalab list returns "Listed 0 items"

Has anyone else seen this?

[chatttybox]

I have the same exact issue on Cloud Console , I cant believe that GCP is unable to fix simple issues and dreams of competing with AWS and Azure , so much for nepotism in hiring in GCP , you can not get hired unless you are from a preferred country and from a specific majority religion from that country !!!

[bnguyen82]

I got same issue. It turn out the user I login doesn't have enough roles/permission to create instance.
`Your selected zone: 34
Creating the network vpc-network-1

write() argument must be str, not bytes`

[xpbabu]

I got the same error .. Here is a verbose message.

Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 897, in ensure_disk_exists
args, gcloud_compute, get_cmd, report_errors=False)
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/utils.py", line 142, in call_gcloud_quietly
gcloud_surface(args, cmd, stdout=stdout, stderr=stderr)
File "/usr/bin/../lib/google-cloud-sdk/platform/datalab-cli/datalab.py", line 201, in gcloud_compute
cmd, stdin=stdin, stdout=stdout, stderr=stderr)
File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gcloud', 'compute', '--verbosity=debug', '--quiet', 'disks', 'describe', 'myprojectName-pd', '--format', 'value(name)', '--zone', 'us-east1-b']' returned non-zero exit status 1.

[datascienceqing]

I tried to create a datalab instance from google cloud shell couple of months ago, it failed. Today, I tried it again. No surprise. It failed again. I am wasting my time. Below is the Tracebacks.

Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 897, in ensure_disk_exists
args, gcloud_compute, get_cmd, report_errors=False)
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/utils.py", line 142, in call_gcloud_quietly
gcloud_surface(args, cmd, stdout=stdout, stderr=stderr)
File "/usr/bin/../lib/google-cloud-sdk/platform/datalab-cli/datalab.py", line 201, in gcloud_compute
cmd, stdin=stdin, stdout=stdout, stderr=stderr)
File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gcloud', 'compute', '--verbosity=debug', '--quiet', 'disks', 'describe', 'job_bot-pd', '--format', 'value(name)', '--zone', 'us-central1-c']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/utils.py", line 142, in call_gcloud_quietly
gcloud_surface(args, cmd, stdout=stdout, stderr=stderr)
File "/usr/bin/../lib/google-cloud-sdk/platform/datalab-cli/datalab.py", line 201, in gcloud_compute
cmd, stdin=stdin, stdout=stdout, stderr=stderr)
File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gcloud', 'compute', '--verbosity=debug', '--quiet', 'disks', 'create', '--zone', 'us-central1-c', '--size', '200GB', '--descript
ion', 'Persistent disk for a Google Cloud Datalab instance', 'job_bot-pd']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/../lib/google-cloud-sdk/platform/datalab-cli/datalab.py", line 434, in run
sdk_version=sdk_version, datalab_version=datalab_version)
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 1012, in run
disk_cfg = prepare(args, gcloud_compute, gcloud_repos)
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 968, in prepare
ensure_disk_exists(args, gcloud_compute, disk_name)
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 899, in ensure_disk_exists
create_disk(args, gcloud_compute, disk_name)
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/create.py", line 877, in create_disk
utils.call_gcloud_quietly(args, gcloud_compute, create_cmd)
File "/usr/lib/google-cloud-sdk/platform/datalab-cli/commands/utils.py", line 148, in call_gcloud_quietly
sys.stderr.write(stderr.read())
TypeError: write() argument must be str, not bytes
write() argument must be str, not bytes

[johnf1004]

When I switched my instance name from test_instance to test-instance, the error stopped happening for me! So maybe Datalab doesn't like underscores.

A new error cropped up, but I guess thats for a different issue...

[estatico3]

I had the same issue. In my case, it turned out that I was not logged in on gcloud inside my terminal, so even when I enabled all the APIs and checked whether the account is billing or not, datalab could not see it. Basically, $gcloud auth login did the trick, then I went through the authentication procedure, and then the command datalab create instance was executed without any problem. By the way, after the authentication, it will show you your current project, so make sure you are creating a new instance for the right project. Hope it will help!

[dwy6626]

I think the main issue here is that the error of child process cannot be correctly displayed.

[camilojaure]

When I switched my instance name from test_instance to test-instance, the error stopped happening for me! So maybe Datalab doesn't like underscores.

A new error cropped up, but I guess thats for a different issue...

The same happened to me! Can't f* believe the underscore is a bug...

[EricHodgins]

It did not like the camel case name either for me. testLAB did not work, but testlab did. Well it first gave me an error and a link in the console to enable the API. But after that it worked.

datalab create testlab

[OneOfOne]

We're having this issue as well, how is this not fixed after almost 2 years?

[petera2c]

Has anyone found a solution to this? After selecting region, I get write() argument must be str, not bytes. Billing and API are enabled. datalab list gives me Listed 0 items.

Edit: I fixed this!!!! When I ran $gcloud auth login It initialized with the wrong project and I had to manually select the right project with gcloud config set project [project-id]

[KevinSig]

I have this same issue, but have Compute Engine API enabled.

datalab list returns "Listed 0 items"

Has anyone else seen this?

I think I was able to work .... by not including any capitals in the name of the file and changing the zone