Settings.yml generation for PyDrive

[snekiam]

Describe the bug
PyDrive won't work since it only looks for settings.yaml, not settings.yml

To Reproduce
Steps to reproduce the behavior:

1. Setup your environment with the correct env variables.
2. Run python3 setup_special_files_files_from_env.py
3. Run python3 flask_api.py
4. Make a request to /new_data/wakeword
5. You'll get an error in the flask console: " 'Error opening file', 'client_secrets.json', 'No such
   file or directory'"

Expected behavior
No error appears in the flask console

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
The missing file error is misleading, because the file is not actually missing; we use settings.yaml for the client secrets, and the client_secrets.json file is the default for PyDrive.

[sarthakvk]

hello @snekiam, I would like take on this issue as my first contribution to this project. I am new to the open-source contributions.
Can you help me how do I Setup my environment with the correct env variables?

[snekiam]

Hello @sarthakchaudhary13 - You actually shouldn't need to have real values for the environment variables to make this change. If you look at lines 25-36 of setup_special_files_from_env.py, you can see a list of environment variables that the script looks for

assert environ.get("DATABASE_PASSWORD", None) is not None, BAD_CONFIG_MSG
assert environ.get("DATABASE_USERNAME", None) is not None, BAD_CONFIG_MSG
assert environ.get("DATABASE_NAME", None) is not None, BAD_CONFIG_MSG
assert environ.get("PYDRIVE_CLIENT_ID", None) is not None, BAD_CONFIG_MSG
assert environ.get("PYDRIVE_CLIENT_SECRET", None) is not None, BAD_CONFIG_MSG
# fmt: off
assert environ.get("GOOGLE_DRIVE_CREDENTIALS", None) is not None, BAD_CONFIG_MSG  # noqa
assert environ.get("GOOGLE_DRIVE_FOLDER_ID", None) is not None, BAD_CONFIG_MSG  # noqa
assert environ.get("GOOGLE_CLOUD_NLP_CREDENTIALS", None) is not None, BAD_CONFIG_MSG  # noqa
assert environ.get("GOOGLE_CLOUD_NLP_MODEL_NAME", None) is not None, BAD_CONFIG_MSG  # noqa

If you set those variables to something other than the empty string, you should be able to run the "setup_special_environment_variables_from_env.py" script, which is where the bug exists. You probably won't be able to run the api locally without valid environment variables. Unfortunately, we are not able to give out all our environment variables, as they contain things like database passwords and cloud credentials. We are working on making it so the project can be run locally without the need for some of the more sensitive environment variables (#126) , but unfortunately we are not there yet.

Let me know if this is enough to get you started, and welcome :)

[sarthakvk]

@snekiam - Hi there, thanks to your help I was able to set the env variables but while running the "setup_special_environment_variables_from_env.py" script it's getting the error at line 141 credentials_json = json.loads(config[GOOGLE_DRIVE_CREDENTIALS_KEY])

Traceback (most recent call last): File "setup_special_files_from_env.py", line 141, in <module> credentials_json = json.loads(config[GOOGLE_DRIVE_CREDENTIALS_KEY]) File "/usr/lib/python3.7/json/__init__.py", line 348, in loads return _default_decoder.decode(s) File "/usr/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Is this the bug that needs to be fixed?

[snekiam]

Sorry for my late response on this -- so the reason for that error is that the script is failing to load the credentials.json file because it probably doesn't exist on your system. Here is some bogus data that you can stick into a credentials.json file in order to test this:

{"access_token": "bougs", "client_id": "bogus", "client_secret": "bugus", "refresh_token": "bogus", "token_expiry": "bogus", "token_uri": "https://accounts.google.com/o/oauth2/token", "user_agent": null, "revoke_uri": "https://oauth2.googleapis.com/revoke", "id_token": null, "id_token_jwt": null, "token_response": {"access_token": "bogus", "expires_in": 3600, "refresh_token": "bogus", "scope": "bogus", "token_type": "bogus"}, "scopes": ["bogus"], "token_info_uri": "https://oauth2.googleapis.com/tokeninfo", "invalid": true, "_class": "OAuth2Credentials", "_module": "oauth2client.client"}

This is not the bug that needs to be fixed. The bug that needs to be fixed is the naming of the output file from this script (its currently settings.yml, and should be settings.yaml). Let me know if this fixes your issue.

[snekiam]

This was addressed at some point in setup_special_files_from_env