-
Notifications
You must be signed in to change notification settings - Fork 345
feat: added system tests for the asyncIO auth changes and async id_token credentials #574
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 34 commits
ef6d872
0a63b3c
1fb2bce
06606ff
cd57fad
a38e333
57d6d10
1db84f2
cc14082
80d005f
aa6ece2
d88839b
dfc6251
c62dd1a
7080d14
8122dbb
b8b585a
77ca4c7
b4c306f
9ddb911
8f2d0ef
9ec8277
8f254de
aa04dc9
0c4c3b6
92175f2
2b29a54
337c772
77d7b6e
fd30685
0ba2dda
50f7fb7
1d7c8dc
f7b49ed
dd92972
c8a61c0
53567f2
1ed6e11
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,267 @@ | ||
| # Copyright 2020 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| """Google ID Token helpers. | ||
|
|
||
| Provides support for verifying `OpenID Connect ID Tokens`_, especially ones | ||
| generated by Google infrastructure. | ||
|
|
||
| To parse and verify an ID Token issued by Google's OAuth 2.0 authorization | ||
| server use :func:`verify_oauth2_token`. To verify an ID Token issued by | ||
| Firebase, use :func:`verify_firebase_token`. | ||
|
|
||
| A general purpose ID Token verifier is available as :func:`verify_token`. | ||
|
|
||
| Example:: | ||
|
|
||
| from google.oauth2 import id_token_async | ||
| from google.auth.transport import aiohttp_requests | ||
|
|
||
| request = aiohttp_requests.Request() | ||
|
|
||
| id_info = await id_token_async.verify_oauth2_token( | ||
| token, request, 'my-client-id.example.com') | ||
|
|
||
| if id_info['iss'] != 'https://accounts.google.com': | ||
| raise ValueError('Wrong issuer.') | ||
|
|
||
| userid = id_info['sub'] | ||
|
|
||
| By default, this will re-fetch certificates for each verification. Because | ||
| Google's public keys are only changed infrequently (on the order of once per | ||
| day), you may wish to take advantage of caching to reduce latency and the | ||
| potential for network errors. This can be accomplished using an external | ||
| library like `CacheControl`_ to create a cache-aware | ||
| :class:`google.auth.transport.Request`:: | ||
|
|
||
| import cachecontrol | ||
| import google.auth.transport.requests | ||
| import requests | ||
|
|
||
| session = requests.session() | ||
| cached_session = cachecontrol.CacheControl(session) | ||
| request = google.auth.transport.requests.Request(session=cached_session) | ||
|
|
||
| .. _OpenID Connect ID Token: | ||
| http://openid.net/specs/openid-connect-core-1_0.html#IDToken | ||
| .. _CacheControl: https://cachecontrol.readthedocs.io | ||
| """ | ||
|
|
||
| import json | ||
| import os | ||
|
|
||
| import six | ||
| from six.moves import http_client | ||
|
|
||
| from google.auth import environment_vars | ||
| from google.auth import exceptions | ||
| from google.auth import jwt | ||
| from google.auth.transport import requests | ||
| from google.oauth2 import id_token as sync_id_token | ||
|
|
||
|
|
||
| async def _fetch_certs(request, certs_url): | ||
| """Fetches certificates. | ||
|
|
||
| Google-style cerificate endpoints return JSON in the format of | ||
| ``{'key id': 'x509 certificate'}``. | ||
|
|
||
| Args: | ||
| request (google.auth.transport.Request): The object used to make | ||
| HTTP requests. This must be an aiohttp request. | ||
| certs_url (str): The certificate endpoint URL. | ||
|
|
||
| Returns: | ||
| Mapping[str, str]: A mapping of public key ID to x.509 certificate | ||
| data. | ||
| """ | ||
| response = await request(certs_url, method="GET") | ||
|
|
||
| if response.status != http_client.OK: | ||
| raise exceptions.TransportError( | ||
| "Could not fetch certificates at {}".format(certs_url) | ||
| ) | ||
|
|
||
| data = await response.data.read() | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this a network op?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, response.data returns a stream coroutine, and therefore needs to be awaited in order to read the content of the response. |
||
|
|
||
| return json.loads(json.dumps(data)) | ||
|
|
||
|
|
||
| async def verify_token( | ||
| id_token, request, audience=None, certs_url=sync_id_token._GOOGLE_OAUTH2_CERTS_URL | ||
| ): | ||
| """Verifies an ID token and returns the decoded token. | ||
|
|
||
| Args: | ||
| id_token (Union[str, bytes]): The encoded token. | ||
| request (google.auth.transport.Request): The object used to make | ||
crwilcox marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| HTTP requests. This must be an aiohttp request. | ||
| audience (str): The audience that this token is intended for. If None | ||
| then the audience is not verified. | ||
| certs_url (str): The URL that specifies the certificates to use to | ||
| verify the token. This URL should return JSON in the format of | ||
| ``{'key id': 'x509 certificate'}``. | ||
|
|
||
| Returns: | ||
| Mapping[str, Any]: The decoded token. | ||
| """ | ||
| certs = await _fetch_certs(request, certs_url) | ||
|
|
||
| return jwt.decode(id_token, certs=certs, audience=audience) | ||
|
|
||
|
|
||
| async def verify_oauth2_token(id_token, request, audience=None): | ||
| """Verifies an ID Token issued by Google's OAuth 2.0 authorization server. | ||
|
|
||
| Args: | ||
| id_token (Union[str, bytes]): The encoded token. | ||
| request (google.auth.transport.Request): The object used to make | ||
| HTTP requests. This must be an aiohttp request. | ||
| audience (str): The audience that this token is intended for. This is | ||
| typically your application's OAuth 2.0 client ID. If None then the | ||
| audience is not verified. | ||
|
|
||
| Returns: | ||
| Mapping[str, Any]: The decoded token. | ||
|
|
||
| Raises: | ||
| exceptions.GoogleAuthError: If the issuer is invalid. | ||
| """ | ||
| idinfo = await verify_token( | ||
| id_token, | ||
| request, | ||
| audience=audience, | ||
| certs_url=sync_id_token._GOOGLE_OAUTH2_CERTS_URL, | ||
| ) | ||
|
|
||
| if idinfo["iss"] not in sync_id_token._GOOGLE_ISSUERS: | ||
| raise exceptions.GoogleAuthError( | ||
| "Wrong issuer. 'iss' should be one of the following: {}".format( | ||
| sync_id_token._GOOGLE_ISSUERS | ||
| ) | ||
| ) | ||
|
|
||
| return idinfo | ||
|
|
||
|
|
||
| async def verify_firebase_token(id_token, request, audience=None): | ||
| """Verifies an ID Token issued by Firebase Authentication. | ||
|
|
||
| Args: | ||
| id_token (Union[str, bytes]): The encoded token. | ||
| request (google.auth.transport.Request): The object used to make | ||
| HTTP requests. This must be an aiohttp request. | ||
| audience (str): The audience that this token is intended for. This is | ||
| typically your Firebase application ID. If None then the audience | ||
| is not verified. | ||
|
|
||
| Returns: | ||
| Mapping[str, Any]: The decoded token. | ||
| """ | ||
| return await verify_token( | ||
| id_token, | ||
| request, | ||
| audience=audience, | ||
| certs_url=sync_id_token._GOOGLE_APIS_CERTS_URL, | ||
| ) | ||
|
|
||
|
|
||
| async def fetch_id_token(request, audience): | ||
| """Fetch the ID Token from the current environment. | ||
|
|
||
| This function acquires ID token from the environment in the following order: | ||
|
|
||
| 1. If the application is running in Compute Engine, App Engine or Cloud Run, | ||
| then the ID token are obtained from the metadata server. | ||
| 2. If the environment variable ``GOOGLE_APPLICATION_CREDENTIALS`` is set | ||
| to the path of a valid service account JSON file, then ID token is | ||
| acquired using this service account credentials. | ||
| 3. If metadata server doesn't exist and no valid service account credentials | ||
| are found, :class:`~google.auth.exceptions.DefaultCredentialsError` will | ||
| be raised. | ||
|
|
||
| Example:: | ||
|
|
||
| import google.oauth2.id_token_async | ||
| import google.auth.transport.aiohttp_requests | ||
|
|
||
| request = google.auth.transport.aiohttp_requests.Request() | ||
| target_audience = "https://pubsub.googleapis.com" | ||
|
|
||
| id_token = await google.oauth2.id_token_async.fetch_id_token(request, target_audience) | ||
|
|
||
| Args: | ||
| request (google.auth.transport.Request): A callable used to make | ||
| HTTP requests. | ||
| audience (str): The audience that this ID token is intended for. | ||
|
|
||
| Returns: | ||
| str: The ID token. | ||
|
|
||
| Raises: | ||
| ~google.auth.exceptions.DefaultCredentialsError: | ||
| If metadata server doesn't exist and no valid service account | ||
| credentials are found. | ||
| """ | ||
| # 1. First try to fetch ID token from metada server if it exists. The code | ||
crwilcox marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| # works for GAE and Cloud Run metadata server as well. | ||
| try: | ||
| from google.auth import compute_engine | ||
|
|
||
| request_new = requests.Request() | ||
| credentials = compute_engine.IDTokenCredentials( | ||
| request_new, audience, use_metadata_identity_endpoint=True | ||
| ) | ||
| credentials.refresh(request_new) | ||
|
|
||
| return credentials.token | ||
|
|
||
| except (ImportError, exceptions.TransportError, exceptions.RefreshError): | ||
| pass | ||
|
|
||
| # 2. Try to use service account credentials to get ID token. | ||
|
|
||
| # Try to get credentials from the GOOGLE_APPLICATION_CREDENTIALS environment | ||
| # variable. | ||
| credentials_filename = os.environ.get(environment_vars.CREDENTIALS) | ||
| if not ( | ||
| credentials_filename | ||
| and os.path.exists(credentials_filename) | ||
| and os.path.isfile(credentials_filename) | ||
| ): | ||
| raise exceptions.DefaultCredentialsError( | ||
| "Neither metadata server or valid service account credentials are found." | ||
| ) | ||
|
|
||
| try: | ||
| with open(credentials_filename, "r") as f: | ||
| info = json.load(f) | ||
| credentials_content = ( | ||
| (info.get("type") == "service_account") and info or None | ||
| ) | ||
|
|
||
| from google.oauth2 import service_account_async as service_account | ||
|
|
||
| credentials = service_account.IDTokenCredentials.from_service_account_info( | ||
| credentials_content, target_audience=audience | ||
| ) | ||
| except ValueError as caught_exc: | ||
| new_exc = exceptions.DefaultCredentialsError( | ||
| "Neither metadata server or valid service account credentials are found.", | ||
| caught_exc, | ||
| ) | ||
| six.raise_from(new_exc, caught_exc) | ||
|
|
||
| await credentials.refresh(request) | ||
| return credentials.token | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| # Copyright 2020 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| import json | ||
| import os | ||
|
|
||
| from google.auth import _helpers | ||
| import google.auth.transport.requests | ||
| import google.auth.transport.urllib3 | ||
| import pytest | ||
| import requests | ||
| import urllib3 | ||
|
|
||
| import aiohttp | ||
| import google.auth.transport.aiohttp_requests | ||
|
|
||
|
|
||
| HERE = os.path.dirname(__file__) | ||
| DATA_DIR = os.path.join(HERE, "data") | ||
|
||
| IMPERSONATED_SERVICE_ACCOUNT_FILE = os.path.join( | ||
| DATA_DIR, "impersonated_service_account.json" | ||
| ) | ||
| SERVICE_ACCOUNT_FILE = os.path.join(DATA_DIR, "service_account.json") | ||
| AUTHORIZED_USER_FILE = os.path.join(DATA_DIR, "authorized_user.json") | ||
| URLLIB3_HTTP = urllib3.PoolManager(retries=False) | ||
| REQUESTS_SESSION = requests.Session() | ||
|
|
||
| ASYNC_REQUESTS_SESSION = aiohttp.ClientSession() | ||
|
|
||
| REQUESTS_SESSION.verify = False | ||
| TOKEN_INFO_URL = "https://www.googleapis.com/oauth2/v3/tokeninfo" | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def service_account_file(): | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If these are copied from system_tests/conftest.py, could they be moved to the root conftest.py to de-duplicate?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same inter-dependency reasoning as above, open to changing if we are okay with having the dependency. |
||
| """The full path to a valid service account key file.""" | ||
| yield SERVICE_ACCOUNT_FILE | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def impersonated_service_account_file(): | ||
| """The full path to a valid service account key file.""" | ||
| yield IMPERSONATED_SERVICE_ACCOUNT_FILE | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def authorized_user_file(): | ||
| """The full path to a valid authorized user file.""" | ||
| yield AUTHORIZED_USER_FILE | ||
|
|
||
| @pytest.fixture(params=["aiohttp"]) | ||
| async def http_request(request): | ||
| """A transport.request object.""" | ||
| yield google.auth.transport.aiohttp_requests.Request(ASYNC_REQUESTS_SESSION) | ||
|
|
||
| @pytest.fixture | ||
| async def token_info(http_request): | ||
| """Returns a function that obtains OAuth2 token info.""" | ||
|
|
||
| async def _token_info(access_token=None, id_token=None): | ||
| query_params = {} | ||
|
|
||
| if access_token is not None: | ||
| query_params["access_token"] = access_token | ||
| elif id_token is not None: | ||
| query_params["id_token"] = id_token | ||
| else: | ||
| raise ValueError("No token specified.") | ||
|
|
||
| url = _helpers.update_query(TOKEN_INFO_URL, query_params) | ||
|
|
||
| response = await http_request(url=url, method="GET") | ||
| data = await response.data.read() | ||
|
|
||
| return json.loads(data.decode("utf-8")) | ||
|
|
||
| yield _token_info | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| async def verify_refresh(http_request): | ||
| """Returns a function that verifies that credentials can be refreshed.""" | ||
|
|
||
| async def _verify_refresh(credentials): | ||
| if credentials.requires_scopes: | ||
| credentials = credentials.with_scopes(["email", "profile"]) | ||
|
|
||
| await credentials.refresh(http_request) | ||
|
|
||
| assert credentials.token | ||
| assert credentials.valid | ||
|
|
||
| yield _verify_refresh | ||
|
|
||
|
|
||
| def verify_environment(): | ||
| """Checks to make sure that requisite data files are available.""" | ||
| if not os.path.isdir(DATA_DIR): | ||
| raise EnvironmentError( | ||
| "In order to run system tests, test data must exist in " | ||
| "system_tests/data. See CONTRIBUTING.rst for details." | ||
| ) | ||
|
|
||
|
|
||
| def pytest_configure(config): | ||
| """Pytest hook that runs before Pytest collects any tests.""" | ||
| verify_environment() | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would this be the same for async?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I made this specific request call sync as an async version of this would result in a substantial re-working of the compute_engine credentials classes (as the current design is not compatible with asynchronous http requests). My reasoning for this was that this authentication trace isn't used in storage, but we could open up an issue for it as a future modification to make for the auth library.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please open a tracking bug
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok will do.