[Cosmos] use python3 typehints and move options to kwargs on async client (#23933)

* initial commit

* Client Constructor (#20310)

* Removed some stuff

* Looking at constructors

* Updated request

* Added client close

* working client creation

Co-authored-by: simorenoh <[email protected]>

* read database

database read works, but ignored exception is returned:
Fatal error on SSL transport
NoneType has no attribute 'send' (_loop._proactor.send)
RuntimeError: Event loop is closed
Unclosed connector/ connection

* Update simon_testfile.py

* with coroutine

Added methods needed to use async with when initializing client, but logs output "Exception ignored... Runtime Error: Event loop is closed"

* Update simon_testfile.py

* small changes

* async with returns no exceptions

* async read container

* async item read

* cleaning up

* create item/ database methods

* item delete working

* docs replace functionality

missing upsert and other resources

* upsert functionality

missing read_all_items and both query methods for container class

* missing query methods

* CRUD for udf, sproc, triggers

* initial query logic + container methods

* missing some execution logic and tests

* oops

* fully working queries

* small fix to query_items()

also fixed README and added examples_async

* Update _cosmos_client_connection_async.py

* Update _cosmos_client_connection.py

* documentation update

* updated MIT dates and get_user_client() description

* Update CHANGELOG.md

* Delete simon_testfile.py

* leftover retry utility

* Update README.md

* docs and removed six package

* changes based on comments

still missing discussion resolution on SSL verification and tests for async functionality under test module (apart from samples which are basically end to end tests)

* small change in type hints

* updated readme

* fixes based on conversations

* added missing type comments

* update changelog for ci pipeline

* added typehints, moved params into keywords, added decorators, made _connection_policy private

* changes based on sync with central sdk

* remove is_system_key from scripts (only used in execute_sproc)

is_system_key verifies that an empty partition key is properly dealt with if ['partitionKey']['systemKey'] exists in the container options - however, we do not allow containers to be created with empty partition key values in the python sdk, so the functionality is needless

* Revert "remove is_system_key from scripts (only used in execute_sproc)"

Reverting last commit, will find way to init is_system_key for now

* async script proxy using composition

* pylint

* capitalized constants

* Apply suggestions from code review

Clarifying comments for README

Co-authored-by: Gahl Levy <[email protected]>

* closing python code snippet

* last doc updates

* Update sdk/cosmos/azure-cosmos/CHANGELOG.md

Co-authored-by: Simon Moreno <[email protected]>

* version update

* cosmos updates for release

* public surface area python3 typehints

* fix connection string comma

* Update CHANGELOG.md

* fixing extra await keyword in sample

* Update CHANGELOG.md

* Update CHANGELOG.md

* simplified short types into one line and moved many options to kwargs

* missed additional Callable definitions

* pylint

* addressed several comments, thank you Travis

* Update CHANGELOG.md

* Callable typehints

* Update CHANGELOG.md

* bets practices recommends using string partition keys

* Revert "bets practices recommends using string partition keys"

This reverts commit d44502f.

* Update sdk/cosmos/azure-cosmos/azure/cosmos/container.py

Co-authored-by: Anna Tisch <[email protected]>

* changes from meeting

* Update CHANGELOG.md

* update query types

* Update dev_requirements.txt

* Update setup.py

* type checking for tokencredential

* Revert "type checking for tokencredential"

This reverts commit 4eadc97.

* update shared_requirements

* anna comments

* pylint?

* Update _scripts.py

Co-authored-by: annatisch <[email protected]>
Co-authored-by: Gahl Levy <[email protected]>
Co-authored-by: Travis Prescott <[email protected]>

Author: 4 people

sdk/cosmos/azure-cosmos/CHANGELOG.md

@@ -3,11 +3,12 @@
 ### 4.3.0b5 (Unreleased)
 
 #### Breaking Changes
+- Method signatures have been updated to use keyword arguments instead of positional arguments for most method options in the async client.
 - Bugfix: Automatic Id generation for items was turned on for `upsert_items()` method when no 'id' value was present in document body.
 Method call will now require an 'id' field to be present in the document body.
 
 #### Other Changes
-- Marked the GetAuthorizationMethod for deprecation since it will no longer be public in a future release.
+- Marked the GetAuthorizationHeader method for deprecation since it will no longer be public in a future release.
 - Added samples showing how to configure retry options for both the sync and async clients.
 - Deprecated the `connection_retry_policy` and `retry_options` options in the sync client.
 

sdk/cosmos/azure-cosmos/azure/cosmos/aio/__init__.py

@@ -19,11 +19,11 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .container import ContainerProxy
+from ._container import ContainerProxy
 from .cosmos_client import CosmosClient
-from .database import DatabaseProxy
-from .user import UserProxy
-from .scripts import ScriptsProxy
+from ._database import DatabaseProxy
+from ._user import UserProxy
+from ._scripts import ScriptsProxy
 
 __all__ = (
     "CosmosClient",

sdk/cosmos/azure-cosmos/azure/cosmos/aio/container.py renamed to sdk/cosmos/azure-cosmos/azure/cosmos/aio/_container.py

@@ -24,15 +24,15 @@
 """Create, read, update and delete users in the Azure Cosmos DB SQL API service.
 """
 
-from typing import Any, List, Dict, Union, cast, Optional
+from typing import Any, Dict, Union, cast
 from azure.core.async_paging import AsyncItemPaged
 
 from azure.core.tracing.decorator_async import distributed_trace_async
 from azure.core.tracing.decorator import distributed_trace
 
 from ._cosmos_client_connection_async import CosmosClientConnection
 from .._base import build_options
-from ..permission import Permission as _permission
+from ..permission import Permission
 
 
 class UserProxy(object):
@@ -42,19 +42,22 @@ class UserProxy(object):
     :func:`DatabaseProxy.get_user_client` method.
     """
 
-    def __init__(self, client_connection, id, database_link, properties=None):  # pylint: disable=redefined-builtin
-        # type: (CosmosClientConnection, str, str, Dict[str, Any]) -> None
+    def __init__(
+            self,
+            client_connection: CosmosClientConnection,
+            id: str,  # pylint: disable=redefined-builtin
+            database_link: str,
+            properties: Dict[str, Any] = None
+    ) -> None:
         self.client_connection = client_connection
         self.id = id
         self.user_link = u"{}/users/{}".format(database_link, id)
         self._properties = properties
 
-    def __repr__(self):
-        # type () -> str
+    def __repr__(self) -> str:
         return "<UserProxy [{}]>".format(self.user_link)[:1024]
 
-    def _get_permission_link(self, permission_or_id):
-        # type: (Union[Permission, str, Dict[str, Any]]) -> str
+    def _get_permission_link(self, permission_or_id: Union[Permission, str, Dict[str, Any]]) -> str:
         if isinstance(permission_or_id, str):
             return u"{}/permissions/{}".format(self.user_link, permission_or_id)
         try:
@@ -63,21 +66,20 @@ def _get_permission_link(self, permission_or_id):
             pass
         return u"{}/permissions/{}".format(self.user_link, cast("Dict[str, str]", permission_or_id)["id"])
 
-    async def _get_properties(self):
-        # type: () -> Dict[str, Any]
+    async def _get_properties(self) -> Dict[str, Any]:
         if self._properties is None:
             self._properties = await self.read()
         return self._properties
 
     @distributed_trace_async
-    async def read(self, **kwargs):
-        # type: (Any) -> Dict[str, Any]
-        """Read user propertes.
+    async def read(self, **kwargs: Any) -> Dict[str, Any]:
+        """Read user properties.
 
-        :keyword Callable response_hook: A callable invoked with the response metadata.
-        :returns: A dictionary of the retrieved user properties.
+        :keyword response_hook: A callable invoked with the response metadata.
+        :paramtype response_hook: Callable[[Dict[str, str], Dict[str, Any]], None]
         :raises ~azure.cosmos.exceptions.CosmosHttpResponseError: If the given user couldn't be retrieved.
-        :rtype: dict[str, Any]
+        :returns: A dictionary of the retrieved user properties.
+        :rtype: Dict[str, Any]
         """
         request_options = build_options(kwargs)
         response_hook = kwargs.pop('response_hook', None)
@@ -91,17 +93,18 @@ async def read(self, **kwargs):
         return cast('Dict[str, Any]', self._properties)
 
     @distributed_trace
-    def list_permissions(self, max_item_count=None, **kwargs):
-        # type: (Optional[int], Any) -> AsyncItemPaged[Dict[str, Any]]
+    def list_permissions(self, **kwargs: Any) -> AsyncItemPaged[Dict[str, Any]]:
         """List all permission for the user.
 
-        :param max_item_count: Max number of permissions to be returned in the enumeration operation.
-        :keyword Callable response_hook: A callable invoked with the response metadata.
+        :keyword int max_item_count: Max number of permissions to be returned in the enumeration operation.
+        :keyword response_hook: A callable invoked with the response metadata.
+        :paramtype response_hook: Callable[[Dict[str, str], AsyncItemPaged[Dict[str, Any]], None]
         :returns: An AsyncItemPaged of permissions (dicts).
-        :rtype: AsyncItemPaged[dict[str, Any]]
+        :rtype: AsyncItemPaged[Dict[str, Any]]
         """
         feed_options = build_options(kwargs)
         response_hook = kwargs.pop('response_hook', None)
+        max_item_count = kwargs.pop('max_item_count', None)
         if max_item_count is not None:
             feed_options["maxItemCount"] = max_item_count
 
@@ -115,26 +118,27 @@ def list_permissions(self, max_item_count=None, **kwargs):
     @distributed_trace
     def query_permissions(
         self,
-        query,
-        parameters=None,
-        max_item_count=None,
-        **kwargs
-    ):
-        # type: (str, Optional[List[str]], Optional[int], Any) -> AsyncItemPaged[Dict[str, Any]]
+        query: Union[str, Dict[str, Any]],
+        **kwargs: Any
+    ) -> AsyncItemPaged[Dict[str, Any]]:
         """Return all permissions matching the given `query`.
 
-        :param query: The Azure Cosmos DB SQL query to execute.
-        :param parameters: Optional array of parameters to the query. Ignored if no query is provided.
-        :param max_item_count: Max number of permissions to be returned in the enumeration operation.
-        :keyword Callable response_hook: A callable invoked with the response metadata.
+        :param Union[str, Dict[str, Any]] query: The Azure Cosmos DB SQL query to execute.
+        :keyword parameters: Optional array of parameters to the query. Ignored if no query is provided.
+        :paramtype parameters: Optional[List[Dict[str, Any]]]
+        :keyword int max_item_count: Max number of permissions to be returned in the enumeration operation.
+        :keyword response_hook: A callable invoked with the response metadata.
+        :paramtype response_hook: Callable[[Dict[str, str], AsyncItemPaged[Dict[str, Any]]], None]
         :returns: An AsyncItemPaged of permissions (dicts).
-        :rtype: AsyncItemPaged[dict[str, Any]]
+        :rtype: AsyncItemPaged[Dict[str, Any]]
         """
         feed_options = build_options(kwargs)
         response_hook = kwargs.pop('response_hook', None)
+        max_item_count = kwargs.pop('max_item_count', None)
         if max_item_count is not None:
             feed_options["maxItemCount"] = max_item_count
 
+        parameters = kwargs.pop('parameters', None)
         result = self.client_connection.QueryPermissions(
             user_link=self.user_link,
             query=query if parameters is None else dict(query=query, parameters=parameters),
@@ -148,16 +152,21 @@ def query_permissions(
         return result
 
     @distributed_trace_async
-    async def get_permission(self, permission, **kwargs):
-        # type: (Union[str, Dict[str, Any], Permission], Any) -> Permission
+    async def get_permission(
+            self,
+            permission: Union[str, Dict[str, Any], Permission],
+            **kwargs: Any
+    ) -> Permission:
         """Get the permission identified by `id`.
 
         :param permission: The ID (name), dict representing the properties or :class:`Permission`
             instance of the permission to be retrieved.
-        :keyword Callable response_hook: A callable invoked with the response metadata.
-        :returns: A dict representing the retrieved permission.
+        :type permission: Union[str, Dict[str, Any], ~azure.cosmos.Permission]
+        :keyword response_hook: A callable invoked with the response metadata.
+        :paramtype response_hook: Callable[[Dict[str, str], Dict[str, Any]], None]
         :raises ~azure.cosmos.exceptions.CosmosHttpResponseError: If the given permission couldn't be retrieved.
-        :rtype: dict[str, Any]
+        :returns: The retrieved permission object.
+        :rtype: ~azure.cosmos.Permission
         """
         request_options = build_options(kwargs)
         response_hook = kwargs.pop('response_hook', None)
@@ -169,7 +178,7 @@ async def get_permission(self, permission, **kwargs):
         if response_hook:
             response_hook(self.client_connection.last_response_headers, permission_resp)
 
-        return _permission(
+        return Permission(
             id=permission_resp["id"],
             user_link=self.user_link,
             permission_mode=permission_resp["permissionMode"],
@@ -178,17 +187,18 @@ async def get_permission(self, permission, **kwargs):
         )
 
     @distributed_trace_async
-    async def create_permission(self, body, **kwargs):
-        # type: (Dict[str, Any], Any) -> Permission
+    async def create_permission(self, body: Dict[str, Any], **kwargs: Any) -> Permission:
         """Create a permission for the user.
 
-        To update or replace an existing permision, use the :func:`UserProxy.upsert_permission` method.
+        To update or replace an existing permission, use the :func:`UserProxy.upsert_permission` method.
 
         :param body: A dict-like object representing the permission to create.
-        :keyword Callable response_hook: A callable invoked with the response metadata.
-        :returns: A dict representing the new permission.
+        :type body: Dict[str, Any]
+        :keyword response_hook: A callable invoked with the response metadata.
+        :paramtype response_hook: Callable[[Dict[str, str], Dict[str, Any]], None]
         :raises ~azure.cosmos.exceptions.CosmosHttpResponseError: If the given permission couldn't be created.
-        :rtype: dict[str, Any]
+        :returns: A permission object representing the new permission.
+        :rtype: ~azure.cosmos.Permission
         """
         request_options = build_options(kwargs)
         response_hook = kwargs.pop('response_hook', None)
@@ -200,7 +210,7 @@ async def create_permission(self, body, **kwargs):
         if response_hook:
             response_hook(self.client_connection.last_response_headers, permission)
 
-        return _permission(
+        return Permission(
             id=permission["id"],
             user_link=self.user_link,
             permission_mode=permission["permissionMode"],
@@ -209,18 +219,19 @@ async def create_permission(self, body, **kwargs):
         )
 
     @distributed_trace_async
-    async def upsert_permission(self, body, **kwargs):
-        # type: (Dict[str, Any], Any) -> Permission
+    async def upsert_permission(self, body: Dict[str, Any], **kwargs: Any) -> Permission:
         """Insert or update the specified permission.
 
         If the permission already exists in the container, it is replaced. If
         the permission does not exist, it is inserted.
 
         :param body: A dict-like object representing the permission to update or insert.
-        :param Callable response_hook: A callable invoked with the response metadata.
-        :returns: A dict representing the upserted permission.
+        :type body: Dict[str, Any]
+        :keyword response_hook: A callable invoked with the response metadata.
+        :paramtype response_hook: Callable[[Dict[str, str], Dict[str, Any]], None]
         :raises ~azure.cosmos.exceptions.CosmosHttpResponseError: If the given permission could not be upserted.
-        :rtype: dict[str, Any]
+        :returns: A dict representing the upserted permission.
+        :rtype: ~azure.cosmos.Permission
         """
         request_options = build_options(kwargs)
         response_hook = kwargs.pop('response_hook', None)
@@ -232,7 +243,7 @@ async def upsert_permission(self, body, **kwargs):
         if response_hook:
             response_hook(self.client_connection.last_response_headers, permission)
 
-        return _permission(
+        return Permission(
             id=permission["id"],
             user_link=self.user_link,
             permission_mode=permission["permissionMode"],
@@ -241,20 +252,27 @@ async def upsert_permission(self, body, **kwargs):
         )
 
     @distributed_trace_async
-    async def replace_permission(self, permission, body, **kwargs):
-        # type: (str, Union[str, Dict[str, Any], Permission], Any) -> Permission
+    async def replace_permission(
+            self,
+            permission: Union[str, Dict[str, Any], Permission],
+            body: Dict[str, Any],
+            **kwargs: Any
+    ) -> Permission:
         """Replaces the specified permission if it exists for the user.
 
         If the permission does not already exist, an exception is raised.
 
         :param permission: The ID (name), dict representing the properties or :class:`Permission`
             instance of the permission to be replaced.
+        :type permission: Union[str, Dict[str, Any], ~azure.cosmos.Permission]
         :param body: A dict-like object representing the permission to replace.
-        :keyword Callable response_hook: A callable invoked with the response metadata.
-        :returns: A dict representing the permission after replace went through.
+        :type body: Dict[str, Any]
+        :keyword response_hook: A callable invoked with the response metadata.
+        :paramtype response_hook: Callable[[Dict[str, str], Dict[str, Any]], None]
         :raises ~azure.cosmos.exceptions.CosmosHttpResponseError: If the replace failed or the permission
             with given id does not exist.
-        :rtype: dict[str, Any]
+        :returns: A permission object representing the permission after the replace went through.
+        :rtype: ~azure.cosmos.Permission
         """
         request_options = build_options(kwargs)
         response_hook = kwargs.pop('response_hook', None)
@@ -266,7 +284,7 @@ async def replace_permission(self, permission, body, **kwargs):
         if response_hook:
             response_hook(self.client_connection.last_response_headers, permission_resp)
 
-        return _permission(
+        return Permission(
             id=permission_resp["id"],
             user_link=self.user_link,
             permission_mode=permission_resp["permissionMode"],
@@ -275,15 +293,16 @@ async def replace_permission(self, permission, body, **kwargs):
         )
 
     @distributed_trace_async
-    async def delete_permission(self, permission, **kwargs):
-        # type: (Union[str, Dict[str, Any], Permission], Any) -> None
+    async def delete_permission(self, permission: Union[str, Dict[str, Any], Permission], **kwargs: Any) -> None:
         """Delete the specified permission from the user.
 
         If the permission does not already exist, an exception is raised.
 
         :param permission: The ID (name), dict representing the properties or :class:`Permission`
             instance of the permission to be deleted.
-        :keyword Callable response_hook: A callable invoked with the response metadata.
+        :type permission: Union[str, Dict[str, Any], ~azure.cosmos.Permission]
+        :keyword response_hook: A callable invoked with the response metadata.
+        :paramtype response_hook: Callable[[Dict[str, str], None], None]
         :raises ~azure.cosmos.exceptions.CosmosHttpResponseError: The permission wasn't deleted successfully.
         :raises ~azure.cosmos.exceptions.CosmosResourceNotFoundError: The permission does not exist for the user.
         :rtype: None