Skip to content

[Blob][STG78]Object Imutability Policy (Version Level Worm) #19098

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
xiafu-msft merged 15 commits into from
Jul 22, 2021
Merged

[Blob][STG78]Object Imutability Policy (Version Level Worm) #19098

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
3ffabe6
[Blob][STG78]Object Imutability Policy (Version Level Worm)
xiafu-msft Jun 4, 2021
90b6c9a
edit import mgmt plane models
xiafu-msft Jun 4, 2021
3939a3f
fix doc pylint dependencies
xiafu-msft Jun 7, 2021
aa14b21
fix test
xiafu-msft Jun 8, 2021
261986c
Merge branch 'feature/storage-stg78' into vlw
xiafu-msft Jul 19, 2021
c581de0
rename immutability_policy permission
xiafu-msft Jul 19, 2021
86b677f
rename legal_hold
xiafu-msft Jul 19, 2021
c2d9026
Update _deserialize.py
xiafu-msft Jul 20, 2021
197c339
rename legal_hold
xiafu-msft Jul 20, 2021
2593525
rename legal_hold
xiafu-msft Jul 20, 2021
8dea092
Create ImmutabilityPolicy class
xiafu-msft Jul 22, 2021
880758c
consume BlobAccountPreparer
xiafu-msft Jul 22, 2021
1611deb
add import
xiafu-msft Jul 22, 2021
859208a
fix import
xiafu-msft Jul 22, 2021
9cb1d56
fix tests
xiafu-msft Jul 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion sdk/storage/azure-storage-blob/azure/storage/blob/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@
UserDelegationKey
)
from ._generated.models import (
RehydratePriority
RehydratePriority,
BlobImmutabilityPolicyMode
)
from ._models import (
BlobType,
Expand Down Expand Up @@ -195,6 +196,7 @@ def download_blob_from_url(
'StandardBlobTier',
'PremiumPageBlobTier',
'SequenceNumberAction',
'BlobImmutabilityPolicyMode',
'PublicAccess',
'BlobAnalyticsLogging',
'Metrics',
Expand Down
162 changes: 161 additions & 1 deletion sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,7 @@ def _upload_blob_options( # pylint:disable=too-many-statements
kwargs['blob_settings'] = self._config
kwargs['max_concurrency'] = max_concurrency
kwargs['encryption_options'] = encryption_options

if blob_type == BlobType.BlockBlob:
kwargs['client'] = self._client.block_blob
kwargs['data'] = data
Expand Down Expand Up @@ -644,6 +645,28 @@ def upload_blob( # pylint: disable=too-many-locals
:keyword ~azure.storage.blob.StandardBlobTier standard_blob_tier:
A standard blob tier value to set the blob to. For this version of the library,
this is only applicable to block blobs on standard storage accounts.
:keyword ~datetime.datetime immutability_policy_expiry_time:
Specifies the date time when the blobs immutability policy is set to expire.
Currently this parameter of upload_blob() API is for BlockBlob only.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword immutability_policy_mode:
Specifies the immutability policy mode to set on the blob.
Currently this parameter of upload_blob() API is for BlockBlob only.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:paramtype immutability_policy_mode: ~azure.storage.blob.BlobImmutabilityPolicyMode or str
:keyword bool legal_hold:
Specified if a legal hold should be set on the blob.
Currently this parameter of upload_blob() API is for BlockBlob only.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword int maxsize_condition:
Optional conditional header. The max length in bytes permitted for
the append blob. If the Append Block operation would cause the blob
Expand Down Expand Up @@ -1384,6 +1407,63 @@ def set_blob_metadata(self, metadata=None, **kwargs):
except HttpResponseError as error:
process_storage_error(error)

@distributed_trace
def set_immutability_policy(self, **kwargs):
# type: (**Any) -> Dict[str, str]
"""The Set Immutability Policy operation sets the immutability policy on the blob.

.. versionadded:: 12.10.0
This operation was introduced in API version '2020-10-02'.

:keyword ~datetime.datetime immutability_policy_expiry_time:
Specifies the date time when the blobs immutability policy is set to expire.
:keyword immutability_policy_mode:
Specifies the immutability policy mode to set on the blob.
:paramtype immutability_policy_mode: ~azure.storage.blob.BlobImmutabilityPolicyMode or str
Copy link
Contributor Author

@xiafu-msft xiafu-msft Jul 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @kasobol-msft I think we need your thoughts here.
Gauri @gapra-msft and I were discussing about these two parameters. For Java and .Net these two parameters are grouped into a class called BlobImmutabilityPolicy, while in Python we don't have the overloading concern so usually we just list the parameters. Do you have any thoughts? Should we keep it consistent with Java and .Net or leave it like this?
Also Anna suggested to rename these two parameters to expiry and mode, and I put the reason why I haven't renamed them yet https://apiview.dev/Assemblies/Review/266426541eba45098d08bbddfb5ce8ac/7977909c50af4022868fd86d76bbc17c#azure.storage.blob.aio.BlobClient.set_immutability_policy:async.param(immutability_policy_mode

Thanks for your input!

Copy link
Contributor

@kasobol-msft kasobol-msft Jul 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd group them together in both places. (we seem to do this from time to time for inputs that belong together

azure-sdk-for-python/sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py

Lines 538 to 540 in a65e853

:keyword ~azure.storage.blob.ContentSettings content_settings:
ContentSettings object used to set blob properties. Used to set content type, encoding,
language, disposition, md5, and cache control.
)

Copy link
Contributor Author

@xiafu-msft xiafu-msft Jul 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can group them together just some params are not grouped together and they are all for conditional match

azure-sdk-for-python/sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py

Lines 514 to 530 in a65e853

:keyword ~datetime.datetime if_modified_since:
A DateTime value. Azure expects the date value passed in to be UTC.
If timezone is included, any non-UTC datetimes will be converted to UTC.
If a date is passed in without timezone info, it is assumed to be UTC.
Specify this header to perform the operation only
if the resource has been modified since the specified time.
:keyword ~datetime.datetime if_unmodified_since:
A DateTime value. Azure expects the date value passed in to be UTC.
If timezone is included, any non-UTC datetimes will be converted to UTC.
If a date is passed in without timezone info, it is assumed to be UTC.
Specify this header to perform the operation only if
the resource has not been modified since the specified date/time.
:keyword str etag:
The destination ETag value, or the wildcard character (*). Used to check if the resource has changed,
and act according to the condition specified by the `match_condition` parameter.
:keyword ~azure.core.MatchConditions match_condition:
The destination match condition to use upon the etag.

:keyword int timeout:
The timeout parameter is expressed in seconds.
:returns: Key value pairs of blob tags.
:rtype: Dict[str, str]
"""

return self._client.blob.set_immutability_policy(
immutability_policy_expiry=kwargs.pop('immutability_policy_expiry_time', None),
cls=return_response_headers, **kwargs)

@distributed_trace
def delete_immutability_policy(self, **kwargs):
# type: (**Any) -> None
"""The Delete Immutability Policy operation deletes the immutability policy on the blob.

.. versionadded:: 12.10.0
This operation was introduced in API version '2020-10-02'.

:keyword int timeout:
The timeout parameter is expressed in seconds.
:returns: Key value pairs of blob tags.
:rtype: Dict[str, str]
"""

self._client.blob.delete_immutability_policy(**kwargs)

@distributed_trace
def set_legal_hold(self, legal_hold, **kwargs):
# type: (**Any) -> Dict[str, Union[str, datetime, bool]]
"""The Set Legal Hold operation sets a legal hold on the blob.

.. versionadded:: 12.10.0
This operation was introduced in API version '2020-10-02'.

:param bool legal_hold:
Specified if a legal hold should be set on the blob.
:keyword int timeout:
The timeout parameter is expressed in seconds.
:returns: Key value pairs of blob tags.
:rtype: Dict[str, str]
"""

return self._client.blob.set_legal_hold(legal_hold, cls=return_response_headers, **kwargs)

def _create_page_blob_options( # type: ignore
self, size, # type: int
content_settings=None, # type: Optional[ContentSettings]
Expand Down Expand Up @@ -1432,6 +1512,7 @@ def _create_page_blob_options( # type: ignore
'blob_content_length': size,
'blob_sequence_number': sequence_number,
'blob_http_headers': blob_headers,
'immutability_policy_expiry': kwargs.pop('immutability_policy_expiry_time', None),
'timeout': kwargs.pop('timeout', None),
'lease_access_conditions': access_conditions,
'modified_access_conditions': mod_conditions,
Expand Down Expand Up @@ -1485,6 +1566,25 @@ def create_page_blob( # type: ignore
Required if the blob has an active lease. Value can be a BlobLeaseClient object
or the lease ID as a string.
:paramtype lease: ~azure.storage.blob.BlobLeaseClient or str
:keyword ~datetime.datetime immutability_policy_expiry_time:
Specifies the date time when the blobs immutability policy is set to expire.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword immutability_policy_mode:
Specifies the immutability policy mode to set on the blob.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:paramtype immutability_policy_mode: ~azure.storage.blob.BlobImmutabilityPolicyMode or str
:keyword bool legal_hold:
Specified if a legal hold should be set on the blob.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword ~datetime.datetime if_modified_since:
A DateTime value. Azure expects the date value passed in to be UTC.
If timezone is included, any non-UTC datetimes will be converted to UTC.
Expand Down Expand Up @@ -1563,6 +1663,7 @@ def _create_append_blob_options(self, content_settings=None, metadata=None, **kw
options = {
'content_length': 0,
'blob_http_headers': blob_headers,
'immutability_policy_expiry': kwargs.pop('immutability_policy_expiry_time', None),
'timeout': kwargs.pop('timeout', None),
'lease_access_conditions': access_conditions,
'modified_access_conditions': mod_conditions,
Expand Down Expand Up @@ -1599,6 +1700,25 @@ def create_append_blob(self, content_settings=None, metadata=None, **kwargs):
Required if the blob has an active lease. Value can be a BlobLeaseClient object
or the lease ID as a string.
:paramtype lease: ~azure.storage.blob.BlobLeaseClient or str
:keyword ~datetime.datetime immutability_policy_expiry_time:
Specifies the date time when the blobs immutability policy is set to expire.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword immutability_policy_mode:
Specifies the immutability policy mode to set on the blob.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:paramtype immutability_policy_mode: ~azure.storage.blob.BlobImmutabilityPolicyMode or str
:keyword bool legal_hold:
Specified if a legal hold should be set on the blob.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword ~datetime.datetime if_modified_since:
A DateTime value. Azure expects the date value passed in to be UTC.
If timezone is included, any non-UTC datetimes will be converted to UTC.
Expand Down Expand Up @@ -1767,6 +1887,7 @@ def _start_copy_from_url_options(self, source_url, metadata=None, incremental_co
options = {
'copy_source': source_url,
'seal_blob': kwargs.pop('seal_destination_blob', None),
'immutability_policy_expiry': kwargs.pop('immutability_policy_expiry_time', None),
'timeout': timeout,
'modified_access_conditions': dest_mod_conditions,
'blob_tags_string': blob_tags_string,
Expand Down Expand Up @@ -1851,6 +1972,25 @@ def start_copy_from_url(self, source_url, metadata=None, incremental_copy=False,
.. versionadded:: 12.4.0

:paramtype tags: dict(str, str)
:keyword ~datetime.datetime immutability_policy_expiry_time:
Specifies the date time when the blobs immutability policy is set to expire.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword immutability_policy_mode:
Specifies the immutability policy mode to set on the blob.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:paramtype immutability_policy_mode: ~azure.storage.blob.BlobImmutabilityPolicyMode or str
:keyword bool legal_hold:
Specified if a legal hold should be set on the blob.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword ~datetime.datetime source_if_modified_since:
A DateTime value. Azure expects the date value passed in to be UTC.
If timezone is included, any non-UTC datetimes will be converted to UTC.
Expand Down Expand Up @@ -1917,7 +2057,7 @@ def start_copy_from_url(self, source_url, metadata=None, incremental_copy=False,
:keyword bool requires_sync:
Enforces that the service will not return a response until the copy is complete.
:returns: A dictionary of copy properties (etag, last_modified, copy_id, copy_status).
:rtype: dict[str, str or ~datetime.datetime]
:rtype: dict[str, Union[str, ~datetime.datetime]]

.. admonition:: Example:

Expand Down Expand Up @@ -2407,6 +2547,7 @@ def _commit_block_list_options( # type: ignore
'validate_content': validate_content,
'cpk_scope_info': cpk_scope_info,
'cpk_info': cpk_info,
'immutability_policy_expiry': kwargs.pop('immutability_policy_expiry_time', None),
'tier': tier.value if tier else None,
'blob_tags_string': blob_tags_string,
'headers': headers
Expand Down Expand Up @@ -2447,6 +2588,25 @@ def commit_block_list( # type: ignore
Required if the blob has an active lease. Value can be a BlobLeaseClient object
or the lease ID as a string.
:paramtype lease: ~azure.storage.blob.BlobLeaseClient or str
:keyword ~datetime.datetime immutability_policy_expiry_time:
Specifies the date time when the blobs immutability policy is set to expire.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword immutability_policy_mode:
Specifies the immutability policy mode to set on the blob.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:paramtype immutability_policy_mode: ~azure.storage.blob.BlobImmutabilityPolicyMode or str
:keyword bool legal_hold:
Specified if a legal hold should be set on the blob.

.. versionadded:: 12.10.0
This was introduced in API version '2020-10-02'.

:keyword bool validate_content:
If true, calculates an MD5 hash of the page content. The storage
service checks the hash of the content that has arrived
Expand Down
2 changes: 1 addition & 1 deletion sdk/storage/azure-storage-blob/azure/storage/blob/_container_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -744,7 +744,7 @@ def list_blobs(self, name_starts_with=None, include=None, **kwargs):
:param list[str] or str include:
Specifies one or more additional datasets to include in the response.
Options include: 'snapshots', 'metadata', 'uncommittedblobs', 'copy', 'deleted', 'deletedwithversions',
'tags', 'versions'.
'tags', 'versions', 'immutabilitypolicy', 'legalhold'.
:keyword int timeout:
The timeout parameter is expressed in seconds.
:returns: An iterable (auto-paging) response of BlobProperties.
Expand Down
3 changes: 3 additions & 0 deletions sdk/storage/azure-storage-blob/azure/storage/blob/_deserialize.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,9 @@ def get_blob_properties_from_generated_code(generated):
blob.tags = parse_tags(generated.blob_tags) # pylint: disable=protected-access
blob.object_replication_source_properties = deserialize_ors_policies(generated.object_replication_metadata)
blob.last_accessed_on = generated.properties.last_accessed_on
blob.immutability_policy_expiry_time = generated.properties.immutability_policy_expires_on
blob.immutability_policy_mode = generated.properties.immutability_policy_mode
blob.legal_hold = generated.properties.legal_hold
blob.has_versions_only = generated.has_versions_only
return blob

Expand Down
Loading