-
Notifications
You must be signed in to change notification settings - Fork 1.6k
[Breadth Coverage] Datashare #1476
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 13 commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
aa33a8c
init codegen for datashare
fengzhou-msft 254f368
add manual code
fengzhou-msft 736f686
modify help
fengzhou-msft 7af1a13
modify datashare account
fengzhou-msft ecd3036
modify params
fengzhou-msft 86e4bbd
update metadata
fengzhou-msft f9788ef
add init files
fengzhou-msft b891431
add integration tests
fengzhou-msft 23caf12
full tests for provider and consumer
fengzhou-msft 77b93cb
Merge remote-tracking branch 'upstream/master' into datashare
fengzhou-msft 47fd90d
refactor command interface
fengzhou-msft 2fedea2
add README
fengzhou-msft 306793f
add wait command for datashare consumer share-subscription synchroniz…
fengzhou-msft f74f671
Update README.md
fengzhou-msft 217ef67
fix location
fengzhou-msft ab2b208
modify help
fengzhou-msft aa7cd57
remove --identity
fengzhou-msft d1073be
flattern synchronization setting
fengzhou-msft 6b20902
remove some generated file
fengzhou-msft 9fa2a5c
Merge branch 'master' into datashare
fengzhou-msft 6600ebb
removed unused file
fengzhou-msft 251fb01
Merge branch 'datashare' of github.com:fengzhou-msft/azure-cli-extens…
fengzhou-msft d82d28d
update readme
fengzhou-msft 388e336
modify trigger
fengzhou-msft e77229f
update recordings
fengzhou-msft 8ad724e
remove unsed files
fengzhou-msft 3178a95
Merge branch 'datashare' of github.com:fengzhou-msft/azure-cli-extens…
fengzhou-msft File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -97,3 +97,5 @@ | |
| /src/databox/ @jsntcy | ||
|
|
||
| /src/hpc-cache/ @qianwens | ||
|
|
||
| /src/datashare/ @fengzhou-msft | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| .. :changelog: | ||
|
|
||
| Release History | ||
| =============== | ||
|
|
||
| 0.1.0 | ||
| ++++++ | ||
| * Initial release. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,195 @@ | ||
| # Azure CLI datashare Extension # | ||
| This package is for the 'datashare' extension, i.e. 'az datashare'. More info on what is [Data Share](https://docs.microsoft.com/azure/data-share/). | ||
|
|
||
| ### How to use ### | ||
| Install this extension using the below CLI command | ||
| ``` | ||
| az extension add --name datashare | ||
| ``` | ||
|
|
||
| Register DataShare Resource Provider for your default susbcription. | ||
| ``` | ||
| az provider register -n "Microsoft.DataShare" | ||
| ``` | ||
|
|
||
| ### Included Features | ||
| #### Datashare Account Management: | ||
| *Examples:* | ||
|
|
||
| ##### Create a Datashare Account | ||
|
|
||
| ``` | ||
| az datashare account create \ | ||
| --identity type=SystemAssigned \ | ||
| --location "West US 2" \ | ||
| --tags tag1=Red tag2=White \ | ||
| --name "cli_test_account" \ | ||
| --resource-group "datashare_provider_rg" | ||
| ``` | ||
|
|
||
| ##### Wait for the Datashare Account to be provisioned | ||
| ``` | ||
| az datashare account wait \ | ||
| --name "cli_test_account" \ | ||
| --resource-group "datashare_provider_rg" \ | ||
| --created | ||
| ``` | ||
|
|
||
| #### Datashare Resource Management for a Provider: | ||
| *Examples:* | ||
|
|
||
| ##### Create a Datashare | ||
| ``` | ||
| az datashare create \ | ||
| --account-name "cli_test_account" \ | ||
| --resource-group "datashare_provider_rg" \ | ||
| --description "share description" \ | ||
| --share-kind "CopyBased" \ | ||
| --terms "Confidential" \ | ||
| --name "cli_test_share" | ||
| ``` | ||
|
|
||
| ##### Create a Data Set | ||
| ``` | ||
| az datashare dataset create \ | ||
| --account-name "cli_test_account" \ | ||
| --name "cli_test_data_set" \ | ||
| --resource-group "datashare_provider_rg" \ | ||
| --share-name "cli_test_share" \ | ||
| --dataset "{\"container_name\":\"mycontainer\",\"storage_account_name\":\"mysa\",\"kind\":\"Container\"}" | ||
| ``` | ||
|
|
||
| Please make sure the datashare account has the right permission of the data source when creating a data set upon it. | ||
| For instance, you can use `az datashare account show` to get 'identity.principalId' of the account, then assign the right role to it. | ||
| ``` | ||
| az role assignment create \ | ||
| --role "Storage Blob Data Reader" \ | ||
| --assignee-object-id {DatashareAccountPrincipalId} \ | ||
| --assignee-principal-type ServicePrincipal \ | ||
| --scope {StorageAccountId} | ||
| ``` | ||
|
|
||
| ##### Create a Synchronization Setting | ||
| ``` | ||
| az datashare synchronization-setting create \ | ||
| --account-name "cli_test_account" \ | ||
| --resource-group "datashare_provider_rg" \ | ||
| --share-name "cli_test_share" \ | ||
| --name "cli_test_synchronization_setting" \ | ||
| --setting "{\"recurrenceInterval\":\"Day\",\"synchronizationTime\":\"2020-04-05T10:50:00Z\",\"kind\":\"ScheduleBased\"}" | ||
| ``` | ||
|
|
||
| ##### List Synchronization History | ||
| ``` | ||
| az datashare synchronization list \ | ||
| --account-name "cli_test_account" \ | ||
| --resource-group "datashare_provider_rg" \ | ||
| --share-name "cli_test_share" | ||
| ``` | ||
|
|
||
| ##### Create a Datashare Invitation | ||
| ``` | ||
| az datashare invitation create \ | ||
| --account-name "cli_test_account" \ | ||
| --target-email "myname@microsoft.com" \ | ||
| --name "cli_test_invitation" \ | ||
| --resource-group "datashare_provider_rg" \ | ||
| --share-name "cli_test_share" | ||
| ``` | ||
|
|
||
| ##### List Share Subscriptions | ||
| ``` | ||
| az datashare provider-share-subscription list \ | ||
| --account-name "cli_test_account" \ | ||
| --resource-group "datashare_provider_rg" \ | ||
| --share-name "cli_test_share" | ||
| ``` | ||
| Share subscriptions are created by Datashare consumers when they accept invitations. | ||
|
|
||
| ##### Revoke Datashare for a Share Subscription | ||
| az datashare provider-share-subscription revoke \ | ||
| --account-name "cli_test_account" \ | ||
| --share-subscription "{ProviderShareSubscriptionObjectId}" \ | ||
| --resource-group "datashare_provider_rg" \ | ||
| --share-name "cli_test_share" | ||
|
|
||
| ##### Reinstate Datashare for a Share Subscription | ||
| az datashare provider-share-subscription reinstate \ | ||
| --account-name "cli_test_account" \ | ||
| --share-subscription "{ProviderShareSubscriptionObjectId}" \ | ||
| --resource-group "datashare_provider_rg" \ | ||
| --share-name "cli_test_share" | ||
|
|
||
| #### Datashare Resource Management for a Consumer: | ||
| *Examples:* | ||
|
|
||
| ##### List received Invitations | ||
| ``` | ||
| az datashare consumer invitation list | ||
| ``` | ||
|
|
||
| ##### Create a Share Subscription from an Invitation | ||
| ``` | ||
| az datashare consumer share-subscription create \ | ||
| --account-name "cli_test_consumer_account" \ | ||
| --resource-group "datashare_consumer_rg" \ | ||
| --invitation-id "{InvitationId1}" \ | ||
| --source-share-location "sourceShareLocation" \ | ||
| --name "cli_test_share_subscription" | ||
| ``` | ||
|
|
||
| ##### List Source Data Sets in the Share Subscription | ||
| ``` | ||
| az datashare consumer share-subscription list-source-dataset \ | ||
| --account-name "cli_test_consumer_account" \ | ||
| --resource-group "datashare_consumer_rg" \ | ||
| --share-subscription-name "cli_test_share_subscription" | ||
| ``` | ||
|
|
||
| ##### Create a Data Set Mapping of the Source Data Set | ||
| ``` | ||
| az datashare consumer dataset-mapping create \ | ||
| --account-name "cli_test_consumer_account" \ | ||
| --name "cli_test_data_set_mapping" \ | ||
| --resource-group "datashare_consumer_rg" \ | ||
| --share-subscription-name "cli_test_share_subscription" \ | ||
| --mapping "{\"data_set_id\":\"2036a39f-add6-4347-9c82-a424dfaf4e8d\", \ | ||
| \"container_name\":\"newcontainer\", \"storage_account_name\":\"consumersa\", \ | ||
| \"kind\":\"BlobFolder\",\"prefix\":\"myprefix\"}" | ||
| ``` | ||
| Please make sure the datashare consumer account has the right permission of the data target when creating a data set mapping on it. For instance, you can use `az datashare account show` to get 'identity.principalId' of the account, then assign the right role to it. | ||
| ``` | ||
| az role assignment create \ | ||
| --role "Storage Blob Data Contributor" \ | ||
| --assignee-object-id "{ConsumerDatashareAccountPrincipalId}" \ | ||
| --assignee-principal-type ServicePrincipal \ | ||
| --scope "{ConsumerStorageAccountId}" | ||
| ``` | ||
|
|
||
| ##### List the synchronization settings of the Source Datashare | ||
| ``` | ||
| az datashare consumer share-subscription list-source-share-synchronization-setting \ | ||
| --account-name "cli_test_consumer_account" \ | ||
| --resource-group "datashare_consumer_rg" \ | ||
| --share-subscription-name "cli_test_share_subscription" | ||
| ``` | ||
|
|
||
| ##### Create a trigger for the Share Subsciption | ||
| ``` | ||
| az datashare consumer trigger create \ | ||
| --account-name "cli_test_consumer_account" \ | ||
| --resource-group "datashare_consumer_rg" \ | ||
| --share-subscription-name "cli_test_share_subscription" \ | ||
| --name "cli_test_trigger" \ | ||
| --trigger "{\"kind\":\"ScheduleBased\",\"recurrenceInterval\":\"Day\", \ | ||
| \"synchronizationTime\":\"2020-04-03T08:45:35+00:00\"}" | ||
| ``` | ||
|
|
||
| ##### Start a synchronization for the Share Subscription | ||
| ``` | ||
| az datashare consumer share-subscription synchronization start \ | ||
| --account-name "cli_test_consumer_account" \ | ||
| --resource-group "datashare_consumer_rg" \ | ||
| --share-subscription-name "cli_test_share_subscription" \ | ||
| --synchronization-mode "Incremental" | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # -------------------------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for license information. | ||
| # -------------------------------------------------------------------------------------------- | ||
| # pylint: disable=unused-argument | ||
| # pylint: disable=unused-import | ||
|
|
||
| from azure.cli.core import AzCommandsLoader | ||
| from azext_datashare._help import helps | ||
|
|
||
|
|
||
| class DataShareManagementClientCommandsLoader(AzCommandsLoader): | ||
|
|
||
| def __init__(self, cli_ctx=None): | ||
| from azure.cli.core.commands import CliCommandType | ||
| from azext_datashare.generated._client_factory import cf_datashare | ||
| datashare_custom = CliCommandType( | ||
| operations_tmpl='azext_datashare.manual.custom#{}', # modified | ||
| client_factory=cf_datashare) | ||
| super(DataShareManagementClientCommandsLoader, self).__init__(cli_ctx=cli_ctx, | ||
| custom_command_type=datashare_custom) | ||
|
|
||
| def load_command_table(self, args): | ||
| from azext_datashare.generated.commands import load_command_table | ||
| load_command_table(self, args) | ||
| try: | ||
| from azext_datashare.manual.commands import load_command_table as load_command_table_manual | ||
| load_command_table_manual(self, args) | ||
| except ImportError: | ||
| pass | ||
| return self.command_table | ||
|
|
||
| def load_arguments(self, command): | ||
| from azext_datashare.generated._params import load_arguments | ||
| load_arguments(self, command) | ||
| try: | ||
| from azext_datashare.manual._params import load_arguments as load_arguments_manual | ||
| load_arguments_manual(self, command) | ||
| except ImportError: | ||
| pass | ||
|
|
||
|
|
||
| COMMAND_LOADER_CLS = DataShareManagementClientCommandsLoader |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # -------------------------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for license information. | ||
| # -------------------------------------------------------------------------------------------- | ||
| # pylint: disable=wildcard-import | ||
| # pylint: disable=unused-wildcard-import | ||
|
|
||
| from .generated._help import * # noqa: F403 | ||
| try: | ||
| from .manual._help import * # noqa: F403 | ||
| except ImportError: | ||
| pass | ||
|
fengzhou-msft marked this conversation as resolved.
Outdated
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # -------------------------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for license information. | ||
| # -------------------------------------------------------------------------------------------- | ||
| # pylint: disable=wildcard-import | ||
| # pylint: disable=unused-wildcard-import | ||
|
|
||
| from .generated._params import * # noqa: F403 | ||
| try: | ||
| from .manual._params import * # noqa: F403 | ||
| except ImportError: | ||
| pass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # -------------------------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for license information. | ||
| # -------------------------------------------------------------------------------------------- | ||
| # pylint: disable=wildcard-import | ||
| # pylint: disable=unused-wildcard-import | ||
|
|
||
| from .generated.action import * # noqa: F403 | ||
| try: | ||
| from .manual.action import * # noqa: F403 | ||
| except ImportError: | ||
| pass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "azext.isExperimental": true, | ||
| "azext.minCliCoreVersion": "2.3.1" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # -------------------------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for license information. | ||
| # -------------------------------------------------------------------------------------------- | ||
| # pylint: disable=wildcard-import | ||
| # pylint: disable=unused-wildcard-import | ||
|
|
||
| from .generated.commands import * # noqa: F403 | ||
| try: | ||
| from .manual.commands import * # noqa: F403 | ||
| except ImportError: | ||
| pass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # -------------------------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for license information. | ||
| # -------------------------------------------------------------------------------------------- | ||
| # pylint: disable=wildcard-import | ||
| # pylint: disable=unused-wildcard-import | ||
|
|
||
| from .generated.custom import * # noqa: F403 | ||
| try: | ||
| from .manual.custom import * # noqa: F403 | ||
| except ImportError: | ||
| pass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # coding=utf-8 | ||
| # -------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for | ||
| # license information. | ||
| # | ||
| # Code generated by Microsoft (R) AutoRest Code Generator. | ||
| # Changes may cause incorrect behavior and will be lost if the code is | ||
| # regenerated. | ||
| # -------------------------------------------------------------------------- | ||
|
|
||
| __path__ = __import__('pkgutil').extend_path(__path__, __name__) |
54 changes: 54 additions & 0 deletions
54
src/datashare/azext_datashare/generated/_client_factory.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| # -------------------------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for license information. | ||
| # -------------------------------------------------------------------------------------------- | ||
|
|
||
|
|
||
| def cf_datashare(cli_ctx, *_): | ||
| from azure.cli.core.commands.client_factory import get_mgmt_service_client | ||
| from ..vendored_sdks.datashare import DataShareManagementClient | ||
| return get_mgmt_service_client(cli_ctx, DataShareManagementClient) | ||
|
|
||
|
|
||
| def cf_account(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).account | ||
|
|
||
|
|
||
| def cf_consumer_invitation(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).consumer_invitation | ||
|
|
||
|
|
||
| def cf_data_set(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).data_set | ||
|
|
||
|
|
||
| def cf_data_set_mapping(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).data_set_mapping | ||
|
|
||
|
|
||
| def cf_invitation(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).invitation | ||
|
|
||
|
|
||
| def cf_share(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).share | ||
|
|
||
|
|
||
| def cf_provider_share_subscription(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).provider_share_subscription | ||
|
|
||
|
|
||
| def cf_share_subscription(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).share_subscription | ||
|
|
||
|
|
||
| def cf_consumer_source_data_set(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).consumer_source_data_set | ||
|
|
||
|
|
||
| def cf_synchronization_setting(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).synchronization_setting | ||
|
|
||
|
|
||
| def cf_trigger(cli_ctx, *_): | ||
| return cf_datashare(cli_ctx).trigger |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
todo?