Skip to content

AzureDataShare: Adding version 2020-10-01-preview swagger #12878

Merged
akning-ms merged 7 commits intoAzure:masterfrom
ahermas:ahermas/V5_2020-10-01-preview_swagger
Feb 11, 2021
Merged

AzureDataShare: Adding version 2020-10-01-preview swagger #12878
akning-ms merged 7 commits intoAzure:masterfrom
ahermas:ahermas/V5_2020-10-01-preview_swagger

Conversation

@ahermas
Copy link
Contributor

@ahermas ahermas commented Feb 5, 2021

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Changelog

Please ensure to add changelog with this PR by answering the following questions.

  1. What's the purpose of the update?
    • new service onboarding
    • new API version
    • update existing version for new feature
    • update existing version to fix swagger quality issue in s360
    • Other, please clarify
  2. When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
  3. When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
  4. If it's an update to existing version, please select SDKs of specific language and CLIs that require refresh after swagger is published.
    • SDK of .NET (need service team to ensure code readiness)
    • SDK of Python
    • SDK of Java
    • SDK of Js
    • SDK of Go
    • PowerShell
    • CLI
    • Terraform
    • No, no need to refresh for updates in this PR

Contribution checklist:

If any further question about AME onboarding or validation tools, please view the FAQ.

ARM API Review Checklist

  • Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

    • Adding new API(s)
    • Adding a new API version
    • Adding a new service
  • Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

  • If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist

If there are following updates in the PR, ensure to request an approval from Breaking Change Review Board as defined in the Breaking Change Policy.

  • Removing API(s) in stable version
  • Removing properties in stable version
  • Removing API version(s) in stable version
  • Updating API in stable or public preview version with Breaking Change Validation errors
  • Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.

Please follow the link to find more details on PR review process.

@openapi-workflow-bot
Copy link

Hi, @ahermas Thanks for your PR. I am workflow bot for review process. Here are some small tips.

  • Please ensure to do self-check against checklists in first PR comment.
  • PR assignee is the person auto-assigned and responsible for your current PR reviewing and merging.
  • For specs comparison cross API versions, Use API Specs Comparison Report Generator
  • If there is CI failure(s), to fix CI error(s) is mandatory for PR merging; or you need to provide justification in PR comment for explanation. How to fix?

  • Any feedback about review process or workflow bot, pls contact swagger and tools team. vsswagger@microsoft.com

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Feb 5, 2021

    Swagger Validation Report

    ️️✔️BreakingChange succeeded [Detail] [Expand]
    There are no breaking changes.
    ️⚠️LintDiff: 1 Warnings warning [Detail]

    Rule Message
    ⚠️ R3018 - EnumInsteadOfBoolean Booleans are not descriptive and make them hard to use. Consider using string enums with allowed set of values defined. Property: fillGapWithZero
    Location: Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json#L5135
    ️️✔️Avocado succeeded [Detail] [Expand]
    Validation passes for Avocado.
    ️️✔️ModelValidation succeeded [Detail] [Expand]
    Validation passes for ModelValidation.
    ️️✔️SemanticValidation succeeded [Detail] [Expand]
    Validation passes for SemanticValidation.
    ️⚠️[Staging] Cross Version BreakingChange (Base on preview version): 9 Warnings warning [Detail]

    Rule Message
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/providers/Microsoft.DataShare/ListInvitations' removed or restructured?
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L314:5
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/providers/Microsoft.DataShare/locations/{location}/RejectInvitation' removed or restructured?
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L406:5
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataShare/accounts/{accountName}/shareSubscriptions/{shareSubscriptionName}/ConsumerSourceDataSets' removed or restructured?
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L1988:5
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataShare/accounts/{accountName}/shareSubscriptions/{shareSubscriptionName}/Synchronize' removed or restructured?
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L2265:5
    ⚠️ 1010 - AddingRequiredParameter The required parameter 'providerShareSubscription' was added in the new version.
    New: Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json#L1556:11
    ⚠️ 1032 - DifferentAllOf The new version has a different 'allOf' property than the previous one.
    New: Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json#L4637:5
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L4538:5
    ⚠️ 1034 - AddedRequiredProperty The new version has new required property 'sourceShareLocation' that was not found in the old version.
    New: Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json#L5782:7
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L5674:7
    ⚠️ 1034 - AddedRequiredProperty The new version has new required property 'sourceShareLocation' that was not found in the old version.
    New: Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json#L5747:9
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L5640:9
    ⚠️ 1034 - AddedRequiredProperty The new version has new required property 'sourceShareLocation' that was not found in the old version.
    New: Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json#L5775:5
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L5668:5
    ️️✔️[Staging] Cross Version BreakingChange (Base on stable version) succeeded [Detail] [Expand]
    There are no breaking changes.

    ️️✔️CredScan succeeded [Detail] [Expand]
    There is no credential detected.
    ️⚠️[Staging] SDK Track2 Validation: 1 Warnings warning [Detail]

    Rule Message
    ⚠️ PreCheck/CheckDuplicateSchemas "readme":"datashare/resource-manager/readme.md",
    "tag":"package-2020-10-01-preview",
    "details":"Checking for duplicate schemas,
    this could take a (long) while. Run with --verbose for more detail."
    ️️✔️[Staging] PrettierCheck succeeded [Detail] [Expand]
    Validation passes for PrettierCheck.
    ️️✔️[Staging] SpellCheck succeeded [Detail] [Expand]
    Validation passes for SpellCheck.
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Feb 5, 2021

    Swagger Generation Artifacts

    ️🔄[Staging] ApiDocPreview inProgress [Detail]
    ️️✔️ azure-cli-extensions succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs] Generate from 1e91197d791a96dfe25dda890c039576ec888a17. Azure CLI Automation 14.0.0
      warn	Skip initScript due to not configured
      command	npx n 12
      command	python3 -m venv venv
      command	pip install --upgrade setuptools wheel pip
      command	pip install pathlib jinja2 msrestazure
      command	autorest --az --use=@autorest/az@latest --version=3.0.6271 --clear-output-folder=true --azure-cli-extension-folder=/home/vsts/work/1/s/azure-cli-extensions ../azure-rest-api-specs/specification/datashare/resource-manager/readme.md
      cmdout	[Autorest] WARNING (PreCheck/CheckDuplicateSchemas): Checking for duplicate schemas, this could take a (long) while.  Run with --verbose for more detail.
      cmdout	[Autorest] WARNING (): cli.auto-parameter-hidden is not true, skip visibility cleaner
      cmderr	[Autorest] BELOW TEST SCENARIO SECTION CAN BE USED IN readme.cli.md
      cmderr	[Autorest] --------------------------------------------------------
      cmderr	[Autorest]   test-scenario:
      cmderr	[Autorest]     - name: /Accounts/put/Accounts_Create
      cmderr	[Autorest]     - name: /Accounts/get/Accounts_Get
      cmderr	[Autorest]     - name: /Accounts/get/Accounts_ListByResourceGroup
      cmderr	[Autorest]     - name: /Accounts/get/Accounts_ListBySubscription
      cmderr	[Autorest]     - name: /Accounts/patch/Accounts_Update
      cmderr	[Autorest]     - name: /DataSetMappings/put/DataSetMappings_SqlDB_Create
      cmderr	[Autorest]     - name: /DataSetMappings/put/DataSetMappings_SqlDW_Create
      cmderr	[Autorest]     - name: /DataSetMappings/put/DataSetMappings_SqlDWDataSetToAdlsGen2File_Create
      cmderr	[Autorest]     - name: /Shares/put/Shares_Create
      cmderr	[Autorest]     - name: /Shares/get/Shares_Get
      cmderr	[Autorest]     - name: /Shares/get/Shares_ListByAccount
      cmderr	[Autorest]     - name: /Shares/post/Shares_ListSynchronizationDetails
      cmderr	[Autorest]     - name: /Shares/post/Shares_ListSynchronizations
      cmderr	[Autorest]     - name: /DataSets/put/DataSets_Create
      cmderr	[Autorest]     - name: /DataSets/put/DataSets_KustoCluster_Create
      cmderr	[Autorest]     - name: /DataSets/put/DataSets_KustoDatabase_Create
      cmderr	[Autorest]     - name: /DataSets/put/DataSets_SqlDBTable_Create
      cmderr	[Autorest]     - name: /DataSets/put/DataSets_SqlDWTable_Create
      cmderr	[Autorest]     - name: /DataSets/get/DataSets_Get
      cmderr	[Autorest]     - name: /DataSets/get/DataSets_ListByShare
      cmderr	[Autorest]     - name: /DataSets/delete/DataSets_Delete
      cmderr	[Autorest]     - name: /Invitations/put/Invitations_Create
      cmderr	[Autorest]     - name: /Invitations/get/Invitations_Get
      cmderr	[Autorest]     - name: /Invitations/get/Invitations_ListByShare
      cmderr	[Autorest]     - name: /Invitations/delete/Invitations_Delete
      cmderr	[Autorest]     - name: /ProviderShareSubscriptions/get/ProviderShareSubscriptions_GetByShare
      cmderr	[Autorest]     - name: /ProviderShareSubscriptions/get/ProviderShareSubscriptions_ListByShare
      cmderr	[Autorest]     - name: /ProviderShareSubscriptions/post/ProviderShareSubscriptions_Reinstate
      cmderr	[Autorest]     - name: /ProviderShareSubscriptions/post/ProviderShareSubscriptions_Revoke
      cmderr	[Autorest]     - name: /Shares/delete/Shares_Delete
      cmderr	[Autorest]     - name: /ShareSubscriptions/put/ShareSubscriptions_Create
      cmderr	[Autorest]     - name: /ShareSubscriptions/get/ShareSubscriptions_Get
      cmderr	[Autorest]     - name: /ShareSubscriptions/get/ShareSubscriptions_ListByAccount
      cmderr	[Autorest]     - name: /ShareSubscriptions/post/ShareSubscriptions_CancelSynchronization
      cmderr	[Autorest]     - name: /ShareSubscriptions/post/ShareSubscriptions_ListSourceShareSynchronizationSettings
      cmderr	[Autorest]     - name: /ShareSubscriptions/post/ShareSubscriptions_ListSynchronizationDetails
      cmderr	[Autorest]     - name: /ShareSubscriptions/post/ShareSubscriptions_ListSynchronizations
      cmderr	[Autorest]     - name: /ShareSubscriptions/post/ShareSubscriptions_Synchronize
      cmderr	[Autorest]     - name: /ShareSubscriptions/delete/ShareSubscriptions_Delete
      cmderr	[Autorest]     - name: /SynchronizationSettings/put/SynchronizationSettings_Create
      cmderr	[Autorest]     - name: /SynchronizationSettings/get/SynchronizationSettings_Get
      cmderr	[Autorest]     - name: /SynchronizationSettings/get/SynchronizationSettings_ListByShare
      cmderr	[Autorest]     - name: /SynchronizationSettings/delete/SynchronizationSettings_Delete
      cmderr	[Autorest]     - name: /Triggers/put/Triggers_Create
      cmderr	[Autorest]     - name: /Triggers/get/Triggers_Get
      cmderr	[Autorest]     - name: /Triggers/get/Triggers_ListByShareSubscription
      cmderr	[Autorest]     - name: /Triggers/delete/Triggers_Delete
      cmderr	[Autorest]     - name: /Accounts/delete/Accounts_Delete
      cmderr	[Autorest]     - name: /ConsumerInvitations/get/ConsumerInvitations_Get
      cmderr	[Autorest]     - name: /ConsumerInvitations/get/ConsumerInvitations_ListInvitations
      cmderr	[Autorest]     - name: /ConsumerInvitations/post/ConsumerInvitations_RejectInvitation
      cmderr	[Autorest]     - name: /ConsumerSourceDataSets/get/ConsumerSourceDataSets_ListByShareSubscription
      cmderr	[Autorest]     - name: /DataSetMappings/put/DataSetMappings_Create
      cmderr	[Autorest]     - name: /DataSetMappings/get/DataSetMappings_Get
      cmderr	[Autorest]     - name: /DataSetMappings/get/DataSetMappings_ListByShareSubscription
      cmderr	[Autorest]     - name: /DataSetMappings/delete/DataSetMappings_Delete
      cmderr	[Autorest] --------------------------------------------------------
    • ️✔️datashare [View full logs]  [Preview Azure CLI Changes]
    ️️✔️ azure-sdk-for-go succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs] Generate from 1e91197d791a96dfe25dda890c039576ec888a17. SDK Automation 14.0.0
      command	sh ./initScript.sh ../../../../../azure-sdk-for-go_tmp/initInput.json ../../../../../azure-sdk-for-go_tmp/initOutput.json
      command	go run ./tools/generator/main.go ../../../../../azure-sdk-for-go_tmp/generateInput.json ../../../../../azure-sdk-for-go_tmp/generateOutput.json
    • ️✔️datashare/mgmt/2019-11-01/datashare [View full logs]  [Preview SDK Changes]
      info	[Changelog] No exported changes
    • ️✔️preview/datashare/mgmt/2018-11-01-preview/datashare [View full logs]  [Preview SDK Changes]
      info	[Changelog] No exported changes
    ️️✔️ azure-sdk-for-net succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs] Generate from 1e91197d791a96dfe25dda890c039576ec888a17. SDK Automation 14.0.0
      warn	Skip initScript due to not configured
      command	sudo apt-get install -y dotnet-sdk-5.0
      command	autorest --version=V2 --csharp --reflect-api-versions --license-header=MICROSOFT_MIT_NO_VERSION --use=@microsoft.azure/autorest.csharp@2.3.82 --csharp-sdks-folder=/home/vsts/work/1/s/azure-sdk-for-net/sdk ../azure-rest-api-specs/specification/datashare/resource-manager/readme.md
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
    • ️✔️Microsoft.Azure.Management.DataShare [View full logs]  [Preview SDK Changes]
    ️⚠️ azure-sdk-for-python warning [Detail]
    • ⚠️Warning [Logs] Generate from 1e91197d791a96dfe25dda890c039576ec888a17. SDK Automation 14.0.0
      command	sh scripts/automation_init.sh ../azure-sdk-for-python_tmp/initInput.json ../azure-sdk-for-python_tmp/initOutput.json
      cmderr	[automation_init.sh] WARNING: Skipping azure-nspkg as it is not installed.
      command	sh scripts/automation_generate.sh ../azure-sdk-for-python_tmp/generateInput.json ../azure-sdk-for-python_tmp/generateOutput.json
      cmdout	[automation_generate.sh] [Autorest]
      cmdout	[automation_generate.sh] [Autorest] (node:2898) ExperimentalWarning: The fs.promises API is experimental
      cmdout	[automation_generate.sh] [Autorest]    Loading AutoRest core      '/home/vsts/.autorest/@autorest_core@3.0.6372/node_modules/@autorest/core/dist' (3.0.6372)
      cmdout	[automation_generate.sh] [Autorest] (node:2952) ExperimentalWarning: The fs.promises API is experimental
      cmdout	[automation_generate.sh] [Autorest]   TypeError: Cannot read property 'pass-thru' of undefined
      cmdout	[automation_generate.sh] [Autorest]../azure-sdk-for-python_tmp/generateOutput.json does not exist!!!
      warn	File azure-sdk-for-python_tmp/generateOutput.json not found to read
      warn	No file changes detected after generation
      warn	No package detected after generation
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-workflow-bot
    Copy link

    Hi @ahermas, Your PR has some issues. Please fix the CI sequentially by following the order of Avocado, semantic validation, model validation, breaking change, lintDiff.

    TaskHow to fixPrioritySupport (Microsoft alias)
    AvocadoFix-AvocadoHighruowan
    Semantic validationFix-SemanticValidation-ErrorHighraychen, jianyxi
    Model validationFix-ModelValidation-ErrorHighraychen,jianyxi
    LintDiffFix-LintDiffhighjianyxi, ruoxuan
    If you need further help, please feedback via swagger feedback."

    @openapi-workflow-bot
    Copy link

    Hi, @ahermas your PR are labelled with WaitForARMFeedback. A notification email will be sent out shortly afterwards to notify ARM review board(armapireview@microsoft.com). cc @ChenTanyi

    @openapi-workflow-bot openapi-workflow-bot bot added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Feb 5, 2021
    @ahermas ahermas changed the title Add 2020-10-01-preview:Duplicating the last version's files first AzureDataShare: Adding version 2020-10-01-preview swagger Feb 5, 2021
    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 8, 2021

    @ChenTanyi I see that the PR has cli errors. I pinged the autorest team but I was wondering if you know about such errors and how they could be resolved. Thanks.

    @ChenTanyi
    Copy link
    Contributor

    /azp run

    @azure-pipelines
    Copy link

    Azure Pipelines successfully started running 2 pipeline(s).

    @ChenTanyi
    Copy link
    Contributor

    @ahermas The Cli CI has passed

    @varunkumta
    Copy link

            "SystemAssigned"
    

    is this a bug fix where the schema now reflects what the service supports? or is this an intentional functionality degradation from the previous api version?


    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json:4710 in 0df549e. [](commit_id = 0df549e, deletion_comment = False)

    @varunkumta
    Copy link

        "containerName": {
    

    consider modeling the containerName, location and storageAccountResourceId within a "storageAccount" object with those 3 fields


    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json:6431 in 0df549e. [](commit_id = 0df549e, deletion_comment = False)

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 9, 2021

            "SystemAssigned"
    

    is this a bug fix where the schema now reflects what the service supports? or is this an intentional functionality degradation from the previous api version?

    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json:4710 in 0df549e. [](commit_id = 0df549e, deletion_comment = False)

    @varunkumta UserAssigned was added by mistake. My change is to remove it.

    @varunkumta
    Copy link

        "location": {
    

    as above, consider combining related storage account properties into 1 object


    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json:6492 in 0df549e. [](commit_id = 0df549e, deletion_comment = False)

    @ghost
    Copy link

    ghost commented Feb 9, 2021

        "location": {
    

    as above, consider combining related storage account properties into 1 object

    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json:6492 in 0df549e. [](commit_id = 0df549e, deletion_comment = False)

    @varunkumta location is a read only property exposed to the customer. We would like to keep it a separate entry in the property bag.

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 9, 2021

        "containerName": {
    

    consider modeling the containerName, location and storageAccountResourceId within a "storageAccount" object with those 3 fields

    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/preview/2020-10-01-preview/DataShare.json:6431 in 0df549e. [](commit_id = 0df549e, deletion_comment = False)

    @varunkumta StorageAccountDataSetProperties already contains what you're referring to as a "storageaccount".

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 9, 2021

    @varunkumta @ChenTanyi @ArcturusZhang
    Santhosh and I have addressed/replied to your comments. Please let me know if you have further comments.
    Thanks for reviewing.

    @varunkumta varunkumta added ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review and removed ARMChangesRequested WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required labels Feb 9, 2021
    @ChenTanyi ChenTanyi added the Approved-OkToMerge <valid label in PR review process>add this label when assignee approve to merge the updates label Feb 10, 2021
    @ChenTanyi
    Copy link
    Contributor

    @ahermas please let me know when it ready to merge.

    @ghost
    Copy link

    ghost commented Feb 10, 2021

    @ahermas please let me know when it ready to merge.

    @ChenTanyi we found a minor issue and we are working on fixing it. Please hold merging this.

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 10, 2021

    @ahermas please let me know when it ready to merge.

    @ChenTanyi The minor issue has been addressed.Please merge the PR when you get a chance.
    Thanks

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 11, 2021

    Hi @ArcturusZhang,

    I'm pinging you since I see the Chen will be away till the 17th.
    Please kindly review the pr when you get a chance after the checks complete.

    Thanks

    @akning-ms
    Copy link
    Contributor

    /azp run

    @azure-pipelines
    Copy link

    Azure Pipelines successfully started running 2 pipeline(s).

    @akning-ms akning-ms merged commit 35affad into Azure:master Feb 11, 2021
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Labels

    Approved-OkToMerge <valid label in PR review process>add this label when assignee approve to merge the updates ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review

    Projects

    None yet

    Development

    Successfully merging this pull request may close these issues.

    4 participants