Skip to content

add datafactory track2 config and fix swagger #11563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
00Kai0 wants to merge 3 commits into from
Closed

add datafactory track2 config and fix swagger #11563

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7f1997f
add datafactory track2 config and fix swagger
00Kai0 Nov 6, 2020
c50d4c2
fix config
00Kai0 Nov 9, 2020
61ae5d4
enable track2
00Kai0 Mar 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...esource-manager/Microsoft.DataFactory/preview/2017-09-01-preview/entityTypes/Dataset.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,7 @@
"CompressionLevel": {
"title": "Microsoft.Azure.Management.DataFactories.Models.CompressionLevel",
"description": "All available compression levels.",
"type": "object",
Copy link
Member

@qiaozha qiaozha Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think string is going to work for datafactory. This is meant to be oneOf scenario and swagger 2.0 hasn't supported oneOf yet.

Copy link
Member

@qiaozha qiaozha Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @lirenhe

Copy link
Member

@weidongxu-microsoft weidongxu-microsoft Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has to be string as "modelAsString": true" below.

If it is not string, modelAsString should be false, and format need to be provided.

Copy link
Member

@qiaozha qiaozha Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that, In datafactory, this kind of property can be either enum string or Expression type. we need to use oneOf to express such scenario which is only supported in swagger 3.0, but currently we are using swagger 2.0 and I believe, we don't have plan to upgrade to swagger 3.0 at least in recent months.

Copy link
Member

@weidongxu-microsoft weidongxu-microsoft Mar 15, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I get you. But even if this is supported by swagger, it will not be supported by SDK, at least not Java. We will not return anything like such: it could be enum, it could be string otherwise not included in the enum, or it could be an object of undefined schema, as there is no Expression anywhere in this def.
Hence Java will not release for datafactory.

"type": "string",
"enum": [
"Optimal",
"Fastest"
Expand Down
4 changes: 2 additions & 2 deletions ...e-manager/Microsoft.DataFactory/preview/2017-09-01-preview/entityTypes/LinkedService.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -374,7 +374,7 @@
"Online",
"OnPremisesWithIfd"
],
"type": "object",
"type": "string",
"description": "The deployment type of the Dynamics instance. 'Online' for Dynamics Online and 'OnPremisesWithIfd' for Dynamics on-premises with Ifd. Type: string (or Expression with resultType string)."
},
"hostName": {
Expand Down Expand Up @@ -402,7 +402,7 @@
"Office365",
"Ifd"
],
"type": "object",
"type": "string",
"description": "The authentication type to connect to Dynamics server. 'Office365' for online scenario, 'Ifd' for on-premises with Ifd scenario. Type: string (or Expression with resultType string)."
},
"username": {
Expand Down
18 changes: 9 additions & 9 deletions ...source-manager/Microsoft.DataFactory/preview/2017-09-01-preview/entityTypes/Pipeline.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -371,7 +371,7 @@
},
"readBehavior": {
"description": "The read behavior for the operation. Default is Query.",
"type": "object",
"type": "string",
"enum": [
"Query",
"QueryAll"
Expand Down Expand Up @@ -577,7 +577,7 @@
},
"consistencyLevel": {
"description": "The consistency level specifies how many Cassandra servers must respond to a read request before returning data to the client application. Cassandra checks the specified number of Cassandra servers for data to satisfy the read request. Must be one of cassandraSourceReadConsistencyLevels. The default value is 'ONE'. It is case-insensitive.",
"type": "object",
"type": "string",
"enum": [
"ALL",
"EACH_QUORUM",
Expand Down Expand Up @@ -1148,7 +1148,7 @@
},
"StoredProcedureParameterType": {
"description": "Stored procedure parameter type.",
"type": "object",
"type": "string",
"enum": [
"String",
"Int",
Expand Down Expand Up @@ -1207,7 +1207,7 @@
"properties": {
"writeBehavior": {
"description": "The write behavior for the operation. Default is 'Insert'.",
"type": "object",
"type": "string",
"enum": [
"Insert",
"Update"
Expand All @@ -1231,7 +1231,7 @@
},
"CopyBehaviorType": {
"description": "All available types of copy behavior.",
"type": "object",
"type": "string",
"enum": [
"PreserveHierarchy",
"FlattenHierarchy",
Expand Down Expand Up @@ -1406,7 +1406,7 @@
},
"PolybaseSettingsRejectType": {
"description": "Indicates whether the RejectValue property is specified as a literal value or a percentage.",
"type": "object",
"type": "string",
"enum": [
"value",
"percentage"
Expand Down Expand Up @@ -1501,7 +1501,7 @@
"properties": {
"writeBehavior": {
"description": "Specify the write behavior when upserting documents into Azure Search Index.",
"type": "object",
"type": "string",
"enum": [
"Merge",
"Upload"
Expand Down Expand Up @@ -1539,7 +1539,7 @@
"properties": {
"writeBehavior": {
"description": "The write behavior for the operation.",
"type": "object",
"type": "string",
"enum": [
"Upsert"
],
Expand Down Expand Up @@ -1568,7 +1568,7 @@
"properties": {
"writeBehavior": {
"description": "The write behavior for the operation. Default is Insert.",
"type": "object",
"type": "string",
"enum": [
"Insert",
"Upsert"
Expand Down
8 changes: 4 additions & 4 deletions ...factory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Dataset.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -408,7 +408,7 @@
},
"JsonFormatFilePattern": {
"description": "JSON format file pattern. A property of JsonFormat.",
"type": "object",
"type": "string",
"enum": [
"setOfObjects",
"arrayOfObjects"
Expand Down Expand Up @@ -551,7 +551,7 @@
"CompressionLevel": {
"title": "Microsoft.Azure.Management.DataFactories.Models.CompressionLevel",
"description": "All available compression levels.",
"type": "object",
"type": "string",
"enum": [
"Optimal",
"Fastest"
Expand Down Expand Up @@ -744,7 +744,7 @@
"description": "The location of the parquet storage."
},
"compressionCodec": {
"type": "object",
"type": "string",
"enum": [
"none",
"gzip",
Expand Down Expand Up @@ -798,7 +798,7 @@
"description": "The code page name of the preferred encoding. If miss, the default value is UTF-8, unless BOM denotes another Unicode encoding. Refer to the name column of the table in the following link to set supported values: https://msdn.microsoft.com/library/system.text.encoding.aspx. Type: string (or Expression with resultType string)."
},
"compressionCodec": {
"type": "object",
"type": "string",
"enum": [
"bzip2",
"gzip",
Expand Down
12 changes: 6 additions & 6 deletions ...y/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/LinkedService.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -603,7 +603,7 @@
"Online",
"OnPremisesWithIfd"
],
"type": "object",
"type": "string",
"description": "The deployment type of the Dynamics instance. 'Online' for Dynamics Online and 'OnPremisesWithIfd' for Dynamics on-premises with Ifd. Type: string (or Expression with resultType string)."
},
"hostName": {
Expand Down Expand Up @@ -632,7 +632,7 @@
"Ifd",
"AADServicePrincipal"
],
"type": "object",
"type": "string",
"description": "The authentication type to connect to Dynamics server. 'Office365' for online scenario, 'Ifd' for on-premises with Ifd scenario, 'AADServicePrincipal' for Server-To-Server authentication in online scenario. Type: string (or Expression with resultType string)."
},
"username": {
Expand All @@ -656,7 +656,7 @@
"ServicePrincipalKey",
"ServicePrincipalCert"
],
"type": "object",
"type": "string",
"description": "The service principal credential type to use in Server-To-Server authentication. 'ServicePrincipalKey' for key/secret, 'ServicePrincipalCert' for certificate. Type: string (or Expression with resultType string)."
},
"servicePrincipalCredential": {
Expand Down Expand Up @@ -758,7 +758,7 @@
"ServicePrincipalKey",
"ServicePrincipalCert"
],
"type": "object",
"type": "string",
"description": "The service principal credential type to use in Server-To-Server authentication. 'ServicePrincipalKey' for key/secret, 'ServicePrincipalCert' for certificate. Type: string (or Expression with resultType string)."
},
"servicePrincipalCredential": {
Expand Down Expand Up @@ -860,7 +860,7 @@
"ServicePrincipalKey",
"ServicePrincipalCert"
],
"type": "object",
"type": "string",
"description": "The service principal credential type to use in Server-To-Server authentication. 'ServicePrincipalKey' for key/secret, 'ServicePrincipalCert' for certificate. Type: string (or Expression with resultType string)."
},
"servicePrincipalCredential": {
Expand Down Expand Up @@ -5349,7 +5349,7 @@
"Workernode",
"Zookeeper"
],
"type": "object",
"type": "string",
"description": "The node types on which the script action should be executed."
},
"parameters": {
Expand Down
1 change: 1 addition & 0 deletions specification/datafactory/resource-manager/readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ This is not used by Autorest itself.
swagger-to-sdk:
- repo: azure-sdk-for-net
- repo: azure-sdk-for-python
- repo: azure-sdk-for-python-track2
- repo: azure-sdk-for-java
- repo: azure-sdk-for-go
- repo: azure-sdk-for-js
Expand Down
44 changes: 41 additions & 3 deletions specification/datafactory/resource-manager/readme.python.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ These settings apply only when `--python` is specified on the command line.
Please also specify `--python-sdks-folder=<path to the root directory of your azure-sdk-for-python clone>`.
Use `--python-mode=update` if you already have a setup.py and just want to update the code itself.

``` yaml $(python)
Copy link
Member

@qiaozha qiaozha Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we still need this since we already have readme.python.md.

``` yaml $(python) && !$(track2)
python-mode: create
python:
azure-arm: true
Expand All @@ -15,13 +15,51 @@ python:
package-version: 1.0.0
clear-output-folder: true
```
``` yaml $(python) && $(python-mode) == 'update'

``` yaml $(python) && $(track2)
python-mode: create
azure-arm: true
license-header: MICROSOFT_MIT_NO_VERSION
namespace: azure.mgmt.datafactory
package-name: azure-mgmt-datafactory
package-version: 1.0.0b1
clear-output-folder: true

directive:
- from: Trigger.json
where: $.definitions.BlobEventTypes.items
transform: >
return {
"type": "string",
"enum": [
"Microsoft.Storage.BlobCreated",
"Microsoft.Storage.BlobDeleted"
],
"x-ms-enum": {
"name": "BlobEventTypesEnum",
"modelAsString": true
}
}
```

``` yaml $(python) && $(python-mode) == 'update' && !$(track2)
python:
no-namespace-folders: true
output-folder: $(python-sdks-folder)/datafactory/azure-mgmt-datafactory/azure/mgmt/datafactory
```
``` yaml $(python) && $(python-mode) == 'create'

``` yaml $(python) && $(python-mode) == 'update' && $(track2)
no-namespace-folders: true
output-folder: $(python-sdks-folder)/datafactory/azure-mgmt-datafactory/azure/mgmt/datafactory
```

``` yaml $(python) && $(python-mode) == 'create' && !$(track2)
python:
basic-setup-py: true
output-folder: $(python-sdks-folder)/datafactory/azure-mgmt-datafactory
```

``` yaml $(python) && $(python-mode) == 'create' && $(track2)
basic-setup-py: true
output-folder: $(python-sdks-folder)/datafactory/azure-mgmt-datafactory
```