Verify autorest in pipeline #16667
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| parameters: | ||
| - name: ServiceToTest | ||
| type: string | ||
| default: '' | ||
| - name: SDKType | ||
| type: string | ||
| default: 'client' | ||
| - name: ServiceDirectory | ||
| type: string | ||
| - name: NodeVersion | ||
| type: string | ||
| default: '12.x' | ||
| - name: PythonVersion | ||
| type: string | ||
| default: '$(PythonVersion)' | ||
| - name: auto_rest_clone_url | ||
| type: string | ||
| default: 'https://github.com/Azure/autorest.python.git' | ||
|
|
||
| steps: | ||
| - task: UsePythonVersion@0 | ||
| displayName: Use | ||
| inputs: | ||
| versionSpec: ${{ parameters.PythonVersion }} | ||
|
|
||
| - task: NodeTool@0 | ||
| inputs: | ||
| versionSpec: ${{ parameters.NodeVersion }} | ||
|
|
||
| - script: | | ||
| npm install -g autorest | ||
| autorest --help | ||
| displayName: "Install autorest" | ||
|
|
||
| - script: | | ||
| python --version | ||
| pip install -r eng/autorest_req.txt | ||
| git clone ${{ parameters.auto_rest_clone_url }} | ||
| cd autorest.python | ||
| npm install | ||
| displayName: 'Prepare Environment' | ||
|
|
||
| - task: PythonScript@0 | ||
| displayName: 'Run autorest Python' | ||
| inputs: | ||
| scriptPath: 'scripts/devops_tasks/verify_autorest.py' | ||
| arguments: >- | ||
| --service_directory="${{ parameters.ServiceDirectory }}" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| #!/usr/bin/env python | ||
|
|
||
| # -------------------------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for license information. | ||
| # -------------------------------------------------------------------------------------------- | ||
|
|
||
| import argparse | ||
| from git import Repo | ||
| import os | ||
| import logging | ||
| import sys | ||
|
|
||
| from common_tasks import run_check_call | ||
|
|
||
| logging.getLogger().setLevel(logging.INFO) | ||
|
|
||
| root_dir = os.path.abspath(os.path.join(os.path.abspath(__file__), "..", "..", "..")) | ||
| sdk_dir = os.path.join(root_dir, "sdk") | ||
|
|
||
| SWAGGER_FOLDER = "swagger" | ||
|
|
||
|
|
||
| def run_autorest(service_dir): | ||
| logging.info("Running autorest for {}".format(service_dir)) | ||
|
|
||
| service_dir = os.path.join(sdk_dir, service_dir) | ||
|
|
||
| swagger_folders = find_swagger_folders(service_dir) | ||
|
|
||
| for working_dir in swagger_folders: | ||
| os.chdir(working_dir) | ||
| f = os.path.join(working_dir, "README.md") | ||
| if os.path.exists(f): | ||
| reset_command = ["autorest", "--reset"] | ||
| run_check_call(reset_command, root_dir) | ||
|
|
||
| command = ["autorest", "--python", f, "--verbose"] | ||
| logging.info("Command: {}\nLocation: {}\n".format(command, working_dir)) | ||
| run_check_call(command, working_dir) | ||
| return swagger_folders | ||
|
|
||
|
|
||
| def find_swagger_folders(directory): | ||
| logging.info("Searching for swagger files in: {}".format(directory)) | ||
|
|
||
| ret = [] | ||
| for root, subdirs, files in os.walk(directory): | ||
| for d in subdirs: | ||
| if d == SWAGGER_FOLDER: | ||
| if os.path.exists(os.path.join(root, d, "README.md")): | ||
| ret.append(os.path.join(root, d)) | ||
|
|
||
| logging.info("Found swagger files at: {}".format(ret)) | ||
| return ret | ||
|
|
||
|
|
||
| def check_diff(): | ||
| repo = Repo(root_dir) | ||
| t = repo.head.commit.tree | ||
| d = repo.git.diff(t) | ||
| if d: | ||
| command = ["git", "status"] | ||
| run_check_call(command, root_dir) | ||
| raise ValueError( | ||
| "Found difference between re-generated code and current commit. Please re-generate with the latest autorest." | ||
| ) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| parser = argparse.ArgumentParser( | ||
| description="Run autorest to verify generated code." | ||
| ) | ||
| parser.add_argument( | ||
| "--service_directory", help="Directory of the package being tested" | ||
| ) | ||
|
|
||
| args = parser.parse_args() | ||
| folders = run_autorest(args.service_directory) | ||
|
|
||
| if len(folders): | ||
| check_diff() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit, as I don't believe autorest repo is that big. but can we use a sparse checkout?
I'm like 99% certain this will work. This will basically grab zero history for the commitsh you ask for. I'm adding path "/" to tell it to grab every file in the repo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm testing this locally to confirm that it's good advice. Feel free to ignore this comment for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok I tested this locally.
This will not work for what I'm asking you to do until I update the
sparse-checkout.ymlto NOT grab the folderengby default. It would work if the pattern was simply:But because by default it has a
Run before it adds the path, it actually won't work for git repos without an
engfolder. It just quietly doesn't checkout properly.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR that will enable this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@scbedd I would set the path to
/*instead of/which will work witheng. This is what git sparse-checkout defaults to using anyway before we rungit sparse-checkout setThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah ok. I'll definitely give it a shot locally.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@seankane-msft this works. I updated the example above. I believe it should be a drop in replacement.