Skip to content

{CI} Use azdev and pytest for full test #17383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jiasli merged 7 commits into from
Apr 6, 2021
Merged

{CI} Use azdev and pytest for full test #17383

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
15a347e
use azdev for full test
jiasli Mar 22, 2021
005d747
forked
jiasli Mar 22, 2021
4e4abfa
Merge branch 'dev' into full-test
jiasli Mar 26, 2021
d025b22
azdev setup --debug
jiasli Mar 26, 2021
5fa5597
disable forked
jiasli Mar 26, 2021
09bfc71
Merge branch 'dev' into full-test
jiasli Apr 1, 2021
f8f02d2
Update automation_test.yml
jiasli Apr 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 22 additions & 5 deletions .azure-pipelines/templates/automation_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
# Azure DevOps task for automation test using azdev and pytest

parameters:
pythonVersion: ''
profile: ''
- name: pythonVersion
type: string
default: ''
Copy link
Member

@qwordy qwordy Apr 2, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does '' mean? How about setting default value 3.8?

Copy link
Member Author

@jiasli jiasli Apr 2, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does '' mean?

'' means empty string. I copied it as-is. It also appears here:

azure-cli/.azure-pipelines/templates/azdev_setup.yml

Lines 5 to 7 in 0fa1fb5

- name: CLIExtensionRepoPath
type: string
default: ''

How about setting default value 3.8?

Sure.

- name: profile
type: string
default: ''
- name: fullTest
displayName: 'Run full test?'
type: boolean
default: false

steps:
- task: UsePythonVersion@0
Expand All @@ -13,10 +23,17 @@ steps:

source env/bin/activate

if [[ "$(System.PullRequest.TargetBranch)" != "" ]]; then
azdev test --series --no-exitfirst --repo=./ --src=HEAD --tgt=origin/$(System.PullRequest.TargetBranch) --cli-ci --profile ${{ parameters.profile }}
echo pythonVersion: ${{ parameters.pythonVersion }}
echo profile: ${{ parameters.profile }}
echo fullTest: ${{ parameters.fullTest }}
echo Build.Reason: $(Build.Reason)

if [[ "$(Build.Reason)" == "PullRequest" && "${{ parameters.fullTest }}" == 'False' ]]; then
echo "Running incremental test"
azdev test --no-exitfirst --repo=./ --src=HEAD --tgt=origin/$(System.PullRequest.TargetBranch) --cli-ci --profile ${{ parameters.profile }} --verbose --series
else
azdev test --series --no-exitfirst --profile ${{ parameters.profile }}
echo "Running full test"
azdev test --no-exitfirst --profile ${{ parameters.profile }} --verbose --series
fi
displayName: "Test on Profile ${{ parameters.profile }}"
env:
Expand Down
206 changes: 68 additions & 138 deletions azure-pipelines-full-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,144 +13,74 @@ pr:
- '*'

jobs:
- job: ExtractMetadata
displayName: Extract Metadata
- job: AutomationTest
displayName: Automation Test (Profile Latest)
timeoutInMinutes: 120
pool:
vmImage: 'ubuntu-16.04'
Copy link
Member

@qwordy qwordy Apr 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is 16.04 too old? Consider 18.04 or 20.04

Copy link
Member Author

@jiasli jiasli Apr 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update in #17368 to keep in sync with main pipeline.

strategy:
matrix:
Python36:
python.version: '3.6'
Python38:
python.version: '3.8'
steps:
- template: .azure-pipelines/templates/automation_test.yml
parameters:
pythonVersion: '$(python.version)'
profile: 'latest'
fullTest: true

condition: succeeded()
pool:
vmImage: 'ubuntu-16.04'
steps:
- task: Bash@3
displayName: 'Extract Version'
inputs:
targetType: 'filePath'
filePath: scripts/release/get_version.sh
- task: PublishPipelineArtifact@0
displayName: 'Publish Artifact: metadata'
inputs:
TargetPath: $(Build.ArtifactStagingDirectory)
ArtifactName: metadata
- job: AutomationTest20200901
displayName: Automation Test (Profile 2020-09-01)
timeoutInMinutes: 120
pool:
vmImage: 'ubuntu-16.04'
strategy:
matrix:
Python36:
python.version: '3.6'
Python38:
python.version: '3.8'
Comment on lines +41 to +44
Copy link
Member

@qwordy qwordy Apr 2, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is 3.7? Is 3.7 less important than 3.6 or 3.8?

Copy link
Member Author

@jiasli jiasli Apr 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We package with Python 3.6 and currently we support up to 3.8. @fengzhou-msft

steps:
- template: .azure-pipelines/templates/automation_test.yml
parameters:
pythonVersion: '$(python.version)'
profile: '2020-09-01-hybrid'
fullTest: true

- job: BuildPythonWheel
displayName: Build Python Wheels
dependsOn: ExtractMetadata
condition: succeeded()
pool:
vmImage: 'ubuntu-16.04'
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.7'
inputs:
versionSpec: 3.7
- job: AutomationTest20190301
displayName: Automation Test (Profile 2019-03-01)
timeoutInMinutes: 120
pool:
vmImage: 'ubuntu-16.04'
strategy:
matrix:
Python36:
python.version: '3.6'
Python38:
python.version: '3.8'
steps:
- template: .azure-pipelines/templates/automation_test.yml
parameters:
pythonVersion: '$(python.version)'
profile: '2019-03-01-hybrid'
fullTest: true

- script: |
if [[ "$(Build.Reason)" == "PullRequest" ]]; then
branch=$(System.PullRequest.TargetBranch)
else
branch=$(Build.SourceBranchName)
fi
scripts/release/pypi/build.sh $branch
displayName: 'Run Wheel Build Script'

- task: PublishPipelineArtifact@0
displayName: 'Publish Artifact: pypi'
inputs:
TargetPath: $(Build.ArtifactStagingDirectory)
ArtifactName: pypi

- job: AutomationTest
displayName: Automation Test
dependsOn: BuildPythonWheel
condition: succeeded()
timeoutInMinutes: 90

pool:
vmImage: 'ubuntu-16.04'
strategy:
matrix:
Python36:
python.version: '3.6'
Python38:
python.version: '3.8'
steps:
- task: UsePythonVersion@0
displayName: 'Use Python $(python.version)'
inputs:
versionSpec: '$(python.version)'
- task: DownloadPipelineArtifact@1
displayName: 'Download Python Wheels'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/pypi'
artifactName: pypi
- bash: |
pip install --force-reinstall wheel
- task: Bash@3
displayName: 'Run Test Automation'
inputs:
targetType: 'filePath'
filePath: ./scripts/ci/test_automation.sh

- job: RunAutomationReduced20190301
displayName: Run Automation Reduced, Profile 2019-03-01
dependsOn: BuildPythonWheel
condition: succeeded()
pool:
vmImage: 'ubuntu-16.04'
strategy:
matrix:
Python36:
python.version: '3.6'
Python38:
python.version: '3.8'
steps:
- task: UsePythonVersion@0
displayName: 'Use Python $(python.version)'
inputs:
versionSpec: '$(python.version)'
- task: DownloadPipelineArtifact@1
displayName: 'Download Python Wheels'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/pypi'
artifactName: pypi
- bash: |
pip install --force-reinstall wheel
- task: Bash@3
displayName: 'Run Test Automation'
inputs:
targetType: 'filePath'
filePath: ./scripts/ci/test_automation.sh
env:
REDUCE_SDK: 'True'
AZURE_CLI_TEST_TARGET_PROFILE: '2019-03-01'

- job: RunAutomation20180301
displayName: Run Automation, Profile 2018-03-01
dependsOn: BuildPythonWheel
condition: succeeded()
pool:
vmImage: 'ubuntu-16.04'
strategy:
matrix:
Python36:
python.version: '3.6'
Python38:
python.version: '3.8'
steps:
- task: UsePythonVersion@0
displayName: 'Use Python $(python.version)'
inputs:
versionSpec: '$(python.version)'
- task: DownloadPipelineArtifact@1
displayName: 'Download Python Wheels'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/pypi'
artifactName: pypi
- bash: |
pip install --force-reinstall wheel
- task: Bash@3
displayName: 'Run Test Automation'
inputs:
targetType: 'filePath'
filePath: ./scripts/ci/test_automation.sh
env:
AZURE_CLI_TEST_TARGET_PROFILE: '2018-03-01'
- job: AutomationTest20180301
displayName: Automation Test (Profile 2018-03-01)
timeoutInMinutes: 120
pool:
vmImage: 'ubuntu-16.04'
strategy:
matrix:
Python36:
python.version: '3.6'
Python38:
python.version: '3.8'
steps:
- template: .azure-pipelines/templates/automation_test.yml
parameters:
pythonVersion: '$(python.version)'
profile: '2018-03-01-hybrid'
fullTest: true
57 changes: 0 additions & 57 deletions scripts/ci/test_automation.sh
View file
Open in desktop

This file was deleted.