|
| 1 | +--- |
| 2 | +title: Azure DevOps Pipelines and Test Studio Tests |
| 3 | +page_title: Test Studio Tests in Azure DevOps Build Pipelines |
| 4 | +description: "Learn how to integrate and execute Test Studio automated tests in Azure DevOps pipelines using self-hosted or Microsoft-hosted agents. This guide covers prerequisites, agent setup, artifact management, and limitations for web and desktop test execution." |
| 5 | +position: 0 |
| 6 | +--- |
| 7 | +# Test Studio Tests and Azure DevOps Build Pipeline |
| 8 | + |
| 9 | +You can integrate <a href="https://www.telerik.com/teststudio" target="_blank">Test Studio</a> tests for automated execution within Azure DevOps pipelines. This article outlines the key prerequisites and configuration steps you should consider before setting up your pipeline. |
| 10 | + |
| 11 | +## Environmental Prerequisites |
| 12 | + |
| 13 | +The build machine may be either a physical or virtual system that hosts both the Azure DevOps agent and the Test Studio Runner. For minimum hardware and software requirements, see the <a href="/system-requirements" target="_blank">Test Studio system requirements documentation</a>. |
| 14 | + |
| 15 | +Additional prerequisites: |
| 16 | + |
| 17 | +* __Azure DevOps account__ enabled to configure Agent pools, Microsoft-hosted agent VMs and pipelines: __mandatory__ |
| 18 | +* __Azure DevOps Agent__ : __mandatory__ |
| 19 | +* __Test Studio Run-time Edition__ (ArtOfTest.Runner.exe must be available - preferably latest version): __mandatory__ |
| 20 | +* __Internet Browser__ (Chrome, Edge, Firefox - preferably latest versions) or WPF/desktop application: __mandatory__ |
| 21 | + |
| 22 | +## Upload the Testing Project into a Source Control Repository |
| 23 | + |
| 24 | +Executing the Test Studio automated tests as part of any CI pipeline requires the project to be deployed on the execution machine. The most convenient approach is to manage the project into a source control repository and use it to checkout the current latest version at the time of execution. The type of source control repository to use depends on your company policies and specifics. |
| 25 | + |
| 26 | +## Choose the Configuration to Setup |
| 27 | + |
| 28 | +Azure DevOps provides two options to run pipelines - on a on-premise local machine (physical or virtual) and on a container hosted by Microsoft. Depending on the configuration you choose there are different initial adjustments of the setup. |
| 29 | + |
| 30 | +Running __pipeline with Test Studio automated tests on a local machine__ requires you to: |
| 31 | + |
| 32 | +* [create a new self-hosted agent in the AzDO project](#create-a-self-hosted-agent) |
| 33 | +* [configure the self-hosted agent to always run n interactive mode](#configure-the-self-hosted-agent) |
| 34 | + |
| 35 | +Running __pipeline with Test Studio automated tests on a Microsoft hosted machine__ requires you to: |
| 36 | + |
| 37 | +* [upload Test Studio installer as artifact](#add-test-studio-run-time-installer-as-artifact-in-azdo-project) |
| 38 | +* [check known limitations for using the MS-hosted agent machines](#limitations-in-using-ms-hosted-agent-with-test-studio-tests) |
| 39 | + |
| 40 | + > __Important!__ |
| 41 | + > <br> |
| 42 | + > <br> |
| 43 | + > Microsoft-hosted agent VMs use no GUI session or UI, thus it can be __used only for executing web tests__! |
| 44 | +
|
| 45 | +## Create a Self-Hosted Agent |
| 46 | + |
| 47 | +Test Studio application is a Windows OS based and as such you can execute the automated tests only on a Windows machine. Thus, you need to use the <a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops" target="_blank">AzDO self-hosted Windows agent</a> to execute tests as part of a pipeline. |
| 48 | + |
| 49 | +To create such agent, follow the steps listed below: |
| 50 | + |
| 51 | +1. Open the **Project settings** of the current AzDO project. |
| 52 | + |
| 53 | + ![Open the current AzDO project settings][2] |
| 54 | + |
| 55 | +2. Click on **Pipelines -> Agent Pools**. |
| 56 | + |
| 57 | + ![Agent Pools][3] |
| 58 | + |
| 59 | +3. Add a new pool or open the default one. |
| 60 | + |
| 61 | + ![New Agent Pools][4] |
| 62 | + |
| 63 | +4. Click **New Agent**, |
| 64 | + |
| 65 | + ![New Agent][5] |
| 66 | + |
| 67 | +5. Download the Agent to the corresponding physical or virtual machine as per the installation requirements above. |
| 68 | + |
| 69 | +6. Extract the Agent and run as Administrator. |
| 70 | + |
| 71 | + ![Start Agent][6] |
| 72 | + |
| 73 | +> **Note** |
| 74 | +> <br> |
| 75 | +> To avoid any permissions issues make sure that the user configuring the Agent is part of the following Permission Groups in AzDO project: Build Administrators, Release Administrators, Project Administrators, ProjectName Team. |
| 76 | +
|
| 77 | +<br> |
| 78 | +<div><a style="float:right" href="#choose-the-configuration-to-setup">Back to top of section</a></div> |
| 79 | +<br> |
| 80 | + |
| 81 | +## Configure the Self-Hosted Agent |
| 82 | + |
| 83 | +1. Enter server URL: i.e. https://dev.azure.com/OrganizationName. |
| 84 | + |
| 85 | +2. <a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops#permissions" target="_blank">Create Personal Access Token (PAT)</a> and save it. |
| 86 | + |
| 87 | +3. Choose PAT for authentication type and paste yours. |
| 88 | + |
| 89 | +4. Choose the Agent pool you created earlier. |
| 90 | + |
| 91 | +5. Choose Agent name, the default name is the machine name on which it is hosted. |
| 92 | + |
| 93 | +6. Choose a work folder or press enter for the default `_work` sub-folder. |
| 94 | + |
| 95 | +7. Choose to run the Agent in Interactive mode - select `N` when asked to run agent as a service. |
| 96 | + |
| 97 | +8. Choose whether to configure autologon and run agent on startup - from Test Studio perspective this is optional and depends on the project demands. |
| 98 | + |
| 99 | + ![Configure Agent][7] |
| 100 | + |
| 101 | +9. Upon successful configuration the config console gets closed. Run the `run.cmd` file in console to trigger the Agent. |
| 102 | + |
| 103 | + ![Trigger Agent][8] |
| 104 | + |
| 105 | + |
| 106 | +<br> |
| 107 | +<div><a style="float:right" href="#choose-the-configuration-to-setup">Back to top of section</a></div> |
| 108 | +<br> |
| 109 | + |
| 110 | + |
| 111 | + |
| 112 | + |
| 113 | + |
| 114 | + |
| 115 | + |
| 116 | +## Add Test Studio Run-time Installer as Artifact in AzDO Project |
| 117 | + |
| 118 | +Follow the below steps to upload the Test Studio Run-time installer as artifact in the AzDO project: |
| 119 | + |
| 120 | + |
| 121 | +1. Open the __Artifacts__ view in the Azure platform and choose the feed in which you want to add the Test Studio Run-time installer. |
| 122 | + |
| 123 | + > __Tip!__ |
| 124 | + ><br> |
| 125 | + > You can __download the Test Studio Run-time Edition *.msi installer file__ from your <a href="https://www.telerik.com/account/product-download?product=TESTSTUDIORUNTIME" target="_blank">Telerik account here</a>. |
| 126 | + |
| 127 | +2. Then select the __Connect to feed__ option to add an artifact in the feed. If you need further information about _Feeds in Azure DevOps_, you can visit the <a href="https://docs.microsoft.com/en-us/azure/devops/artifacts/concepts/feeds?view=azure-devops" target="_blank">Microsoft docs page on the topic</a>. |
| 128 | + |
| 129 | +  |
| 130 | + |
| 131 | +3. Choose the __Universal Packages__ option from the list of artifacts. The instructions how to publish a package are listed on the right side of the list. It is necessary to use the Azure CLI to login and publish the installer file - you can find example commands, which need to be adjusted for your project specifics. If you need further information about __Publishing Universal Packages__, you can refer to the <a href="https://docs.microsoft.com/en-us/azure/devops/artifacts/quickstarts/universal-packages?view=azure-devops" target="_blank">Microsoft docs page on the topic</a>. |
| 132 | + |
| 133 | +  |
| 134 | + |
| 135 | +4. After the package is successfully uploaded, it is __listed as an artifact__ and can be used in the Azure pipelines for that project. |
| 136 | + |
| 137 | +  |
| 138 | + |
| 139 | +<br> |
| 140 | +<div><a style="float:right" href="#choose-the-configuration-to-setup">Back to top of section</a></div> |
| 141 | +<br> |
| 142 | + |
| 143 | +## Limitations in Using MS-Hosted Agent with Test Studio Tests |
| 144 | + |
| 145 | +* __Only web tests using Edge and Chrome browsers__ can be executed in a Microsoft-hosted agent. |
| 146 | +* __The <a href="/automated-tests/test-results/step-failure-details#images-tab" target="_blank">images on failure</a>__ are not available in the results generated from the test runs in Microsoft-hosted agent. |
| 147 | + |
| 148 | +<br> |
| 149 | +<div><a style="float:right" href="#choose-the-configuration-to-setup">Back to top of section</a></div> |
| 150 | +<br> |
| 151 | + |
| 152 | +## Continue to Setting Up a Pipeline |
| 153 | + |
| 154 | +You can integrate the Test Studio automated tests execution in both classic and YAML type of pipelines in AzDO. Find reference for the different options and choose the one which suits your project needs: |
| 155 | + |
| 156 | +- <a href="/advanced-topics/build-server/azdo/self-hosted-agent-classic-pipeline" target="_blank">Configure classic pipeline to execute tests on self hosted machine</a> |
| 157 | +- <a href="/advanced-topics/build-server/azdo/ms-hosted-agent-classic-pipeline" target="_blank">Configure classic pipeline to execute tests on Microsoft hosted machine</a> |
| 158 | +- <a href="/advanced-topics/build-server/azdo/self-hosted-agent-yml-pipeline" target="_blank">Configure YAML pipeline to execute tests on self hosted machine</a> |
| 159 | +- <a href="/advanced-topics/build-server/azdo/ms-hosted-agent-yml-pipeline" target="_blank">Configure YAML pipeline to execute tests on Microsoft hosted machine</a> |
| 160 | + |
| 161 | +[2]: /img/advanced-topics/build-server/azure-devops/fig2.png |
| 162 | +[3]: /img/advanced-topics/build-server/azure-devops/fig3.png |
| 163 | +[4]: /img/advanced-topics/build-server/azure-devops/fig4.png |
| 164 | +[5]: /img/advanced-topics/build-server/azure-devops/fig5.png |
| 165 | +[6]: /img/advanced-topics/build-server/azure-devops/fig6.png |
| 166 | +[7]: /img/advanced-topics/build-server/azure-devops/fig7.png |
| 167 | +[8]: /img/advanced-topics/build-server/azure-devops/fig8.png |
0 commit comments