Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent Results in Pipeline #18

Closed
arpalmares opened this issue Mar 23, 2022 · 1 comment
Closed

Inconsistent Results in Pipeline #18

arpalmares opened this issue Mar 23, 2022 · 1 comment

Comments

@arpalmares
Copy link

arpalmares commented Mar 23, 2022

Actions:

  • Test Studio test are working
  • Created test user for accessing the tests with access to app and dataverse (CRUD)
  • MFA and Security defaults disabled
  • Repository cloned in Azure DevOps
  • Updated TestURLs.json
  • Service connection setup
  • Pipeline setup with variables
  • Disabled Firefox task (only running Test on Chrome)

Issue:

  • Running the Test from the pipeline produces inconsistent results (Same tests in test studio, same commit in repo)
  • Sometimes it is successful, but mostly fails. (only 21.43% pass rate)

image

  • Majority of failed runs report the following prompt:
Error Message:
   Test method Microsoft.PowerApps.TestAutomation.Tests.TestAutomation.RunTestAutomation threw exception: 
OpenQA.Selenium.JavaScriptException: javascript error: Cannot read properties of undefined (reading 'Log')
  (Session info: chrome=99.0.4844.74)
  Stack Trace:
      at OpenQA.Selenium.WebDriver.UnpackAndThrowOnError(Response errorResponse)
   at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
   at OpenQA.Selenium.WebDriver.ExecuteScriptCommand(String script, String commandName, Object[] args)
   at OpenQA.Selenium.WebDriver.ExecuteScript(String script, Object[] args)
   at Microsoft.PowerApps.TestAutomation.Browser.SeleniumExtensions.ExecuteScript(IWebDriver driver, String script, Object[] args) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\Extensions\SeleniumExtensions.cs:line 165
   at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.InitiateTest(IWebDriver driver, Uri uri) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 142
   at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.<>c__DisplayClass1_0.<ExecuteTestAutomation>b__0(IWebDriver driver) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 35
   at Microsoft.PowerApps.TestAutomation.Browser.DelegateBrowserCommand`1.ExecuteCommand(IWebDriver driver, Object[] params) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\DelegateBrowserCommand.cs:line 28
   at Microsoft.PowerApps.TestAutomation.Browser.BrowserCommand`1.Execute[T1,T2,T3,T4,T5,T6,T7,T8,T9](IWebDriver driver, T1 p1, T2 p2, T3 p3, T4 p4, T5 p5, T6 p6, T7 p7, T8 p8, T9 p9) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserCommand.cs:line 136
   at Microsoft.PowerApps.TestAutomation.Browser.BrowserCommand`1.Execute(IWebDriver driver) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserCommand.cs:line 36
   at Microsoft.PowerApps.TestAutomation.Browser.BrowserPage.Execute[TResult](BrowserCommandOptions options, Func`2 delegate) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserPage.cs:line 182
   at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.ExecuteTestAutomation(Uri uri, Int32 testRunNumber, Int32 maxWaitTimeInSeconds) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 32
   at Microsoft.PowerApps.TestAutomation.Tests.TestAutomation.RunTestAutomation() in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Tests\TestAutomation\RunTestAutomation.cs:line 193
  Standard Output Messages:
 Power Apps  Test Automation Execution Starting Test #1 of 1
 An error occurred during Test Run #1 of 1: OpenQA.Selenium.JavaScriptException: javascript error: Cannot read properties of undefined (reading 'Log')
  (Session info: chrome=99.0.4844.74)
    at OpenQA.Selenium.WebDriver.UnpackAndThrowOnError(Response errorResponse)
    at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
    at OpenQA.Selenium.WebDriver.ExecuteScriptCommand(String script, String commandName, Object[] args)
    at OpenQA.Selenium.WebDriver.ExecuteScript(String script, Object[] args)
    at Microsoft.PowerApps.TestAutomation.Browser.SeleniumExtensions.ExecuteScript(IWebDriver driver, String script, Object[] args) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\Extensions\SeleniumExtensions.cs:line 165
    at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.InitiateTest(IWebDriver driver, Uri uri) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 142
    at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.<>c__DisplayClass1_0.<ExecuteTestAutomation>b__0(IWebDriver driver) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 35
    at Microsoft.PowerApps.TestAutomation.Browser.DelegateBrowserCommand`1.ExecuteCommand(IWebDriver driver, Object[] params) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\DelegateBrowserCommand.cs:line 28
    at Microsoft.PowerApps.TestAutomation.Browser.BrowserCommand`1.Execute[T1,T2,T3,T4,T5,T6,T7,T8,T9](IWebDriver driver, T1 p1, T2 p2, T3 p3, T4 p4, T5 p5, T6 p6, T7 p7, T8 p8, T9 p9) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserCommand.cs:line 136
    at Microsoft.PowerApps.TestAutomation.Browser.BrowserCommand`1.Execute(IWebDriver driver) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserCommand.cs:line 36
    at Microsoft.PowerApps.TestAutomation.Browser.BrowserPage.Execute[TResult](BrowserCommandOptions options, Func`2 delegate) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Browser\BrowserPage.cs:line 182
    at Microsoft.PowerApps.TestAutomation.Api.TestAutomation.ExecuteTestAutomation(Uri uri, Int32 testRunNumber, Int32 maxWaitTimeInSeconds) in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Api\Pages\TestAutomation.cs:line 32
    at Microsoft.PowerApps.TestAutomation.Tests.TestAutomation.RunTestAutomation() in D:\a\1\s\Microsoft.PowerApps.TestAutomation.Tests\TestAutomation\RunTestAutomation.cs:line 162
 Current results directory location: D:\a\_temp\..\TestResults\TestResults\Deploy_VssAdministrator 2022-03-24 12_50_24\In

@arpalmares
Copy link
Author

This is already fixed. Added code to extend timeouts on TestURLs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant