Update tests to be able to run them outside of AppVeyor (part 2)

[andreasjordan]

Followup to #9406

I want to run all the tests on a local lab using "real world" sql server instances.

These are some fixes to get all -DbaDb tests to run on my local lab against three local instanes without any failures.

[andreasjordan]

For Invoke-DbaWhoisActive.Tests.ps1: Can we add \CommunitySoftware\sp_whoisactive-12.00.zip to the appveyorlabrepo? @potatoqualitee Do you want to add the file from https://github.com/amachanic/sp_whoisactive/archive/refs/tags/v12.00.zip or should I open a pull request to the repo?

[andreasjordan]

I also added some changes to commands to fix some issues to be able to run the tests.

About the change to Connect-DbaInstance: Maybe I have deleted this line some time in the past, I just can not remember. I can move this change to a separate PR for better documentation.

[niphlod]

I don't think adding whoisactive is a problem. I should have enough perms to do it, wait a bit

[niphlod]

done @andreasjordan , check if everything matches

[andreasjordan]

In my local lab, all 699 tests complete in around 90 minutes.

These currently have failing tests, will work on them in the next week:

Add-DbaAgDatabase.Tests.ps1
Add-DbaAgListener.Tests.ps1
Add-DbaAgReplica.Tests.ps1
Copy-DbaAgentJob.Tests.ps1
Copy-DbaLinkedServer.Tests.ps1
Copy-DbaResourceGovernor.Tests.ps1
Export-DbaDacPackage.Tests.ps1
Get-DbaAgDatabase.Tests.ps1
Get-DbaAgListener.Tests.ps1
Get-DbaAgReplica.Tests.ps1
Get-DbaAvailabilityGroup.Tests.ps1
Get-DbaClientProtocol.Tests.ps1
Get-DbaDump.Tests.ps1
Get-DbaLatchStatistic.Tests.ps1
Invoke-DbaBalanceDataFiles.Tests.ps1
Invoke-DbaWhoisActive.Tests.ps1
New-DbaAvailabilityGroup.Tests.ps1
New-DbaLogin.Tests.ps1
New-DbaSsisCatalog.Tests.ps1
Read-DbaAuditFile.Tests.ps1
Read-DbaXEFile.Tests.ps1
Remove-DbaAgDatabase.Tests.ps1
Remove-DbaAgListener.Tests.ps1
Remove-DbaAvailabilityGroup.Tests.ps1
Remove-DbaDatabaseSafely.Tests.ps1
Restore-DbaDatabase.Tests.ps1
Resume-DbaAgDbDataMovement.Tests.ps1
Set-DbaAgReplica.Tests.ps1
Set-DbaAvailabilityGroup.Tests.ps1
Set-DbaErrorLogConfig.Tests.ps1
Set-DbaTcpPort.Tests.ps1
Set-DbaTempDbConfig.Tests.ps1
Start-DbaDbEncryption.Tests.ps1
Suspend-DbaAgDbDataMovement.Tests.ps1
Sync-DbaLoginPermission.Tests.ps1
Test-DbaBackupEncrypted.Tests.ps1
Test-DbaDeprecatedFeature.Tests.ps1
Watch-DbaXESession.Tests.ps1

Update: Updated the list, because some tests passed on the second execution.
Update: Updated the list, because some tests passed after the first instance was installed as an express edition.

[andreasjordan]

Test for Invoke-DbaWhoIsActive is failing on instance2:

Describe : Invoke-DbaWhoisActive Integration Tests
Context  : Should have SPWhoisActive installed correctly
Name     : It Should be installed to master
Result   : Failed
Message  : Expected $null or empty, but got @([07:57:48][Install-DbaWhoIsActive] Failed to install stored procedure. | Log entry string is too long. A string written to the event log cannot exceed 32766 characters.).

Let's see if it works on instance3...

[andreasjordan]

@niphlod - sorry, I need to learn a little more about the tests. I changed Invoke-DbaWhoIsActive tests to use instance3 instead of instannce2 to see if it fails on this instance as well. What do I need to changes also to more the test in the other test group?

[niphlod]

hi @andreasjordan , the "logic" that assigns tests to a specific job within the matrix of concurrent jobs in appveyor (we name it as "scenario") is within appveyor.common.ps1.

Trying to sum up all the things involved....

pester.groups.ps1 is the place where "ties are created" between jobs on appveyor and scenarios, based on:

• explicit assignment (need to manually assign a test to a scenario/matrix editing pester.groups.ps1, this is the case e.g. for Update-DbaServiceAccount, etc that have a specific "confinement" in order to NOT disrupt other tests)
• autodetect assignment where the source code of the test is "scanned" for the presence of "instance1/instance2,etc"

So, as soon as a test has just "instance3" in it, without "instance1", it'll go (or, at least, that's the idea) automatically into the 2016_2017 scenario.
Scenarios (run tests with a set of preconfigured services active, see https://github.com/dataplat/dbatools/blob/development/appveyor.yml#L35 and following) are further split on "parts" in order to run tests concurrently.
Each scenario gets set up the same way via "init scripts", such as \tests\appveyor.SQL2017.ps1.
Each test either lands either on "part1" or "part2" automatically (if parts are present), and they're balanced with a simple count, i.e. the logic tries to keep the number of test files assigned to a Job (which is the "result" of the matrix) even, but still considers an absolute "bound" by scenario to not cross (tests that are assigned to 2016_2017 can't cross the "scenario" bound, just the "part" one).

[potatoqualitee]

Thanks all! I am happy to accept what make sense for the lab materials.