Skip to content

Running Our Tests

Jump to bottom
Matt Bierner edited this page Nov 7, 2016 · 14 revisions

If you are working on a pull request for NTVS, you should run our tests to help with development and verify the changes do not introduce any regressions. The tests are also run as part of our AppVeyor continuous integration process when you submit a pull request.

All new code should also be tested.

Manually Running Our Tests

Our tests use the Visual Studio Unit Testing Framework.

After you have a good build, you can run individual tests suites using the vstest.console command on the windows command prompt:

$ vstest.console .\BuildOutput\Debug14.0\Tests\NodeTests.dll

💡Note: If the vstest.console command does not exist, try running the above command inside of the Visual Studio Command Prompt instead, or you can use the full path the vstest.console executable: 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe' (path will differ depending on Visual Studio version)

The above command will run all tests for the NodeTests project. To run all tests in NTVS, use the runtests.ps1 Powershell script:

$ .\runtests.ps1

If you want run tests only for VS 2015 for example, you could pass vstarget parameters to the runtests.ps1 Powershell script:

$ .\runtests.ps1 -vstarget "14.0"

Test Categories

Some tests are marked with categories that can help you run just a targeted subset of tests during development. New tests should also be marked with correct categories.

You can run all tests of a given category using the TestCaseFilter argument of vstest.console:

$ vstest.console .\BuildOutput\Debug14.0\Tests\NodeTests.dll /TestCaseFilter:"TestCategory=CATEGORY"

# Or, to run all tests

$ .\runtests.ps1 /TestCaseFilter:"TestCategory=CATEGORY"

'Ignore'

Tests that are disabled. The reason for using an Ignore category over the [Ignore] attribute is that using a category makes it explicit that tests are being ignored. Ignored tests are just dead code.

❗️Important: Do not ever add any new tests to this category.

'AppVeyorIgnore'

Tests that are not run as part of AppVeyor CI validation. These tests should pass locally but fail in AppVeyor.

❗️Important: Do not add any new tests to this category unless you really, really have a good reason to. It exists mainly to support legacy tests and will be phased out over time.

Clone this wiki locally