Skip to content

Running Our Tests

Jump to bottom
Matt Bierner edited this page Mar 1, 2016 · 14 revisions

If you are working on a pull request for NTVS, you should run our tests to help with development and verify the changes do not introduce any regressions. The tests are also run as part of our AppVeyor continuous integration process when you submit a pull request.

All new code should also be tested.

Manually Running Our Tests

Our tests use the Visual Studio Unit Testing Framework.

After you have a good build, you can run individual tests suites using the vstest.console command on the windows command prompt:

$ vstest.console .\BuildOutput\Debug14.0\Tests\NodeTests.dll

💡Note: If the vstest.console command does not exist, try running the above command inside of the Visual Studio Command Prompt instead, or you can use the full path the vstest.console executable: 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\M icrosoft\TestWindow\vstest.console.exe' (path will differ depending on Visual Studio version)

The above command will run all tests for the NodeTests project. To run all tests in NTVS, use the tools\runtests.ps1 Powershell script:

$ .\tools\runtests.ps1"

Test Categories

Some tests are marked with categories that can help you run just a targeted subset of tests during development. New tests should also be marked with correct categories.

You can run all tests of a given category using the TestCaseFilter argument of vstest.console:

$ vstest.console .\BuildOutput\Debug14.0\Tests\NodeTests.dll /TestCaseFilter:"TestCategory=CATEGORY"

# Or, to run all tests

$ .\runtests.ps1 /TestCaseFilter:"TestCategory=CATEGORY"

'UnitTest'

Tests a small unit of functionality in isolation. In general, a unit test must:

  • Run quickly.
  • Run reliably.
  • Run in a sandbox (no state and no network or filesystem access).

A good unit test should test a single unit of code with minimal dependencies.

'IntegrationTest'

Tests the end to end flow of the system. These tests may be more expensive and are run in a less strict environment than unit tests, but still should be reliable and as isolated from each other as possible.

'AppVeyorIgnore'

Tests that are not run as part of AppVeyor CI validation. These tests should pass locally but fail in AppVeyor.

❗️Important: Do not add any new tests to this category unless you really, really have a good reason to. It exists mainly to support legacy tests and will be phased out over time.

Clone this wiki locally