docs: add vstest.console CLI options reference and document missing env vars - #16286
Merged
Azat Mukhametshin (azat-msft) merged 20 commits intoJul 17, 2026
Conversation
…nv vars Adds a source-grounded docs/commandline.md covering every user-facing vstest.console.exe option (mirrors --Help / the argument processors), and documents five environment variables that were referenced in src/ but missing from environment-variables.md: - VSTEST_DOTNET_ROOT_PATH - VSTEST_DOTNET_ROOT_ARCHITECTURE - VSTEST_BACKGROUND_DISCOVERY - VSTEST_DISABLE_DOTNET_ROOT_ON_NONWINDOWS - VSTEST_DISABLE_DYNAMICNATIVE_CODECOVERAGE_DEFAULT_SETTING Also links the how-to guides from the README documentation index. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3012d638-7bdb-4b0f-8731-35c9ab2aa258
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 15, 2026 08:43
View session
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves in-repo documentation coverage for the VSTest platform by adding a vstest.console.exe command-line reference, documenting previously-missing environment variables, and indexing key guides from the repository README.
Changes:
- Add a new
docs/commandline.mdreference forvstest.console.exeoptions, aligned with the in-product help text and argument processors. - Document additional environment variables used by the platform in
docs/environment-variables.md. - Add a “Guides” section to
README.mdto make existing documentation easier to discover.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| README.md | Adds a “Guides” subsection linking to the primary how-to docs. |
| docs/environment-variables.md | Documents additional env vars (DOTNET_ROOT-related, background discovery hint, code coverage default behavior flag). |
| docs/commandline.md | Adds a comprehensive command-line options reference for vstest.console.exe. |
Roadmap (docs/releases.md) is already linked from the dedicated ## Roadmap section, so it is removed from the merged list to avoid duplication. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3012d638-7bdb-4b0f-8731-35c9ab2aa258
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 15, 2026 10:11
View session
commandline.md: - /Platform: list the full set of accepted Architecture values (ARM64, S390x, Ppc64le, RiscV64, LoongArch64) not just x86/x64/ARM - @<file>: correct response-file parsing (whitespace-separated, quoting; not one-option-per-line) - clarify short-form options accept a '/' or '-' prefix (e.g. -e, -lt) - soften 'actively maintained reference' wording environment-variables.md: - VSTEST_DISABLE_DOTNET_ROOT_ON_NONWINDOWS: describe the actual behavior (architecture-specific DOTNET_ROOT_<ARCH>, gated on a known dotnet root path) - VSTEST_BACKGROUND_DISCOVERY: it is propagated into the testhost process environment (and also lowers testhost priority) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3012d638-7bdb-4b0f-8731-35c9ab2aa258
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 15, 2026 10:25
View session
Adds an Exit codes section to commandline.md (0 = success, 1 = failure; the process never returns any other value) plus a 'When no tests are found' subsection covering the warning messages emitted for empty filter/selection/ source discovery and the non-uniform exit code across those cases. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3012d638-7bdb-4b0f-8731-35c9ab2aa258
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 15, 2026 12:17
View session
Populates the previously-empty quickstart.md with a CLI-focused getting-started guide: choosing dotnet test vs vstest.console.exe (and the deprecated dotnet vstest), acquiring the standalone runner from the Microsoft.TestPlatform NuGet package, a first run, common tasks, and exit codes. Links it from the README Guides index. Note: quickstart.md content is also touched by another in-flight PR; the two will be reconciled at merge time. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3012d638-7bdb-4b0f-8731-35c9ab2aa258
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 15, 2026 12:30
View session
…d-missing-envvars # Conflicts: # docs/quickstart.md
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 15, 2026 20:49
View session
commandline.md: - scope the intro to the --Help options and note that internal/legacy/hidden switches (/EnableCodeCoverage, /UseVsixExtensions, internal /List*, etc.) are intentionally omitted, so it no longer implies completeness - use forward slashes in wildcard examples for cross-platform correctness - match the real 'no test matches filter' message (filter shown in backticks) - correct the no-tests exit code: it is 0 by default and only 1 when RunConfiguration.TreatNoTestsAsError=true (not a /Tests-vs-filter split) environment-variables.md: - feature flags are enabled for any value other than 0 (not just 'non-zero'); clarify the ambiguous 'to disable' wording - VSTEST_BACKGROUND_DISCOVERY reduces parallelism for both discovery and execution, not discovery only quickstart.md: - use forward slashes in the cross-platform runner path and wildcard examples Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3012d638-7bdb-4b0f-8731-35c9ab2aa258
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 15, 2026 21:08
View session
| [`src/vstest.console/Processors`](../src/vstest.console/Processors) and mirrors the built-in | ||
| `--Help` output. | ||
|
|
||
| `vstest.console.exe` also accepts a number of internal, legacy, or hidden switches that are |
Member
There was a problem hiding this comment.
this is a second paragraph, that should have the most important info, instead we tell them what we will not show them, move this to the bottom of the article, so it is searchable but not in our face.
|
|
||
| Makes `vstest.console.exe` use custom test adapters from the given path in the test run. | ||
|
|
||
| ### `/TestAdapterLoadingStrategy:<strategy>` |
Member
There was a problem hiding this comment.
as mentioned elsewhere, maybe this should be removed, and the option not documented.
Member
Author
There was a problem hiding this comment.
The TestAdapterLoadingStrategy is moved to the "Omitted switches" section.
Should it be removed from the --Help output as well?
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 16, 2026 12:52
View session
Point the /logger section solely at docs/report.md for the full list of loggers and options instead of the external console-logger aka.ms link. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f53a115c-0178-46a7-ac37-2e5ac1676f10
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 16, 2026 13:04
View session
- Re-add the UTF-8 BOM that was accidentally stripped from the 13 vstest.console .xlf files (matches the rest of the repo and avoids noisy diffs/tooling breakage). - Reword the commandline.md intro so it no longer claims to fully mirror --Help, and point readers to the Omitted switches section (avoids the contradiction with /TestAdapterLoadingStrategy). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f53a115c-0178-46a7-ac37-2e5ac1676f10
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 16, 2026 13:18
View session
…processors There is no generator script producing docs/commandline.md; 'derived from' is accurate and avoids implying an automated guarantee. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f53a115c-0178-46a7-ac37-2e5ac1676f10
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 16, 2026 13:34
View session
Add /ListDiscoverers, /ListExecutors, /ListLoggers and /ListSettingsProviders (they use a private CommandName const and were missed earlier), and reword the intro so it no longer claims a single --Help exception, since both /TestAdapterLoadingStrategy and /RunTests have --Help entries. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f53a115c-0178-46a7-ac37-2e5ac1676f10
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 16, 2026 13:53
View session
The PlatformArgumentProcessorTests and FrameworkArgumentProcessorTests assert on the exact PlatformArgumentHelp/FrameworkArgumentHelp resource strings, which were updated to list all architectures and short TFM monikers. Update the expected values (and the generated Designer.cs doc comments) accordingly. Verified all 21 tests pass in Release. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f53a115c-0178-46a7-ac37-2e5ac1676f10
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 16, 2026 14:27
View session
Jakub Jareš (nohwnd)
approved these changes
Jul 16, 2026
- VSTEST_DOTNET_ROOT_PATH: the variable vstest sets depends on the testhost — DOTNET_ROOT_<ARCH> for net8+ (17.14+) hosts, but DOTNET_ROOT(x86) for x86 or the architecture-less DOTNET_ROOT otherwise on older netcoreapp3.1 hosts. - VSTEST_DISABLE_DOTNET_ROOT_ON_NONWINDOWS: note that deriving the dotnet root from DOTNET_ROOT only happens on Windows (the dotnet.exe PE-header architecture probe returns nothing on Linux/macOS). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f53a115c-0178-46a7-ac37-2e5ac1676f10
Copilot started reviewing on behalf of
Azat Mukhametshin (azat-msft)
July 16, 2026 14:40
View session
Azat Mukhametshin (azat-msft)
merged commit Jul 17, 2026
3026de6
into
microsoft:main
20 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of a documentation completeness audit (checking docs against the actual code). This PR fills two gaps not covered by the other open docs PRs (#16277, #16278, #16279):
1. New
docs/commandline.md— vstest.console.exe options referenceThere was no in-repo reference for
vstest.console.execommand line options; they were only discoverable via the runtime--Helpoutput or external Microsoft Learn. This adds a source-grounded reference covering every user-facing option, generated from the argument processors insrc/vstest.console/Processorsand mirroring the--Helptext:/Tests,/TestCaseFilter,/ListTests,/Parallel,/InIsolation,/Platform,/Framework,/Environment,/TestAdapterPath,/Settings,-- [name]=[value],/logger,/Collect,/Blame,/ResultsDirectory,/Diag,/Help,@<file>,/Port,/ParentProcessId. Internal/hidden switches (including/TestAdapterLoadingStrategy) are enumerated in an "Omitted switches" section.2. Missing environment variables in
docs/environment-variables.mdFive variables referenced in
src/were undocumented (verified they are not added by #16279):VSTEST_DOTNET_ROOT_PATHVSTEST_DOTNET_ROOT_ARCHITECTUREVSTEST_BACKGROUND_DISCOVERYVSTEST_DISABLE_DOTNET_ROOT_ON_NONWINDOWSVSTEST_DISABLE_DYNAMICNATIVE_CODECOVERAGE_DEFAULT_SETTING3. README documentation index
Adds a Guides subsection linking the existing how-to docs (command line, filter, runsettings, configure, coverage, diagnostics), which previously were not indexed.
4.
--Helptext corrections (product resource change)While grounding the CLI reference against the argument processors, two
vstest.console--Helpstrings were found to be inaccurate and are corrected insrc/vstest.console/Resources/Resources.resx(with matching*.xlfsource updates, targets flagged for re-translation):/Platformnow lists all supported architectures (x86, x64, ARM, ARM64, S390x, Ppc64le, RiscV64, LoongArch64) instead of onlyx86/x64/ARM./Frameworknow points out the common short TFM monikers (net48,net10.0) alongside the long forms.Apart from those corrected
--Helpresource strings, this is a documentation-only change.