Skip to content

Microsoft.Testing.Platform: not equal operator not working in mstest runner graph query filtering #4302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sergey-rybalkin opened this issue Dec 10, 2024 · 1 comment · Fixed by #4494
Assignees
Labels
Area: MTP Belongs to the Microsoft.Testing.Platform core library In-PR

Comments

@sergey-rybalkin
Copy link

Describe the bug

Using new Microsoft.Testing.Platform library with TUnit framework. It seems like not all of the test filtering operators documented in graph query filtering are supported. For example, when using TUnit with --treenode-filter command line parameter the not equal != operator doesn't seem to be working for me.

Steps To Reproduce

Consider the following code snippet:

public class MyTests
{
    [Test]
    [Category("Smoke")]
    public void TestWithCategory()
    {
        // ...
    }

    [Test]
    public void TestWithoutCategory()
    {
        // ...
    }
}

The following command runs only the first test, as expected dotnet run --treenode-filter /*/*/*/*[Category=Smoke]:

Test run summary: Passed! - bin\Debug\net9.0\Engine.Tests.dll (net9.0|x64)
  total: 1
  failed: 0
  succeeded: 1
  skipped: 0
  duration: 91ms

However, dotnet run --treenode-filter /*/*/*/*[Category!=Smoke] doesn't seem to find anything:

Test run summary: Zero tests ran - bin\Debug\net9.0\Engine.Tests.dll (net9.0|x64)
  total: 0
  failed: 0
  succeeded: 0
  skipped: 0
  duration: 88ms

Expected behavior

Filter should exclude tests with Category property set to Smoke.

Actual behavior

Filter excludes all tests.

Additional context

Seems like filter parser doesn't support != operator at all - https://github.com/microsoft/testfx/blob/main/src/Platform/Microsoft.Testing.Platform/Requests/TreeNodeFilter/TreeNodeFilter.cs#L56
Related issue in TUnit repository - thomhurst/TUnit#1381

SDK version: 9.0.101
OS version: Microsoft Windows [Version 10.0.26100.2454]
TUnit version: 0.4.86
Microsoft.Testing.Platform version: 1.4.3

@Evangelink
Copy link
Member

I don't recall how much was actually implemented in the current version of the tree filter but that's definitely supported by our design: https://github.com/microsoft/testfx/blob/main/docs/mstest-runner-graphqueryfiltering/graph-query-filtering.md

Hence marking this as a bug.

@Evangelink Evangelink added Area: MTP Belongs to the Microsoft.Testing.Platform core library and removed Needs: Triage 🔍 labels Dec 10, 2024
@Evangelink Evangelink added this to the MSTest 3.8 / Platform 1.6 milestone Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: MTP Belongs to the Microsoft.Testing.Platform core library In-PR
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants