Update NUnit3TestAdapter version to 4.0.0-beta.2

[Mohit-Chakraborty]

No description provided.

[Mohit-Chakraborty]

/azp run net - search - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Mohit-Chakraborty]

/azp run net - search - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Mohit-Chakraborty]

/azp run net - search - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Mohit-Chakraborty]

@mikeharder - I ran the live search tests and they passed. I think the change to update the NUnit test adapter has helped.
What other tests should I run here?

[mikeharder]

/azp run net - core - tests

[mikeharder]

/azp run net - storage - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mikeharder]

@Mohit-Chakraborty: I triggered runs of core and storage tests, if these pass I think this is safe enough to merge. If we find any failures after merge we can investigate.

[pakrym]

Any particular reason to get the update especially a beta? We should compare test counts before/after.

[mikeharder]

@pakrym: The latest RTM version is crashing with OOM per this issue: #20509. I don't think @Mohit-Chakraborty has been able to repro locally, so we're not sure exactly what's causing it. But I suspect it's an issue in NUnit3TestAdapter, so I suggested we try the latest version and it does seem to fix the OOM.

If you would rather not use a Beta, we could test GA versions older than 3.17.0 to see if any of them fix the OOM as well.

[pakrym]

As far as I know, 4.0.0-* is a major rewrite of NUnit. I'm not strictly against it but think we should be careful taking it.

Doesn't Test host process crashed apply to the child testhost process?

[pakrym]

Yep, use this test to repro the error message. This might be an actual test/product bug.

using System;
using System.Threading;
using NUnit.Framework;
using System.Collections.Generic;

namespace ooomunit
{
    public class Tests
    {
        [Test]
        public void Test1()
        {
            ThreadPool.QueueUserWorkItem(_ => 
            {
            List<byte[]> b = new();
            while (true)
            {
                b.Add(new byte[100000000]);
            }
            }
            );
            
            Thread.Sleep(1000000);
        }
    }
}

[pakrym]

I would also look into the --blame-* set of parameters for dotnet test. https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test

[weshaggard]

Similar to @pakrym's feedback I would like to avoid taking a dependency on a prerelease if it can be helped.

[mikeharder]

I agree we need to step back and understand what is happening here. If this is indeed a test or product bug causing OOM, then why doesn't it repro with the newer NUnit3TestAdapter?

[pakrym]

. If this is indeed a test or product bug causing OOM, then why doesn't it repro with the newer NUnit3TestAdapter?

Maybe some changes in test ordering/parallelization? I think adding an ability (or learning a way) to collect test crash dumps is a great exercise in general.