Skip to content

Removed Discover Warnings in Test Output | Issue #437 #480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 23, 2018
Merged

Removed Discover Warnings in Test Output | Issue #437 #480

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
bc4ccd7
Fixed issue #437 - Removed discover warnings
Aug 25, 2018
52afbf3
Merge branch 'master' into issue437-discoverwarnings
parrainc Sep 7, 2018
16d5d40
Revert "Fixed issue #437 - Removed discover warnings"
Sep 17, 2018
a189a86
Solved issue #437 - Removed discover warnings
Sep 26, 2018
731c4c8
Merge branch 'master' of https://github.com/parrainc/testfx into issu…
Sep 26, 2018
5a32326
Merge branch 'master' into issue437-discoverwarnings
jayaranigarg Oct 8, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions test/UnitTests/MSTest.CoreAdapter.Unit.Tests/Execution/TestExecutionManagerTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -868,7 +868,7 @@ private void SetTestCaseProperties(TestCase testCase, object[] propertiesValue)

#region Dummy implementation

[UTF.TestClass]
[DummyTestClass]
internal class DummyTestClass
{
public static IDictionary<string, object> TestContextProperties
Expand Down Expand Up @@ -901,7 +901,7 @@ public void IgnoredTest()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassWithCleanupMethods
{
[UTF.ClassCleanup]
Expand All @@ -916,7 +916,7 @@ public void TestMethod()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassForParallelize
{
private static HashSet<int> threadIds = new HashSet<int>();
Expand Down Expand Up @@ -951,7 +951,7 @@ public void TestMethod2()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassForParallelize2
{
private static HashSet<int> threadIds = new HashSet<int>();
Expand Down Expand Up @@ -982,7 +982,7 @@ public void TestMethod2()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassForParallelize3
{
private static HashSet<int> threadIds = new HashSet<int>();
Expand All @@ -1007,7 +1007,7 @@ public void TestMethod1()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassWithDoNotParallelizeMethods
{
private static HashSet<int> parallelizableTestsThreadIds = new HashSet<int>();
Expand Down Expand Up @@ -1096,6 +1096,10 @@ public void TestMethod4()
}
}

private class DummyTestClassAttribute : UTF.TestClassAttribute
{
}

#endregion
}

Expand Down
6 changes: 5 additions & 1 deletion test/UnitTests/MSTest.CoreAdapter.Unit.Tests/Execution/TestMethodFilterTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ public void GetFilterExpressionForDiscoveryContextWithGetTestCaseFilterThrowingE
Assert.AreEqual(TestMessageLevel.Error, recorder.TestMessageLevel);
}

[UTF.TestClass]
[DummyTestClass]
internal class DummyTestClassWithTestMethods
{
public UTFExtension.TestContext TestContext { get; set; }
Expand Down Expand Up @@ -282,5 +282,9 @@ public bool MatchTestCase(TestCase testCase, Func<string, object> propertyValueP
throw new NotImplementedException();
}
}

private class DummyTestClassAttribute : UTF.TestClassAttribute
{
}
}
}
24 changes: 14 additions & 10 deletions test/UnitTests/MSTest.CoreAdapter.Unit.Tests/Execution/TypeCacheTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1236,7 +1236,7 @@ private void SetupMocks()

#region dummy implementations

[UTF.TestClass]
[DummyTestClass]
internal class DummyTestClassWithTestMethods
{
public UTFExtension.TestContext TestContext { get; set; }
Expand Down Expand Up @@ -1308,7 +1308,7 @@ public void TestMethodWithMultipleExpectedException()
}
}

[UTF.TestClass]
[DummyTestClass]
internal class DerivedTestClass : BaseTestClass
{
}
Expand Down Expand Up @@ -1343,7 +1343,7 @@ private class DummyTestClassWithMultipleTestContextProperties : DummyTestClassWi
{
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassWithInitializeMethods
{
public static void AssemblyInit(UTFExtension.TestContext tc)
Expand All @@ -1355,7 +1355,7 @@ public void TestInit()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassWithCleanupMethods
{
public static void AssemblyCleanup()
Expand All @@ -1367,23 +1367,23 @@ public void TestCleanup()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyDerivedTestClassWithInitializeMethods : DummyTestClassWithInitializeMethods
{
public void TestMehtod()
{
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyDerivedTestClassWithCleanupMethods : DummyTestClassWithCleanupMethods
{
public void TestMehtod()
{
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassWithInitAndCleanupMethods
{
public static void AssemblyInit(UTFExtension.TestContext tc)
Expand All @@ -1399,7 +1399,7 @@ public void TestInitOrCleanup()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassWithIncorrectInitializeMethods
{
public static void TestInit(int i)
Expand All @@ -1411,7 +1411,7 @@ public void AssemblyInit(UTFExtension.TestContext tc)
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassWithIncorrectCleanupMethods
{
public static void TestCleanup(int i)
Expand All @@ -1423,7 +1423,7 @@ public void AssemblyCleanup()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassWithIncorrectTestMethodSignatures
{
public static void TestMethod()
Expand All @@ -1435,6 +1435,10 @@ private class DerivedTestMethodAttribute : UTF.TestMethodAttribute
{
}

private class DummyTestClassAttribute : UTF.TestClassAttribute
{
}

#endregion
}
}
8 changes: 6 additions & 2 deletions test/UnitTests/MSTest.CoreAdapter.Unit.Tests/Execution/UnitTestRunnerTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ private MSTestSettings GetSettingsWithDebugTrace(bool captureDebugTraceValue)

#region Dummmy implementations

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClass
{
public UTFExtension.TestContext TestContext { get; set; }
Expand All @@ -358,7 +358,7 @@ public void TestMethodToTestInProgress()
}
}

[UTF.TestClass]
[DummyTestClass]
private class DummyTestClassWithInitializeMethods
{
public static Action AssemblyInitializeMethodBody { get; set; }
Expand Down Expand Up @@ -412,6 +412,10 @@ public void TestMethod()
}
}

private class DummyTestClassAttribute : UTF.TestClassAttribute
Copy link
Member

@jayaranigarg jayaranigarg Oct 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add [DummyTestClass] to class DummyTestClassWithCleanupMethods(Line 388) as well.

Copy link
Contributor Author

@parrainc parrainc Oct 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just did it, but when added the [DummyTestClass] attribute to DummyTestClassWithCleanupMethods class, one of the unit test failed RunCleanupShouldReturnCleanupResultsForAssemblyAndCla ssCleanupMethods because of the AssemblyCleanup method is not being called.

Copy link
Member

@jayaranigarg jayaranigarg Oct 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is weird.

Copy link
Contributor Author

@parrainc parrainc Oct 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed it is. I did even try debugging the unit test that was using the DummyTestClassWithCleanupMethods with the [DummyTestClass] and [UTF.TestClass] but could not observe nothing. and got same behavior except for the part of the assemblyCleanup thing. Notice that when invoking the ClassCleanup it works properly.

Copy link
Member

@jayaranigarg jayaranigarg Oct 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried debugging this. Basically, when we get DeclaredMethods for DummyTestClassWithCleanupMethods, it is not returning AssemblyCleanup() method in the list of methods for that type and hence AssemblyCleanup() is not getting called. Now, the question is why AssemblyCleanup() is not present in the list when we mention [DummyTestClass] and is present in case of [UTF.TestClass]

Copy link
Contributor Author

@parrainc parrainc Oct 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the same question I asked to myself...
I created a mstest project, added a class CustomTestClass that inherits from TestClass, and set up the assembly/class initalize and cleanup methods. It looks like there could be an issue when a test class is inheriting from a custom attribute that inherits from TestClassAttribute.
Have you seem any issues regarding to that?

Here's the debug output with CustomTestClass attribute:
expectedValue is a private static int field declared in the test class

ClassInitialize: before -> 0
ClassInitialize: after -> 321
TestMethod: expectedValue -> 321

Assert.AreEqual failed. Expected:<123>. Actual:<321>.

ClassCleanup: expectedValue -> 321

notice that with the custom attribute, neither the assembly initialize nor the cleanup were executed

Here's the debug output with TestClass attribute:

AssemblyInitialize: before -> 0
AssemblyInitialize: after -> 123
ClassInitialize: before -> 123
ClassInitialize: after -> 123
TestMethod: expectedValue -> 123
ClassCleanup: expectedValue -> 123
AssemblyCleanup: expectedValue -> 123

Copy link
Contributor Author

@parrainc parrainc Oct 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jayaranigarg any updates?
I'm thinking of opening an issue with my inputs in above comments? Would that be alright?

Copy link
Member

@jayaranigarg jayaranigarg Oct 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@parrainc : That should be fine. Please raise a separate issue for this and let's get this PR in.

{
}

#endregion
}
}