[mono][xunit tests] Move skipped tests out of rsp file #2087

MaximLipnin · 2020-01-23T16:14:20Z

Addresses #1980

stephentoub · 2020-01-23T16:39:52Z

Thanks. What's the rhyme or reason for which lines were moved to be ActiveIssue and which ones weren't?

src/mono/netcore/CoreFX.issues.rsp

MaximLipnin · 2020-01-24T06:38:56Z

Thanks. What's the rhyme or reason for which lines were moved to be ActiveIssue and which ones weren't?

I sent it as a draft with just several first tests moved in order to verify the general way. So I'm going to complete this work, at least for CoreFX.issues.rsp file, in the current PR.

MaximLipnin · 2020-01-24T06:45:00Z

Thank you for a quick look!

MaximLipnin · 2020-01-24T19:34:12Z

Given that the CoreFX.issues.rsp file is quite big, it would make sense to divide this work into several PRs and merge the current one because it's in good shape right now.

...untime.InteropServices/tests/System/Runtime/InteropServices/Marshal/GetExceptionCodeTests.cs

src/mono/netcore/CoreFX.issues.rsp

…ibuteTests.StronglyTypedStructureTest

…ontainerTests.GetExportOfTTMetadataView1_TypeAsMetadataViewTypeArgument_IsUsedAsMetadataConstraint

…mblyLoadContext, out of rsp file

…s with no local failures from rsp file

…stentInDefaultContext and System.Runtime.Loader.Tests.DefaultLoadContextTests.LoadInDefaultContext from rsp file because the related issues should be addressed

…Tests.LoadNonExistentInDefaultContext and System.Runtime.Loader.Tests.DefaultLoadContextTests.LoadInDefaultContext because they fail on CI

… an active issues out of rsp file

…ow they behave on CI

…namespace on Mono

…space.

…d on Mono

src/mono/netcore/System.Private.CoreLib/src/System/Text/Utf8Span.cs

akoeplinger

Looks great! I found a few small nits that can be addressed in a follow-up PR.

akoeplinger · 2020-02-18T18:53:35Z

src/libraries/System.ComponentModel.Composition/tests/AssemblyInfo.cs

+
+using Xunit;
+
+[assembly: SkipOnMono("Flaky tests: https://github.com/mono/mono/issues/16417")]


should be

Suggested change

[assembly: SkipOnMono("Flaky tests: https://github.com/mono/mono/issues/16417")]

[assembly: ActiveIssue("Flaky tests: https://github.com/mono/mono/issues/16417", TestRuntimes.Mono)]

Attribute 'ActiveIssue' is not valid on this declaration type. It is only valid on 'class, method' declarations.

Hm interesting. I don't see a good reason not to allow ActiveIssue on assembly, will file a PR in dotnet/arcade.

akoeplinger · 2020-02-18T18:53:55Z

...on/tests/System/ComponentModel/Composition/AttributedModel/AllowNonPublicCompositionTests.cs

@@ -8,9 +8,11 @@

 namespace System.ComponentModel.Composition
 {
+    // [ActiveIssue("https://github.com/mono/mono/issues/16417", TestRuntimes.Mono)]


commented code, can be removed

Suggested change

// [ActiveIssue("https://github.com/mono/mono/issues/16417", TestRuntimes.Mono)]

akoeplinger · 2020-02-18T18:56:07Z

src/mono/netcore/System.Private.CoreLib/src/System/Utf8String.cs

+#else
+using nint = System.Int32;
+using nuint = System.UInt32;
+#endif


nint/nuint isn't used, can be removed.

marek-safar added the area-Infrastructure-libraries label Jan 23, 2020

akoeplinger requested changes Jan 23, 2020

View reviewed changes

src/mono/netcore/CoreFX.issues.rsp Show resolved Hide resolved

MaximLipnin force-pushed the move_skipped_tests_out_of_rsp_file branch from 07ec772 to bc6dd61 Compare January 24, 2020 11:58

MaximLipnin mentioned this pull request Jan 24, 2020

[mono][tests] Microsoft.VisualBasic.Tests.InteractionTests fail #2139

Open

MaximLipnin force-pushed the move_skipped_tests_out_of_rsp_file branch from 37a2154 to 7a0b55a Compare January 27, 2020 08:23

jkotas reviewed Jan 27, 2020

View reviewed changes

...untime.InteropServices/tests/System/Runtime/InteropServices/Marshal/GetExceptionCodeTests.cs Outdated Show resolved Hide resolved

MaximLipnin force-pushed the move_skipped_tests_out_of_rsp_file branch from 905511b to 092faed Compare January 29, 2020 08:23

MaximLipnin commented Jan 29, 2020

View reviewed changes

src/mono/netcore/CoreFX.issues.rsp Show resolved Hide resolved

This was referenced Jan 30, 2020

[mono] Tests failed: System.Reflection.Emit.ILGeneration.Tests #2383

Open

[mono] Tests failed: System.Reflection.Emit.Tests #2389

Open

MaximLipnin force-pushed the move_skipped_tests_out_of_rsp_file branch from 3523e4b to 25119d3 Compare January 31, 2020 09:17

MaximLipnin added 9 commits February 18, 2020 08:20

Add an active issue to System.ComponentModel.Composition.MetadataAttr…

b7aa5d3

…ibuteTests.StronglyTypedStructureTest

Add an active issue to System.ComponentModel.Composition.CompositionC…

3b74374

…ontainerTests.GetExportOfTTMetadataView1_TypeAsMetadataViewTypeArgument_IsUsedAsMetadataConstraint

Disable the whole System.ComponentModel.Composition.Tests namespace

5514c90

Move some System.Runtime.Loader.Tests, which rely on collectible Asse…

05fe86d

…mblyLoadContext, out of rsp file

Remove three System.Runtime.Loader.Tests.AssemblyLoadContextTest test…

07afeb2

…s with no local failures from rsp file

Remove System.Runtime.Loader.Tests.DefaultLoadContextTests.LoadNonExi…

6a0d104

…stentInDefaultContext and System.Runtime.Loader.Tests.DefaultLoadContextTests.LoadInDefaultContext from rsp file because the related issues should be addressed

Add an active issue to System.Runtime.Loader.Tests.DefaultLoadContext…

3ce15ea

…Tests.LoadNonExistentInDefaultContext and System.Runtime.Loader.Tests.DefaultLoadContextTests.LoadInDefaultContext because they fail on CI

Move several System.Runtime.Loader.Tests.AssemblyLoadContextTest with…

7779a6b

… an active issues out of rsp file

Remove System.Utf8String.Experimental.Tests from rsp file to verify h…

315a4c8

…ow they behave on CI

MaximLipnin force-pushed the move_skipped_tests_out_of_rsp_file branch from 08cf306 to 315a4c8 Compare February 18, 2020 06:09

MaximLipnin added 4 commits February 18, 2020 11:20

Clean up rsp file to check all remaining failures

16a55ef

Skip the entire System.ComponentModel.Composition.Registration.Tests …

4e28d11

…namespace on Mono

Disable a set of classes in System.Utf8String.Experimental.Tests work…

0d3f163

…space.

Add stubs for Utf8String and Utf8Span.cs classes to fix the test buil…

5172b49

…d on Mono

akoeplinger reviewed Feb 18, 2020

View reviewed changes

src/mono/netcore/System.Private.CoreLib/src/System/Text/Utf8Span.cs Outdated Show resolved Hide resolved

MaximLipnin added 2 commits February 18, 2020 17:43

Remove nint/nunit aliases unused in PNSE version of Utf8Span

7d0bbc6

Remove CoreFX.issues.rsp file because it has been cleaned up

19a5e04

MaximLipnin marked this pull request as ready for review February 18, 2020 17:45

MaximLipnin requested review from ahsonkhan, EgorBo, marek-safar, steveisok and vargaz as code owners February 18, 2020 17:45

marek-safar approved these changes Feb 18, 2020

View reviewed changes

akoeplinger approved these changes Feb 18, 2020

View reviewed changes

akoeplinger merged commit b74422f into dotnet:master Feb 18, 2020

MaximLipnin deleted the move_skipped_tests_out_of_rsp_file branch February 19, 2020 05:22

MaximLipnin mentioned this pull request Feb 19, 2020

[mono] Fix several nits #32529

Merged

akoeplinger mentioned this pull request Mar 9, 2020

Mono rsp files should be removed and replaced by xunit arguments #1980

Closed

ghost locked as resolved and limited conversation to collaborators Dec 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[mono][xunit tests] Move skipped tests out of rsp file #2087

[mono][xunit tests] Move skipped tests out of rsp file #2087

MaximLipnin commented Jan 23, 2020

stephentoub commented Jan 23, 2020

MaximLipnin commented Jan 24, 2020

MaximLipnin commented Jan 24, 2020

MaximLipnin commented Jan 24, 2020

akoeplinger left a comment

akoeplinger Feb 18, 2020

MaximLipnin Feb 19, 2020

akoeplinger Feb 19, 2020

akoeplinger Feb 18, 2020

akoeplinger Feb 18, 2020


		using Xunit;

		[assembly: SkipOnMono("Flaky tests: https://github.com/mono/mono/issues/16417")]

	[assembly: SkipOnMono("Flaky tests: https://github.com/mono/mono/issues/16417")]
	[assembly: ActiveIssue("Flaky tests: https://github.com/mono/mono/issues/16417", TestRuntimes.Mono)]

[mono][xunit tests] Move skipped tests out of rsp file #2087

[mono][xunit tests] Move skipped tests out of rsp file #2087

Conversation

MaximLipnin commented Jan 23, 2020

stephentoub commented Jan 23, 2020

MaximLipnin commented Jan 24, 2020

MaximLipnin commented Jan 24, 2020

MaximLipnin commented Jan 24, 2020

akoeplinger left a comment

Choose a reason for hiding this comment

akoeplinger Feb 18, 2020

Choose a reason for hiding this comment

MaximLipnin Feb 19, 2020

Choose a reason for hiding this comment

akoeplinger Feb 19, 2020

Choose a reason for hiding this comment

akoeplinger Feb 18, 2020

Choose a reason for hiding this comment

akoeplinger Feb 18, 2020

Choose a reason for hiding this comment