Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mono rsp files should be removed and replaced by xunit arguments #1980

Closed
ViktorHofer opened this issue Jan 21, 2020 · 5 comments
Closed

Mono rsp files should be removed and replaced by xunit arguments #1980

ViktorHofer opened this issue Jan 21, 2020 · 5 comments
Assignees
@steveisok
Labels
area-Infrastructure untriaged New issue has not been triaged by the area owner

Comments

@ViktorHofer
Copy link
Member

Tracking issue to replace the mono rsp files with xunit test arguments in libraries.

cc @safern @marek-safar @akoeplinger
@ViktorHofer ViktorHofer self-assigned this Jan 21, 2020
@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added the untriaged New issue has not been triaged by the area owner label Jan 21, 2020
@ViktorHofer ViktorHofer removed their assignment Jan 21, 2020
@ViktorHofer
Copy link
Member Author

Accidentally assigned myself. Removing assignment.

@steveisok steveisok mentioned this issue Jan 22, 2020
Closed
5 tasks
@MaximLipnin MaximLipnin mentioned this issue Jan 23, 2020
Merged
akoeplinger pushed a commit that referenced this issue Feb 18, 2020
@MaximLipnin
[mono][xunit tests] Move skipped tests out of rsp file (#2087)
b74422f 
Addresses #1980

* [xunit tests][mono] Remove disabled Microsoft.VisualBasic.Core.Tests tests from rsp file.

* [xunit tests][mono] Remove System.Collections.Specialized.Tests.Ctor_NegativeCapacity_ThrowsArgumentOutOfRangeException from rsp file

* Remove System.Collections.Concurrent.Tests.ConcurrentQueueTests.ReferenceTypes_NulledAfterDequeue from rsp file because it passes on release mono

* Remove BasicEventSourceTests.TestsManifestNegative.Test_GenerateManifest_InvalidEventSources from rsp file because it passes locally

* Remove ManagedTests.DynamicCSharp.Conformance.dynamic.overloadResolution.Methods.Oneclass2methods.twoprms004.twoprms004.Test.DynamicCSharpRunTest from rsp file

* Remove System.IO.Tests.*.CopyFileWithData_MemberData from rsp file

* Move a set of skipped System.Linq.Expressions.Tests out of rsp file.

* Annotate System.Collections.Concurrent.Tests.ConcurrentQueueTests.ReferenceTypes_NulledAfterDequeue with ActiveIssue attribute

* Move another set of skipped System.Linq.Expressions.Tests out of rsp file.

* Move skipped System.Linq.Parallel.Tests.PlinqModesTests.WithExecutionMode_Multiple test out of rsp file

* Move skipped System.Memory.Tests out of rsp file

* Move skipped System.Net.Http.Functional.Tests.PlatformHandler_HttpClientHandler.* tests out of rdp file

* Remove System.Net.Sockets.Tests.SocketOptionNameTest.MulticastInterface_Set_AnyInterface_Succeeds because it passes locally.

* Move System.Reflection tests with an active issue out of rsp file

* Move skipped System.Runtime.InteropServices.Tests out of rsp file

* Replace closed CoreFX GH issue with the active Runtime one

* Move skipped System.Runtime.Serialization.Formatters.Tests out of rsp file

* Move skipped System.Runtime.Tests with an active issue out of rsp file

* Move skipped System.Threading.Tests with an active issue out of rsp file

* Remove skipped System.Data.Common.Tests because the related issue was addressed

* Move skipped System.Data.Common.Tests.DbConnectionTests.ProviderFactoryTest with an active issue out of rsp file

* Move skipped System.Diagnostics.StackTrace.Tests with an active issue out of rsp file

* Move skipped System.Numerics.Vectors.Tests with an active issue out of rsp file

* Move skipped System.Reflection.Context.Tests with an active issue out of rsp file

* Move skipped System.Threading.ThreadPools.Tests.ThreadPoolTests.SetMinMaxThreadsTest with an active issue out of rsp

* Move skipped System.ComponentModel.Composition and Tests.Integration namespaces with an active issue out of rsp file

* Update a message in skipped System.Runtime.InteropServices.Tests

* Move skipped System.Reflection.Tests.MetadataTokenTests.SuccessImpliesNonNilWithCorrectTable with an active issue out of rsp file

* Move skipped/no repro System.Threading.Overlapped.Tests with an active issue out of rsp file

* Move skipped System.Runtime.Extensions with an active issue out of rsp file

* Re-enable skipped System.Runtime.Handles.Tests because the related issue was addressed

* Move skipped System.Reflection.TypeExtensions.Tests with an active issue out of rsp file

* Move skipped System.Reflection.Emit.Lightweight.Tests with an active issue out of rsp file

* Move skipped System.Reflection.MetadataLoadContext.Tests with an active issue out of rsp file

* Move skipped System.Xml.Xsl.XslTransformApi.Tests with an active issue out of rsp file

* Move skipped System.Security.Cryptography.Encoding.Tests with an active issue out of rsp file

* Move System.Drawing.Imaging.Tests.ImageAttributesTests.SetColorMatrix_InvalidFlags_ThrowsArgumentException out of rsp file

* Add an active issue attribute to System.Drawing.Imaging.Tests.ImageAttributesTests.SetColorMatrix_InvalidFlags_ThrowsArgumentException test; add a comment about libgdiplus dependency to PlatformDetection.IsDrawingSupported method.

* Move System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ClientCertificates_Test.AutomaticOrManual_DoesntFailRegardlessOfWhetherClientCertsAreAvailable out of rsp because it has an active corefx issue; see https://github.com/dotnet/corefx/issues/37336

* Remove System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_DangerousAcceptAllCertificatesValidator_Test.SetDelegate_ConnectionSucceeds from rsp because it's already disabled as part of system.net.* tests; see #2318

* Move skipped System.Reflection.Emit.ILGeneration.Tests with an active issue out of rsp file

* Move skipped System.Reflection.Emit.Tests with an active issue out of rsp file

* Remove skipped System.Reflection.Tests from rsp file because they pass locally

* Try to re-enabled skipped System.Reflection.Tests.AssemblyTests.LoadFromStream_Location_IsEmpty

* Remove broken namespace

* Add an active issue to System.Reflection.Tests.AssemblyTests.LoadFromStream_Location_IsEmpty

* Add an active issue to several System.Reflection tests

* Remove skipped System.Tests.ExitCodeTests.SigTermExitCode because it passes locally.

* Remove skipped System.Tests.Types.VoidTests.IsByRef_Get_ReturnsExpected from rsp file because it passes locally

* Remove skipped System.Tests.GCTests.LatencyRoundtrips from rsp file because it passes locally

* Remove skipped System.Tests.GCExtendedTests.GetGCMemoryInfo from rsp-file because the related functionality has been implemented (see mono/mono#15236)

* Remove skipped System.Tests.GetEnvironmentVariable.EnvironmentVariablesAreHashtable from rsp file because it passes locally

* Turn System.Threading.Tasks.Tests.ExecutionContextFlowTest.TaskCompletionSourceDoesntCaptureExecutionContext into a conditional theory because it requires precise GC

* Try to re-enable System.Threading.Tasks.Tests.TaskContinueWithTests.LongContinuationChain_ContinueWith_DoesNotStackOverflow

* Remove skipped System.Threading.ThreadPools.Tests.ThreadPoolTests.SetMinThreadsTo0Test from rsp file because it passes locally

* Remove System.Threading.ThreadPools.Tests.ThreadPoolTests.SetMinMaxThreadsTest_ChangedInDotNetCore from rsp file because it's explicitly skipped on Mono using SkipOnTargetFramework attribute

* Add an active issue to System.Tests.ExitCodeTests.SigTermExitCode and System.Tests.GCTests.LatencyRoundtrips tests

* Remove skipped System.Data.Tests.SqlTypes.SqlCharsTest.ReadWriteXmlTest from rsp file because it passes locally

* Add an active issue to System.Tests.GCExtendedTests.GetGCMemoryInfo because it fails on helix

* Add an active issue

* Add an active issue

* Re-add an active issue to System.Tests.ExceptionTests.ThrowStatementDoesNotResetExceptionStackLineOtherMethod

* Move skipped System.ComponentModel.Composition.MetadataViewProviderTests.GetMetadataView_InterfaceWithIndexer_ShouldThrowNotSupportedException with an active issue out of rsp file (see mono/mono#15169)

* Remove skipped Tests.Integration.ExportFactoryTests.ExportFactoryStandardImports_ShouldWorkProperly from rsp file because it's already disabled within the entire Tests.Integration.ExportFactoryTests class (see mono/mono#16417)

* Remove several skipped System.ComponentModel.Composition.Tests from rsp file because they pass locally.

* Move skipped System.ComponentModel.Composition.ExportCollectionTests.ImportCollectionsFromContainerOnly with an active issue out of rsp file

* Move skipped System.Diagnostics.Tests.StackTraceTests.Ctor_Exception_SkipFrames and System.Diagnostics.Tests.StackTraceTests.Ctor_Exception_SkipFrames_FNeedFileInfo tests with an active issue out of rsp file.

* Move skipped System.Diagnostics.Tests.StackTraceTests.ToString_Invoke_ReturnsExpected with an active issue out of rsp file

* Remove skipped System.Reflection.Tests.MetadataLoadContextTests.RelocatableAssembly from rsp file because it passes locally.

* Move skipped System.Runtime.Loader.Tests.RefEmitLoadContextTests.LoadRefEmitAssembly with an active issue out of rsp file

* Moved skipped ThreadPoolBoundHandleTests.PreAllocatedOverlapped_NullAsCallback_ThrowsArgumentNullException with an active issue out of rsp file (see mono/mono#15313)

* Move skipped System.Runtime.Tests.ProfileOptimizationTest.ProfileOptimization_CheckFileExists with an active issue out of rsp file

* Move skipped System.Reflection.Emit.Tests.DynamicILInfoTests with an active issue out of rsp file

* Move skipped System.Reflection.Emit.Tests.DynamicILInfoTests.SetX_NullInput_ThrowsArgumentNullException and System.Reflection.Emit.Tests.DynamicMethodctor1.InvalidOwner_ThrowsArgumentException with an active issue out of rsp file

* Add an active issue to System.Threading.Tasks.Tests.TaskContinueWithTests.LongContinuationChain_ContinueWith_DoesNotStackOverflow because it's flaky

* Add an active issue to Tests.Integration.DiscoveryTests.DiscoverAddinsWithCombinedCustomExportAndMetadataAttribute

* Add an active issue to System.Threading.ThreadPools.Tests.ThreadPoolTests.SetMinMaxThreadsTest_ChangedInDotNetCore

* Add an active issue to System.ComponentModel.Composition.MetadataAttributeTests.StronglyTypedStructureTestWithTransparentViews

* Replace TargetFrameworkMonikers.Mono with TestRuntimes.Mono

* Remove redundant space

* Remove all -notrait category

* Add an active issue to System.ComponentModel.Composition.MetadataAttributeTests.StronglyTypedStructureTest

* Add an active issue to System.ComponentModel.Composition.CompositionContainerTests.GetExportOfTTMetadataView1_TypeAsMetadataViewTypeArgument_IsUsedAsMetadataConstraint

* Disable the whole System.ComponentModel.Composition.Tests namespace

* Move some System.Runtime.Loader.Tests, which rely on collectible AssemblyLoadContext, out of rsp file

* Remove three System.Runtime.Loader.Tests.AssemblyLoadContextTest tests with no local failures from rsp file

* Remove System.Runtime.Loader.Tests.DefaultLoadContextTests.LoadNonExistentInDefaultContext and System.Runtime.Loader.Tests.DefaultLoadContextTests.LoadInDefaultContext from rsp file because the related issues should be addressed

* Add an active issue to System.Runtime.Loader.Tests.DefaultLoadContextTests.LoadNonExistentInDefaultContext and System.Runtime.Loader.Tests.DefaultLoadContextTests.LoadInDefaultContext because they fail on CI

* Move several System.Runtime.Loader.Tests.AssemblyLoadContextTest with an active issues out of rsp file

* Remove System.Utf8String.Experimental.Tests from rsp file to verify how they behave on CI

* Clean up rsp file to check all remaining failures

* Skip the entire System.ComponentModel.Composition.Registration.Tests namespace on Mono

* Disable a set of classes in System.Utf8String.Experimental.Tests workspace.

* Add stubs for Utf8String and Utf8Span.cs classes to fix the test build on Mono

* Remove nint/nunit aliases unused in PNSE version of Utf8Span

* Remove CoreFX.issues.rsp file because it has been cleaned up
This was referenced Feb 20, 2020
Merged
Merged
@ViktorHofer
Copy link
Member Author

@akoeplinger @steveisok what's the status on this? I would really want to remove the rsp file as it bloats non Mono test builds as well.

@akoeplinger
Copy link
Member

We removed the main .rsp file with #2087, the macOS one with #32585 and Windows is in progress here: #32592.

The remaining .rsp files are just a couple lines so there's no "bloat" :)

@ViktorHofer
Copy link
Member Author

ViktorHofer commented Mar 9, 2020 •
edited
Loading

Thanks for the update. Is somebody working on removing the remaining ones: linux, interpreter and interpreter_arm64?

The remaining .rsp files are just a couple lines so there's no "bloat" :)

I wasn't just talking about file size bloat here but also about infrastructure:

runtime/eng/testing/tests.targets

Lines 79 to 90 in 5aff3be

<!-- Generate Mono .rsp file -->
<!-- TODO: remove when rsp files are removed: https://github.com/dotnet/runtime/issues/1980. -->
<ItemGroup>
<_rspFile Include="$(TestRspFile)" Condition="'$(TestRspFile)' != ''" />
<_rspFile Include="$(MonoProjectRoot)netcore\CoreFX.issues_linux.rsp" Condition="'$(TargetOS)' == 'Linux'" />
<_rspFile Include="$(MonoProjectRoot)netcore\CoreFX.issues_windows.rsp" Condition="'$(TargetOS)' == 'Windows_NT'" />
</ItemGroup>
<ItemGroup>
<_rspFileContent Include="$([System.IO.File]::ReadAllText(%(_rspFile.Identity)))" Condition="'$(TargetOS)' != 'OSX'" />
</ItemGroup>
<WriteLinesToFile File="$(MonoRspFile)" Lines="@(_rspFileContent)" Overwrite="true" />
. This is currently running always, as we share built libraries test assemblies between coreclr and mono.

@steveisok
Copy link
Member

Yeah, @MaximLipnin is working on removing the rest.

@MaximLipnin MaximLipnin mentioned this issue Mar 23, 2020
Merged
@ViktorHofer ViktorHofer mentioned this issue Mar 25, 2020
Merged
@monojenkins monojenkins mentioned this issue Mar 30, 2020
Closed
monojenkins pushed a commit to monojenkins/mono that referenced this issue Mar 30, 2020
@MaximLipnin
[mono] Remove CoreFX.issues_windows.rsp file
a416a73 
Part of dotnet/runtime#1980
@ghost ghost locked as resolved and limited conversation to collaborators Dec 11, 2020
ViktorHofer pushed a commit to dotnet/winforms that referenced this issue Dec 5, 2022
@MaximLipnin
[mono] Remove CoreFX.issues_windows.rsp file (dotnet/runtime#32592)
f0dec44 
Changes:
- enabled `runtime (Libraries Test Run release mono Windows_NT x64 Debug)` CI lane (Windows Nano Server CI leg is disabled);
- enabled the tests that pass;
- marked failing test assemblies/classes/methods with `ActiveIssue` attribute using respective GH issues;
- removed CoreFX.issues_windows.rsp file and related mono rsp arguments.

Fixes dotnet/runtime#1980

Commit migrated from dotnet/runtime@ab6e225
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@steveisok steveisok

Labels
area-Infrastructure untriaged New issue has not been triaged by the area owner
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@steveisok @akoeplinger @ViktorHofer @Dotnet-GitSync-Bot