Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure baseservices/exceptions/unhandled/unhandledTester/unhandledTester.cmd #96114

Closed
v-wenyuxu opened this issue Dec 18, 2023 · 8 comments
Closed

Test failure baseservices/exceptions/unhandled/unhandledTester/unhandledTester.cmd #96114

v-wenyuxu opened this issue Dec 18, 2023 · 8 comments
Labels
arch-arm64 area-crossgen2-coreclr blocking-clean-ci-optional Blocking optional rolling runs os-mac-os-x macOS aka OSX

Comments

@v-wenyuxu
Copy link

Failed in: runtime-coreclr crossgen2 20231216.1

Failed tests:

R2R-CG2 osx arm64 Checked no_tiered_compilation @ OSX.1200.ARM64.Open
    - baseservices/exceptions/unhandled/unhandledTester/unhandledTester.cmd

Error message:

/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/baseservices-exceptions/../unhandled/unhandledTester/unhandledTester.sh: line 315: -r:/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/IL-CG2/*.dll: No such file or directory

cmdLine:/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/baseservices-exceptions/../unhandled/unhandledTester/unhandledTester.sh Timed Out (timeout in milliseconds: 1800000 from variable __TestTimeout, start: 12/16/2023 11:12:37 PM, end: 12/16/2023 11:42:37 PM)

Return code:      -100
Raw output file:      /tmp/helix/working/B1E20958/w/A92E0955/uploads/unhandled/unhandledTester/output.txt
Raw output:
BEGIN EXECUTION
in takeLock
/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/IL-CG2/TestLibrary.dll
23:12:37
Response file: /private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/TestLibrary.dll.rsp
/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/IL-CG2/TestLibrary.dll
-o:/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/TestLibrary.dll
-r:/tmp/helix/working/B1E20958/p/System.*.dll
-r:/tmp/helix/working/B1E20958/p/Microsoft.*.dll
-r:/tmp/helix/working/B1E20958/p/xunit.*.dll
-r:/tmp/helix/working/B1E20958/p/mscorlib.dll
--verify-type-and-field-layout
--method-layout:random
--targetarch:arm64
--targetos:osx
Running CrossGen2:  dotnet /tmp/helix/working/B1E20958/p/crossgen2/crossgen2.dll @/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/TestLibrary.dll.rsp  
Emitting R2R PE file: /private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/TestLibrary.dll
Running R2RDump:  dotnet /tmp/helix/working/B1E20958/p/R2RDump/R2RDump.dll --header --sc --in /private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/TestLibrary.dll --out /private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/TestLibrary.dll.r2rdump --val
23:12:38
/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/IL-CG2/unhandled.dll
23:12:38
Response file: /private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/unhandled.dll.rsp
/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/IL-CG2/unhandled.dll
-o:/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/unhandled.dll
-r:/tmp/helix/working/B1E20958/p/System.*.dll
-r:/tmp/helix/working/B1E20958/p/Microsoft.*.dll
-r:/tmp/helix/working/B1E20958/p/xunit.*.dll
-r:/tmp/helix/working/B1E20958/p/mscorlib.dll
--verify-type-and-field-layout
--method-layout:random
--targetarch:arm64
--targetos:osx
Running CrossGen2:  dotnet /tmp/helix/working/B1E20958/p/crossgen2/crossgen2.dll @/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/unhandled.dll.rsp  
Emitting R2R PE file: /private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/unhandled.dll
Running R2RDump:  dotnet /tmp/helix/working/B1E20958/p/R2RDump/R2RDump.dll --header --sc --in /private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/unhandled.dll --out /private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/unhandled.dll.r2rdump --val
23:12:38
/private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/IL-CG2/unhandledTester.dll
23:12:38
Response file: /private/tmp/helix/working/B1E20958/w/A92E0955/e/baseservices/exceptions/unhandled/unhandledTester/unhandledTester.dll.rsp
/private/tmp/helix/working/B1E20958/w/A92E09

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 146
   at Xunit.Assert.True(Boolean condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 128
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor16|0_17(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)
Queued Pipeline Pipeline_Configuration OS Arch Test Outcome Properties BuildReason
2023-11-01T09:14:34.267Z runtime Checked-no_tiered_compilation osx.1200.arm64.open arm64 baseservices/exceptions/unhandled/unhandledTester/unhandledTester.cmd Failed {"Project":"public","System.StageAttempt":"1","Reason":"Schedule","BuildId":"457297","BuildNumber":"20231101.1","System.PhaseName":"run_test_p0_coreclr__osx_arm64_checked","AzurePipelinesTestRunId":"10298760","DefinitionId":"129","architecture":"arm64","DefinitionName":"runtime","System.PhaseAttempt":"1","System.JobId":"53d8f4c3-9599-5751-a535-e5a7f4a4a506","System.JobName":"__default","configuration":"Checked-no_tiered_compilation","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","System.StageName":"Build","operatingSystem":"OSX.1200.ARM64.Open"} Schedule
@v-wenyuxu v-wenyuxu added arch-arm64 os-mac-os-x macOS aka OSX blocking-clean-ci-optional Blocking optional rolling runs labels Dec 18, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Dec 18, 2023
@BruceForstall
Copy link
Member

@dotnet/crossgen-contrib

@janvorli
Copy link
Member

It seems that the issue is caused by the helper process that intentionally crashes with an unhandled exception also triggers core dump creation and it somehow causes the test to timeout. I am adding a new flavor of the unhanded exception test in #96138 and I have added disabling the createdump invocation for the crashing process. I think it should fix this issue.

@JulieLeeMSFT
Copy link
Member

CC @janvorli, it failed again in Pipelines - Runs for runtime-jit-experimental (azure.com)

cmdLine:/private/tmp/helix/working/93D20867/w/9DAE0883/e/baseservices/exceptions/baseservices-exceptions/../unhandled/unhandledTester/unhandledTester.sh Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 3/10/2024 8:01:51 PM, end: 3/10/2024 8:11:51 PM)
Printing stacktrace from '/cores/crashdump_4294.dmp.crashreport.json'
Invoking llvm-symbolizer --pretty-print
Errors while running llvm-symbolizer --pretty-print
System.ComponentModel.Win32Exception (2): An error occurred trying to start process 'llvm-symbolizer' with working directory '/private/tmp/helix/working/93D20867/w/9DAE0883/e/baseservices/exceptions/baseservices-exceptions'. No such file or directory
   at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs:line 496
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs:line 456
   at CoreclrTestLib.CoreclrTestWrapperLib.TryPrintStackTraceFromCrashReport(String crashReportJsonFile, TextWriter outputWriter)
Test Harness Exitcode is : -100
To run the test:
Set up CORE_ROOT and run.
> /private/tmp/helix/working/93D20867/w/9DAE0883/e/baseservices/exceptions/baseservices-exceptions/../unhandled/unhandledTester/unhandledTester.sh
   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 146
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor135|0_136(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)
20:14:22.674 Failed test: baseservices/exceptions/unhandled/unhandledTester/unhandledTester.cmd
20:14:23.117 Running test: baseservices/exceptions/unittests/Baadbaad/Baadbaad.cmd

@v-wenyuxu
Copy link
Author

Failed in: runtime-coreclr jitstress 20240617.2

Failed tests:

coreclr windows x86 Checked tailcallstress @ Windows.10.Amd64.Open
    - GC\\Scenarios\\FinalizeTimeout\\FinalizeTimeout\\FinalizeTimeout.cmd

Error message:

 
cmdLine:C:\h\w\A49808EC\w\A4EA08C7\e\GC\Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.cmd Timed Out (timeout in milliseconds: 1800000 from variable __TestTimeout, start: 6/18/2024 5:58:17 AM, end: 6/18/2024 6:28:17 AM)

Return code:      -100
Raw output file:      C:\h\w\A49808EC\w\A4EA08C7\uploads\Reports\GC.Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.output.txt
Raw output:
BEGIN EXECUTION
 "C:\h\w\A49808EC\p\corerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  FinalizeTimeout.dll 
Main start
Finalizer start
Finalizer end
Main end

cmdLine:C:\h\w\A49808EC\w\A4EA08C7\e\GC\Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.cmd Timed Out (timeout in milliseconds: 1800000 from variable __TestTimeout, start: 6/18/2024 5:58:17 AM, end: 6/18/2024 6:28:17 AM)
Test Harness Exitcode is : -100
To run the test:
> set CORE_ROOT=C:\h\w\A49808EC\p
> C:\h\w\A49808EC\w\A4EA08C7\e\GC\Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.cmd

Stack trace:

   at GC_Scenarios._FinalizeTimeout_FinalizeTimeout_FinalizeTimeout_._FinalizeTimeout_FinalizeTimeout_FinalizeTimeout_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

@v-wenyuxu
Copy link
Author

Failed in: runtime-coreclr outerloop 20240619.3

Failed tests:

coreclr windows x86 Checked @ Windows.10.Amd64.Open
    - GC\\Scenarios\\FinalizeTimeout\\FinalizeTimeout\\FinalizeTimeout.cmd

Error message:

 
cmdLine:C:\h\w\ACB90932\w\BCDE0A0D\e\GC\Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.cmd Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 6/19/2024 8:00:16 PM, end: 6/19/2024 8:10:16 PM)

Return code:      -100
Raw output file:      C:\h\w\ACB90932\w\BCDE0A0D\uploads\Reports\GC.Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.output.txt
Raw output:
BEGIN EXECUTION
 "C:\h\w\ACB90932\p\corerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  FinalizeTimeout.dll 
Main start
Finalizer start
Finalizer end
Main end

cmdLine:C:\h\w\ACB90932\w\BCDE0A0D\e\GC\Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.cmd Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 6/19/2024 8:00:16 PM, end: 6/19/2024 8:10:16 PM)
Test Harness Exitcode is : -100
To run the test:
> set CORE_ROOT=C:\h\w\ACB90932\p
> C:\h\w\ACB90932\w\BCDE0A0D\e\GC\Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.cmd

Stack trace:

   at GC_Scenarios._FinalizeTimeout_FinalizeTimeout_FinalizeTimeout_._FinalizeTimeout_FinalizeTimeout_FinalizeTimeout_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

@v-wenyuxu
Copy link
Author

Failed in: runtime-coreclr jitstress 20240619.1

Failed tests:

coreclr windows arm64 Checked jitstress2 @ Windows.11.Arm64.Open
    - GC\\Scenarios\\FinalizeTimeout\\FinalizeTimeout\\FinalizeTimeout.cmd
coreclr windows arm64 Checked tailcallstress @ Windows.11.Arm64.Open
    - GC\\Scenarios\\FinalizeTimeout\\FinalizeTimeout\\FinalizeTimeout.cmd

Error message:

 
cmdLine:C:\h\w\A8BB0904\w\B3BB0945\e\GC\Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.cmd Timed Out (timeout in milliseconds: 1800000 from variable __TestTimeout, start: 6/20/2024 5:56:22 AM, end: 6/20/2024 6:26:22 AM)

Return code:      -100
Raw output file:      C:\h\w\A8BB0904\w\B3BB0945\uploads\Reports\GC.Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.output.txt
Raw output:
BEGIN EXECUTION
 "C:\h\w\A8BB0904\p\corerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  FinalizeTimeout.dll 
Main start
Finalizer start
Finalizer end
Main end

cmdLine:C:\h\w\A8BB0904\w\B3BB0945\e\GC\Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.cmd Timed Out (timeout in milliseconds: 1800000 from variable __TestTimeout, start: 6/20/2024 5:56:22 AM, end: 6/20/2024 6:26:22 AM)
Test Harness Exitcode is : -100
To run the test:
> set CORE_ROOT=C:\h\w\A8BB0904\p
> C:\h\w\A8BB0904\w\B3BB0945\e\GC\Scenarios\FinalizeTimeout\FinalizeTimeout\FinalizeTimeout.cmd

Stack trace:

   at GC_Scenarios._FinalizeTimeout_FinalizeTimeout_FinalizeTimeout_._FinalizeTimeout_FinalizeTimeout_FinalizeTimeout_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

@steveisok
Copy link
Member

@mangod9 is this more a GC issue?

@jkotas
Copy link
Member

jkotas commented Jul 2, 2024

There are multiple unrelated failures logged above:

@jkotas jkotas closed this as completed Jul 2, 2024
@dotnet-policy-service dotnet-policy-service bot removed the untriaged New issue has not been triaged by the area owner label Jul 2, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Aug 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
arch-arm64 area-crossgen2-coreclr blocking-clean-ci-optional Blocking optional rolling runs os-mac-os-x macOS aka OSX
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@steveisok @jkotas @janvorli @BruceForstall @JulieLeeMSFT @v-wenyuxu