Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure: baseservices/TieredCompilation/BasicTestWithMcj/BasicTestWithMcj.cmd #106949

Closed
v-wenyuxu opened this issue Aug 26, 2024 · 5 comments · Fixed by #107064
Closed

Test failure: baseservices/TieredCompilation/BasicTestWithMcj/BasicTestWithMcj.cmd #106949

v-wenyuxu opened this issue Aug 26, 2024 · 5 comments · Fixed by #107064
Labels
area-VM-coreclr blocking-outerloop Blocking the 'runtime-coreclr outerloop' and 'runtime-libraries-coreclr outerloop' runs in-pr There is an active PR which will close this issue when it is merged
Milestone

Comments

@v-wenyuxu
Copy link

Failed in: runtime-coreclr outerloop 20240825.2

Failed tests:

coreclr linux arm Checked no_tiered_compilation @ (Alpine.316.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.17-helix-arm32v7
    - baseservices/TieredCompilation/BasicTestWithMcj/BasicTestWithMcj.cmd

Error message:

 
Assert failure(PID 212 [0x000000d4], Thread: 212 [0x00d4]): Consistency check failed: Crst Level violation: Can't take level 6 lock CrstUnresolvedClassLock because you already holding level 0 lock CrstNativeImageEagerFixups
FAILED: false
    File: /__w/1/s/src/coreclr/vm/crst.cpp:765
    Image: /root/helix/work/correlation/crossgen2/crossgen2

[createdump] The app model does not support crash report generation
[createdump] Failure took 0ms
waitpid() returned successfully (wstatus 00000086) WEXITSTATUS 0 WTERMSIG 6
/root/helix/work/workitem/e/baseservices/baseservices/../TieredCompilation/BasicTestWithMcj/BasicTestWithMcj.sh: line 211:   212 Aborted                 (core dumped) "$CORE_ROOT/crossgen2/crossgen2" --out r2r/BasicTestWithMcj.dll BasicTestWithMcj.dll -r $CORE_ROOT/*.dll
BEGIN: coreclr_initialize failed - Error: 0x80070057
END: coreclr_initialize failed - Error: 0x80070057

Return code:      1
Raw output file:      /root/helix/work/workitem/uploads/TieredCompilation/BasicTestWithMcj/output.txt
Raw output:
BEGIN EXECUTION
[createdump] Target process is alive
./RunBasicTestWithMcj.sh --runCustomTest /root/helix/work/workitem/e/baseservices/TieredCompilation/BasicTestWithMcj/ BasicTestWithMcj.dll ''
Collect profile without R2R, use profile without R2R
Collect profile with R2R, use profile with R2R
Exe path: /root/helix/work/correlation/corerun
Properties:
    TRUSTED_PLATFORM_ASSEMBLIES = /root/helix/work/correlation/System.ServiceModel.Web.dll:/root/helix/work/correlation/System.Composition.Hosting.dll:/root/helix/work/correlation/System.Net.dll:/root/helix/work/correlation/System.ServiceProcess.ServiceController.dll:/root/helix/work/correlation/Microsoft.CodeAnalysis.dll:/root/helix/work/correlation/System.Net.WebSockets.dll:/root/helix/work/correlation/System.Threading.Tasks.dll:/root/helix/work/correlation/System.Net.Primitives.dll:/root/helix/work/correlation/System.DirectoryServices.dll:/root/helix/work/correlation/Microsoft.Diagnostics.Tracing.TraceEvent.dll:/root/helix/work/correlation/Microsoft.Extensions.Options.ConfigurationExtensions.dll:/root/helix/work/correlation/xunit.execution.dotnet.dll:/root/helix/work/correlation/System.Net.Sockets.dll:/root/helix/work/correlation/Microsoft.Extensions.Hosting.Systemd.dll:/root/helix/work/correlation/System.Runtime.Serialization.Json.dll:/root/helix/work/correlation/System.Threading.Channels.dll:/root/helix/work/correlation/System.Diagnostics.DiagnosticSource.dll:/root/helix/work/correlation/Microsoft.Extensions.Configuration.Xml.dll:/root/helix/work/correlation/xunit.runner.utility.netcoreapp10.dll:/root/helix/work/correlation/System.ComponentModel.DataAnnotations.dll:/root/helix/work/correlation/System.Security.Cryptography.Cng.dll:/root/helix/work/correlation/System.Security.AccessControl.dll:/root/helix/work/correlation/System.Formats.Cbor.dll:/root/helix/work/correlation/System.Buffers.dll:/root/helix/work/correlation/System.Runtime.CompilerServices.Unsafe.dll:/root/helix/work/correlation/System.IO.Pipes.dll:/root/helix/work/correlation/System.ServiceProcess.dll:/root/helix/work/correlation/System.Runtime.Extensions.dll:/root/helix/work/correlation/System.Threading.Thread.dll:/root/helix/work/correlation/System.Dynamic.Runtime.dll:/root/helix/work/correlation/WindowsBase.dll:/root/helix/work/correlation/Microsoft.Extensions.Logging.EventSource.dll:/root/helix/work/correlation/Microsoft.Extensions.FileProviders.Physical.dll:/root/helix/work/correlation/System.ComponentModel.Composition.Registration.dll:/root/helix/work/correlation/System.IO.FileSystem.Primitives.dll:/root/helix/work/correlation/System.Collections.dll:/root/helix/work/correlation/System.Security.Principal.Windows.dll:/root/helix/work/correlation/xunit.console.dll:/root/helix/work/correlation/Microsoft.Extensions.FileProviders.Composite.dll:/root/helix/work/correlation/System.Runtime.Serialization.Primitives.dll:/root/helix/work/correlation/System.Configuration.ConfigurationManager.dll:/root/helix/

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 146
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath, String testPathPrefix)
   at Program.<<Main>$>g__TestExecutor23|0_24(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)
@v-wenyuxu v-wenyuxu added arch-arm32 os-linux Linux OS (any supported distro) blocking-outerloop Blocking the 'runtime-coreclr outerloop' and 'runtime-libraries-coreclr outerloop' runs labels Aug 26, 2024
@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Aug 26, 2024
@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Aug 26, 2024
@jkotas
Copy link
Member

jkotas commented Aug 26, 2024

Introduced by #80154?

cc @am11 @jakobbotsch

@dotnet-policy-service dotnet-policy-service bot added the in-pr There is an active PR which will close this issue when it is merged label Aug 26, 2024
@am11
Copy link
Member

am11 commented Aug 27, 2024

@jkotas #106965 (comment), this is failing in main as well:
Screenshot 2024-08-27 at 17 56 07
https://dev.azure.com/dnceng-public/public/_build?definitionId=134&_a=summary

so this was unrelated to #80154 change.

@am11
Copy link
Member

am11 commented Aug 27, 2024

Can't say for sure when it started to happen because old logs are deleted on helix side, but by the number of failures on those jobs, it seems like it's between 1st and 2nd of august: e3684bc...30b34e6, i.e. e3684bc had fewer failures and next run on the next day against 30b34e6 had many.

Copy link
Contributor

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

@jkotas
Copy link
Member

jkotas commented Aug 27, 2024

This was introduced by #105671

We may need to add a new lock dependency to https://github.com/dotnet/runtime/blob/main/src/coreclr/inc/CrstTypes.def

cc @AaronRobinsonMSFT

@AaronRobinsonMSFT AaronRobinsonMSFT added area-crossgen2-coreclr and removed arch-arm32 os-linux Linux OS (any supported distro) untriaged New issue has not been triaged by the area owner needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners area-crossgen2-coreclr labels Aug 27, 2024
@AaronRobinsonMSFT AaronRobinsonMSFT added this to the 10.0.0 milestone Aug 27, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Sep 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-VM-coreclr blocking-outerloop Blocking the 'runtime-coreclr outerloop' and 'runtime-libraries-coreclr outerloop' runs in-pr There is an active PR which will close this issue when it is merged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants
@am11 @jkotas @jeffschwMSFT @AaronRobinsonMSFT @v-wenyuxu and others