Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd #85234

Closed
v-wenyuxu opened this issue Apr 24, 2023 · 16 comments · Fixed by #97109
Closed

Test failure Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd #85234

v-wenyuxu opened this issue Apr 24, 2023 · 16 comments · Fixed by #97109
Assignees
Labels
area-GC-coreclr disabled-test The test is disabled in source code against the issue os-windows
Milestone

Comments

@v-wenyuxu
Copy link

Failed in: runtime-coreclr gcstress-extra 20230423.1

Failed tests:

coreclr windows x64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x86 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x86 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd

Error message:

 
Assert failure(PID 3424 [0x00000d60], Thread: 1892 [0x0764]): !"Fatal Error in HandleTable."

CORECLR! BlockVerifyAgeMapForBlocks + 0x12C (0x00007fff`4507d98c)
CORECLR! TableScanHandles + 0x24D (0x00007fff`4507e15d)
CORECLR! HndVerifyTable + 0xE2 (0x00007fff`4507b4e2)
CORECLR! Ref_VerifyHandleTable + 0x11F (0x00007fff`45081ecf)
CORECLR! WKS::gc_heap::verify_heap + 0xD2A (0x00007fff`4510333a)
CORECLR! WKS::gc_heap::gc1 + 0x95F (0x00007fff`450e531f)
CORECLR! WKS::gc_heap::garbage_collect + 0x74A (0x00007fff`450e495a)
CORECLR! WKS::GCHeap::GarbageCollectGeneration + 0x452 (0x00007fff`450cc592)
CORECLR! WKS::GCHeap::GarbageCollect + 0x246 (0x00007fff`450cc0c6)
CORECLR! WKS::GCHeap::StressHeap + 0x5F0 (0x00007fff`450cf4f0)
    File: D:\a\_work\1\s\src\coreclr\gc\handletablescan.cpp Line: 929
    Image: C:\h\w\BCAA0A12\p\corerun.exe


Return code:      1
Raw output file:      C:\h\w\BCAA0A12\w\A2980904\uploads\Reports\Interop.MarshalAPI\IUnknown\IUnknownTest\IUnknownTest.output.txt
Raw output:
BEGIN EXECUTION
 "C:\h\w\BCAA0A12\p\corerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  IUnknownTest.dll 
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:\cores since 4/23/2023 11:15:48 PM
Processing C:\cores\corerun.exe.3424.dmp
Invoking: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe -c "$<C:\h\w\BCAA0A12\t\tmpscz23x.tmp" -z "C:\cores\corerun.exe.3424.dmp"
stdout: 
Microsoft (R) Windows Debugger Version 10.0.18362.1 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\cores\corerun.exe.3424.dmp]
User Mini Dump File with Full Memory: Only application data is available


************* Path validation summary **************
Response                         Time (ms)     Location
OK                                             C:\h\w\BCAA0A12\p\PDB
Symbol search path is: C:\h\w\BCAA0A12\p\PDB
Executable search path is: 
Windows 10 Version 14393 MP (4 procs) Free x64
Product: Server, suite: TerminalServer DataCenter SingleUserTS
10.0.14393.5125 (rs1_release.220429-1732)
Machine Name:
Debug session time: Sun Apr 23 23:15:52.000 2023 (UTC + 0:00)
System Uptime: 0 days 0:42:04.209
Process Uptime: 0 days 0:00:04.000
...............................

************* Symbol Loading Error Summary **************
Module name            Error
ntdll                  The system cannot find the file specified

You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded.
You should also verify that your symbol search path (.sympath) is correct.
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(d60.764): Unknown exception - code c0000602 (first/second chance not available)
For analysis of this file, run !analyze -v
*** WARNING: Unable to verify checksum for coreclr.dll
KERNELBASE!RaiseFailFastException+0xae:
00007fff`72bb98ce 803d4d76120000  cmp     byte ptr [KERNELBASE!local_unwind+0x7fd2 (00007fff`72ce0f22)],0 ds:00007fff`72ce0f22=00
0:006> cdb: Reading initial command '$<C:\h\w\BCAA0A12\t\tmpscz23x.tmp'
0:006> .load C:\Users\runner\.dotnet\sos\sos.dll
0:006> ~*k

   0  Id: d60.f48 Suspend: 0 Teb: 00000078`8d824000 Unfrozen
Child-SP          RetAddr           Call Site
00000078`8db7dd08 00007fff`72b34d7f ntdll!ZwWaitForMultipleObjects+0x14
00000078`8db7dd10 00007fff`44c55a76 KERNELBASE!WaitForMultipleObjectsEx+0xef
00000078`8db7e010 00007fff`44c56023 coreclr!Thread::DoAppropriateAptStateWait+0x116
00000078`8db7e0e0 00007fff`44c5fc5d coreclr!Thread::DoAppropriateWaitWorker+0x3b3
00000078`8db7e330 00007fff`44c55bb6 coreclr!`Thread::DoAppropriateWait'::`11'::__Body::Run+0x9d
00000078`8db7e390 00007

Stack trace:

    at Interop_MarshalAPI._IUnknown_IUnknownTest_IUnknownTest_._IUnknown_IUnknownTest_IUnknownTest_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Apr 24, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Apr 24, 2023
@kunalspathak kunalspathak added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI blocking-clean-ci-optional Blocking optional rolling runs labels Apr 24, 2023
@ghost
Copy link

ghost commented Apr 24, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Failed in: runtime-coreclr gcstress-extra 20230423.1

Failed tests:

coreclr windows x64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x86 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x86 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd

Error message:

 
Assert failure(PID 3424 [0x00000d60], Thread: 1892 [0x0764]): !"Fatal Error in HandleTable."

CORECLR! BlockVerifyAgeMapForBlocks + 0x12C (0x00007fff`4507d98c)
CORECLR! TableScanHandles + 0x24D (0x00007fff`4507e15d)
CORECLR! HndVerifyTable + 0xE2 (0x00007fff`4507b4e2)
CORECLR! Ref_VerifyHandleTable + 0x11F (0x00007fff`45081ecf)
CORECLR! WKS::gc_heap::verify_heap + 0xD2A (0x00007fff`4510333a)
CORECLR! WKS::gc_heap::gc1 + 0x95F (0x00007fff`450e531f)
CORECLR! WKS::gc_heap::garbage_collect + 0x74A (0x00007fff`450e495a)
CORECLR! WKS::GCHeap::GarbageCollectGeneration + 0x452 (0x00007fff`450cc592)
CORECLR! WKS::GCHeap::GarbageCollect + 0x246 (0x00007fff`450cc0c6)
CORECLR! WKS::GCHeap::StressHeap + 0x5F0 (0x00007fff`450cf4f0)
    File: D:\a\_work\1\s\src\coreclr\gc\handletablescan.cpp Line: 929
    Image: C:\h\w\BCAA0A12\p\corerun.exe


Return code:      1
Raw output file:      C:\h\w\BCAA0A12\w\A2980904\uploads\Reports\Interop.MarshalAPI\IUnknown\IUnknownTest\IUnknownTest.output.txt
Raw output:
BEGIN EXECUTION
 "C:\h\w\BCAA0A12\p\corerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  IUnknownTest.dll 
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:\cores since 4/23/2023 11:15:48 PM
Processing C:\cores\corerun.exe.3424.dmp
Invoking: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe -c "$<C:\h\w\BCAA0A12\t\tmpscz23x.tmp" -z "C:\cores\corerun.exe.3424.dmp"
stdout: 
Microsoft (R) Windows Debugger Version 10.0.18362.1 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\cores\corerun.exe.3424.dmp]
User Mini Dump File with Full Memory: Only application data is available


************* Path validation summary **************
Response                         Time (ms)     Location
OK                                             C:\h\w\BCAA0A12\p\PDB
Symbol search path is: C:\h\w\BCAA0A12\p\PDB
Executable search path is: 
Windows 10 Version 14393 MP (4 procs) Free x64
Product: Server, suite: TerminalServer DataCenter SingleUserTS
10.0.14393.5125 (rs1_release.220429-1732)
Machine Name:
Debug session time: Sun Apr 23 23:15:52.000 2023 (UTC + 0:00)
System Uptime: 0 days 0:42:04.209
Process Uptime: 0 days 0:00:04.000
...............................

************* Symbol Loading Error Summary **************
Module name            Error
ntdll                  The system cannot find the file specified

You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded.
You should also verify that your symbol search path (.sympath) is correct.
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(d60.764): Unknown exception - code c0000602 (first/second chance not available)
For analysis of this file, run !analyze -v
*** WARNING: Unable to verify checksum for coreclr.dll
KERNELBASE!RaiseFailFastException+0xae:
00007fff`72bb98ce 803d4d76120000  cmp     byte ptr [KERNELBASE!local_unwind+0x7fd2 (00007fff`72ce0f22)],0 ds:00007fff`72ce0f22=00
0:006> cdb: Reading initial command '$<C:\h\w\BCAA0A12\t\tmpscz23x.tmp'
0:006> .load C:\Users\runner\.dotnet\sos\sos.dll
0:006> ~*k

   0  Id: d60.f48 Suspend: 0 Teb: 00000078`8d824000 Unfrozen
Child-SP          RetAddr           Call Site
00000078`8db7dd08 00007fff`72b34d7f ntdll!ZwWaitForMultipleObjects+0x14
00000078`8db7dd10 00007fff`44c55a76 KERNELBASE!WaitForMultipleObjectsEx+0xef
00000078`8db7e010 00007fff`44c56023 coreclr!Thread::DoAppropriateAptStateWait+0x116
00000078`8db7e0e0 00007fff`44c5fc5d coreclr!Thread::DoAppropriateWaitWorker+0x3b3
00000078`8db7e330 00007fff`44c55bb6 coreclr!`Thread::DoAppropriateWait'::`11'::__Body::Run+0x9d
00000078`8db7e390 00007

Stack trace:

    at Interop_MarshalAPI._IUnknown_IUnknownTest_IUnknownTest_._IUnknown_IUnknownTest_IUnknownTest_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
Author: v-wenyuxu
Assignees: -
Labels:

arch-arm64, arch-x86, os-windows, GCStress, arch-x64, area-CodeGen-coreclr, untriaged, blocking-clean-ci-optional, needs-area-label

Milestone: -

@vcsjones vcsjones removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Apr 24, 2023
@JulieLeeMSFT JulieLeeMSFT removed the untriaged New issue has not been triaged by the area owner label May 1, 2023
@JulieLeeMSFT JulieLeeMSFT added this to the 8.0.0 milestone May 1, 2023
@JulieLeeMSFT
Copy link
Member

@markples, PTAL GC stress test failure.

@v-wenyuxu
Copy link
Author

Failed again in: runtime-coreclr gcstress-extra 20230430.1

Failed tests:

coreclr windows x64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd

Error message:

 
Assert failure(PID 1164 [0x0000048c], Thread: 1436 [0x059c]): !"Fatal Error in HandleTable."

CORECLR! BlockVerifyAgeMapForBlocks + 0x11C (0x00007ffb`403b95dc)
CORECLR! TableScanHandles + 0x274 (0x00007ffb`403b9c04)
CORECLR! HndVerifyTable + 0xB8 (0x00007ffb`403b7398)
CORECLR! Ref_VerifyHandleTable + 0x100 (0x00007ffb`403bdac8)
CORECLR! WKS::gc_heap::verify_heap + 0xCA8 (0x00007ffb`4043fe98)
CORECLR! WKS::gc_heap::gc1 + 0xA38 (0x00007ffb`40421710)
CORECLR! WKS::gc_heap::garbage_collect + 0x67C (0x00007ffb`40420c9c)
CORECLR! WKS::GCHeap::GarbageCollectGeneration + 0x4AC (0x00007ffb`4040883c)
CORECLR! WKS::GCHeap::GarbageCollect + 0x1C4 (0x00007ffb`40408314)
CORECLR! WKS::GCHeap::StressHeap + 0x5C4 (0x00007ffb`4040b404)
    File: D:\a\_work\1\s\src\coreclr\gc\handletablescan.cpp Line: 929
    Image: C:\h\w\BD140A06\p\corerun.exe


Return code:      1
Raw output file:      C:\h\w\BD140A06\w\B5ED09F3\uploads\Reports\Interop.MarshalAPI\IUnknown\IUnknownTest\IUnknownTest.output.txt
Raw output:
BEGIN EXECUTION
 "C:\h\w\BD140A06\p\corerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  IUnknownTest.dll 
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:\cores since 4/30/2023 10:57:46 PM
Processing C:\cores\corerun.exe.1164.dmp
Unable to find cdb.exe at C:\Program Files (x86)\Windows Kits\10\Debuggers\arm64\cdb.exe
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=C:\h\w\BD140A06\p
> C:\h\w\BD140A06\w\B5ED09F3\e\Interop\MarshalAPI\IUnknown\IUnknownTest\IUnknownTest.cmd
Expected: True
Actual:   False

Stack trace:

    at Interop_MarshalAPI._IUnknown_IUnknownTest_IUnknownTest_._IUnknown_IUnknownTest_IUnknownTest_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

@BruceForstall
Copy link
Member

This test continues to fail in GCStress where DOTNET_HeapVerify=1 is set.

@dotnet/gc Issue with verify_heap?

@dotnet/interop-contrib Failure is in an interop test.

BruceForstall added a commit to BruceForstall/runtime that referenced this issue May 8, 2023
@AaronRobinsonMSFT AaronRobinsonMSFT added the disabled-test The test is disabled in source code against the issue label May 9, 2023
@AaronRobinsonMSFT
Copy link
Member

Let me also rewrite this test. It is very odd and I'm not ever sure it is worth having. I will check it out today, but I signed off on disabling it too.

@AaronRobinsonMSFT
Copy link
Member

This looks like a GC issue. I have a very simple repro and I'm not sure how other COM tests are passing with GCStress at this point. I will update this issue as I uncover more details.

@AaronRobinsonMSFT
Copy link
Member

The following is where the logic is failing.

if (dwClumpMask & GEN_CLUMP_0_MASK)
{
// for each clump, check whether any object is younger than the age indicated by the clump
uint8_t minAge = ((uint8_t *)pSegment->rgGeneration)[uClump];
for ( ; pValue < pLast; pValue++)
{
if (!HndIsNullOrDestroyedHandle(*pValue))
{
VerifyObjectAndAge((*pValue), (*pValue), minAge);

The object being verified is valid according to all tools I know of. The returned minAge value is 0xff in the failing case. This is passed into VerifyObjectAndAge() below.

if (minAge >= GEN_MAX_AGE || (minAge > thisAge && thisAge < static_cast<int>(g_theGCHeap->GetMaxGeneration())))
{
_ASSERTE(!"Fatal Error in HandleTable.");
GCToEEInterface::HandleFatalError(COR_E_EXECUTIONENGINE);
}

The minAge is 0xff and GEN_MAX_AGE is defined as 0x3f so we fail.

#define GEN_MAX_AGE (0x3F)

There is a slew of debug code in VerifyObjectAndAge() that I will uncomment and see if I can understand how we got 0xff for minAge.

/cc @dotnet/gc

@AaronRobinsonMSFT AaronRobinsonMSFT added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI and removed area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI labels May 9, 2023
@ghost
Copy link

ghost commented May 9, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Failed in: runtime-coreclr gcstress-extra 20230423.1

Failed tests:

coreclr windows x64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x86 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x86 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd

Error message:

 
Assert failure(PID 3424 [0x00000d60], Thread: 1892 [0x0764]): !&quot;Fatal Error in HandleTable.&quot;

CORECLR! BlockVerifyAgeMapForBlocks + 0x12C (0x00007fff`4507d98c)
CORECLR! TableScanHandles + 0x24D (0x00007fff`4507e15d)
CORECLR! HndVerifyTable + 0xE2 (0x00007fff`4507b4e2)
CORECLR! Ref_VerifyHandleTable + 0x11F (0x00007fff`45081ecf)
CORECLR! WKS::gc_heap::verify_heap + 0xD2A (0x00007fff`4510333a)
CORECLR! WKS::gc_heap::gc1 + 0x95F (0x00007fff`450e531f)
CORECLR! WKS::gc_heap::garbage_collect + 0x74A (0x00007fff`450e495a)
CORECLR! WKS::GCHeap::GarbageCollectGeneration + 0x452 (0x00007fff`450cc592)
CORECLR! WKS::GCHeap::GarbageCollect + 0x246 (0x00007fff`450cc0c6)
CORECLR! WKS::GCHeap::StressHeap + 0x5F0 (0x00007fff`450cf4f0)
    File: D:\a\_work\1\s\src\coreclr\gc\handletablescan.cpp Line: 929
    Image: C:\h\w\BCAA0A12\p\corerun.exe


Return code:      1
Raw output file:      C:\h\w\BCAA0A12\w\A2980904\uploads\Reports\Interop.MarshalAPI\IUnknown\IUnknownTest\IUnknownTest.output.txt
Raw output:
BEGIN EXECUTION
 &quot;C:\h\w\BCAA0A12\p\corerun.exe&quot; -p &quot;System.Reflection.Metadata.MetadataUpdater.IsSupported=false&quot; -p &quot;System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true&quot;  IUnknownTest.dll 
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:\cores since 4/23/2023 11:15:48 PM
Processing C:\cores\corerun.exe.3424.dmp
Invoking: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe -c &quot;$&lt;C:\h\w\BCAA0A12\t\tmpscz23x.tmp&quot; -z &quot;C:\cores\corerun.exe.3424.dmp&quot;
stdout: 
Microsoft (R) Windows Debugger Version 10.0.18362.1 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\cores\corerun.exe.3424.dmp]
User Mini Dump File with Full Memory: Only application data is available


************* Path validation summary **************
Response                         Time (ms)     Location
OK                                             C:\h\w\BCAA0A12\p\PDB
Symbol search path is: C:\h\w\BCAA0A12\p\PDB
Executable search path is: 
Windows 10 Version 14393 MP (4 procs) Free x64
Product: Server, suite: TerminalServer DataCenter SingleUserTS
10.0.14393.5125 (rs1_release.220429-1732)
Machine Name:
Debug session time: Sun Apr 23 23:15:52.000 2023 (UTC + 0:00)
System Uptime: 0 days 0:42:04.209
Process Uptime: 0 days 0:00:04.000
...............................

************* Symbol Loading Error Summary **************
Module name            Error
ntdll                  The system cannot find the file specified

You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded.
You should also verify that your symbol search path (.sympath) is correct.
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(d60.764): Unknown exception - code c0000602 (first/second chance not available)
For analysis of this file, run !analyze -v
*** WARNING: Unable to verify checksum for coreclr.dll
KERNELBASE!RaiseFailFastException+0xae:
00007fff`72bb98ce 803d4d76120000  cmp     byte ptr [KERNELBASE!local_unwind+0x7fd2 (00007fff`72ce0f22)],0 ds:00007fff`72ce0f22=00
0:006&gt; cdb: Reading initial command &#39;$&lt;C:\h\w\BCAA0A12\t\tmpscz23x.tmp&#39;
0:006&gt; .load C:\Users\runner\.dotnet\sos\sos.dll
0:006&gt; ~*k

   0  Id: d60.f48 Suspend: 0 Teb: 00000078`8d824000 Unfrozen
Child-SP          RetAddr           Call Site
00000078`8db7dd08 00007fff`72b34d7f ntdll!ZwWaitForMultipleObjects+0x14
00000078`8db7dd10 00007fff`44c55a76 KERNELBASE!WaitForMultipleObjectsEx+0xef
00000078`8db7e010 00007fff`44c56023 coreclr!Thread::DoAppropriateAptStateWait+0x116
00000078`8db7e0e0 00007fff`44c5fc5d coreclr!Thread::DoAppropriateWaitWorker+0x3b3
00000078`8db7e330 00007fff`44c55bb6 coreclr!`Thread::DoAppropriateWait&#39;::`11&#39;::__Body::Run+0x9d
00000078`8db7e390 00007

Stack trace:

    at Interop_MarshalAPI._IUnknown_IUnknownTest_IUnknownTest_._IUnknown_IUnknownTest_IUnknownTest_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
Author: v-wenyuxu
Assignees: markples
Labels:

arch-arm64, arch-x86, disabled-test, os-windows, GCStress, arch-x64, area-CodeGen-coreclr, blocking-clean-ci-optional

Milestone: 8.0.0

@BruceForstall BruceForstall added area-GC-coreclr and removed area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI labels May 9, 2023
@ghost
Copy link

ghost commented May 9, 2023

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

Failed in: runtime-coreclr gcstress-extra 20230423.1

Failed tests:

coreclr windows x64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x86 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x86 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows x64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.10.Amd64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_jitminopts_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd
coreclr windows arm64 Checked gcstress0xc_zapdisable_heapverify1 @ Windows.11.Arm64.Open
    - Interop\\MarshalAPI\\IUnknown\\IUnknownTest\\IUnknownTest.cmd

Error message:

 
Assert failure(PID 3424 [0x00000d60], Thread: 1892 [0x0764]): !&quot;Fatal Error in HandleTable.&quot;

CORECLR! BlockVerifyAgeMapForBlocks + 0x12C (0x00007fff`4507d98c)
CORECLR! TableScanHandles + 0x24D (0x00007fff`4507e15d)
CORECLR! HndVerifyTable + 0xE2 (0x00007fff`4507b4e2)
CORECLR! Ref_VerifyHandleTable + 0x11F (0x00007fff`45081ecf)
CORECLR! WKS::gc_heap::verify_heap + 0xD2A (0x00007fff`4510333a)
CORECLR! WKS::gc_heap::gc1 + 0x95F (0x00007fff`450e531f)
CORECLR! WKS::gc_heap::garbage_collect + 0x74A (0x00007fff`450e495a)
CORECLR! WKS::GCHeap::GarbageCollectGeneration + 0x452 (0x00007fff`450cc592)
CORECLR! WKS::GCHeap::GarbageCollect + 0x246 (0x00007fff`450cc0c6)
CORECLR! WKS::GCHeap::StressHeap + 0x5F0 (0x00007fff`450cf4f0)
    File: D:\a\_work\1\s\src\coreclr\gc\handletablescan.cpp Line: 929
    Image: C:\h\w\BCAA0A12\p\corerun.exe


Return code:      1
Raw output file:      C:\h\w\BCAA0A12\w\A2980904\uploads\Reports\Interop.MarshalAPI\IUnknown\IUnknownTest\IUnknownTest.output.txt
Raw output:
BEGIN EXECUTION
 &quot;C:\h\w\BCAA0A12\p\corerun.exe&quot; -p &quot;System.Reflection.Metadata.MetadataUpdater.IsSupported=false&quot; -p &quot;System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true&quot;  IUnknownTest.dll 
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:\cores since 4/23/2023 11:15:48 PM
Processing C:\cores\corerun.exe.3424.dmp
Invoking: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe -c &quot;$&lt;C:\h\w\BCAA0A12\t\tmpscz23x.tmp&quot; -z &quot;C:\cores\corerun.exe.3424.dmp&quot;
stdout: 
Microsoft (R) Windows Debugger Version 10.0.18362.1 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\cores\corerun.exe.3424.dmp]
User Mini Dump File with Full Memory: Only application data is available


************* Path validation summary **************
Response                         Time (ms)     Location
OK                                             C:\h\w\BCAA0A12\p\PDB
Symbol search path is: C:\h\w\BCAA0A12\p\PDB
Executable search path is: 
Windows 10 Version 14393 MP (4 procs) Free x64
Product: Server, suite: TerminalServer DataCenter SingleUserTS
10.0.14393.5125 (rs1_release.220429-1732)
Machine Name:
Debug session time: Sun Apr 23 23:15:52.000 2023 (UTC + 0:00)
System Uptime: 0 days 0:42:04.209
Process Uptime: 0 days 0:00:04.000
...............................

************* Symbol Loading Error Summary **************
Module name            Error
ntdll                  The system cannot find the file specified

You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded.
You should also verify that your symbol search path (.sympath) is correct.
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(d60.764): Unknown exception - code c0000602 (first/second chance not available)
For analysis of this file, run !analyze -v
*** WARNING: Unable to verify checksum for coreclr.dll
KERNELBASE!RaiseFailFastException+0xae:
00007fff`72bb98ce 803d4d76120000  cmp     byte ptr [KERNELBASE!local_unwind+0x7fd2 (00007fff`72ce0f22)],0 ds:00007fff`72ce0f22=00
0:006&gt; cdb: Reading initial command &#39;$&lt;C:\h\w\BCAA0A12\t\tmpscz23x.tmp&#39;
0:006&gt; .load C:\Users\runner\.dotnet\sos\sos.dll
0:006&gt; ~*k

   0  Id: d60.f48 Suspend: 0 Teb: 00000078`8d824000 Unfrozen
Child-SP          RetAddr           Call Site
00000078`8db7dd08 00007fff`72b34d7f ntdll!ZwWaitForMultipleObjects+0x14
00000078`8db7dd10 00007fff`44c55a76 KERNELBASE!WaitForMultipleObjectsEx+0xef
00000078`8db7e010 00007fff`44c56023 coreclr!Thread::DoAppropriateAptStateWait+0x116
00000078`8db7e0e0 00007fff`44c5fc5d coreclr!Thread::DoAppropriateWaitWorker+0x3b3
00000078`8db7e330 00007fff`44c55bb6 coreclr!`Thread::DoAppropriateWait&#39;::`11&#39;::__Body::Run+0x9d
00000078`8db7e390 00007

Stack trace:

    at Interop_MarshalAPI._IUnknown_IUnknownTest_IUnknownTest_._IUnknown_IUnknownTest_IUnknownTest_cmd()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
Author: v-wenyuxu
Assignees: AaronRobinsonMSFT
Labels:

arch-arm64, arch-x86, disabled-test, os-windows, GCStress, arch-x64, area-GC-coreclr, blocking-clean-ci-optional

Milestone: 8.0.0

@BruceForstall BruceForstall removed the blocking-clean-ci-optional Blocking optional rolling runs label May 9, 2023
@AaronRobinsonMSFT
Copy link
Member

After we root cause this test, I'm going to need some help from @dotnet/gc, we can remove this test entirely. All the APIs here are being tested in libraries already.

@EgorBo
Copy link
Member

EgorBo commented May 9, 2023

It should be related to frozen objects for which we report generation (age) as 0x7fffffff (int.MaxValue) so presumably it's just the assert needs to be improved 🤔

@AaronRobinsonMSFT
Copy link
Member

presumably it's just the assert needs to be improved 🤔

I don't think so. If you look above after the assert the runtime is told to record an EE exception (that is,COR_E_EXECUTIONENGINE). I guess the predicate can be update in some manner, but I'm not sure the right way to do that. Also, I don't think a uint8_t is sufficient anymore to capture the 0x7fffffff value. Unless I am missing some logic here.

@AaronRobinsonMSFT
Copy link
Member

/cc @cshung

@AaronRobinsonMSFT
Copy link
Member

AaronRobinsonMSFT commented Jan 17, 2024

This issue reproduces reliably on Windows x64 under a Checked runtime.

  1. Build the Interop.csproj test under - src\tests\Interop\Interop.csproj

  2. Set the following environment variables.

    set DOTNET_TieredCompilation=0
    set DOTNET_HeapVerify=1
    
  3. Run the Interop.cmd script for the test.

Assert failure(PID 27064 [0x000069b8], Thread: 34264 [0x85d8]): !"Fatal Error in HandleTable."

CORECLR! BlockVerifyAgeMapForBlocks + 0x12C (0x00007ffc`054152cc)
CORECLR! TableScanHandles + 0x24D (0x00007ffc`05415a9d)
CORECLR! HndVerifyTable + 0xE2 (0x00007ffc`05412e32)
CORECLR! Ref_VerifyHandleTable + 0x168 (0x00007ffc`0541a028)
CORECLR! WKS::gc_heap::verify_heap + 0xFA8 (0x00007ffc`054a4448)
CORECLR! WKS::gc_heap::gc1 + 0x937 (0x00007ffc`054867c7)
CORECLR! WKS::gc_heap::garbage_collect + 0x73A (0x00007ffc`05485e2a)
CORECLR! WKS::GCHeap::GarbageCollectGeneration + 0x4E3 (0x00007ffc`0546bbc3)
CORECLR! WKS::GCHeap::GarbageCollect + 0x1F6 (0x00007ffc`0546b666)
CORECLR! GCInterface_Collect + 0x182 (0x00007ffc`050a53c2)
    File: ...\runtime\src\coreclr\gc\handletablescan.cpp Line: 929
    Image: ...\runtime\artifacts\tests\coreclr\windows.x64.Checked\Tests\Core_Root\corerun.exe

It looks like the predicate in VerifyObjectAndAge() needs to be updated.

static void VerifyObjectAndAge(_UNCHECKED_OBJECTREF from, _UNCHECKED_OBJECTREF obj, uint8_t minAge)
{
VerifyObject(from, obj);
int thisAge = g_theGCHeap->WhichGeneration(obj);
//debugging code
//if (minAge > thisAge && thisAge < g_theGCHeap->GetMaxGeneration())
//{
// if ((*pValue) == obj)
// printf("Handle (age %u) %p -> %p (age %u)", minAge, pValue, obj, thisAge);
// else
// printf("Handle (age %u) %p -> %p -> %p (age %u)", minAge, pValue, from, obj, thisAge);
// // for test programs - if the object is a string, print it
// if (obj->GetGCSafeMethodTable() == g_pStringClass)
// {
// wprintf("'%s'\n", ((StringObject *)obj)->GetBuffer());
// }
// else
// {
// printf("\n");
// }
//}
if (minAge >= GEN_MAX_AGE || (minAge > thisAge && thisAge < static_cast<int>(g_theGCHeap->GetMaxGeneration())))
{
_ASSERTE(!"Fatal Error in HandleTable.");
GCToEEInterface::HandleFatalError(COR_E_EXECUTIONENGINE);
}
}

@AaronRobinsonMSFT
Copy link
Member

Multiple tests will need to be reenabled when this is fixed.

See #96364

@AaronRobinsonMSFT
Copy link
Member

@cshung I am disabling this test in #97118. Please make this a priority to address as the issue appears to be a simple validation check.

@AaronRobinsonMSFT AaronRobinsonMSFT modified the milestones: Future, 9.0.0 Jan 17, 2024
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jan 18, 2024
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jan 20, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Feb 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-GC-coreclr disabled-test The test is disabled in source code against the issue os-windows
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants