Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

System.Net.Sockets.Tests fails on Android #50568

Open
mdh1418 opened this issue Apr 1, 2021 · 5 comments
Open

System.Net.Sockets.Tests fails on Android #50568

mdh1418 opened this issue Apr 1, 2021 · 5 comments
Labels
area-System.Net.Sockets disabled-test The test is disabled in source code against the issue os-android
Milestone
Future

Comments

@mdh1418
Copy link
Member

mdh1418 commented Apr 1, 2021 •
edited
Loading

Several tests from System.Net.Sockets.Tests fail on Android

System.Net.Sockets.Tests.dll Failed: 10

System.Net.Sockets.Tests.SendReceive_SpanSync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
System.Net.Sockets.Tests.SendReceive_SpanSync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
System.Net.Sockets.Tests.UnixDomainSocketTest.UnixDomainSocketEndPoint_UsingAbstractSocketAddressOnUnsupported_Throws
System.Net.Sockets.Tests.SendReceive_SyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
System.Net.Sockets.Tests.SendReceive_SyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
System.Net.Sockets.Tests.OSSupportTest.IOControl_SIOCATMARK_Unix_Success
System.Net.Sockets.Tests.SendReceive_Sync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
System.Net.Sockets.Tests.SendReceive_Sync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
System.Net.Sockets.Tests.SendReceive_SpanSyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
System.Net.Sockets.Tests.SendReceive_SpanSyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)

Updated based on local run as of 349d811
@mdh1418 mdh1418 added this to the 6.0.0 milestone Apr 1, 2021
@ghost
Copy link

ghost commented Apr 1, 2021

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Several tests from System.Net.Sockets.Tests fail on Android

  • System.Net.Sockets.Tests.SendReceive_SyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
  • System.Net.Sockets.Tests.SendReceive_SyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
  • System.Net.Sockets.Tests.UnixDomainSocketTest.UnixDomainSocketEndPoint_UsingAbstractSocketAddressOnUnsupported_Throws
  • System.Net.Sockets.Tests.OSSupportTest.IOControl_SIOCATMARK_Unix_Success
  • System.Net.Sockets.Tests.SendReceive_SpanSyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
  • System.Net.Sockets.Tests.SendReceive_SpanSyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
  • System.Net.Sockets.Tests.SendReceive_SpanSync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
  • System.Net.Sockets.Tests.SendReceive_SpanSync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
  • System.Net.Sockets.Tests.SendReceive_Sync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
  • System.Net.Sockets.Tests.SendReceive_Sync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
  • System.Net.Sockets.Tests.SocketOptionNameTest.GetSetRawSocketOption_Roundtrips(family: InterNetwork)
  • System.Net.Sockets.Tests.SocketOptionNameTest.GetSetRawSocketOption_Roundtrips(family: InterNetworkV6)
  • System.Net.Sockets.Tests.DualModeAcceptAsync.AcceptAsyncV4BoundToAnyV4_Success
  • System.Net.Sockets.Tests.DualModeAcceptAsync.AcceptAsyncV4BoundToSpecificV4_Success
  • System.Net.Sockets.Tests.DualModeAcceptAsync.AcceptAsyncV4BoundToAnyV6_Success
  • System.Net.Sockets.Tests.DualModeAcceptAsync.AcceptAsyncV6BoundToSpecificV6_Success
  • System.Net.Sockets.Tests.DualModeConnectAsync.ConnectAsyncV4IPEndPointToV4Host_Success
  • System.Net.Sockets.Tests.DualModeConnectAsync.ConnectAsyncV6IPEndPointToV6Host_Success
  • System.Net.Sockets.Tests.DualModeConnectAsync.ConnectAsyncV6IPEndPointToV4Host_Fails
  • System.Net.Sockets.Tests.DualModeConnectAsync.ConnectAsyncV4IPEndPointToDualHost_Success
  • System.Net.Sockets.Tests.DualModeConnectAsync.ConnectAsyncV4IPEndPointToV6Host_Fails
  • System.Net.Sockets.Tests.DualModeConnectAsync.ConnectAsyncV6IPEndPointToDualHost_Success
  • System.Net.Sockets.Tests.IPPacketInformationTest.Equals_NonDefaultValue_Success
  • System.Net.Sockets.Tests.IPPacketInformationTest.GetHashCode_NonDefaultValue_Succes

Based on CI runs from #50095

Author: mdh1418
Assignees: -
Labels:

area-System.Net.Sockets, os-android
Milestone: 6.0.0

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Apr 1, 2021
@karelz karelz added test-bug Problem in test source code (most likely) and removed untriaged New issue has not been triaged by the area owner test-bug Problem in test source code (most likely) labels Apr 6, 2021
@antonfirsov
Copy link
Member

antonfirsov commented Jul 26, 2021 •
edited
Loading

@mdh1418 :

It looks like these tests have been mass-disabled in #50800. Is there any log / stack trace from the time before? (Can't find on kusto). Some of them might be caused by a problem described in #1481.

Is this something you still plan to investigate for 6.0? If not shouldn't we move it out to 7.0?

@antonfirsov antonfirsov added the disabled-test The test is disabled in source code against the issue label Jul 26, 2021
@mdh1418
Copy link
Member Author

mdh1418 commented Jul 27, 2021

@antonfirsov I don't think I have the log/stack trace from before.

I just reran the tests, and out of the original 26, only 12 still fail now based off of 34094b3 and I updated the description accordingly.

The following is the failure log/stack trace from my local run.

System.Net.Sockets.Tests.dll   Failed: 12

Test collection for System.Net.Sockets.Tests.SendReceive_SpanSync
System.Net.Sockets.Tests.SendReceive_SpanSync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
    System.AggregateException : One or more errors occurred. (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null))\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)
System.Net.Sockets.Tests.SendReceive_SpanSync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
    System.AggregateException : One or more errors occurred. (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null))\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)

Test collection for System.Net.Sockets.Tests.UnixDomainSocketTest
System.Net.Sockets.Tests.UnixDomainSocketTest.UnixDomainSocketEndPoint_UsingAbstractSocketAddressOnUnsupported_Throws
    Assert.Throws() Failure\nExpected: typeof(System.Net.Sockets.SocketException)\nActual:   (No exception was thrown)

Test collection for System.Net.Sockets.Tests.SendReceive_SyncForceNonBlocking
System.Net.Sockets.Tests.SendReceive_SyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
    System.AggregateException : One or more errors occurred. (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null))\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)
System.Net.Sockets.Tests.SendReceive_SyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
    System.AggregateException : One or more errors occurred. (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null))\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)

Test collection for System.Net.Sockets.Tests.OSSupportTest
System.Net.Sockets.Tests.OSSupportTest.IOControl_SIOCATMARK_Unix_Success
    System.Net.Sockets.SocketException : Permission denied

Test collection for System.Net.Sockets.Tests.SocketOptionNameTest
System.Net.Sockets.Tests.SocketOptionNameTest.GetSetRawSocketOption_Roundtrips(family: InterNetwork)
    Microsoft.DotNet.XUnitExtensions.SkipTestException : Unknown platform
System.Net.Sockets.Tests.SocketOptionNameTest.GetSetRawSocketOption_Roundtrips(family: InterNetworkV6)
    Microsoft.DotNet.XUnitExtensions.SkipTestException : Unknown platform

Test collection for System.Net.Sockets.Tests.SendReceive_Sync
System.Net.Sockets.Tests.SendReceive_Sync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
    System.AggregateException : One or more errors occurred. (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null))\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)
System.Net.Sockets.Tests.SendReceive_Sync.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
    System.AggregateException : One or more errors occurred. (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null))\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)

Test collection for System.Net.Sockets.Tests.SendReceive_SpanSyncForceNonBlocking
System.Net.Sockets.Tests.SendReceive_SpanSyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: False, dualModeClient: True)
    System.AggregateException : One or more errors occurred. (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null))\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)
System.Net.Sockets.Tests.SendReceive_SpanSyncForceNonBlocking.TcpReceiveSendGetsCanceledByDispose(receiveOrSend: True, ipv6Server: True, dualModeClient: False)
    System.AggregateException : One or more errors occurred. (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)) (Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null))\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)\n---- Assert.Equal() Failure\nExpected: ConnectionReset\nActual:   (null)

@mdh1418 mdh1418 mentioned this issue Jul 27, 2021
Merged
@mdh1418 mdh1418 modified the milestones: 6.0.0, 7.0.0 Aug 6, 2021
@directhex
Copy link
Contributor

This isn't really any better today than a year ago. How many of these are cases where Android should be passing, vs. cases where the OS means the test can never pass? e.g. SIOCATMARK is explicitly disabled on Android (https://android.googlesource.com/platform/system/sepolicy/+/master/public/domain.te)

@SamMonoRT
Copy link
Member

moving to 8.0.0

@SamMonoRT SamMonoRT modified the milestones: 7.0.0, 8.0.0 Aug 12, 2022
@steveisok steveisok modified the milestones: 8.0.0, Future Aug 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-System.Net.Sockets disabled-test The test is disabled in source code against the issue os-android
Projects
None yet
Milestone
Future
Development

No branches or pull requests
6 participants
@directhex @steveisok @karelz @antonfirsov @mdh1418 @SamMonoRT