Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: [mono][tests] Enable trimming for libraries and functional tests on CI #92645

Closed
wants to merge 6 commits into from

Conversation

ivanpovazan
Copy link
Member

@ivanpovazan ivanpovazan commented Sep 26, 2023

This PR is currently marked as a DRAFT and is intended just for testing.

Description

The PR will be used to incrementally test enabling trimming for libraries and functional tests when targeting ios/tvos platforms on CI. To enable trimming during test builds we should set: -p:EnableAggressiveTrimming=true, however it was reported that some tests depend on not being trimmed in: #91923 which we should investigate and root/fix the failing tests accordingly.

To run a libraries test locally:

  • build the runtime for a specific target:
    ./build.sh mono+libs -c Release -os ios -arch arm64
  • build/run the test on a specific target (use correct Apple Team ID instead of XXXXXXXX):
    ./dotnet.sh build -c Release src/libraries/System.Runtime/tests/System.Runtime.Tests.csproj -p:TargetOS=ios -p:TargetArchitecture=arm64 -t:Test -p:MonoEnableLLVM=true -p:DevTeamProvisioning=XXXXXXXX

!IMPORTANT!

The initial commits have to be reverted as they disable the default CI triggers, which was done to avoid polluting the queues.
Incremental testing can be done by running: /azp run runtime-ioslike

Tentative plan

  • Fix build on Helix configuration with trimming enabled
  • Fix functional and runtime tests failures with trimming enabled
    • Fix System.Runtime.Numerics.Tests - 3 failing tests
    • Fix System.Runtime.Tests - 11 failing tests
    • Fix iOS.Device.Aot.Test
    • Fix iOS.Device.ExportManagedSymbols.Test
    • Fix any other test failures
  • Measure build/trim times and compare how much we save with this change

Current state ios-arm64 with trimming

Failing work items:

  • System.Runtime.Numerics.Tests - failed tests: 3 - logs
  • System.Runtime.Tests - failed tests: 11 - logs
  • iOS.Device.Aot.Test* - failed test: 1 - logs
  • iOS.Device.ExportManagedSymbols.Test* - failed test: 1 - logs

NOTE: both functional test from above seem to run the correctly and return the expected return value, but XHarness reports XHarness exit code: 92 (TCP_CONNECTION_FAILED) marking the test as failed.

  • QUESTION This should not be related to trimming, why are we experiencing that in this set up?

Should fix: #91923

@ivanpovazan ivanpovazan added NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) NO-REVIEW Experimental/testing PR, do NOT review it labels Sep 26, 2023
@ivanpovazan ivanpovazan self-assigned this Sep 26, 2023
@ghost
Copy link

ghost commented Sep 26, 2023

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

This PR is currently marked as a DRAFT and is intended just for testing.

Description

The PR will be used to incrementally test enabling trimming for libraries and functional tests when targeting ios/tvos platforms on CI. To enable trimming during test builds we should set: -p:EnableAggressiveTrimming=true, however it was reported that some tests depend on not being trimmed in: #91923 which we should investigate and root/fix the failing tests accordingly.

To run a libraries test locally:

  • build the runtime for a specific target:
    ./build.sh mono+libs -c Release -os ios -arch arm64
  • build/run the test on a specific target (use correct Apple Team ID instead of XXXXXXXX):
    ./dotnet.sh build -c Release src/libraries/System.Runtime/tests/System.Runtime.Tests.csproj -p:TargetOS=ios -p:TargetArchitecture=arm64 -t:Test -p:MonoEnableLLVM=true -p:DevTeamProvisioning=XXXXXXXX

IMPORTANT!!!

The initial commit has to be reverted as it disables the default CI trigger so that we do not polute the queues.
Incremental testing can be done by running: /azp run runtime-ioslike


Should fix: #91923

Author: ivanpovazan
Assignees: ivanpovazan
Labels:

NO-MERGE, NO-REVIEW, area-Infrastructure-libraries

Milestone: -

@ivanpovazan
Copy link
Member Author

/cc: @simonrozsival @mdh1418

@ghost
Copy link

ghost commented Sep 26, 2023

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

This PR is currently marked as a DRAFT and is intended just for testing.

Description

The PR will be used to incrementally test enabling trimming for libraries and functional tests when targeting ios/tvos platforms on CI. To enable trimming during test builds we should set: -p:EnableAggressiveTrimming=true, however it was reported that some tests depend on not being trimmed in: #91923 which we should investigate and root/fix the failing tests accordingly.

To run a libraries test locally:

  • build the runtime for a specific target:
    ./build.sh mono+libs -c Release -os ios -arch arm64
  • build/run the test on a specific target (use correct Apple Team ID instead of XXXXXXXX):
    ./dotnet.sh build -c Release src/libraries/System.Runtime/tests/System.Runtime.Tests.csproj -p:TargetOS=ios -p:TargetArchitecture=arm64 -t:Test -p:MonoEnableLLVM=true -p:DevTeamProvisioning=XXXXXXXX

IMPORTANT!!!

The initial commit has to be reverted as it disables the default CI trigger so that we do not polute the queues.
Incremental testing can be done by running: /azp run runtime-ioslike


Should fix: #91923

Author: ivanpovazan
Assignees: ivanpovazan
Labels:

NO-MERGE, NO-REVIEW, area-Infrastructure-libraries, area-Infrastructure-mono

Milestone: -

@ivanpovazan
Copy link
Member Author

/azp run runtime-ioslike

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ivanpovazan
Copy link
Member Author

/azp run runtime-ioslike

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ghost
Copy link

ghost commented Oct 27, 2023

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@ghost ghost locked as resolved and limited conversation to collaborators Nov 26, 2023
This pull request was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-Infrastructure-mono NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) NO-REVIEW Experimental/testing PR, do NOT review it
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[mono] Update tests to be trim-compatible on ios/tvos platforms
1 participant