Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider upgrading Xcode version on size regression tracking lanes for iOS applications #79879

Open
ivanpovazan opened this issue Dec 21, 2022 · 7 comments
Assignees
Labels
area-Infrastructure-mono perf-pipeline Issues with dotnet-runtime-perf, or runtime-wasm-perf pipelines
Milestone

Comments

@ivanpovazan
Copy link
Member

Description

A notable difference has been detected in the SOD measurements for the HelloiOS Mono sample app with different versions of Xcode.
The table below shows the difference in the executable binary HelloiOS of around 7% in favour of Xcode 14.1.0 when compared to Xcode 13.3.0

SOD comparison for HelloiOS sample app

File Xcode 13.3.0 Xcode 14.1.0
HelloiOS 23126720 21593736
Info.plist 980 979
PkgInfo 8 8
Program.aotdata 4792 4792
Program.deps.json 12411 12411
Program.dll 6656 6656
Program.runtimeconfig.json 1064 1064
System.Console.aotdata 177912 177912
System.Console.dll 17408 17408
System.Private.CoreLib.aotdata 1169640 1169640
System.Private.CoreLib.dll 1047552 1047552
icudt.dat 1859040 1859040
icudt_CJK.dat 956416 956416
icudt_EFIGS.dat 550320 550320
icudt_no_CJK.dat 1106560 1106560

Since .NET performance pipelines seem to be using Xcode 13.2.1, consider upgrading the Xcode version to get more accurate measurements.

Repro

  1. Clean checkout current main
  2. Set Xcode version to 13.3.0
  3. Build the runtime for iOS
./build.sh mono+libs -c Release -os iOS -arch arm64
  1. Build the HelloiOS sample
cd src/mono/sample/iOS/
export DevTeamProvisioning=-;make build-appbundle TARGET=iOS MONO_ARCH=arm64 MONO_CONFIG=Release AOT=true USE_LLVM=true DEPLOY_AND_RUN=false
ls -al bin/ios-arm64/publish/app/HelloiOS/Release-iphoneos/HelloiOS.app
  1. Record results
  2. Repeat the steps by setting the Xcode version to 14.1.0 in step (2)
@ivanpovazan ivanpovazan added area-Infrastructure-mono perf-pipeline Issues with dotnet-runtime-perf, or runtime-wasm-perf pipelines labels Dec 21, 2022
@ivanpovazan ivanpovazan added this to the 8.0.0 milestone Dec 21, 2022
@ghost
Copy link

ghost commented Dec 21, 2022

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

A notable difference has been detected in the SOD measurements for the HelloiOS Mono sample app with different versions of Xcode.
The table below shows the difference in the executable binary HelloiOS of around 7% in favour of Xcode 14.1.0 when compared to Xcode 13.3.0

SOD comparison for HelloiOS sample app

File Xcode 13.3.0 Xcode 14.1.0
HelloiOS 23126720 21593736
Info.plist 980 979
PkgInfo 8 8
Program.aotdata 4792 4792
Program.deps.json 12411 12411
Program.dll 6656 6656
Program.runtimeconfig.json 1064 1064
System.Console.aotdata 177912 177912
System.Console.dll 17408 17408
System.Private.CoreLib.aotdata 1169640 1169640
System.Private.CoreLib.dll 1047552 1047552
icudt.dat 1859040 1859040
icudt_CJK.dat 956416 956416
icudt_EFIGS.dat 550320 550320
icudt_no_CJK.dat 1106560 1106560

Since .NET performance pipelines seem to be using Xcode 13.2.1, consider upgrading the Xcode version to get more accurate measurements.

Repro

  1. Clean checkout current main
  2. Set Xcode version to 13.3.0
  3. Build the runtime for iOS
./build.sh mono+libs -c Release -os iOS -arch arm64
  1. Build the HelloiOS sample
cd src/mono/sample/iOS/
export DevTeamProvisioning=-;make build-appbundle TARGET=iOS MONO_ARCH=arm64 MONO_CONFIG=Release AOT=true USE_LLVM=true DEPLOY_AND_RUN=false
ls -al bin/ios-arm64/publish/app/HelloiOS/Release-iphoneos/HelloiOS.app
  1. Record results
  2. Repeat the steps by setting the Xcode version to 14.1.0 in step (2)
Author: ivanpovazan
Assignees: -
Labels:

area-Infrastructure-mono, perf-pipeline

Milestone: 8.0.0

@ivanpovazan
Copy link
Member Author

ivanpovazan commented Dec 21, 2022

/cc @kotlarmilos @LoopedBard3

@am11
Copy link
Member

am11 commented Feb 2, 2023

performance pipelines

The consumer pipeline (running dotnet publish) needs to be updated, right? (and not the pipeline building the ILCompiler package because that only builds unlinked objects)

@ivanpovazan
Copy link
Member Author

ivanpovazan commented Feb 3, 2023

@am11 I was referring to dotnet performance CI/CD lanes tracking size regressions for Mono. Maybe the title is a bit misleading, I will correct it.

@ivanpovazan ivanpovazan changed the title Consider upgrading Xcode version on .NET performance pipelines Consider upgrading Xcode version on size regression tracking lanes for iOS applications Feb 3, 2023
@ivanpovazan
Copy link
Member Author

/cc @SamMonoRT

@am11
Copy link
Member

am11 commented Feb 3, 2023

@ivanpovazan, there was no confusion. :) I was just confirming if you noticed a difference in the published binary size when the ilcompiler package was also built with the new toolchain?

AFAIK, the official build pipeline which publishes ilcompiler packages is also using the older Xcode.

@ivanpovazan
Copy link
Member Author

oh I see (I think the title is better like this anyway).
No, I did not check that with ILC. I will add it to my TODO and will come back to you. Thank you.

@akoeplinger akoeplinger modified the milestones: 8.0.0, Future May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-Infrastructure-mono perf-pipeline Issues with dotnet-runtime-perf, or runtime-wasm-perf pipelines
Projects
None yet
Development

No branches or pull requests

4 participants