Severe memory usage

[gutaker]

Issue:
I have a really big project with mixed Swift, SwiftUI. There are ~9k unit tests. When I run unit testing, the memory usage of xcbeautify grows up to 7GBs...

Test are ran with fastlane, with invocation fastlane scan --workspace mobileApp.xcworkspace --scheme UnitTests --derived_data_path "./DerivedData" --device "iPhone 15 Pro" -c true --output_files "mobileapp.unit.test.report.html,mobileapp.unit.test.report.junit"

Here's screenshot

fastlane env output below, however, the issue is there for macOS Sonoma 14.1.1/Xcode 15.0.1 and for good, old, stable macOS Ventura 13.6/Xcode 14.2 as well.

🚫 fastlane environment 🚫

Stack

Key	Value
OS	14.1.1
Ruby	3.1.4
Bundler?	false
Git	git version 2.39.3 (Apple Git-145)
Installation Source	~/.local/share/rtx/installs/ruby/3.1.4/bin/fastlane
Host	macOS 14.1.1 (23B81)
Ruby Lib Dir	~/.local/share/rtx/installs/ruby/3.1.4/lib
OpenSSL Version	OpenSSL 3.1.4 24 Oct 2023
Is contained	false
Is homebrew	false
Is installed via Fabric.app	false
Xcode Path	/Applications/Xcode15.1.app/Contents/Developer/
Xcode Version	15.1
Swift Version	5.9.2

System Locale

Error
No Locale with UTF8 found 🚫

fastlane files:

No Fastfile found

No Appfile found

fastlane gems

Gem	Version	Update-Status
fastlane	2.217.0	✅ Up-To-Date

Loaded fastlane plugins:

No plugins Loaded

Loaded gems

Gem	Version
error_highlight	0.3.0
did_you_mean	1.6.1
public_suffix	5.0.4
addressable	2.8.5
artifactory	3.0.15
jmespath	1.6.2
aws-partitions	1.855.0
aws-eventstream	1.3.0
aws-sigv4	1.7.0
aws-sdk-core	3.188.0
aws-sdk-kms	1.73.0
aws-sdk-s3	1.139.0
babosa	1.0.4
bundler	2.3.26
rexml	3.2.5
CFPropertyList	3.0.6
colored	1.2
highline	2.0.3
commander	4.6.0
dotenv	2.8.1
emoji_regex	3.2.3
excon	0.104.0
faraday-em_http	1.0.0
faraday-em_synchrony	1.0.0
faraday-excon	1.1.0
faraday-httpclient	1.0.1
multipart-post	2.3.0
faraday-multipart	1.0.4
faraday-net_http	1.0.1
faraday-net_http_persistent	1.2.0
faraday-patron	1.0.0
faraday-rack	1.0.0
faraday-retry	1.0.3
ruby2_keywords	0.0.5
faraday	1.10.3
faraday_middleware	1.2.0
domain_name	0.6.20231109
http-cookie	1.0.5
faraday-cookie_jar	0.0.7
fastimage	2.2.7
gh_inspector	1.1.3
uber	0.1.0
declarative	0.0.20
trailblazer-option	0.1.2
representable	3.2.0
retriable	3.1.2
mini_mime	1.1.5
jwt	2.7.1
multi_json	1.15.0
os	1.1.4
signet	0.18.0
googleauth	1.8.1
httpclient	2.8.3
webrick	1.8.1
google-apis-core	0.11.2
google-apis-androidpublisher_v3	0.53.0
google-apis-playcustomapp_v1	0.13.0
google-cloud-env	1.6.0
google-cloud-errors	1.3.1
google-cloud-core	1.6.0
google-apis-iamcredentials_v1	0.17.0
google-apis-storage_v1	0.29.0
rake	13.0.6
digest-crc	0.6.5
google-cloud-storage	1.45.0
json	2.6.1
mini_magick	4.12.0
naturally	2.2.1
optparse	0.1.1
plist	3.7.0
rubyzip	2.3.2
security	0.1.3
simctl	1.6.10
terminal-notifier	2.0.0
unicode-display_width	2.5.0
terminal-table	3.0.2
tty-screen	0.8.1
tty-cursor	0.7.1
tty-spinner	0.9.3
word_wrap	1.0.0
atomos	0.1.3
claide	1.1.0
colored2	3.1.2
nanaimo	0.3.0
xcodeproj	1.23.0
rouge	2.0.7
xcpretty	0.3.0
xcpretty-travis-formatter	1.0.1
set	1.0.2
forwardable	1.3.2
logger	1.5.0
pathname	0.2.0
shellwords	0.1.0
cgi	0.3.6
date	3.2.2
timeout	0.2.0
stringio	3.0.1
securerandom	0.2.0
uri	0.12.1
openssl	3.0.1
digest	3.1.0
io-nonblock	0.1.0
ipaddr	1.2.4
io-wait	0.2.1
zlib	2.1.1
resolv	0.2.1
time	0.2.2
open-uri	0.2.0
mutex_m	0.1.1
net-http	0.3.0
net-protocol	0.1.2
ostruct	0.5.2
english	0.7.1
erb	2.2.3
strscan	3.0.1
abbrev	0.1.0
io-console	0.5.11
tempfile	0.1.2
delegate	0.2.0
fileutils	1.6.0
tmpdir	0.1.2
base64	0.1.1
singleton	0.1.1
open3	0.1.1
nkf	0.1.1
prettyprint	0.1.1
pp	0.3.0
find	0.1.1
yaml	0.2.0
psych	4.0.4

generated on: 2023-11-27

[github-actions]

Hola 👋,

We want to inform you that the issue has been marked as stale. This means that there hasn't been any activity or updates on it for quite some time, and it's possible that it may no longer be relevant or actionable.
If you still believe that this issue is valid and requires attention, please provide an update or any additional information that can help us address it. Otherwise, we may consider closing it in the near future.
Thank you for your understanding.

[victor-sarda]

Hi 👋

I'm seeing the same issue with Xcode 15.2 & macOS Sonoma (14.2.1). It sometimes goes up to 4GB.
Is there any update on this?

Thanks

[cpisciotta]

Hi @gutaker and @victor-sarda! Thank you for filing this issue, and I'm sorry for the delayed response. I do have some follow-up questions, so I can better understand the potential root cause.

As a general note, I expect gradual performance improvements over time. As we introduce missing regexes, we'll reduce the amount of searches. Similarly, as we improve the regexes themselves, we'll reduce their associated search complexity.

[cpisciotta]

@gutaker Can you share the generated xcodebuild/xcbeautify command? More specifically, can you share what flags - if any - you're passing to xcbeautify?

I see you're using JUnit in your fastlane invitation. I'd like to see if you're using xcbeautify's JunitReporter too.

[cpisciotta]

Hi @victor-sarda. Thanks for the follow-up. Can you share your xcodebuild/xcbeautify command? I'd specifically like to see what flags - if any - you're passing to xcbeautify.

[victor-sarda]

Hey @cpisciotta sure!

Here's the xcodebuild command I use (from Fastlane):

set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace <redacted>.xcworkspace -scheme UnitTests -derivedDataPath /Users/victor.sarda/Library/Developer/Xcode/DerivedData/<redacted>-dvyxozgptgpxumegogztmewovxws -destination 'platform=iOS Simulator,name=iPhone 14 Pro,OS=17.2' -resultBundlePath './fastlane/test_output/UnitTests.xcresult' -parallel-testing-worker-count 1 -enableCodeCoverage YES -testPlan 'UnitTests' COMPILER_INDEX_STORE_ENABLE='NO' -skipPackagePluginValidation build test | tee '/Users/victor.sarda/Library/Logs/scan/UnitTests-UnitTests.log' | xcbeautify

This ran on an M1 Pro Macbook Pro on Sonoma (14.3.1) with Xcode 15.2.
The project has around 1700 unit tests.

When running, I can see that xcbeautify gradually consumes RAM, usually up to 4GB.

Let me know if you need more details!

[cpisciotta]

Thanks @victor-sarda! I don't have an immediate fix, but I'll continue to investigate this and address overall performance. Please let me know if its performance gets noticeably worse (or better) with future releases.

As a note, I intend to release v2 this weekend, so I expect the upcoming architectural changes (and the ability to refactor more quickly) will help address performance-related concerns.

[cpisciotta]

Great news @gutaker and @victor-sarda! After investigation, I've narrowed down a huge bottleneck in xcbeautify's performance. You can track the upcoming changes in #253.

The improvements are... drastic. From my testing on my machines, on average, I'm seeing a 90% time savings and a 99% memory savings.

Thanks for your patience and your help flagging this issue! Huge improvements coming coon!

[victor-sarda]

Hey @cpisciotta that's awesome news, thanks for having a look!
Can't wait to try it out 🎉

[cpisciotta]

This issue should be fixed starting with the recently released version, 1.5.0. If you continue to see this issue, please let me know.

[victor-sarda]

Thank you! I'll try first thing tomorrow and will let you know 👍

[gutaker]

Sorry guys for such late reply - I was totally off-grid. I can see however you've already solved the issue. Thanks a bunch!

[victor-sarda]

Hey @cpisciotta, I can confirm the issue is fixed on our runners with v1.6.0!
Much better, thanks again for fixing 🙌