Fix pre warmed app start measurement

[philipphofmann]

Description

Since iOS 15, the system might decide to pre-warm your app before the user tries to open it. In such cases, we can't reliably measure the app start, and we started to drop reporting app start measurements with #1896. Otherwise, we would get app start times ranging from one minute to even days.

According to a former Apple engineer, you should use MetricKit to collect statistics on app start because MetricKit uses internal OS values, which are not exposed otherwise. MetricKit works on iOS 13 and above. Maybe it's time to replace our own SentryAppStartTracker.

See also #1661. [Internal Notion page] (https://www.notion.so/sentry/Detecting-iOS-15-App-Prewarming-c892f118bdd34b558eb187796ed5e424) on how to detect pre-warmed app starts with data we can't share in public, and another Notion page for a DACI on how to report pre-warmed app starts again.

Related PRs:

• Stop reporting pre warmed app starts #1896
• Set max app start duration to 60s #1899
• Detect prewarmed starts with env variable #1927
• Report pre-warmed app starts #1969

[bruno-garcia]

Can we get values from MetricKit about the startup related to the transaction that launched the app?

[philipphofmann]

Can we get values from MetricKit about the startup related to the transaction that launched the app?

No, we can't because we don't get launch data in real-time. You get it once per day: see #1661 (comment)

[philipphofmann]

According to the Apple docs

Prewarming an app results in an indeterminate amount of time between when the prewarming phase completes and when the user, or system, fully launches the app. Because of this, use MetricKit to accurately measure user-driven launch and resume times instead of manually signposting various points of the launch sequence.

And

Prewarming executes an app’s launch sequence up until, but not including, when main() calls UIApplicationMain(::::).

So we could check if the runtime init and the process start time are some threshold in the past to assume it is an pre warmed app start. We could then just ditch Pre Main and UIKit and Application Init and create a new app start type pre warmed. MetricKit has a special histogram for pre warmed app starts. Taken from our docs:

• Pre Main: From the process start time to the runtime init.
• UIKit and Application Init: From the runtime init to the didFinishLaunchingNotification.
• Initial Frame Render: From the didFinishLaunchingNotification to UIWindowDidBecomeVisibleNotification.

[philipphofmann]

Prewarming can execute code up to viewDidLoad of a UIViewController, and keep your app in the background. It can pause at any stage from creating the process up to viewDidLoad of a UIViewController or something similar. This is a contradiction to the Apple docs Prewarming executes an app’s launch sequence up until, but not including, when main() calls UIApplicationMain(::::).

I dug into some transactions of customers and found a few transactions similar to this one:

The above screenshot shows missing instrumentation of around 4 hours in between viewDidLoad and viewWillAppear. This can't be the case if the app was in the foreground because the watchdog would kill the app.

Discover query to reveal such transactions: event.type:transaction measurements.app_start_warm:>500 measurements.app_start_warm:<2000 transaction.duration:>10000

An answer on StackOverflow supports this claim.

More screenshots

Discover query used:event.type:transaction measurements.app_start_warm:>10000

Pausing after process creation

Pausing after process creation and runtime init