MetricKit was introduced last year with iOS 13, offering a relatively simple API to access a wide range of on-device power and performance metrics for your app. With the introduction of iOS 14, MetricKit added three new trackable metrics: CPU instructions, animation hitches, and exit reasons.
Out of those three new metrics, the exit reasons metric is the most interesting to me. Exit reasons track the various ways an app could be terminated while running in either the foreground or the background. Those include things like terminations due to memory pressure, watchdog terminations, as well as crashes.
Why is this interesting? Because crash reports on iOS right now can be collected by either using a third-party SDK or relying on Apple's limited Xcode Crashes organizer. This is the first time Apple has offered a solution to build complex crash reporting pipelines with data provided by a first-party framework, and without relying on a bunch of messy techniques.
This blog post is an attempt to answer some of the questions that I think are going to be common for anyone looking at MetricKit and comparing it to any of the current crash reporters.
Before getting started, I recommend reading this blog post by Matt Massicotte first. It talks about a lot of the implementation details of crash reporting in MetricKit. Here are some of the key points Matt made that are relevant to this blog post.
- In-process crash reporting, which is how all current commercial and open-source crash reporters are implemented, is very complex. While most crash reporters are stable and work reliably, the techniques they use to achieve that are messy.
- MetricKit offers raw diagnostics data via MXDiagnosticPayload and MXCrashDiagnostic. It’s up to you to do any analysis of that data.
- MetricKit crash reporting is designed to support multiple consumers of the diagnostics data, unlike methods currently used by in-process crash reporting which can leave multiple crash reporters competing for the data.
- Stack traces collected by MetricKit are unsymbolicated and will require further processing before being human-readable.
How is the data from MetricKit different from Xcode Organizer?
When it comes to crashes, in theory, data shown in Xcode Organizer should be identical to that returned by MetricKit. The only difference is that data in Xcode Organizer will be presented in a human-readable format and fully symbolicated.
Should I start collecting crashes using MetricKit in my app?
Probably not. Unless you have a very good reason to build a completely custom crash reporting pipeline (guess what? It’s a lot of work!), you should just rely on crash reports coming from the in-process crash reporter of your choice plus crash reports in Xcode Organizer.
What are MetricKit's limits?
MetricKit is useful. It's also constrained in ways that matter for production crash reporting.
- Delivered at most once a day: MetricKit delivers diagnostic payloads to your app through the MXMetricManagerSubscriber delegate, called once per day when the device is on Wi-Fi and charging. If something breaks on a Monday morning, MetricKit data arrives on Tuesday. For a production incident with users churning in real time, a 24-hour delay is not actionable.
- Only from users who opted in: MetricKit diagnostic data (crash logs, hang reports) is delivered only from users who have allowed "Share iPhone Analytics" in their privacy settings. The opt-in rate varies but is typically well below your full user base. This means MetricKit crash data represents a sample, not the whole picture, and the sample is self-selected toward users with certain device settings.
- Hang diagnostics can be unreliable: MetricKit's hang reports use a 250ms threshold and are generated by the OS, which means the diagnostic data depends on the OS's internal scheduling and may not capture all hang scenarios. Some hang types that are clearly visible in real-user session data do not appear consistently in MetricKit hang reports.
- Limited context per event: MetricKit delivers a structured report with stack traces and some metadata. It does not deliver the session context around the crash: the user's navigation history, the network calls that fired, the screen state at the moment of failure. That context is what makes a crash reproducible without a user reporting it.
- No real-time alert path: MetricKit has no mechanism to trigger an immediate alert when a crash occurs. It is a reporting channel, not a monitoring channel. Real-time alerting requires in-process instrumentation that can fire as the failure happens.
Who is this for then?
Apple doesn’t specifically say this in the documentation or any of the WWDC videos talking about MetricKit, but I’m presuming that it’s mainly targeted at services offering crash reporting and other performance and diagnostics tools.
Is this the end of in-process crash reporting?
I highly doubt it. MetricKit is very promising, but it also currently has a few limitations that I think are going to curb its adoption.
MetricKit will only collect data for users that have opted-in to share diagnostics & usage data with developers. This is great for privacy, yet it also means that you will not be getting the full picture of how a certain crash impacts your users since you only have a (supposedly small) percentage of the diagnostics reports. More obscure crashes affecting a small percentage of your userbase may go completely unnoticed.
Another limitation with MetricKit is that you can request diagnostics data about the previous 24 hours only once per day. This means that sudden spikes in crashes may go unnoticed for a few hours too long.
Finally, crash reporting in MetricKit requires iOS 14. Depending on how fast iOS 14 gets adopted by users, this limitation alone could mean that it could take 2 to 3 years before MetricKit sees meaningful levels of adoption.
Apple could address some of those limitations in future versions of MetricKit, but I highly doubt that anybody will be ditching current in-process crash reporters for MetricKit any time soon.
Why do in-process tools still matter?
MetricKit and in-process crash reporting are not alternatives. They capture different things and serve different purposes in a crash reporting stack.
In-process tools register signal handlers (for SIGABRT, SIGSEGV, SIGBUS and others) and exception handlers (NSUncaughtExceptionHandler on iOS) that fire at the moment the crash occurs, before the process terminates. This gives them two capabilities MetricKit doesn't have:
- Immediate capture: The in-process handler runs before the app exits. It can capture the stack trace, the thread state and whatever session context has been recorded up to that point, and persist it to disk for delivery on next launch. This happens within milliseconds of the failure, not 24 hours later.
- Session context: Because the in-process tool has been running throughout the session, it has the complete record: every screen navigated, every network call made, the state of the app at the moment of failure. MetricKit sees only the crash event. An in-process tool sees everything that led up to it.
Main-thread watchdogs are a second category of in-process monitoring that MetricKit's hang diagnostics partially overlap with, but don't replace. A main-thread watchdog runs on a background thread and monitors whether the main thread is responsive. If the main thread is blocked past a threshold (typically 1-3 seconds in production tools), the watchdog fires immediately, captures the main thread's stack trace at that moment and records a hang event. This produces actionable data, the exact callstack causing the blockage, in real time. MetricKit's hang reports are delivered later and may not capture the same events.
The right setup for most iOS apps is both: MetricKit for its Apple-native coverage of certain crash types, and in-process instrumentation for real-time capture, session context and the hang detection that matters for user experience.
Is Apple planning on limiting the use of in-process crash reporting?
It’s not entirely implausible that MetricKit could be part of a plan to limit the use of methods currently used by in-process crash reporters and have tighter control over the crash reporting lifecycle and data. It’s very hard to speculate on this, but I doubt that this is what Apple is planning to do.
If this were to happen, it would give Apple the ability to enforce the privacy of user’s data at an API level. While the idea, in theory, sounds appealing, the amount of private data in a crash report is very little, if not none at all.
If Apple was to take such a heavy-handed approach, I believe it would start tackling SDKs and services that collect a lot more sensitive data, like analytics SDKs, where this would have a much higher impact.
Combine MetricKit data with full session context
Luciq's Signals pairs crash data with the session behind it, including the user's steps, screen and network state, so iOS crashes are easy to reproduce and fix.