Ever wonder why certain mobile apps you use crash so much?
It turns out there are many possible reasons. And it can vary particularly depending on whether you are using an Apple iOS device such as an iPhone or iPad, or an Android device.
One of the reasons for app crashes is the proliferation of mobile operating systems on iOS and Android. As Apple and Google have released more new operating systems, each with multiple updates, app developers face more operating systems to test apps on. In data that mobile app monitoring startup Crittercism compiled for app crashes between December 1 and 15, there were at least 23 different iOS operating systems on which apps had crashed and 33 Android operating systems on which apps had crashed. (See the graphs above.) Note that the graphs that separate out Android and iOS show these number of operating systems and the graph that combines both iOS and Android shows less–22 iOS and 17 Android.
The largest proportion of app crashes from both iOS and Android platforms were on iOS 5.01 with 28.64% of overall crashes (in a normalized data set). That makes sense since iOS 5 was still relatively new at that time and many apps still need to work out the kinks with the new OS. But there are also older iOS versions that have a significant proportion of app crashes. For example, iOS 4.2.10 had 12.64% of app crashes, iOS 4.3.3 had 10.66% and iOS 4.1 had 8.24%. One other point that this made clear to me is that many people apparently take their time updating their iPhone software or never update it at all.
The data raises two main questions for me: why do apps on these operating systems crash so much, and do iOS apps crash more than Android apps? On the first question of why apps crash, the reasons are many, says Crittercism CEO Andrew Levy. This can be due to hardware issues, such as the use of location or GPS services or cameras; it could be due to the Internet connection, that is, how a phone connects to 3G or WiFi, or that the device is not connected to the Internet at a certain moment, or that something happens during the switch between 3G and WiFi. There could also be issues with language support on certain devices. There can also be memory problems if an app uses too much memory.
Problems can also occur with third-party services that developers use in their apps, from analytics to advertising systems. For example, there were reports that Apple’s iAds system gave some developers problems if they did not adhere to certain standards. ”It can be a mix of both hardware and software issues that developers may or may not be responding to,” Levy says.
In addition, developers also constantly create new updates to their apps to create new features or fix bugs. But again, people often don’t update their apps–just as they don’t update their operating system. (Android, unlike iOS, allows users to auto-update their apps, which can eliminate some of the problems.) So developers often test all previous versions of their apps with each version of the different operating systems. ”The permutations go on forever,” Levy says. “That’s a large reason for creating our platform.” Particularly with a new OS, developers have to test their app to make sure it still works on the new OS. Often they will seek to test their apps in a test environment, but often that isn’t possible.
The Apple iOS operating system app crashes accounted for more of the app crashes in Crittercism’s data than did Android-based phones, as mentioned above. In the pie graph “Crashes by OS Version Normalized” you can see that iOS accounted for close to three-fourths of the app crashes, with Android making up the rest. But is that just because Crittercism has more iOS phones in its network? Crittercism parsed some data to answer that question.
Crittercism analyzed a total of more than 214 million app launches from November and December 2011 from apps that use its service (see graph at top of this article). There were about 3 times more app launches for iOS that Crittercism analyzed, about 162 million to 52 million. But the analysis examined app crashes as a percentage of each app launch, so this data takes out the issue of there being more iOS than Android apps. In other words for each iOS app and each Android app how often percentage-wise do they crash?
In the top quartile of apps, Android apps crashed 0.15% of the time they launched, while top quartile iOS apps crashed 0.51% of the time. In the second quartile of apps, Android apps crashed 0.73% of the time and iOS apps crashed 1.47% of their launches. In the third quartile of apps, Android apps crashed 2.97% of the time, while iOS apps crashed 3.66% of the time.
So what does all this data mean? On a basic level, you can see that iOS apps crashed more than Android apps during this time period. But Crittercism’s Levy cautions that this doesn’t necessarily mean that overall iOS apps crash more than Android apps. That’s because Apple had recently released a new version iOS 5 in October. Android’s new Ice Cream Sandwich operating system (Android 4.0), meanwhile, had not been widely released on phones yet at the time of this study. “I expect as Ice Cream Sandwich just launched and the new Nexus S phone launched (during the study), we’ll expect the same situation to occur (with Android) as what happened (with iOS),” Levy says.
Still, the data shows that apps on iOS did crash substantially more than Android apps. Anecdotally, I know that certain apps I use on my iPhone crash and they crash often. Will that change as Ice Cream Sandwich rolls out and as more developers improve on iOS 5? We’ll see.
One other thing about the data: in the best apps, that is, the top quartile, the apps crashed much less than in the third quartile. Levy emphasized that that shows the difference that developers can make with their apps by analyzing the data and improving their apps. Splitting up the data by quartiles also removes apps with massive user bases that can skew the averages.
My own point: when you get to the top apps, Android’s lower app crash rate than iOS makes less of a difference because they are both well below 1%. However, there was a bigger difference between iOS and Android app crashes in the top quartile of apps than in the third quartile. In other words, the best apps in Android crashed about one third as many times as the best iOS apps, while the second best quartile Android apps crashed about half as much as comparable iOS apps, and in the quartile, the difference between the two operatings systems was even less. So the very top Android apps are achieving a crash rate that, at least in this time period, the best iOS apps can’t match. Why that is, I’m not entirely clear.
However, Android, it should be noted, allows developers to push updates faster than Apple. With Android developers can just send an update to its code, which can show up almost in real-time. But for iOS it can takes days or a week for an update to show up. That means there can be more app crashes while those updates are waiting to happen. Whereas with Android, presumably if developers know there’s a bug they can immediately fix it.
One final piece of interesting data from Crittercism: The performance of apps is not only different on various operating systems but also on different devices. About 74.41% of the iOS crashes Crittercism tracked were on the iPhone, 14.81% were on the iPod Touch, and 10.72% were on the iPad.
Crittercism, which is backed by Google Ventures, Kleiner Perkins Caufield & Byers, AngelPad, AOL Ventures, Opus Capital and Shasta Ventures, provides crash reporting to app developers. The company provides developers with a wide range of data besides the mobile device and operating system, including, for example, the length of time between when an app is loaded and when it crashes, or how a user is holding a phone–portrait or landscape–when it crashes. It can also help evaluate whether an app’s own code has caused it to crash or whether a third party service SDK being used is causing the problem. This kind of data is important for helping developers plug all the holes in the landscape of operating systems. Clients using Crittercism include: Aston Martin Explore, Bullet Time and Hipster.
Protecting apps from crashes is not only important for app developers, who by definition live and die by the ability of their apps to work smoothly. Increasingly, many types of companies rely on apps to run their businesses, even if there isn’t a large consumer user base using the apps. Banks have mobile apps that enable check deposits, sales people use apps in the field, and so on. “It’s about protecting your brand,” says Crittercism cofounder Rob Kwok. “More and more business-critical functions are moving to apps.”
Crittercism is signing up a number of large customers to its platform as a result. The company has already ramped way up from the 214 million app launches that it monitored in this data from November and December. If you’re hoping to end those annoying crashes on your phone or tablet, that should be welcome news.