8 Common Mobile Analytics Challenges And Pitfalls — Tips On How To Avoid Them

1. App installation data is not consistent

A common case: the app installation data from your analytics system and the console (App Store Connect or Google Play Console) do not match. What can cause these discrepancies? Most of the time, the reason is as follows: in App Store Connect, we see the number of installs, that is, how many times your app has been downloaded, the Installations metric. On the other hand, analytics systems show how many times your app has been opened. They start tracking when the SDK is initialized, i.e., when your app is launched. The thing is that not all of the users who have downloaded your app will run it. And that’s where discrepancies begin.

Charts showing installs of the same app in App Store Connect and Appsflyer

2. App event data is not tracked by your analytics system

Analytics systems might not track certain events you would like them to. There are several reasons for this:

  • The event is configured incorrectly. Because of this, it is not sent to your tracking and analytics system to begin with.
  • The event is not triggered by users. This sometimes happens when your app has a small audience or if the event occurs rarely: users just do not get to it. To avoid this, you need to test and make sure your events are sent correctly before you release the app. When adding new events to the analytics, it’s important to check that they do in fact arrive into the system and are triggered when they are supposed to. Ask your QA engineer to ensure the correctness of the event triggering before the release.
  • Your analytics system sends events in ‘batches’. To avoid server overload, analytics services might send event data in ‘batches’ or use timeouts (e.g., when there are 10 events in-store or when 10, 30 etc. seconds have passed). Assume we have an app with a sign-up process that has 5 events. The analytics system will wait for several more events before it sends the whole ‘batch’. It’s possible that the number of events never reaches the required number, so the ‘batch’ will never be sent.
    How do we address this? One solution is to force the system to send the events. In most analytics systems, you can force the sending of one or several events. However, we do not recommend using this approach all the time since it can disrupt the initial sequence when you receive them. This is also not common for analytics systems. By using separate requests to ‘force send’ every single event, you increase the server and network load.
  • Something has broken down in the app itself. You might have just missed a bug in the latest update: some features might in fact be working incorrectly and the issue has not been discovered during the testing stage. It might be worth it to check the app itself. For example, if, due to an error, your app does not open the user’s profile screen, the event of viewing the profile screen will not be sent to the analytics system. Sometimes abrupt drops of a certain metric to zero may be a sign of overlooked bugs and other technical issues. The advice here is quite obvious: to release a bug fix to address those, and quickly.

3. Events are not matched with users

Another common problem is when your analytics system does not know which user a certain event is related to. In that case, the system either creates a new user (like Amplitude does) or the event is just ignored (like in AppsFlyer).

4. Event stats duplication

It’s fairly easy to notice when event data is not tracked at all: the stats will show zero. However, determining if a certain event is duplicated is more difficult to do by just glancing at the dashboard. In our work, there have been at least a few cases of the same data about a certain event coming into the analytics system twice.

5. App event data differ from system to system

Event data in different analytics systems might not match. The most common case is that their services are run at different times. Those are not triggered simultaneously. For instance, a user launches an app, and for that user, Amplitude has enough time to run and send the event. A second user’s app launch, however, is not tracked because in their case, the system did not have enough time to send the information.

6. Income data are added up in different currencies

The next problem is that your analytics system might add up your income data in different currencies. This issue is difficult to identify without comparing each sum using the console. If you attract traffic from multiple countries, you need to know the currency used by your analytics system when receiving money and the currency when sending it.

7. Time-delayed conversions and different attribution windows are not taken into account

Yet another situation that might lead to data discrepancies is when different systems have different attribution windows.

8. Incorrect data comparison

Different analytics systems may use the same term for different things. The most basic example is from the first mistake on this list, about comparing the installation data from the console and our tracker. As we have seen, this is not really consistent: after all, we are actually comparing different metrics: the installations and the launches.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Proba

Proba

proba.ai is a tool for A/B testing in mobile apps. Carry out experiments faster, and at a better price — using the mobile app product hypothesis testing tool.