When Keeping Analytics Real Time Goes Wrong

Virtually every piece of software I get pitched puts near to or real time analytics at my fingertips 24/7/365. It sounds fantastic and in some cases, is true – but the reality of ‘real time’ data is that very few products actually deliver data in real time due to the need to process raw data or dependency on other not-so-real-time technologies.

But, this article is not about whether or not real time data actually comes in real time – what’s more important the than latency of delivery is the accuracy and subsequent value of real time versus aggregated data sets.

In order to understand the time and place for real time versus aggregated data, it’s worth taking a step back to understand why Web analytics haven’t always come in real time. A common practice in analytics software is to have two types of databases (note: this is simplified but directionally accurate) – one containing raw data and another with aggregated data.

While new performance data pours into the raw database in batches, the raw data is cleaned up and populated into the aggregated database, which can then be queried against – populating performance data in the software UI. Streamlining the flow of data in and out of the raw database keeps performance stable and mitigates the likelihood of major issues stemming from too many queries stalling the raw database.

In order to provide real time data, you need to remove the aggregation process and have enough bandwidth and processing power to churn through new data as fast as possible while managing the demand to query against the raw database.

As a result, it’s understandably very expensive to maintain and scale true real time analytics without succumbing to heavy data sampling or limiting the types or queries against the database to mitigate risk of an epic failure.

Google Analytics (the free, non-premium, version; which does heavy data sampling), is a perfect example of a horrible real time dashboard. Sampling incomplete data never leads to accurate KPIs – it usually takes about 24 hours to get the data right in GA.

What Are The Benefits Of Real Time Analytics?

It’s perfect for the VP of Sales to watch revenue coming rolling in, constantly refreshing the internal dashboard to see if tonight will bring tequila shots of victory or preparation for a meeting with the CEO to figure out what’s going wrong. I used to have a dashboard like that…and I can confidently say that nothing productive came from anyone constantly refreshing the screen.

The dangers over overreacting to real time analytics are highlighted by the instinct to make significant changes to marketing campaigns using insight from real time intra-day data. If daily revenue is lagging, it’s easy to overreact by bidding down on paid search display ads as a means of maintaining hitting ad cost goals.

This can lead to a downward spiral because most ad servers are not designed to report and be managed in real time. The analytics behind performance marketing are built on statistically significant data sets leading to a logical decision making process.

The experienced analysts knows that patterns and trend lines are not made in real time. Understanding the value of a visit to your website on a Monday at 4 pm takes thousands and thousands of similar visits aggregated and normalized to provide an average value.

Creating projections modeling accurate historical trend lines, while not nearly as sexy as the ‘Real Time Dashboard’, are more reliable and can predict within acceptable variance for the purposes of a projection – leaving little to assumptions and chance.

As performance marketers, we are not in the business of rash decisions. We thrive on statistically significant data sets and the knowledge that acting rash on incomplete data could have detrimental effects on the marketing program. We take good days and bad days in stride knowing that if we make the right decisions there will be more good than bad, we’ll hit our numbers, and life will be good.

So while the bossman is freaking out that the numbers look soft because his real time dashboard says so, you head out to happy hour knowing that tomorrow morning, it’ll be all good.