However, this is not the best way to control your budget for two reasons:

(1) If you have hit your campaign budget limit, it means that you are not competing in auctions for part of the day and hence are missing out on potentially profitable clicks.

(2) You are paying a higher keyword CPC than necessary to get the same number of clicks. The reason for this is a bit nuanced. When you bid high for a certain keyword(s), Google lets you participate in more auctions with the result that you pay a higher average CPC and also get a higher rate of new clicks.

As a result, your budget is spent quickly, and the campaign caps before the day ends. A lower bid would would to participation in fewer auctions and also get you clicks at a lower CPC. However, since you participate in auctions for a longer period of time, you could potentially get the same or even greater number of clicks at a higher CPC.

The profit of a keyword, ad group or campaign can be represented as:

Profit = (RPC – CPC) x Clicks 

Where RPC Is The Revenue Per Click

Hal Varian, Google’s chief economist, has shown that  RPC is position independent. Thus, a lower bid will not lead to lower RPC. However, a lower bid leads to a lower CPC and hence more profit if the number of clicks are the same. Thus advertisers, especially those maximizing profit, need to pay special attention to campaign caps because this has direct bearing on the performance of their campaigns.

Many advertisers prefer to set very high campaign budget limits. They feel that rather than risking hitting the campaign budget limit and losing out on profitable clicks it is best to keep the campaign budgets very high and then find the right keyword level bid that will enable them to spend on keywords profitably without the hinderance of the campaign budget limit.

While this is the right approach, it can also be dangerous. This example shows why:

For the sake of simplicity, lower numbers have been used to demonstrate the point; but imagine the implications on your overall budget when considering tens of thousands, or even hundreds of thousands to millions of clicks or dollars spent.

Consider three different days or scenarios of campaign performance.

On Day 1, a bid of $1 gets you a 10 cent CPC and 500 clicks by midnight the same day. Since the revenue per click is 20 cents you make $50 in profit. You also did not miss out on potential revenue as the potential clicks that you would have received for the day were the same as the actual revenue.

On Day 2, your campaign hits its limit earlier in the day and as a result you received 50 clicks less than possible. As a result, you lost $5 in potential profit.

On Day 3, however, two things happen. For a slightly lower bid, your CPC jumps to 30 cents due to competition and at the same time, the number of potential clicks jumps to 1000. In this case, the campaign budget limit will kick in quickly and you will only receive 167 clicks before your ads stop showing. In this scenario, since every click is unprofitable, you will make a loss of $16.67. However, the campaign budget limit in this case protected you from an additional loss of $83.33.

Many advertisers only think about the second scenario, eg, potentially missing out on profitable clicks when they set campaign budget limits. However, scenario 3 can and does happen. In highly seasonal retail and travel periods there is a lot of consumer interest and hence higher-than-average traffic.

At the same time, there are more advertisers vying for the same click with the result of rising  CPCs. Thus, if the RPCs are lower than CPCs at this point in time one could lose a lot of money.

How To Set The Right Campaign Budget Limit

I like to think of the campaign budget as your insurance. You don’t want to use it in your daily life, but should something untoward happen it should kick in and protect you from catastrophic loss.

Here are some tips on helping you set the right campaign budget:

1. If you are controlling your spend by controlling the campaign budgets you are probably paying a higher CPC than you need to and are probably not participating in keyword auctions 24/7. Remedy this by finding the right tradeoff of campaign budget and keyword level bid that enables you to get the same traffic and also participate in auctions all day.
2. If you have very high campaign budgets and are not hitting the caps, I would recommend setting a budget that is a multiple of the daily average spend. In this case, if your campaign spend is say $100 on average, you can set a campaign budget of $150 or $200. In this case, the worse case spend multiple is 1.5 times more than expected. Again, the multiple depends on your risk appetite.
3. In periods of high seasonality or short term spikes -  like Black Friday or Cyber Monday – intra day monitoring of performance would be helpful as it will enable you to understand if the advertising landscape has dramatically changed in the short term and take remedying action quickly if needed.

I hope these tips help you set your campaign budgets wisely – high enough to drive maximum performance but low enough to protect you, should something unexpected occur.

Performance data from the past three years reveals that search marketers are indeed leaving money on the table. Spend drops during the week in anticipation of Black Friday and then picks up on Black Friday.

The ROI increases by 20% over baseline too. Spend on Cyber Monday increased 50% over baseline in the past three years but the ROI more than doubled too.

Conversion rate trends tell a similar story, as conversion rates on Cyber Monday are more than double the baseline.

These trends suggest that marketers have room to maximize their performance during Black Friday week.

First, they should increase their investments on Cyber Monday to get to the consumer who is twice as willing to buy on that day than the preceding week. Second, if the overall budget is fixed then spend can be lowered on Thanksgiving day as ROI and conversion rates are lower than average.

Finally, if retailers want to analyze their own data, I would highlight a few potential caveats. Cyber Monday conversion rates are higher because many retailers lower their prices. As a result, retailers will see a lot more revenue but not necessarily margin. Needless to say, if you are margin focused then measure performance by looking at margins by day.

Further, conversion rates on Cyber Monday might be higher because consumers are researching products they want to buy over the weekend and making the purchase on Monday when the prices drop. This “priming” would help Cyber Monday’s performance.  Still, the high discrepancy between Cyber Monday and the other days around it suggest an opportunity for optimization.

If you are a retailer, I would advise looking at performance numbers by day for the days around Cyber Monday for the past two three years and if you find trends similar to the ones I find, you should consider increasing search budgets on Cyber Monday. After all, there is money on the table, don’t you want it?

Note: I would like to acknowledge Manoj Jacob, one of the crack analysts at Efficient Frontier who compiled all this data. Thank you Manoj!

In that report, I also projected that by the end of 2011, mobile spend would be between 7 and 9.5% of all paid Search spend. Well, it seems my projections are right on track with current market trends.

When I looked at mobile spend for September 2011, the share is aggressively on the rise.

Here is how Mobile usage stacked up against Desktop usage in September for retail spenders:

CTRs on mobile devices were 37% more than desktop, which is not surprising as there are fewer ads on mobile pages than on desktops. Mobile spend was 7.4% of desktop (or 6.9% of total spend).

When we dig into the granular mobile data that Google Adwords provides, we can look at the traffic split by tablets and smartphones:

What I found as that ad spend on tablets is 77% of all mobile devices! I found this was quite surprising. Further, they also represent 60% of all mobile impressions and clicks.

What This Means To You An Advertisers

1. Expect more tablet traffic. With more and more tablet devices like Kindle Fire coming on the market, as well as the continued growth of the iPad market, tablet traffic is bound to increase further. While there is little data tracking conversions from tablets, the data we do have suggests that tablet traffic converts identically to desktops.
2. Get your website mobile ready. As smartphones increase, it will become imperative for your website to be mobile optimized. Google announced recently that keyword quality score for mobile campaigns will be influenced by the level of mobile optimization for your website. So, if your website is not mobile optimized yet, it’s time you think about it.
3. Google+ and better mobile ROI attribution. From a last-click perspective, mobile campaigns seldom do well. One of the main reasons for this is that surfers on smartphones are often researching products. They convert on desktop or tablets later. However, tracking users across smartphones and then on desktops was not possible. Until now. With Google+ ,the surfer is always logged in to the Google eco system, be it on their smartphone or on their desktop. This will enable Google to better attribute conversions across devices. I expect them to provide cross device attribution reports in the future.

Finally, 2011 has indeed been the year of mobile and the trends that I am seeing are backing all the hype. As an advertiser, its best for you to take note of these trends, prepare your website for the mobile customer to better engage and monetize them.

Marketers also frequently ask if they should design ad copies solely with click though rate (CTR) in mind or not. The thinking goes like this: since the incremental effort in liking a product or brand is small and there is no monetary transaction involved, a person who clicks on an ad will likely also like your fan page.

However, the answers to these questions are a bit nuanced.

The graph above shows data from a Like acquisition campaign with anonymized campaign names for a technical product. The size of the bubbles indicate reach while the color represents the cost per like (the darker the color, the lower the CPL). I have only labeled some of the campaigns in the interest of clarity.

Several things become readily apparent from the graph:

• There is one campaign — Extreme Geeks — that has a very high CTR and conversion rate. While the CPLs are very low on this campaign, it does have a small reach.
• There is a relationship between CTR and Conversion Rate (CVR). The higher the CTR the higher the CVR. However, the relationship is only moderately strong.
• The broad targeted campaigns have good reach but poor conversion rates.
• There are some campaigns that have good reach, good conversion rates but poor CTR. These are campaigns above the regression line.

Thus, the data highlights the typical issue in scaling many campaigns. While one can highly target audiences and get great CPLS (such as the Extreme Geeks campaign here), the reach of these campaigns is small. On the other side, if one targets broad audiences, it attracts a variety of consumers and as a result the conversion rates drop.

Note that one of the broad campaigns has a good CTR but the CVR is still poor. Efforts to improve CTRs of these campaigns would only benefit partially, as high CTRs do not ensure high CVRs.

Finally, from an ad copy optimization lens, the campaigns with high CVRs and low CTRs are promising. If the marketer could improve the CTRs of these campaigns, he or she could potentially get more “likes” economically at scale.

7 Tips For Managing A Like Acquisition Campaign On Facebook

Here are some tips if you are building or plan to build a like acquisition campaign on Facebook.

1. Start by targeting audiences that have a high affinity for your product. These campaigns will have small reach but will tend to have the best CTRs and conversion rates.
2. Build out broadly targeted audiences to get additional traffic but do not bid on them too aggressively. This will help you scale initially while you build out your campaigns.
3. Build new segments that are similar to the segments that are working well initially. In the above example one would go from targeting extreme geeks to geeks.
4. Focus on improving ad copies for those campaigns that have a low CTR but high conversion rate.
5. Leverage Sponsored Stories, especially when you already have many fans. Sponsored Stories leverage the virality inherent in Facebook and can help you acquire Fans at a significantly lower CPL. I have seen several instances where sponsored stories have helped Facebook like acquisition campaigns scale without compromising on CPL.
6. CTRs of ads on Facebook will drop if they are not changed regularly. I have discussed this in detail in a previous post. To prevent this problem, I recommend you constantly test new ad copy and so that new copies are ready to go when your current ads CTRs start to decline.
7. There are times when broad targeting can work better than highly targeted segments. If your brand or service has a large social audience, Facebook appears to do a better job targeting than it would if you targeted to small segments. Hence, it is always worth experimenting with a broadly reaching campaign.

Following these steps will help you scale your campaigns without compromising on CPL.

Since this is a vast topic, I am going to focus on three specific ways in which you can analyze this data and drive insights. Each of these techniques will help you understand a different facet of consumer behavior.

For this piece, I took a dataset from a large retail advertiser and classified the keywords into four buckets: generic (“shirts”), branded (“Ralph Lauren”), generic+branded (“Ralph Lauren shirts”), sale (“discount Ralph Lauren shirts”).

I then looked at 20,000 transactions over a period of 2 months. Each transaction was tracked for up to 10 Search touch points. How did the transactions breakout by the number of touch points?

Approximately 98% percent of transactions for this retailer take place within 5 Search touch points. The average path length was 1.72. This is a bit surprising and interesting. When similar studies (most notably a RKG one) were done in the past, the path length hovered in the 1.2-1.4 range. Similar analyses now reveal longer path lengths, in the 1.7-2 range. This indicates a savvier customer, more intent in researching a product before purchase.

Measuring The Branding Effect

Many advertisers are trying to measure the branding effect of generic keywords. The theory goes that many users start with a generic keyword and end up converting with a branded keyword. Does the data bear that theory out?

The graph above shows how transactions beginning with a certain keyword type ended. For instance, 89% of transactions that began with a generic+brand type keyword converted with the same keyword type, 5% of generic+brand type keyword converted with a brand keyword and so on.

Notice the strong diagonal (89%, 95%, 89% and 90%) indicating that for the most part users starting a transaction with a certain keyword type will end their transactions with the same keyword type.

What is the non-brand to brand spillover here? It is 7%. (NOTE: in the brand column, 22% of  117%, e.g. 7% of all brand conversions, began with a non branded term).  In other words, the branding effect is small for this retailer.

Key Takeaways For The Advertiser

Clickstream analysis can provide valuable insights into consumer behavior. In this article, I have presented one way in which the data can be mined to provide useful insights.

If your average click path is long, it reveals a more bargain- and research-centric consumer. Ad copy must reflect this mindset. However, a shorter path length reveals a customer with immediate intent to buy. Ad copies for these types of terms must reflect the urgency in the consumers’ mindset.

While the branding effect for this dataset is small, it might be different for you. If you do see a strong branding effect, you must account for this in your bidding decisions for the generic terms. While looking at generic terms from a purely last-click perspective might make you more efficient, it might undercut the performance of your brand terms.

(a)  Identify the keyword/adgroup sets for which the new landing page will be tested.

(b)  Set up identical ad copies in the ad group but with destination URLs pointing to the original as well as the test landing pages.

(c)  Set up the ads to rotate evenly.

Most of the time, this setup requires you to change the account structure to run the experiment. This is seen schematically below.

Figure 1: Original Adgroup structure

Figure 2: Testing Adgroup structure

There are a few problems with this approach:

• It is not scalable: If there are M ad copies in an ad group, you have to create 2M ad copies in the new structure to do the A/B landing page test.
• You cannot control impression serving (beyond a point). By enabling an even rotation, you split traffic evenly across the ad copies. In the above example, each landing page gets 50% of the impressions.

While one can overcome the first problem with automation tools, the second issue is problematic esspecially when testing landing pages in conjunction with high traffic brand keywords.

If the new landing page doesn’t perform well, you run the risk of poor quality of conversions for half the traffic. There are many ways one can over come this, but they tend to be cumbersome to set up and also make the testing process complicated. The ideal solution, would be to:

• Not change the mix of ad copies across the impressions that the adgroup gets.
• Show a user defined percentage of impressions the new ad copy. If you are risk averse, only a small percentage of clicks should be served with the new landing page.
• Require minimal work on part of the experimenter to set up the experiment.

5 Steps To Using Adwords Campaign Experiments

The Google Adwords Campaign Experiments (ACE) platform offers a nice way to execute these landing page experiments. While detailed instructions on their setup have been provided here, I am going to skim over the mechanics in connection to the landing page experiment. The ACE platform setup has four necessary steps, and one followup action to take based on the results.

1.  Experimental setup

This can be done in the Campaign settings tab under the Advanced Settings option:

The key input here is the Control/Experiment split which enables you to decide what percentage of traffic goes to the experimental landing page vs. control. In this example, I have picked only 10% traffic to the experiment, as I am risk averse.

2. Make  the experimental change

Next, you need to decide the experimental change. In our example, we create an identical adgroup to the one we want to test, and change the destination URLs of the adcopies to the new landing page. Once the ad group is created, you have to choose the original ad group as the control and the new duplicate ad group as the experimental one. To do this, click on the empty beaker next to the adgroup column in the campaign screen.

3. Run the experiment

This can be done manually by clicking on the Settings>Advanced Settings> Experiment> chose Start Running Experiment button or automatically if the start and stop date of the experiment was selected in step 1.

4. Analyze the results

Here is an example of the type of results you would see:

Note that the test ad group got approximately 10% of all impressions which is what we wanted. You can run a test (Z test) to check if the conversion rates are different with statistical confidence.

In this case, the test does show that the conversion rates are statistically different and the test landing page is better than the existing one. Incidentally, the Adwords editor does provide statistical measures of confidence in their reporting itself and you can find details in Google’s help pages.

5. Deploy your results

You can now test the results on a larger sample (50/50 or 30/70 split) to ensure that the new landing page will perform when more traffic is diverted to it. If you feel confident however, you can divert all traffic to the new landing page by going to the settings tab of your campaign and clicking on Advanced Settings>Experiment> Apply: Launch changes fully button.

In this way, you can A/B test new landing pages on high volume keywords without risking significant downside as well as account restricting headaches.

While, ACE is an excellent and flexible platform for advertisers for many types of A/B tests, it would be great to see  more testing features such as multi-variant testing capabilities in the future.

I would like to thank Rob Levetsky, Sr. Account Manager at Efficient Frontier for providing me with several examples and use cases of Landing Page testing with ACE.

In this post, I shall glean insights about purchasing behavior from transaction data. For the analysis, I classified purchases between $1-$300 as small ticket and $300-$1000 as mid ticket items and over $1000 as big ticket items. I then looked at one month worth of data spanning over 50,000 transactions. Several significant trends emerged including:

People Are Price Sensitive… Somewhat

When we calculate the distribution of items by average order size, an interesting pattern emerges. While the biggest peak is around $65, the distribution shows spikes at regular intervals. For small order sizes, the blips occur every $20 or so while for the larger ones they occur every $50.

This interesting effect is due to two possible reasons (1) Items being priced close to certain “threshold” points ($99.99) and (2) Customers having different pricing thresholds in their mind. They might not be inclined to make a $210 purchase but a $195 purchase might be fine with them.

While the true cause is debatable, the decreased price sensitivity with higher order values is clear i.e. the more expensive the item the less sensitive the person is to the price.

What about purchasing time? One surmises that consumers would take a longer time to purchase bigger ticket items. Which brings me to my next insight…

The More Expensive An Item = A Longer Purchase Cycle

The graph shows that this is indeed true. If you regress the median order value vs average transaction time, you would get a nice relationship between transaction time and order value.

But what about the last bar? It shows that the median transaction time for very big ticket items is only 4 hours; 5 times lower than the very low ticket item transaction times. What is going on there?

The Offline-Online Effect

A deeper investigation revealed that many purchases in this category took less than 10 minutes. Further, while they were few and far between infrequency they contributed substantially to the revenues for the campaign.

Thus, just as discussed last time, big ticket items are a study in outliers. The data also suggests that many of these “quick” transactions happened because of offline activity. Consumers went offline to look at the products in the store and purchased the product online because of an incentive or because of the convenience in the purchase process.

Takeaways For Marketers

The order size distribution curve suggests that consumers like to buy at arbitrary price points for smaller size orders. It also suggests price consciousness for smaller ticket items. Advertisers will be well advised to experiment with price information for smaller ticket items.

As consumers take a longer time to purchase more expensive items, different strategies of SEM bid management should be employed for small and big ticket items. As we discussed last time (albeit from a different point of view), small ticket items should be monitored closely to gain maximum efficiency from the campaigns while keywords related to big ticket items should be monitored with a longer time window.

Further, ad copies for big ticket items should be more upper funnel, inviting the consumer to research the product. However, this should be back with solid content onsite giving all information possible.

We have often heard of online searches converting to offline transactions but for very expensive items the opposite might be true. This is tantalizing as it opens several new questions. Should store reps who engaged with customers for a high value product consider their efforts as lost when consumers leave the store? Perhaps not.

But then retailers could provide CRM tools for their sales reps to connect with their prospects once they leave the store. This would let brands measure this effect.  I would love to hear your thoughts on this idea.

An SEM manager for such a retailer is often faced with several campaign management questions.

Should the merchant focus on drawing more volume from small ticket items or from sparse but high value big ticket items? If she is tasked with profit maximization should she look at keyword performance for all keywords the same way? Do her customers first search online then purchase offline or vice versa?

As I analyzed a large amount of transactional data, I first classified the transactions in the categories of small, mid and big ticket items. Small ticket items were those that had an order size between $1 and $300, mid ticket $300 to $1000 and big ticket items are above $1000. When classified this way, the revenue and transactions break up as shown below.

Small ticket items contribute to 91% of all transactions but only 43% of revenue. Big ticket items on the other hand contribute only 3% of all transactions but 37% of all revenue. Clearly, while big ticket items are small in number, they are big revenue drivers.

When a marketer tracks daily transactions coming from small and big ticket items, another pattern becomes very apparent. Small, mid and big ticket items are statistically different in behavior. Small ticket items, show a definite pattern: Strong performance on Mondays, decreasing steadily over the remainder of the week.

The difference between best and worst performance over this time period was 25%. Mid-ticket items exhibit more variance but is quite steady. Big ticket on the other hands exhibit significant volatility. The best performance in this time period was over $130,000 while the worst performance was $20,000, over a 6x difference.

To use a football example, small ticket items are like your trusty running back, plowing through in every play getting a few yards, while the big ticket items are like the temperamental star wide receiver who can make huge plays but also fumble the ball.

Keyword Management Tips

Just as people with varying temperaments should be managed differently, keywords for small, mid and big ticket items should be managed with a different mindset.

Keywords relating to small and mid ticket items should be managed tightly with a shorter look back window. They exhibit low variance and steady performance and hence one would do well to apply short term reactive approach in managing them.

Keywords relating to big ticket items will deliver but they will take more time on average.  They also exhibit a lot of performance variance and hence might burn through a lot of cash before making that big conversion that makes it worth the effort.

Ideally, they should be managed with sophisticated statistical models but if you are using spreadsheets to manage these keywords, I would suggest using significantly longer lookback windows. You would be prudent to err on the side of bidding aggressively on these keywords taking a short term loss in exchange for a big long term gain.

This analysis opens up many questions on the behavioral side. Is there is a relationship between order size and time between click and conversion? Do people exhibit sensitivity to certain price points?

All this and much more in my next post.

However, there remain some fundamental differences in the two platforms which can be categorized under two broad umbrellas – strategic objectives and tactical campaign management.

While much can and has been written about the strategic differences in Facebook advertising, I shall focus on a key tactical aspect of Facebook campaign management – Ad Fatigue.

What Is Ad Fatigue?

Search marketing is a “pull” type of advertising. The searcher declares intent and the marketer shows her ad based on the expressed intent. Facebook (and display) are more of a “push”. Typical Facebook users are not looking for a product or a service but are looking to interact with the platform. As a result, they are less likely to click on the ad. This fundamental difference has two consequences:

(a)    The CTRs of Facebook ads are much lower than Search ads – an obvious effect.

(b)   Your ads will potentially be less effective over time as your ads will be shown to a user base repeatedly. Fatigue is less of an issue in search because the lower funnel users are always changing and are also more intent on purchasing. However, Facebook users, on average see the ad many more times before a purchase and hence are more likely to get tired of the creative. This sounds good in theory but does the data bear it out?

The above graph shows the click, impressions and CTR of a Facebook ad over a period of 12 days. While the bid was exactly the same the number of clicks it received dropped by over 30% in a span of two weeks. So is this ad fatigue? Yes and No.

If it were purely ad fatigue, the number of clicks would drop in proportion to the drop in click through rates as the proportion of users inclined to click on the ad would drop over time. The graph shows that CTRs have dropped but not as much the drop in clicks. So while there is some ad fatigue there must be another factor in play.

A closer look at the impression volume reveals that a substantial trend in the drop in clicks can be explained by the drop in impressions. The drop in the number of impressions is not a function of user interest but is a function of the Facebook platform. Facebook in a sense decides that an ad is getting old and lets the ad participate in few auctions. As a result the overall impression volume drops over time.

Tips To Combat Ad Fatigue

While the actual causes of ad fatigue are debatable the consequences are clear. If left alone, Facebook ads will quickly decline in performance. Here are some tips to mitigate the problem:

(1)    Set most of your budget for performance but a small fraction of it for creative testing. As you identify new winners in your creative test, you can apply the learnings to the performance oriented campaigns.

(2)    Refresh your creative with the winners from your test campaigns on a weekly or bi-weekly basis to combat ad fatigue.  As a general rule, if you see impression or click volume drop at the same bid and budget levels, it will be a good time to change your ad.

(3)    Test creatives in a systematic manner. The testing has to be done in three fronts – the image, the message and the demographic. Ideally you want to identify the right creative, image combination for every targeted demographic.

(4)    Unlike search, setting up a systematic ad testing process on Facebook campaigns is a little involved as Facebook does not currently allow you to rotate ads evenly. To overcome this, you can place your creative/target combinations in different campaigns and allocate separate budgets. This will force spend on the combinations you want to test.

These tips will help you continuously optimize the creative on your Facebook campaigns while combating ad fatigue.

Statistics tell us about general trends and properties about our data but often they hide information that would give key insights.

The goal of every analyst then is to make statistics skimpy – but at the same time, make them as revealing as possible while not overwhelming the decision maker with unnecessary numbers.

In this post, I will focus on working the average and median statistics. While widely used to describe trends and data, they have a few limitations which can give you a wrong impression of what is going on.

Consider the revenue from three campaigns shown below:

The following patterns are readily apparent:

1. The average revenue for Campaign 1 is close to $1000 a day.
2. For campaign 2, the average revenue appears to be increasing and it has also had a few exceptional days.
3. Campaign 3’s performance appears to fluctuate a lot.

Taking the average and median revenue in the time period for all three campaigns, we get the following graph:

The average revenue of all three campaigns is comparable. Campaign 2 appears to have a significantly higher average revenue.

However, when we look at the median revenue (i.e. the middle value of the revenue data), we see that Campaign 3 has the best performance, while Campaign 2 has the worst.

Since we have daily data, we know that Campaign 2’s apparently superior average is due to two primary reasons:

• Performance has gradually improved with time
• It has had a few exceptional days of performance.

Moreover, Campaign 3’s apparently superior median is due to its performance volatility. Clearly, both the mean and median statistics provide a partial picture about performance and can lead you to mistaken conclusions.

The simplest solution that comes to mind is to not provide statistics but provide time series data like the first graph.

Unfortunately, this clearly defeats the purpose of providing statistics. That is, to compress large volumes data into simple, digestible metrics that provide clear and concise information about the nature of your data.

One of the simplest ways to overcome the problem with the mean statistic is to use boxplots. I have drawn a boxplot of the 3 campaigns below.

In a box plot:

• The left end of the left line represents the minimum value
• The left line of the darker box represents the 25^th percentile
• The middle line separating the 2 shaded boxes represents the median
• The right end of the lighter box represents the 75^th percentile
• The right end of the right line represents the maximum value

This box plot shows:

(a)    While the median value of Campaign 2 is the lowest, there is a high degree of variance as indicated by the large box size between the 25^th and the 75^th percentile. The high max value hints at the days of exceptional performance (outliers).

(b)   Campaign 1 has the steadiest performance as the box width is the least of all three campaigns.

(c)    Campaign 3 has moderate volatility but has a lot of days of poor performance as the dark shaded box is larger than the lighter shaded one.

Thus, all key performance trends have been captured in this chart. An analyst reading the chart will draw the right inferences and analyze the data to confirm her hypothesis. Also note that the real estate occupied by the boxplot is the same as a bar graph. So it also achieves the goal of conciseness.

While making boxplots in Excel is not straightforward, there are various resources online to teach you how to make them relatively easily. These plots will enhance the mean statistics of your data and ensure that you will seldom miss out on key trends and insights on your data.