The What, Whys & Hows Of Multiple Metric Optimization

The beauty of online marketing is that every step in the click path is visible. A searcher clicks on an ad, lands on a merchant’s site and navigates to purchase—all of which can be viewed and analyzed.

This process is often referred to as multiple metric optimization. However, this term is often misused so it is worth spending some time on explaining the process and how marketers can use it to their advantage.

Simply put, multiple metric optimization refers to the class of methods that seek to maximize a marketing goal (for example, ROI/Revenue) using more than one event in the click path. Examples of these metrics are clicks, leads, revenue, purchase, visit time, times of day etc. It is important to note the word “optimization” in the context of bid management necessitates the need for statistical algorithms. Applying arbitrary rules or filters to five or so metrics does not mean you are optimizing to multiple metrics. Perhaps an example will make it clear.

Consider that you run the marketing campaign for an online magazine. You let visitors get a free one month subscription and after a month, many of them sign-up for a year. You realize revenue only when a free trial converts to a subscription. As a smart ROI conscious advertiser, you want to advertise to maximize subscriptions. Mathematically speaking, you want to maximize subscriptions subject to a budget/CPA goal by using a constraint such as not exceeding a certain amount. However, you then need only maximize to the revenue event i.e. subscriptions so there is no need for multi-metric optimization. So why bother?

The trouble is that between a trial and a subscription there is a 30-day delay. When you have enough data, you can predict the revenue generating potential of every keyword. However, this is seldom the case for long-tail search terms and sometimes even for mid-tier terms. For example: If 10% of all clicks convert to registrations and 10% of those registrations convert to subscriptions in a 30-day period, we are in effect saying that one in a thousand clicks becomes a subscription in a 30-day period. In other words, if a keyword on average got 30 clicks a day in a 30-day period, you could expect one subscription from it. By most definitions 30 clicks per day would be considered a mid-tier term. Hence, building good revenue predictions for most mid and tail terms using only subscription data would be difficult if not impossible. Enter multiple metric optimization into the equation. If a marketer uses another richer dataset to predict subscriptions, he or she might be able to build better revenue predictions.

A richer dataset can be created by combining the registration and subscription data into a new metric which acts as a proxy metric for subscriptions. The question is, how to create this metric. Here are the steps:

Step 1: Identify the relationship between the metrics

A simple way to identify the relationship between the metrics is to apply linear regression to the metrics. For 2 metrics, as in this case, it requires plotting registrations vs subscriptions.

In this chart, the registration line tells us that on average 6.3% of registrations become subscriptions. In other words, if you were to use registrations as a proxy on the tail terms, you should consider a registration worth only 0.063 a subscription. The example in the table should make this clear

Thus, we estimate keyword 2 will do better than KW1 even though KW 2 does not have a subscription as of now.

Step 2: Create the proxy metric

The example above should make it evident that, in this case, the proxy metric will look like:

Proxy Metric = Subscriptions + 0.063*Registratons

Step 3: Build models to the proxy metric and set bids after optimization

Once the proxy metric is created, every keyword should be modeled with the proxy metric as the revenue metric. After this, optimize the keyword set to maximize the proxy metric.

In the interest of brevity and to keep it from becoming too mathematical, I have made several simplifications in this analysis. However, several key points should become readily apparent. First, the metrics and the weights came from sophisticated statistical analysis. Second, even this simple case is relatively mathematically involved. We only looked at two proxy metrics. For more metrics multivariate regression methods would have to be used. Lastly, even though the relative weights were determined, we did not discussed the bids that would actually optimize to both. It requires optimization algorithms as any manual method would be a half-baked heuristic.

So the question is do you really need to optimize to multiple metrics? If you have a simple business model with a short sales cycle (less than a day for over 80% of your transactions) and where you are maximizing one metric (say revenue) then the answer is usually no. Remember that you usually have one metric and several predictor metrics. If the dataset is rich enough, then its better to optimize to the real thing isn’t it?

However, when the sales cycle is long and you are using long tail keywords, its an option worth considering. Also, remember that just because you are tracking 50 metrics you shouldn’t optimize to all. More is not always better. Metric selection for optimization should be done systematically via regression modeling and keeping only as metrics needed for the estimator metric to make your predictions accurate and robust.

In summary, multiple metric optimization is a powerful optimization technique that can benefit many advertisers. Among Its key benefits are predicting the conversion/transactional metrics when you do not have enough data or predicting performance when sales cycles are long. This is especially useful for the long tail where data is sparse.

However, the technique is not for everyone and is to be used with caution using sophisticated algorithms. Developing a sound modeling methodology coupled with a good optimization engine is key to make this strategy successful.