Analytics news and expert advice every Thursday.
6 Interview Questions For Conversion Optimization Hires
So you’re looking to hire someone to do conversion optimization on your team? Here are six interview questions that you might consider asking. The answers aren’t necessarily black-and-white, but these kinds of questions can help reveal how a candidate thinks about conversion optimization.
1. How would you approach deciding what to optimize first?
For most businesses, the potential universe of optimization opportunities — from their high-level home page down through highly targeted campaigns that have yet to be conceived — is extremely large. How would the candidate decide what to prioritize?
There are many ways to tackle this. Which e-commerce categories or demand generation campaigns have been most successful? Which have been least successful? Which search queries are generating the most traffic? Which of those have the lowest bounce rate, the highest bounce rate? How do these data points intersect with the company’s strategic direction?
In my opinion, a good answer reveals an analytical approach to identifying opportunities, while at the same time demonstrating an openness to finding the right analytical lens for your specific business — not necessarily a cookie-cutter checklist. A great candidate should ask questions about your existing analytics and other factors driving your business priorities.
2. How do you think about audience segmentation?
There’s a natural flow of conversation from priorities to audience segmentation. Which audiences are the most important to the business and why? How do you identify them? How do you serve them best?
What are the possible dimensions by which the candidate even thinks to segment people? New customers versus existing customers is an obvious (but important) one. Vertical markets too.
Demographics have a rich legacy in marketing, so segmentation options along that axis can open up some good possibilities. But I’d love to hear a candidate open the door to more modern segmentation approaches too, such as personas or Clay Christensen’s insightful question, “What job are you hiring this product to do?”
Upon brainstorming different segmentations, a good follow-up is to discuss ways in which those segments can be identified. By answers they fill out on a form? Previous purchase histories? The ads they clicked on — and the keywords or site contexts in which those ads were placed? Behavioral choices on landing pages? Data from services empowered by cookies or IP address look-ups? What are the strengths and weaknesses of these different mechanisms?
3. What was one of the most surprising outcomes to a test you’ve run?
A/B testing and multivariate testing (MVT) are the workhorses of conversion optimization. Experienced conversion professionals should have a rich history of tests to draw upon. Which ones were the most memorable? And why? While this is certainly an opportunity for a candidate to (humbly?) note some of their biggest successes, I think the best answers to this question reveal courage and creativity.
In particular, how bold has the candidate been in trying big and imaginative ideas? Did they push the envelope, or did they play it safe?
What were some of the hypotheses that motivated their testing, and how did their perspective evolve over a sequence of tests? Were there any tests that were spectacular failures? What did they learn from them? If they haven’t taken any risks that have failed — or if they’re uncomfortable admitting them — that would be a red flag.
4. In your testing, what variables do you try to control? What potentially confounding variables are outside of your control?
There are two alarming answers to this question. One is an unawareness of any variables to be controlled in conversion optimization. The other is the opposite extreme: confidence that their experiments are always perfectly controlled.
The reality is that confounding variables are some of the biggest — yet rarely discussed — challenges in conversion optimization. Sometimes respondents come from different traffic sources — the same ad served for two different search queries can send traffic with very different intent.
The timeframe during which a test occurs might have any number of externalities, from holidays to major events in the news. Other events that are happening with your company or within your industry — especially in our Age of Social Media — can wildly impact your results.
There’s no perfect solution. However, savvy conversion professionals control the variables that they can, within reason, while remaining vigilant to a wide range of external factors that can skew their results.
5. How do you weigh the trade-offs between brand consistency and experimentation?
There’s no single right answer to this question — it depends a lot on your organization, your brand, your audience. But since conversion optimization usually plays the role of championing change to the status quo — while still leveraging, and hopefully enhancing, the value of your underlying brand — it’s insightful to hear how a candidate thinks about this trade-off.
How far should you experiment with the look-and-feel of your landing pages? When you should engage in special offers (and what kinds of special offers)?
In addition to considering different points along this continuum, a good conversion optimization professional should have suggestions for mitigating the risks of experiments that “poke the box.”
6. Did you ever have a time when you had to deal with a HiPPO?
Everyone in conversion optimization should be familiar with the HiPPO — Avinash Kaushik’s label for the highest-paid person’s opinion, emphatically presented without regard to testing or analytical data.
In practice, it’s not just an affliction of executives and senior managers. Almost anyone on the marketing team or the company at large may voice strongly-held opinions on the direction of your conversion optimization programs.
The question is how a candidate deals with such situations. A masterful conversion optimization professional is able to gracefully channel such input into productive efforts. It’s important to not give in or give up in the face of thundering viewpoints.
At the same time, a diplomatic touch goes a long way to keeping people engaged in a positive way. Great interpersonal skills are golden here. Listening to a candidate describe their previous experiences managing these dynamics can tell you a lot about what it will be like to work with them.
Are there other suggestions for good conversion optimization interview questions? Share them in the comments.
Image courtesy of Wikimedia Commons.
Some opinions expressed in this article may be those of a guest author and not necessarily Search Engine Land. Staff authors are listed here.