Recruiting: 12 Questions For Uncomfortably Hands-On SEO Interviews

I was shocked by the widespread, deeply negative reaction to the introductory paragraph of my column last month, in which I described beginning every interview with a math problem.  The response is probably best typified by EricM:

Despite missing the entire point of the article (that you can use simple free tools to calculate confidence intervals for A/B split tests), Eric did raise an interesting challenge:  “are you going to let them go online and use this tool at the interview?”

The answer? Of course I am. Why the hell wouldn’t I?

I wouldn’t expect anyone to do their job without access to the Web, so why on earth would I conduct an interview devoid of a computer and Web connection? That would be liking hiring a jockey without seeing him ride or a heart surgeon based on her MCAT scores.

If you really want to assess how someone works, attach a laptop to a projector during the interview and give them free reign to use the Web as they see fit. You’ll see proficiency with Excel, intimacy with third party tools and quickly build a map of how their mind functions. Nothing serves better in separating theory from practice than watching how someone works.

Over the past 7 years working in search, I’ve refined my interviewing process down to a two page cheat sheet. The result – I’ve hired some of the best co-workers imaginable. Smarter than me. More technical. More detail oriented. More experienced. More creative.

Depending on which side of the desk you are on, what follows is either a “how to nail a search interview” or “how to hire awesome search talent.”  Note that this is admittedly through the heavily biased lens of a business guy within an aggressive mid sized business – one size won’t fit all.

My first question is always a math problem. This sets an immediate precedent that the job is highly analytical and weeds out the marketers who would rather talk about what kind of tree their brand would be if it were a tree.

It also lets me assess if this is going to be a quick, polite 20-minute conversation or an in-depth hour long dialogue. I actually watch the person’s physical reaction to the first question, if he or she is uncomfortable with analytics its going to show. Any answer that hints “I’m not really good at math” = 20 minutes and a handshake.

Below are a set of questions I draw from:

Question 1

“You are A/B testing two different ad creatives . . . here are their impressions and clicks. Imagine on your first day, I tell you to optimize the campaign of 50 keywords, what do you do?”

Better yet, put data in front of them from a small Adwords account and just ask a simple, “What do you do?”

While this isn’t natural search specific, its an easily understood simplistic pre-Algebra problem. The (insufficient) math-only answer is to calculate CTR and declare a winner (per EricM above).

Years of schooling have trained us to identify a math problem, calculate the answer and move on to the next problem. This is the wrong approach and yields the wrong answer.

What I’m really looking for is some holistic, out of the box thinking that is driven by a business orientation and technical experience.

The right answers (yes plural answers) include:

  • Consideration of sample size and confidence intervals.
  • Looking downstream for differences in conversions in the creative.
  • How can I further segment the 50 keywords to get more granular groups.
  • For anyone with any PPC in their background, I’d expect some discussion around Quality Scores and landing page optimization testing.

Question 2

“In the above example, calculate the ROI of Ad A.”

This is a great, albeit obvious test for business orientation. The engineer who can get this question is surprisingly rare, but amazingly valuable. A “marketing person” who can’t come up with the basic formula isn’t going anywhere.

Also note that the data I’ve provide the candidate so far doesn’t include anything around revenue or costs. I’m looking for the candidate who will quickly ID those missing elements, then put them together.

Question 3

“How would I use an analytics package to track reservations made on Urbanspoon?”

I’m looking for an understanding of how conversion tracking works – ideally within a large analytics package, like GA or Omniture.

Question 4

“According to Comscore, Company XXXX seems to been outpacing our growth in search over the past 12 months, why do you think that’s the case?”

I’m looking for answers that utilize some of the paid or unpaid tools on the market. I’d expect someone to recommend doing a comparative link evaluation using tools from Majestic, SeoMOZ, Raven or even something free like Blekko.

This is a great time to let the candidate actually log on to a tool and walk you through how they use it. Having a Majestic account is very different from being able to use one; in the same way that having a gym membership has not slowed the progression of my increasingly soft midsection.

Question 5

“Tell me about Caffeine or May Day.”

I’m not looking for my favorite beverage or descriptions of young girls dancing with ribbons. These are two named algo updates from 2009/2010 and familiarity with them indicates some duration of hands-on search. If you are looking for someone with serious longevity, ask about the Google Dance.

Question 6

“Here is an excel file with data for all of the 600,000 restaurants in the United States from Urbanspoon. Turn this file into a list of cities with more than 1,000 restaurants.”

The answer involves pivot tables and should take about 1-3 minutes.

Question 7

“Now, imagine we purchase a list of restaurant emails and want to append our huge database with those emails … how would we go about doing this?”

The answer involves a VLookup, which can be persnickety and tedious – fair to assume if they know the answer, they can work through the actual implementation.

Question 8

“We’re testing television advertising to evaluate our ability to extend our brand from a directory to a reservations destination. How would you evaluate the ROI of this?”

This question requires a fairly complex answer and I’m looking for nuances that include:

  • What is your business objective?
  • How is the test set up . . . what is the control I’m comparing this to?
  • Some concept around lift and then extended lift (this is more of a branding answer, but ties into the notion of multiple touch points instead of focusing entirely on last click attribution – which is a very common mindset.)
  • Search bonus for looking for lift uptick in branded search as well as direct traffic.

Question 9

“How can we rank better for ‘Romantic Italian Restaurants in Redmond’ – here’s the page, but it ranks on the second page for that term.”  (Assume said page has standard on page optimization in place.)

This is a red herring question - what I really want is for someone to tell me something along the lines of:

  • “ranking reports are a waste of time and here’s why”
  • “you should look at the inbound search traffic instead of the ranking report for a specific term”
  • “this is a long tail term, if you spend time chasing every long tail term, you’ll miss the forest for the trees.”

Question 10

At some point, I’m going to dig really deep on something where I have a lot of experience (say local or news search) where I know the candidate is going to hit the ceiling of her knowledge.

In this case, the right answer is, “I don’t know.”  There’s a lot of changes in search, making it impossible to know everything. Being able to tell your interviewer (or boss) “I don’t know” is very important. There are already enough people trying to BS their way through the search industry and I don’t want them working for me.

Conversely, (and just as importantly) I want the candidate to enlighten me. A great candidate should be able to teach me something.

If all else fails to draw this out, try:  “Tell me something I don’t know about search.”  The close cousin of the tell me something question is forward looking: ”what is the next big thing we should be thinking about?” or “My CEO is all atwitter about Pintrest, how should this play into our search strategy?”  

Question 11

“Tell me what you know about me.”

This isn’t a narcissistic probing into my favorite topic; but I’m really asking to see if a) the candidate has done any preparation and b) if the candidate is social. The integration of social into search makes this question increasingly relevant, especially for smaller entrepreneurial companies that may not have a “social” department or dedicated function.

Smart candidates avoid this question by dropping hints about the interviewer’s background during the interview: “I saw you played rugby”, “lived in Ireland”, “have kids” to let the interviewer know they’ve done some research. This can also take the form of, “In the New York times article last week about Urbanspoon…”

Question 12

Finally, the best question in any interview is a series of “whys”. I interrupt repeatedly with why.

Why? Because it challenges statements. Why? Because some candidates try to gloss over hard questions with assumptions. Why? Frequently, I find explaining assumptions leads to discussions on fascinating tangents. Why? Because their experience is very different to mine and I want to know what they’ll bring to the table.

Onerous interview process? Uncomfortable? Maybe. But, I’d much rather have a tough hour long interview than a tough month or two or six with a poor fit. And let’s be honest, anyone worth his or her salt is going to have to answer much harder, on the spot questions across the organization every week.

For more on hiring awesome SEOs, check out Luc Levesque’s post:  How to Sniff out Rockstar SEO Talent.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: Channel: Search Marketing | In House Search Marketing

Sponsored


About The Author: is the founder of Atticus Marketing - a search agency dedicated exclusively to the legal profession. Prior to Atticus, Conrad ran marketing for Urbanspoon and the legal directory Avvo, which rose from concept to market leader under his watch.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://twitter.com/andrew_rayner Andrew Rayner

    Just a small point on Q9, this could be addressed best with Google+ Local (formerly Google Places – can’t believe it’s changed it’s name again!)

  • Bill Hunt

    Conrad – great post on how to do a good technical position interview.  I do think companies should adopt this sort of hands on process since it does show how prospects think and solve problems.  I have asked people to bring their laptop if they have one to see what tools they use and how they think since people your looking for most likely have done this before. 

    On Question 4 – I would be discouraged by someone who went to Majestic or SEOMoz to answer this question since that would tell me they are a link building fan boy and not a problem solver.  Your question “why are they outpacing our growth in search” and the data from Comscore is a function of traffic.  I would want them to be more focused on the delta in traffic and provide a process to compare top 1000 words us to them in paid and and organic and see where the largest gaps are vs. where we have fewer links.  That step would come after they know which keywords we have a gap on.  A majority of the time the gap can be closed by fixing crap snippets for high ranking terms.  Then we move into improving rank via template changes and link building.  That to me shows the difference between a small site search pro and one that can work at the scale of the enterprise. 

    Are you having success finding people that can work in paid and organic and be an Excel master?  I find people are good at one or the other and few other than those who have had their one sites or been a 1 person show at a smaller site have these blended skills. 

    And for those that will copy this as your process – if you find a person who can actually make it through this gauntlet don’t insult them by offering them a $35k salary.  

  • Christopher Regan

    Honestly, I’d expected another Fortune/Huff Post crappy count-down/up fluff piece when I clicked into this article. But, alas, great points — and, well thought-out and well written. Question #11, when asked, will astound you most often with silence, though….
    Thanks.

  • Rachelle King

    When I read question 11, I thought How Cocky!  Then I realized once you stated “a tough month or two or six with a poor fit”, this question made a lot more sense. It not
    only touches on social integration of search but if someone can answer this
    then it means they took the time to care about the person they could be working
    with day and day out. No matter how great a person’s skills and knowledge are
    if you clash on a personal level it can make the work environment strained
    which takes away from productivity, not to mention actually enjoying your job.

  • http://profile.yahoo.com/3NIT4IMLYV5FGVTCXYWEPTKYDQ Christian

    You had me at “Ranking reports are a waste of time…”

  • http://twitter.com/kirstyshark Kirsty Shark

    As someone who is relatively new to the industry and would love to get to the level where I could make it through that interview in one piece, but how the hell does someone learn all those things?

    I’ve been devouring everything I can online but I still feel a million miles away from what you describe above. Any courses in my area are way too basic. 

    So maybe for a follow up post, how can people learn to be an SEO pro? Is it all networking and reading the right things online, if its experience how do you structure your experience in the right way and are there actually courses out there that cover advanced SEO well?

  • http://www.facebook.com/AlessiaCorriere Alessia Corriere

    I am totally with Kirsty on this one… I mean, I don’t think that I’d be able to answer most of those questions, apart from a few. But, I’d love to be able to get to a level where all of these questions are a walk in the park. 

    Thanks :)

  • http://twitter.com/cryptblade cryptblade

    Hard to even source technical SEOs. The “SEOs” these days are pathetic. I’ve had to sit through many interviews with SEOs who never touched HTML. WTH???

    Sat through 1 interview with a jobseeker – she claimed confidence in knowing SEO and learning quickly. Claimed she did SEO work on title tags previously. So I asked her, in general, what were the accepted character limits on title tags? She had no idea.

    …I think she knew she bombed the interview from that point.

    Sooo…. mazeltov to Conrad for even SOURCING candidates that are technical enough for him to get through any of those questions.

  • http://www.facebook.com/profile.php?id=507536131 Michael Shostack

     While not on the SEO side, I interview lots of people for the paid search side of things from entry-level up through manager level. Many of the advanced things we know are from hands on experience, and living and breathing the business every day. It is also about being connected with the right folks so this info gets shared with you, and of course religiously reading blogs like this one because there are no good schools that teach this stuff.

    In general though, if you are more junior, I think everyone would agree that you are not expected to know all of this.  The more important skills to show your interviewer (and what a good interviewer will be looking for) would be how you approach the problem when you don’t know the answer.  Most of our day is spent addressing problems we don’t know the answer to, and being able to find those answers is what makes someone valuable int his field.

    Of course if you don’t know answers to these sorts of questions and are going for a higher-level position, you might need to reevaluate where your experience places you in the market as SEM/SEO is unfortunately one of those areas where having a ton of related experience in say, display, or email marketing, doesn’t qualify you for more senior roles (or even mid-level roles) in these particular disciplines due to how technical they are.

  • http://www.facebook.com/profile.php?id=507536131 Michael Shostack

    I think an important thing to add to this thorough process for a technical interview is to clarify what level of candidate these questions are for.  I would never expect a junior person to know all of these things (although I would be watching to see how they approached figuring them out to measure problem solving ability).

    That said, for technical roles, it is often disappointing since the majority of candidates for higher-level roles I’ve found to be lacking on the technical side lately. Definitely important to have some sort of pre-interview screening checklist or test to weed them out so you aren’t wasting even 20 minutes of your time with someone interviewing for a senior role who doesn’t know how to calculate ROI.

    Depending on where they come from, I might be more lenient on the tracking side, particularly depending on if they come from certain agencies that are known for having silos between the analytics/tracking team that implements and handles all the technical tracking stuff, and the people that do the work on the accounts.

  • http://www.facebook.com/profile.php?id=749852742 Conrad Saam

    Kirsty and Alessia – First, thanks for making it through the post.  Secondly – sadly I find very very little education that is timely and detailed in the search world.  Colleges and training centers just aren’t picking up the right people who are advanced and current in search.  

    Thirdly – there’s lots of bad information out there.  Lots of it.  Many companies with solid reputations in the industry are either shady or misguided.  I’ve found that many agencies cater to their clients (which makes sense at face value – but few of their clients really grok what it takes to be successful in search.)

    So what to do?  Now this is my opinion, and I’m sure I’ll be flamed for this, but . . .  if I were advising someone on how to get into the industry, I strongly believe the best way to do it is to work in house for someone who really gets it and is current.  Someone who believes in in-sourcing this skill b/c they take a very long term view of search.  Someone from the old guard who has gone in-house.  Someone who works for a brand that has never been burned (goodbye JC Penny and Findlaw).  Where do you find these people?  Every industry has some shining stars in search – in legal its Gyi Tsakalakis and Steve Matthews for example – it’s just a matter of IDing them through some painstaking research.    

    And remember during an interview, you are interviewing them too.  I’d have a list of questions to see how they respond to theory espoused by people like Vanessa Fox, Matt McGee and Danny Sullivan.  I’d probe to see if they think they are smart enough to outwit the engines (bad sign).  I’d want to know if they are focused on ranking reports (another bad sign).  

    The best opportunity is one that will let you get hands on dirty with data and tactics.    

    Hope this helps.  

  • http://www.facebook.com/seo.jaipur Seo Jaipur

    If You Want Right Answer Q9- Rank better for ‘Romantic Italian Restaurants in Redmond Add in Google+ & Google Local Places.

  • MeyerKaty

    my roomate’s st ep-mot her brought ho me $1 3342 the previous mo nth. s he g ets pa id on the computer and bought a $369300 condo. All she did was get lucky and profit by the advice uncove red on this web page ===>> ⇛⇛⇛⇛► http://getitmust.blogspot.com/m

  • http://twitter.com/kirstyshark Kirsty Shark

    That is good advice Conrad, thanks. 

 

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide