SEM Tools: To Build Or Not To Build

To build, or not to build, that is the question (with apologies to Shakespeare):

Whether ’tis nobler in the mind to suffer The slings and arrows of outrageous SEM Tools, Or to take arms against a sea of in-house engineers……

I know more than a few in-house SEMers. Many of them insist on building in-house SEM tools, as opposed to using what’s out there in the market. Here are the most popular reasons:

“Our volume is so high that the economics justify building our own tools.”

“Our SEM needs are specific to our business.”

“SEM is our core competency.”

“We won’t share our data with anyone.”

All valid points, to be sure. So what’s right for you? The key in making the build/buy decision is to understand your core needs and the tradeoffs involved with any scenario. Here are some questions you should ask yourself that should point you in the right direction.

What kind of SEM programs are you running?

If you are trying to solve what I would call a ‘normal’ SEM problem, you should avoid building if you can. By ‘normal’, I mean something that can be tracked by a conventional javascript or pixel-redirect conversion tracking system. Examples include subscriptions, trials, e-commerce shopping carts, and lead generation. If you can track a landing and a conversion, and you know what that conversion is worth to your business (even if it varies, as in the case of a shopping cart), you should definitely look around to see what’s available on the open market.

What other media are you running in addition to paid search?

If paid search is but one component of your media strategy, you’ll want to use a system that can track all your media, and de-dupe your conversions based on a convention like last click attribution. Last-click (giving credit of a conversion event to the media channel that drove the last click – or impression – prior to conversion) is one of the most common attribution systems due to its simplicity, not its accuracy. But that’s for another column. The goal here is to avoid double counting your conversions. For sites advertising on search and display media, the big third-party ad servers have integrated SEM management into their services, making it easier for advertisers to handle attribution issues without lots of extra work. If you’re not running other media, then look for an SEM-only tool, or something that’s integrated into your web analytics platform.

How big is your paid search program?

How many search engine accounts to you have? How many keywords? How many keywords do you need to manage actively at any given time? The truth is, most off-the-shelf SEM management solutions aren’t built for large-scale operations. And by large-scale, I mean millions of keywords and dozens of search engine accounts. So if you’re in this league, you have some difficult decisions to make. At this stage, if size is the only consideration, you can interview a few vendors to see if they can get the job done for you. You’ll need to put together an evaluation framework and/or test plan or request for proposal (RFP), something that will really help you make a smart decision. While this exercise is difficult and time consuming, I recommend doing it anyway. At the very least, you’ll need to show it to your managements to justify whichever path you decide to take. Be careful with tools vendors. Many will tell you they can handle your program without considering all the issues that are specific to your program. A good RFP will help prepare you for the ‘slings and arrows’ you’re likely to encounter with SEM tools.

Are you a publisher?

We talked about ‘normal’ SEM problems. So what’s an ‘abnormal’ SEM problem? Websites where the value of a conversion is dynamic in nature, and unknown when the session is in progress, require additional consideration. Examples of this type of site are media-driven or listings-driven sites, where the value you derive from a user is driven by the fact that you are a publisher of some kind. In this environment, the value of a user will vary widely according to that user’s behavior on your site. How many pages did the user view and which ads were shown? Perhaps you’re publishing paid links on your page and the value of those links is driven by a dynamic auction model. Either (or both) of these cases will generally require a more sophisticated, and potentially home-grown tracking system. In addition, gathering all that data and making sense of it may require that you build, at the very least, some basic infrastructure to handle the scale and complexity of the task.

Not so fast

So, what if you do decide to build? You have complex dynamic revenue events, big SEM campaigns, and you’re ready to dig in. Here comes the tough part. Once you’ve started building the basic infrastructure, how far do you take it? Do you make the commitment to staff a dedicated development team to support your ongoing efforts? Or do you only go so far as building out a datafeed and porting it over to a tools or service provider?  They answer lies in level of commitment you can sustain for the project you are about to take on. Before you attempt to navigate the tricky waters of tools development, keep in mind that building SEM management tools in-house is a serious commitment. It generally requires a (large) team of dedicated developers, and a very capable product manager, not to mention a lot of funding. All of these are scarce commodities these days, which raises the premium on in-house tools. And, to make things tougher, committing to an in-house solution means you’ll forever be chasing the latest features out there, which will add to your already significant price tag. So the more you commit to building, the more support you are going to need.

Channeling your inner Hamlet

In summary, I recommending buying—or renting—what you can, and building what you have to. The decision to build should come out of the fact that you have a very unusual business model, or an extremely large SEM campaign, or both. Building SEM tools requires a significant commitment at all levels of the organization, so make sure you have your story straight (and the buy-in of your execs) before you draft that first requirements document. After all, the uncharted waters of SEM tools development might just make the slings and arrows of SEM tools seem like child’s play.