What Is Search Terms View?

The AdWords Search Terms View shows us the performance metrics on the Search Terms that matched against our Keywords (more detail on the difference between Search Terms and Keywords is coming up later in this post). Advertisers can use this data to optimize the Keywords and Negatives in their account.

Optimization in this context usually includes adding Keywords (both new Keywords on an any Match Type and existing Keywords but on a new Match Type), and adding Negatives (again, both new Negatives and existing Negatives on a new Match Type).

The data can also be used to help remove Keywords and Negatives, optimize bids and ad copy, and to inform the organization of AdGroups and Campaigns. In this article, we will focus on the two primary optimization tasks: adding Keywords, and adding Negatives.

Why Optimize Keywords? How Often?

Most of our clients are doing this on a weekly basis at least. I consider this activity to be like gardening for PPC; we want to pull out the weeds and make sure we fertilize the pretty flowers, all the while considering the overall organization and appearance of the garden… and the process never ends.

This core and ongoing practice is essential because it helps us continually refine the traffic we are paying for and improve Quality Score, and also because it helps us find new ideas for AdGroups, Ads, and Landing Pages.

Generating The Search Term View

To navigate to the Search Term View, start by logging in to AdWords, then click the Keywords tab, then click on the “See Search Terms…” button and choose either “All.” This will generate a view of the Search Terms triggered by all of the Keywords in our account.

We can optionally narrow the list using the usual navigation with AdWords, and by selecting individual Keywords and choosing “Selected” instead of “All” from the drop-down.

Recommended Keyword Selection

I recommend choosing our AdGroup with the most clicks in the last 30 days (All Campaigns, then AdGroup tab, then sort descending by Clicks), then the Keyword with the most Impressions (Click on the AdGroup with the most clicks, this takes we to the Keywords tab, now sort descending by Impressions).

We will consider other approaches for narrowing down the list later in this post, but this will do for now. Check the box next to the Keyword with the most Impressions, then click the “See Search Terms…” button. When we click the button, it will drop a menu, choose “Selected.” This will take us to Search Terms View.

Getting Back

When in Search Terms view, there is now a link to get us “Back to Keywords.” This is how we return to the Keyword list we were just at. We can also navigate away from Search Term view by any of the other traditional means, such as the tabs or the quick menu on the left.

Columns In Search Terms View

In Search Terms view, the first column will show us the Search Term, and the next column will be the (Search Term) Match Type. The next two columns will be optionally included by AdWords. If your original Keyword tab was at the “All” level or represented more than one Campaign, then the Search Term view will show a Campaign column next. Likewise, if your Keyword tab represented more than one AdGroup, then it will show an AdGroup column.

One thing to note about the Search Term view is that a given Search Term and Match Type may have matched against multiple Keywords. In this report, that Keyword detail will not be shown; Each Search Term and Match Type combination is reported in one row, regardless of how many different Keywords it matched.

For example, if we have the Keyword “bike” in our account on Broad Match, Phrase Match, and Exact Match, and many different users searched on “bike”, the report will have one row for the “bike” Search Term, and it will be Exact Match. Behind the scenes, our Keywords may have each gotten credit for some of the traffic on their various Match Types, but this report will not show that.

Search Terms vs. Keywords

Search Terms are what the user typed into Google Search. Keywords are added to our account by us, and Google matches them to Search Terms based on their Match Type. For example, if we have a Keyword “bike” in our account on Broad Match, some Search Terms that might match include: “beach bikes,” “road bikes,” “bicycle,” etc.

Match Type in Search Term View

Match Type in Search Term View is the Match Type that Google used to match the Search Term to a Keyword in our account. This is not necessarily the same as the Match Type of the Keyword in our account. For example, if we have a Broad Match Keyword “bike” in our account, and the Search Term report is showing traffic for “bike”, the Match Type in this report would be Exact Match because the Search Term is an Exact Match with the Keyword.

Customizing Columns

The remaining columns can be customized in the usual way (Click the “Columns” drop-down-button, then choose “Customize Columns.”) See references at the end of this entry for a link to get help customizing columns.

Recommended Columns

My recommendation is to start with the following columns:

• Impressions
• Clicks
• Conv. (many-per-click)
• CTR
• Conv. rate (many-per-click)
• Cost
• Total Conv. Value
• Avg. CPC
• Avg. Pos.

In my experience, these columns provide the information I need, in an order that makes sense to me for the task at hand.

Added & Excluded Terms

The Search Term view will also show us which Search Terms already exist as Keywords “Added” or negatives “Excluded” in our account.

Using Added and Excluded Terms in practice

While this feature is helpful, there are some practical limitations to be aware of. It does not tell us if the Keyword is “Added” to the correct AdGroup according to your account’s design structure – we need to check that ourselves.

Another limitation to be aware of is that this feature looks for literal matches. It does not consider coverage provided by the Match Type functionality in AdWords. Meaning, if we have a Phrase Match “bikes” Keyword, while it would cover the Search Term “fast bikes,” it will not be marked as “Added.” We have to check that ourselves.

As a rule of thumb, we want to be pushing mature accounts towards Search Term Reports that show every Search Term is “Added” or “Excluded.” (Note to AdWords: this would be a nice feature improvement.)

Adding Keywords

We can add Keywords directly from the Search Term View. To do so, check the box next to the Search Term(s) we wish to add, and click the “Add as Keyword” button. We will be given an opportunity to set a specific bid and destination URL. The Campaign and Adgroup are indicated as well.

The Keyword field accepts power posting syntax for Broad Match (no punctuation), “Phrase Match”, and [Exact Match] Keywords. For example, to add “fast bikes” on Exact Match, edit the Keyword field to be [fast bikes]. Click “Save” and the Keywords are added to our account immediately.

Adding Keywords in Practice

My recommendation is that we should almost always be adding Keywords on Phrase or Exact Match. This is based on my best-practice recommendation of using Broad Match to help us find additional terms, and assumes this AdGroup already has a Broad Match Keyword (or maybe a select few), that is doing that research for us. If so, then we want to run as much of our traffic as we can through Phrase and Exact Match Keywords.

Adding Negatives

We can add negative Keywords in a similar way. Check the box next to the Search Term(s) we wish to add, then click “Add as negative Keyword.” When adding Negatives, we have the option to add them as Campaign or AdGroup Negatives. Click “Save” and the Keywords are added to our account immediately.

Adding Negatives in Practice

My rule of thumb for Negatives is to always add them on Exact Match, maybe add them on Phrase Match, and to only add them on Broad Match in exceptional cases. With the tools we have available today for Keyword targeting and Search Term reporting, we have the ability to be very targeted with our Negatives. We can add Keywords on Broad Match, then use Negatives on Exact Match or Phrase Match to filter unwanted traffic.

In my experience, using Broad Match Negatives runs too high of a risk of blocking words unintentionally. For example, if we have a Keyword for “beach bike” on Broad Match, and are matching the Search Term “bike beach” (which, in this example, we do not want), then we can block that by adding “bike beach” as a Negative on Phrase Match. We would not want to add “bike beach” as a Negative on Broad Match because we would most likely end up unintentionally blocking the good searches for “beach bike.”

Pro Tip: If you have a dedicated rep, he can produce a “Negative Blocked Report” – a report showing you the Search Terms that were blocked by your negatives, and the potential traffic you are missing. It is a good idea to review this occasionally to check if you are inadvertently blocking good traffic.

For example, I recently had a situation where we copied an AdGroup and did a find/replace on the keywords, but neglected to review the negatives. The new AdGroup was blocking itself based on the negatives from the original AdGroup.

Download The Search Term View

We can also download the Search Term view; Just click the “Download” button.

Working Online vs. Downloading and Working in Bulk

Once we start working with this report, we may be tempted to start working it in bulk, by downloading it. Once downloaded we could manipulate the report, generate Keywords and Negatives and upload them in bulk via AdWords Editor, for example.

While I am a big fan of doing things in bulk in Excel then uploading them via AdWords Editor, this is one workflow that I prefer to do directly on the website. I find that I am constantly changing my view from one AdGroup to the next, changing the dates, etc., and that the instant gratification of adding Keywords and Negatives in place, instantly, works for me. our mileage may vary, of course.

Choosing Which Keywords To Work With

When choosing which Keywords to work with, there are many different approaches. While getting started, my recommendation would be to start with 30 days, the AdGroup with the most Clicks, then the Keyword with the most Impressions, as recommended above. This is a great bellweather.

If the Search Terms are all over the place here, then we have a lot of opportunity for optimization. Whereas if our biggest term is generating almost all Exact Match and Phrase Match Search Terms, then we are doing a great job keeping our Keyword Targeting tight. I like to work Keyword by Keyword in this way for my most-trafficked Keywords.

We can also work AdGroup by AdGroup, which will pool more data into the report for us to work with, while still keeping the theme tight. I do not usually find it useful to work with this report at the All or Campaign level, because the Keywords tend to be all over the place. One notable exception; It can definitely be useful to work at the All or Campaign level when adding Negatives. You may want to discipline yourself to cycle through all of your AdGroups over time.

Choosing where to go next will really depend on how much data we have in our Account, and how much time we wish to invest. As a rule of thumb, optimize 5-10% of our traffic weekly, and be sure to work through those less-trafficked AdGroups and Keywords over time as well.

Benefits of Optimizing Keywords &Negatives

The benefits of optimizing in this way should include increased traffic (and/or more targeted traffic), improved Quality Score (through improved Keyword and Ad CTR), improved bidding efficiency (we will be able to refine our bids for a tighter subset of traffic), and often improved Conversion Rate (by filtering out unwanted traffic).

We will also find this is a great way to find Keywords we may not already have coverage for, and for getting insight into how our customers search for our products, which can lead us to create new Ads and Landing Pages. Good luck out there.

Resources

• Google AdWords Help on this topic
• Daydreaming About Paid Search: How About Airtight Ad Groups?
• How do I customize the data in my tables and reports?

To be clear, the Department of Justice investigation and settlement with Google is a done deal and there’s nothing new happening specific to those legalities. But the WSJ article shines a new light on how the U.S. government used a con man (and currently a convicted felon) in a sting operation against Google in early 2009.

As the WSJ reports, David Whitaker first tipped federal agents to Google’s complicity with his pharmaceutical activities when he was arrested in Mexico in 2008 and sent back to the U.S. to face wire fraud, conspiracy and commercial bribery charges. That case wasn’t directly related to an online pharmacy that he had set up in 2006 — an operation that sold steroids and human growth hormone to U.S. residents and advertised through Google’s AdWords program. Whitaker tells the Journal that Google was aware of what he was doing.

“It was very obvious to Google that my website was not a licensed pharmacy,” Mr. Whitaker wrote to the Journal. “Understanding this, Google provided me with a very generous credit line and allowed me to set my target advertising directly to American consumers.”

Federal prosecutors used Whitaker as their point man in a four-month sting against Google in early 2009. While in custody and being guarded by federal agents, Whitaker used a pseudonym (Jason Corriente) to begin using AdWords to promote a website, www.sportsdrugs.net, that the government had created to sell HGH and steroids.

Google first rejected it, along with an anti-aging website called www.NotGrowingOldEasy.com. But the company’s ad executives worked with Mr. Whitaker to find a way around Google rules, according to prosecutors and Mr. Whitaker’s account.

The undercover team removed a link to buy the drugs directly—instead requiring customers to submit an online request form—and Google approved it. “The site generated a flood of email traffic from customers wanting to buy HGH and steroids,” Mr. Whitaker said.

A quick WHOIS check reveals that notgrowingoldeasy.com is, indeed, registered currently in the name of Whitaker’s fake “Jason Corriente” character.

The sting expanded to include new websites, including one that sold the controversial abortion pill RU-486 — a pill that can only be taken in doctors’ offices. The WSJ reports that Google approved ads for that site, too.

Google’s ad team in Mexico approved the site, so U.S. consumers searching for “RU 486″ would see an ad for the site. Google ad executives allowed the agents to add the phrase “no prescription needed.”

The WSJ article paints a very damning case against Google’s behavior prior to the $500 million settlement with the DOJ.

Kent Walker, Google’s general counsel, gave this statement to the WSJ:

“We ban not just ads but also advertisers who abuse our platform, and we work closely with law enforcement and other government authorities to take action against bad actors.”

In a month that’s already seen Google suffer black eyes for a mess involving Google Chrome sponsored blog posts, along with negative reaction and requests for an FTC investigation related to Google+, and an incident where Google had to apologize to a Kenyan local business directory after trying to poach its customers, the details coming to light in Wednesday’s Wall Street Journal are about the last thing the company needs right now.

There’s more discussion on Techmeme.

The new features will allow advertisers to target to specific versions of operating systems, to ensure the ads are going to the right users.

Additionally, AdWords users will now be able to target based on whether they are using a faster Wi-Fi connection. This functionality resides in the area where advertisers select targeting by mobile carrier. Google says targeting by Wi-Fi will let advertisers expand their reach, and also will come in handy if the campaign or landing page features high-bandwidth content like video.

Google’s new features come at a time when use of mobile devices, including tablets, is growing tremendously.

Previously, impression share metrics were available only at the campaign level, which didn’t help when advertisers were trying to determine which particular ad group should be allocated more budget.

“Today you know that your campaign overall is profitable and you know that it has ‘runway’ but when you get down to the Ad Group levels you don’t know much,” Jon Diorio, product manager at Google, told Search Engine Land.

In the Ad Groups tab, advertisers will now be able to see:

• Impression share: the percentage of impressions received divided by the estimated number of impressions you were eligible to get
• Lost impression share (Rank): the percentage of impressions lost due to Ad Rank. If you were budget constrained — if your budget, rather than your bid price or quality score was keeping you from getting more impressions — this won’t be shown.
• Exact match impression share: the share of impressions you received for searches that were an exact match for your keyword divided by the estimated number you were eligible to receive.

Google is also updating the algorithms it uses to calculate these impression share metrics, so it will update all existing campaign-level information back to May 2011. Advertisers won’t be able to see impression share data prior to that time.

All of the impression share metrics will be updated only once a day, around noon Pacific Time, so advertisers will have to wait until after noon Pacific to get full data from the previous day.

The Ad Group impression share metrics will begin being rolled out on January 30 to all advertisers globally, and will probably be released to everyone by the first few days of February.

The tool helps us generate the core concepts of a PPC Account in an easy-to-understand visual layout, without having to burden business users with all of the details. We call it the PPC Planning Matrix.

I’m going to gloss over the absolute basics of Campaign creation, and aim for a quick “How to Get Started Using the PPC Planning Matrix Today” entry.

If you are an expert with Paid Search, this tool may help you work with your customers to generate great PPC Campaigns. You might notice that while this tool covers all the required basics, it leaves room for your own additions and fine tuning. The real value in this tool is that it helps visually organize and collect what is needed to create PPC Campaigns.

Even if you are not a Paid Search expert, this article is still for you. This tool will help you organize your Account into meaningful Campaigns and AdGroups, and produce a bulk sheet that can be pasted into Google AdWords Editor or into adCenter Desktop.

AdGroups & Campaigns

Campaigns are like an organizational folder for holding AdGroups. In addition to the Campaign Name, the one obligatory setting is the Campaign Daily Budget. The tool provides a default of $500, which you are free to override.

Campaigns also provide the ability to change some advanced settings like geography targeting, or device targeting (i.e.: computers or mobile devices), which you are free to adjust after the tool generates a Bulk Sheet for you.

AdGroups contain Keywords and Text Ads. The one compulsory setting is the default Maximum CPC. This will apply to all of your keywords. If you apply a Max CPC for any keyword, then it trumps the AdGroup Default. The tool provides a default AdGroup CPC of $1.00, which you are free to override.

If you want to set keyword level CPC’s you can edit the bulk sheet when the tool is done.

Ads

Ads are what your customers see on their Search Results Page. You provide the Headline, the two lines of Description (combined into one for adCenter), a Display URL that your customers will see, and a Destination URL that your customers’ browser navigates to when they click on your ad. The tool provides some examples.

AdWords restricts the length of the various fields in an ad. The tool will highlight them red if they are too long. You can add up to 3 ads per AdGroup in this tool (AdWords allows more and you can add them later).

Keywords

Keywords are what AdWords and adCenter use to match your ads to actual user searches. Add keywords that are related to the AdGroup you are creating, and the Ads you already provided. The tool supports “Power Posting” keywords (I’m dating myself, for those of you who remember when Power Posting first came out).

You can add Broad, “Phrase”, [Exact], or –negative match keywords directly in the tool, and it will parse them into the bulk sheet as expected.

Errors?

The tool tries to help you fill it out by highlighting errors like a missing Campaign Name, missing Ads or Keywords, etc. The tool is also “protected” to prevent inadvertently breaking things. The adventurous at heart can use the password “StoneTemple” to unlock the worksheet.

Generate Bulk Sheet

Click the button. If nothing happens, be sure you have enabled macros. Change Macro Settings in Microsoft Excel

Import into AdWords Editor

Choose: File | Import CSV | Paste Text… Review and Accept the Proposed Changes. Don’t forget to Post your new Campaigns!

Import Into adCenter Desktop

Desktop doesn’t yet support importing from the Clipboard. You can save the bulk sheet to a new CSV file, or as of Desktop Editor Version 8.1.11291.0, you can import directly from AdWords (Beta).

Choose: Import | From Google Select Import From a File, and browse to the file on your Computer. Verify the columns line up and click “Import.” Don’t forget to Post your new Campaigns!

Take It For A Spin

If you would like to get started with the PPC Planning Matrix, please feel free to send me an email. You will get an automated response with a download link and you can get started right away. You could also adapt these ideas to your own tools. Have fun out there.

The story is bound to sound familiar. Google recently agreed to pay a $500 million penalty to the U.S. Department of Justice to settle charges it knowingly accepted ads from Canadian pharmacies illegally selling to American customers. The company admitted wrongdoing and agreed to government monitoring to ensure it wouldn’t happen again. Back in 2007, Google paid $3 million in a similar case involving illegal gambling ads, in which Yahoo and Microsoft paid even more significant penalties.

The BBC investigation stemmed from a consumer who tried to buy Olympics tickets from an unauthorized reseller, LiveOlympicTickets, believing that a company that came up first in Google’s advertising results had to be legitimate. When LiveOlympicTickets asked her to fax in a copy of her signature to finalize the sale, she began to be suspicious and contacted the BBC investigative reporters.

“It was a sponsored ad at the top of the page, so we presumed it was a trusted official site,” the victim told the BBC, “and we spent £750 on two tickets for my mum and dad to see the 1500m, which is what my dad really wanted.”

Though UK police apparently also contacted Google, according to the report, the ads for LiveOlympicTickets continued to appear for more than a week after that, the BBC report said, until the investigative reporters contacted the search company.

This latest dust-up highlights a perennial problem at Google. Though the company uses both algorithmic and human filters to try to weed out illegal advertising, it seems that some of the noxious AdWords inevitably make it through. Even more problematic is the fact that Google appears to have a conflict of interest, because it profits from the very ads it says it’s trying hard to police. And the dollar amounts in question are not inconsequential. The $500 million penalty it paid in the DOJ illegal pharmacy case was estimated to be the value of what Google earned from the ads, along with the amount earned by the pharmacies.

On Thursday, January 5th, U.S. District Judge Edward Davila turned down the class action suit brought against Google. He said he would not allow the suit because “it was more appropriate for companies who had bought the ads to show any alleged harm on an individual basis,” according to paidContent.

Here is the 25-page court document with the ruling:

Google Parked Domain Class Action

We covered the suit back when it was first filed in July 2008.

Related Stories:

• Updated: Google Settles DOJ Pharmacy Ad Investigation With $500 Million
• Australian Watchdog Loses Case Against Google Over Deceptive Ads
• Google Working With U.S. Treasury Department To Shut Down AdWords Mortgage Scams
• Google Instant Costs Google $65,000 In France
• RIAA Scolds Google, Wants Search Algorithm Changed To Fight Online Piracy
• Google Braces For Early 2012 (400-Page) EU Antitrust Report
• US Senators Call For FTC Investigation Into Google’s Search Results
• Google Sued By buySAFE Over Trusted Stores Program
• EU Signaling It May Tolerate Google’s “Dominance” Of Search
• Google Wins Potentially Significant Antitrust Victory In Ohio

Image credit to ShutterStock for lawsuit graphic.

We’ve seen the format used by daily deal provider bloomspot, which also collected ZIP codes via the form, presumably to add subscribers to the appropriate city-specific deals newsletters. Email service providers like AWeber and Vertical Response also seem to be trying out the format. Previous versions of the test included the ability for users to request phone calls from marketers, as well.

A Google spokesperson sent a statement reading: “This ad format is still in testing phase, but our team is getting a lot of interest from advertisers and we’ve been slowly expanding the trial over recent months.  These ads help businesses gather new leads and enable users to easily get relevant information and ask for quotes.”

Indeed, the format seems designed for marketers peddling high-consideration goods or services, which often require significant nurturing and follow-up by a salesperson. On the consumer side, one could seem them being useful for car manufacturers and retailers, and business-to-business leads seem a natural use case.

Advertisers pay per lead collected and bid the same way they would for the normal auction. When users sign up, at least in the experience of Justin Premick, director of education marketing at AWeber, advertisers receive individual emails as people sign up via the AdWords ad. Interestingly, Google is providing the advertiser with the real email address of the user. (When people see the email submission box, a link titled “Privacy” tells them their information will be given to the advertiser. Previous tests (outlined in this presentation) anonymized the email addresses and phone numbers.

It’s not easy to get emails collected one-by-one into email marketing software –- other than by using cut and paste –- so the offering currently seems more tailored to leads intended for individual follow-up. Another challenge is the fact that signing up may be too easy –- when a searcher is signed into a Google account, the email address associated with the account is pre-populated, meaning a single click is enough to represent an opt-in.

“That’s the classic tradeoff between reducing friction on the conversion funnel and qualifying leads,” said Premick.

AWeber’s Premick says the email marketing company has only been using the email capture ads for a little over a month, so he couldn’t yet say how well they were performing in comparison to his other creatives. Premick says the company will perform testing to determine if it will continue with this type of ad.

Google has been testing these lead-generation AdWords formats, called Communication extensions, for much of 2011, and has even been testing similar functionality on its display network. The company suggests marketers who are interested in testing the format should contact their Google representative to ask to participate in the trial.

There’s a great deal of data already in the market about SMB adoption of social media. What they show is that between 45 percent and 70 percent of SMBs say they already have a presence on social media sites (mostly Facebook).

Borrell reports that between 60 and 64 percent of SMBs have a formal presence on social media sites. An earlier 2011 study by Palore found that 58.2 percent of SMBs are on either Facebook or Twitter. And a late-November survey from MerchantCircle found that about 70 percent of SMBs said they promoted themselves using Facebook.

Borrell also found that social media marketing was just behind paid-search for SMBs in 2011. Given the ambivalence that many SMBs feel about paid search (though not organic) one could expect that social media advertising and other promotional spending would surpass paid search in 2012.

Borrell’s report estimates that roughly $6.2 billion was spent in 2011 on social media advertising (all in) and that Facebook captured or saw about 65 percent of that. The SMB-specific component of social media spending is smaller, roughly $1.14 billion, according to the report.

Another interesting piece of data in the report is the way that SMBs measure social media success or ROI. Most use “new customers” as the key metric (it’s not clear how many actively or successfully track that however). Additional fans/followers comes in at number two.

Borrell says, “On average each [SMB] has a network of more than 4,000 friends and followers. But this statistic is skewed by a few respondents who claim tens of thousands or more. Perhaps a better gauge is the median reported: about 250 followers.”

Yet even 250 fans/followers is probably more than a substantial percentage of SMBs have on their pages. The mid-2011 Palore study argued that about 38 percent of SMBs on Facebook had very few fans/Likes and very little engagement. The percentage of SMBs showing limited follower activity was even larger on Twitter (44.5 percent).

The Borrell report illustrates the increasing demand for social media marketing among SMBs. However it doesn’t explore the gap between that demand and the often ineffectual or inept social media efforts of those same businesses.

Related Entries

• Survey Says SEO The Single Most Important Marketing Channel For SMBs
• Report: 58 Percent of SMBs On Social Media Sites, Most Have Only Limited Engagement
• SMBs Need Most Help With Keyword Selection, Tracking
• 3 Ways Small Businesses Can Use Social Media To Drive Customer Loyalty
• Local Search Complexity = SMB Frustration
• Search + Social Media Increases CTR By 94 Percent: Report
• Nifty Hard Core Local SEO Tactics From SMX Advanced
• Local Search Marketers Share Ranking Factors
• Infographic: Local Search Evolved
• Recent Trends Should Guide How Businesses Grow Local Search Strategies

Canonical Form

The canonical form of a keyword refers to the form of the keyword that Paid Search Engines use behind the scenes to match keywords to actual search queries. It is sometimes referred to as Normal Form (Normalized Form) or Equivalent Form. For this article, let’s call this Canonical Form, or Canonicalization.

Wikipedia has a good canonicalization reference, in case you are curious about the origin or use of the word. Every Search Engine does this a bit differently, but the basic principles are similar. So let’s cover this a bit theoretically, without dwelling on the details or the particular differences between Search Engines. We can start with case (e.g.: upper-case vs. lower-case letters).

Upper Case vs. Lower Case

Case is insignificant in paid search (at least, from a keyword matching perspective). Search engine canonicalization will match a user query for “nasa” with an exact-match paid-keyword “NASA.”

Search Engines regard the canonical form of “NASA” to be “nasa,” and they both are considered to match the user query exactly. For that matter, “NaSa” would also be an exact match, as well as every other combination of upper and lower letters. Similar things happen for punctuation.

Punctuation

In general, the rule is that punctuation is replaced with a space to translate to the canonical form. For example, you may have noticed that searches for “bikes com” will match your exact-match paid-keyword “bikes.com” and vice-versa. Likewise, leading, trailing, and double-spaces are all insignificant.

A user-query for “bicycle  store” will match a paid-keyword “ bicycle store” (with a leading-space and a “  ” double-space). AdCenter provides a list of extraneous characters on their help site. AdWords provides a list of ignored symbols on their help site.

Possessives

AdCenter addresses most of the high-volume and regular possessives directly (but not all of them). For example, the search query “Mike’s Bike” is equivalent to the canonical form “mike bike.”

In AdWords, it would be “mike s bike.” In adCenter’s parlance, adCenter normalizes the possessive form of words, such as Mike’s to Mike.

Plurals

Canonical form can collapse plurals together (but will not always do so). A user-query for “bikes” could match an exact-match paid-keyword “bike.” (Please note: I am aware this example is in direct contrast to the information provided via the link below with regards to plurals of the word “bike.” It is just an example for illustration. Check your own user query report to find examples where plurals are treated as equivalent and delivered as exact-match.)

Likewise for non-standard plurals, like “battery” and “batteries.” They may be treated as equivalent. Of the canonicalizations covered so far, this one seems to be the most inconsistently applied across search engines and over time.

Noise Words

Canonicalization can remove “noise words” from the mix as well. For example, Ad Center will canonicalize a paid keyword “bike for the beach” to be “bike beach.”

The noise words “for” and “the” are not considered when AdCenter matches the canonical form of your paid-keyword to the user-query. AdCenter provides a list of extraneous words on their help site. (I didn’t find an equivalent list on AdWords help – maybe the community will add it to the comments, below?)

So Far…

So far we have: (letter) case, punctuation, whitespace, and plurality, and possession, but there is more.

Did you notice that we have crossed into territory where canonicalization might start to modify the intent of the original search query? “Bike for the beach” implies a different user intent than “bike beach.” The former quite clearly looking for a bike, while the latter would most likely be looking for a place. This does not stop here – there is more.

Misspellings & Closely-Related Words

Taking this one step further, canonicalization will sometimes collapse misspellings, and even seemingly different words to be the same. I am going to use theoretical, illustrative examples here, without claiming that either engine actually canonicalizes these particular keywords in this exact way.

So, an example then; Consider a paid-keyword “bike mart.” Canonicalization could collapse misspellings like “bikemarte” to be equivalent. Similarly synonym substitutions can be made. “Cycle mart” could conceivably be canonicalized to “bike mart” (Again, this is an example meant to be illustrative. I don’t think the search engines have ever actually canonicalized “cycle” to “bike.”)

These canonicalizations happen in particular with brands that happen to be slight misspellings, and also as we reach into the tail for more specific keywords.

AdWords Specific Notes: “site:” & Broad-Match Modifier In Negatives

AdWords will remove “site:” words from your keyword as part of canonicalization. For example, if you add “site:SearchEngineLand.com Crosby” as a keyword, AdWords will consider that equivalent to a keyword “crosby.” It will ignore the rest.

Likewise, if you use “+” either accidentally or in an attempt to trigger broad-match-modifier functionality in a negative keyword, the “+” is ignored as an extraneous symbol. It has no effect.

When & Where is Canonicalization Happening?

Canonicalization applies to negatives and all match types. Canonicalization happens prior to matching via match type, it is like a pre-filter for comparing keywords and user queries. It is always on; You can’t turn it off.

Gather Your Own Data

Don’t take my word for it. You can gather your own evidence. Pull a search query report from a Search Engine that includes both the paid-keyword and paid-match-type, and the user-query it matched. Better yet, pull it from your own analytics source. You may be surprised at what you find.

Why?

Paid Search Engines are businesses (and that is a good thing, believe it or not.) As businesses, they monetize searches by collecting fees from advertisers who pay-per-click in a competitive auction market for each keyword. They are motivated to generate the most value from those searches.

In an admittedly simplistic view, they may seek to “maximize profit, ” “maximize user value,” or “maximize advertiser value,” or some combination of all three. Let’s consider the “keyword market” for each user query the Search Engine receives.

On one hand, Search Engines could provide literal interpretation of the user-queries, and require advertisers to discover and manage all of the various forms of punctuation, capitalization, etc. to match each user-query literally.

In our example above, this would require an advertiser to run 2^4 variations of “NASA” to cover the various ways people could search for “NASA” using different capitalization (e.g.: “Nasa”, “nASA”,etc.). Clearly, this is way too granular, provides minimal incremental value, and would be quite burdensome on the advertisers. Advertisers would stop short of full coverage because it just wouldn’t be worth it. So Advertiser burdens would detract from user-value, and ultimately, Search Engine value.

On the other extreme, Search Engines could collapse everything. Advertisers would have one thing to manage, and would be eligible to appear on every SERP (Search Engine Results Page) for any user-query. Selling travel? Bid $5.25 for “run of site” on Google.com. Selling bird feeders? Bid $5.15 for run of site on Google.com…

Obviously, that would not provide anywhere near the value generated by breaking up the keyword markets in a more granular way. We need to draw a line somewhere. That is the game Search Engines play, and thankfully they play as rational businesses.

In this context, the low-level canonicalizations of case, punctuation, etc. are readily explained. But what about the more interesting cases? Now that we have set the stage, let’s consider a more interesting example; “bike” and “cycle” (theoretically, of course).

Let’s say that searches for “bike” monetize for the Search Engines at $.15 CPC, and searches for “cycle” monetize at $.10. If we could collapse the two keywords, we’d be looking at an incremental $.5 per click every time a user clicks on an ad after searching for “cycle.” Granted, this gets complicated fast as we could argue that the value is diminished, so the advertisers would adjust their bids down, which would reduce the effective CPC and mitigate the expected gains. Yes, they probably would.

We could also consider CTR, ad relevance, etc. They would all be impacted. It is a moving target to be sure. The point is; the Search Engine has a mechanism for collapsing keyword markets (or leaving them distinct). They play this game according to whatever their goals and values are, and just as with most human endeavors, they play it imperfectly.

So What?

This is the fun part. What can you, the discerning PPC Advertiser that you are, do about all of this? You can use it to your advantage to save time and to optimize your accounts.

For starters, you are already reaping the rewards of matching all the different combinations of capitalization, punctuation, misspellings, and other variations on your keywords that just don’t matter. Now that you know why and how, there are also some things you may start to notice, and some things you can do more actively.

For example, have you ever wondered why adCenter Desktop is kicking out words as duplicates, when they don’t appear to actually be duplicates? AdCenter added a canonicalization filter to the Desktop Editor. It stops words from being uploaded before they even make it to adCenter. The same thing would happen if you tried to add them via the Web interface. AdWords tends to allow you to add them regardless, and then sorts it out later by dividing up the traffic between them. While adCenter can be a bit obtrusive in this process, I personally like knowing that every keyword is a unique keyword in adCenter. This brings us to our next opportunity.

You can also save yourself the effort of adding all the different variations of “YourSite.com”, “YourSite com”, “www.YourSite.com”, “www YourSite com”, etc. Just because AdWords or adCenter lets you add them, doesn’t mean they are adding coverage or doing good things to your account. A generalized best practice is to manage all of your keywords in lower case, replacing all punctuation with ” “, and trimming all leading, trailing and double spaces.

If you want to be really complete, you could even remove all the extraneous noise words; this helps you make sure you are not bloating your account with effective duplicates. One possible exception would be if you are using Dynamic Keyword Insertion and have a word like “NASA” that should appear in all caps. In this case, you would of course want to add the keyword with all caps.

Let’s take that a step further and actively remove effective duplicates from your account (e.g.: words that you have been able to add, but that have equivalent canonical forms). If you have them in your account now, you are effectively dividing your traffic arbitrarily between them.

You have an opportunity to collapse that data down into one keyword, removing bloat and giving you more direct control over bids, ads, destination URLs, etc. For the coders out there, adCenter provides an API call GetNormalizedStrings Service Function to assist with this process.

Here is an Excel formula that does much of the basic canonicalization work for you:

=TRIM((SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(CLEAN(LOWER(A1)),"'"," "),"."," "),","," "),"-"," ")))

You could safely use this on the majority of your keyword and negative-keyword operations and improve the manageability of your accounts.

Here is one last handy trick (and if you have read this far, you deserve some gold stars). You can reset AdWords Quality Score on a keyword by adding it with different capitalization. Try it out in your account.

Go find a keyword with a terrible Quality Score (4 or lower), then add that keyword with different capitalization. You should start out with a default (hopefully higher) Quality Score. Here is your chance to breathe new life into that dying keyword! Now make sure you have the best ads possible, great negatives, and a healthy bid to get this one back on the starting lineup.

Good Luck out there, and Happy Holidays!