The rise appears to be due to a combination of factors — first, there are just more bad ads out there.

“My interpretation is that the better we get, the faster we get at shutting them down, the more persistent they get,” David Baker, Google’s director of engineering for advertising told me.

Secondly, Google appears to be getting better at detecting them over time.

“Despite the rising numbers that you see, the bottom line is that things are working,” Baker said. “Things are improving over time.”

Google has been criticized for everything from gambling ads, to pharmaceutical ads, to ads related to human trafficking, with some saying the company is incentivized — by advertising revenue — to overlook such ads. But the company argues that many bad actors don’t pay anyway, because they use fraudulent credit cards, and, execs say, Google benefits more from a clean, trustworthy ad environment.

On the other end, advertisers complain about it taking too long to approve their ads, and a lack of transparency about problems.

“What I’m hoping is that by being a little more transparent, advertisers will understand what we’re doing and the challenges we have,” said Baker. “A good advertiser doesn’t want their ads to be seen next to bad ads.”

The good news is that Google has reduced the percentage of its ads that are bad, the company says. Though he won’t reveal exact numbers or percentages, Baker says Google reduced the percentage of bad ads showing by 50% between 2010 and 2011 — according to human benchmarking that the team does every two weeks.

The broad modifier feature was rolled out in July 2010 in the U.S and was mostly aimed at scaling up those accounts not already using regular broad match type. Search marketers have to then build new broad keywords using “+” signs to effectively unlock the broad modifier feature.

Because it requires some time and effort, not all advertisers have actually implemented this feature – particularly those advertisers already using broad match type. Some of our clients reluctant to use standard broad match type did test broad modifier, and it turned out that it performed surprisingly well – with an incremental revenue volume up to 15% while maintaining efficiency on target.

The new matching behavior for exact and phrase match types announced on April 17th  (and just rolled out last week) is going one step further since it potentially impacts all advertisers by automatically updating the default matching behavior from standard exact and phrase matching to a more lenient matching behavior including plurals, misspellings, and other close variants.

In that sense, it can be seen as a logical sequel to the broad modifier feature. Advertisers have the option to opt out – however, most of them will allow the update to occur.

There has been a lot of speculation and skepticism about the latter one, as it seems it is just another way for Google to generate more ad revenue. Hence the question: what are the first takeaways a couple of days after the new matching behavior roll-out?

Where Can I See The Impact In AdWords?

While I couldn’t find any details in AdWords about those incremental queries corresponding to “plurals, misspellings and other close variants” in AdWords, you can now see that the “Other search terms” section shows impressions and clicks even for exact keywords:

In this particular case (a strong trademark keyword in exact match), we have measured an impressive 18% lift in impressions at a stable CTR. Conversions did follow with a slightly higher conversion rate and a slightly lower cost per order – not significantly though.

Overall Impact On Traffic & Conversion Volume

In this section, I will attempt to answer two questions: what is the average impact on traffic? And what is the average impact on conversion volume?

According to Google: “on average, the new matching behavior increased AdWords search clicks by 3%, with comparable CPCs”. Looking at 15 top brands managed through eSearchVision’s proprietary search query report, early findings show that clicks from queries not containing the actual keywords actually increased by 3.5/4.5% on average with comparable CPCs.

More specifically, the below graph shows that the percentage of clicks generated through exact and phrase without close variants has slightly decreased, while the percentage of “All Other Queries”, i.e. when the keyword is not included in the query, has increased from 23% on average the week before the roll-out up to roughly 26-29% the week after.

In the meantime, conversions have increased by 2.0/3.0% on average. However we can expect this number to go slightly up over time due to the post-click effect – since users who clicked over the last couple of days are likely to convert in the next couple of days or even weeks.

Main Takeaway

The new exact and phrase matching behavior seems relatively promising so far. As expected (and even a bit more than expected) we’re seeing more impressions and clicks at a stable CPC, as well as incremental conversions, even though conversions are not following as fast as the clicks for the time being.

As a result, search marketers no longer need to create additional keywords using the broad modifier feature. The new matching behavior seems to be doing pretty much the same job with no effort. Just keep an eye on search query performance and maybe add more negative exact and phrase keywords if you identify any poor performing or irrelevant queries since this roll-out.

“For a given keyword, the report tells you in aggregate how you’re doing as compared to other participants,” Bhanu Narasimhan, group product manager for AdWords, told us.

While the report lists the landing page domains for all of the other competitors in a particular auction, Google stresses that the information provided about competitors is no more than a marketer would get by performing the search on Google.com.

The reports won’t give the competitors’ keywords, quality or settings. For example, you can see that a competitor’s ad will be triggered by your keyword, but the other advertiser may not be actively bidding on that specific keyword — perhaps they are using broad match, for example.

The new report provides new data at the keyword level on five different statistics:

• Impression share: the percentage of total impressions the ads will get based on current settings.
• Average position: the average rank of your ad, compared to other domains appearing for the auction.
• Overlap rate: how often your ads overlap with those of another domain.
• Position above rate: how often your ads are above a certain other domain.
• Top of page percent: how often your ads appear in the ad block on the top of the page.

The report, rolling out globally in the next few days, is only available for highly-trafficked keywords, as there’s not enough data available for less-popular ones. Marketers can only see one keyword at a time, currently.

To find the report, go to the Keywords tab, click the box next to the keyword — available keywords will show an icon that indicates the report is available for that keyword. Click the “Keyword Details” tab and choose “Selected (single keyword)” under “Auction insights.”

The intent, says a Google spokesperson, is to give users more information: “As part of our ongoing efforts to show ads that are relevant and informative, we are including more information about why users are seeing certain search ads.”

If I know Google, this effort at transparency is also meant to spur click-through rates — and has likely done so in extensive testing. Perhaps users are more likely to click when they understand better why they’re getting the ads they’re getting.

Google already provides information on why certain ads appear if users click on “Why These Ads?” at the top right of the yellow ad block, but this text provides the information at a glance.

The change will allow advertisers to model changes even when there’s not enough data to do so at the keyword or ad group level. They can also see what would happen if they raised or lowered all of their bids within the campaign by a certain percentage — say, 5%. Advertisers can see what would result if you changed all campaign bids to the same fixed value.

If a change looks promising, the system will be able to say whether an advertiser would need to increase the campaign budget to avoid being limited at the new bid value.

There’s also the opportunity to download the bid simulation data at the account or campaign level, and users can also download an AdWords Editor-compatible file with bid amounts at the simulated level, along with the ad groups and keywords to which they should be applied.

Campaign bid simulator can be found in the Opportunities tab within the left navigation tree.

That’s their expectation. The reality is AdWords for even a moderate budget could really be a full-time, 40-hour-per-week job. There are many intricacies in an AdWords account, and it’s becoming more complicated by the day. Analytics plays a vital role in deciding how to spend your money effectively.

Buying a number one ad position in Google can bankrupt even the heftiest of budgets, and honestly, do you need that number one position to make more money?  Could number 4 make you more, or just as much, for a much reduced investment?  Once you determine the best position for your keywords, how do you make sure they stay there?

In this article, we’ll show you how to find the most profitable positions, and how to create automated rules in AdWords to ensure your keywords show ads in the most desirable places on the search results pages.

Now we can see in which positions our keywords become the most profitable – profitable – not necessarily the highest earners. I can spend $10,000 on a number one position for 30 days, and make $20,000. I made a 100% return on my investment. If I buy the same keyword into the number 4-6 positions, spend $5,000 for 30-days, and receive that same $20,000 in revenue – I just increased my ROI to 300%.

Here’s how to see the best position for the investment in Analytics.

First select “Advertising” from the sidebar menu.  Then choose “AdWords” and “Keyword Positions” from that sub-menu.

From this report view, you can see how many AdWords visits your keywords received, and the Site Usage for each of those keywords. This can help you determine which positions and keywords provide the lowest bounce rate, longest time on site, etc.

If you don’t track revenue with your AdWords account, this is still extremely important information to have.

If you do track revenue, you can take this report one step further. You can see how much revenue you generated per keyword, by position. This is really important when evaluating the necessity of a number one ranking.

First choose “Ecommerce” just below the Explorer tab above the graph:

Then choose “Revenue” from the dropdowns. You can do one or both here; it is good to see position performance for the keywords that make the most money and for the keywords that get the most visits.

In the example above, you can see that the keyword “Pink Widgets” made the most money in the 8^th position in the sidebar. This is odd, but it’s not unusual. You’ll see results like this all the time.

Here’s another example where position 3 at the top brought in 5x the revenue that position 1, 2, or 4.

So what do we learn? We now know that position one, for these keywords, is not necessarily the top converting position. The formula you use to determine ROI for any given position is:

I can now adjust my CPC bid for this keyword and use an automated rule to increase or reduce my budget to keep me in a particular position. Previously, we used the “Position Preference” feature, but that was retired last year…so we now create rules to handle positioning of keywords.

To create an automated rule, choose the keyword(s) in your AdWords account and pick “Automate” and then “Change max. CPC bids when” from the drop down menu:

From here, you set the position and bid where you think they need to be.  You’ll need to set a rule to raise the bid when the “Avg. position worse than” is triggered, and a separate rule for “Avg. position better than.”

This is definitely something that needs to be monitored and tweaked until it’s working perfectly. I strongly recommend setting a max bid for each click so the rule cannot keep escalating the bid without a limit.  I also think the Frequency setting will take a bit of work.

You can set 2 or 3 rules and apply them to multiple keywords at once, or pick them from the list of existing rules, so once your rules are set up – adding them across the account becomes much easier.

The “Automated rules” section of AdWords will list all of your rules and let you tweak them, and see when rules ran on particular keywords or ads. Be sure you read and understand how rules work before you start. I recommend the Automated Rules best practices document from Google.

TIP: Take a look at your “Matched Search Queries” report and compare it to your Keyword Report in analytics.  Avinash Kaushik has a great article about how Matched Search Queries are truth, wherein the Keyword Report is “lies.”  This can also help you fine-tune your AdWords bidding process, as well as understand what phrases are actually bringing traffic.

Once you’ve set up your rules, be sure to annotate the change in your analytics account so you can watch and see if the ROI from your AdWords account improves over time.

If things aren’t going well, tweak your bid or your position until they improve. This technique will definitely take time, but it can offer huge returns when done correctly. This is just one more reason your AdWords account should not be an afterthought in your entire online marketing strategy.

“We’re changing the design layout of our hotel, flight, credit card and bank account results, which help users complete actions such as booking flights quickly and easily,” a Google spokesperson told us in a statement. “We’ve always disclosed that Google may be paid when a user completes such an action; we want to be clear and consistent in how we do that.”

The New Look

The comparison units appear in the US when people do these types of searches:

• Flight search
• Hotel search
• Financial product search for checking accounts, savings accounts, credit cards or certificates of deposits

In the UK, the units only appear for financial products, specifically for current accounts, savings accounts and credit cards.

Again, the units aren’t new. They’ve existed for over a year for some products. They’re simply getting a new format. Below is an example of the old-style look, which some may still see now:

Here’s an example of how they are changing to over the coming days:

In the new format, the background color that’s used for Google’s traditional AdWords units is gone. The comparison units also carry a “Sponsored” disclaimer rather than an “Ads” one, as with AdWords ads. This seems part of Google’s positioning the new units as something different than ads.

Not Ads, Not Organic Listings But A “Third Kind Of Thing”

Indeed, even though Google’s called these “comparison ads” in the past, it pushed back on that label for them now. What are they called? We’ve yet to get a formal name for them. In talking with us about them today, Google referred to the units as a “third type of thing” — not organic listings, and not ads but something in between.

Clicking on the comparison link will take users to a results page in the relevant vertical search product, be it Google Hotel Finder, Google Flight Search or Google Advisor.

The distinction between these and ads, Google told us, was that advertisers control the keywords, the copy and the links in AdWords. In the results generated by the comparison units, Google decides what listings get displayed and how they get displayed, based on aggregate data that advertisers provide. In most cases, Google gets paid for leads it sends.

While the comparison ads site is still up and shows the program as in beta, participation isn’t open to any advertiser as with AdWords, not does it seem likely to be.

Google says those who want to be in the flight area, or the hotel area or in the financial products area will either already be approached by the right team in Google or know the team to contact. In short, if you don’t know the right place to talk to, apparently you aren’t the right company for these types of ads.

Organic, Paid Placement & Paid Inclusion Listings

This “third Kind of thing” will sound familiar to veteran search marketers. It’s paid inclusion. For those new to the concept, a refresher.

For the most part, Google (as well as Bing) has two different types of search listings. The first are “editorial” or “natural” or “organic” listings, the “main” listings that people tend to think of as the search engine’s results. Google doesn’t charge for people to show up in this space. Its search algorithms try to determine the most relevant sites to list for any particular search.

There are also paid listings, the listing powered by AdWords, where advertisers bid against each other to appear above or to the right of the organic listings. Because these ads grew out of advertisers trying to gain prominent placement, they’ve historically been called “paid placement” ads, even though with Google, advertisers can’t guarantee that their ads will rank well for any particular term, even if they’re willing to pay the most. An ad algorithm takes payment along with overall relevancy into account.

Paid inclusion was once a popular way that the major search engines like Yahoo or Bing’s predecessor MSN Search charged sites to help increase the odds they might perform well within organic search results.

You couldn’t buy a top ranking, but you could pay to ensure more of your pages were gathered up or revisited on a regular basis. It was kind of like buying more tickets for a lottery. You aren’t guaranteed to win, but you can buy more chances.

Google was long the major search engine that stood against paid inclusion, even calling out against paid inclusion as part of its 2004 IPO filing. Microsoft and Ask, feeling the pressure, dropped their paid inclusion programs that year. Yahoo — the last holdout — dropped its program in 2009.

So what’s up with paid inclusion happening at Google, which fought against it before?

Paid Inclusion In The Vertical Space

It’s important to note that paid inclusion is not happening in Google’s main web search results. At the time Google fought against paid inclusion, that was largely where it was happening. Since then, paid inclusion has moved into the province of smaller specialty search engines, where it remains common. Other search engines in vertical spaces, like Kayak.com and Mint.com, include data from companies with which they have financial relationships. Even Bing does this.

Google has come close to paid inclusion in the past with some mixture of sponsored listings in things like shopping and local results, but in talking today with the company, it seems it may be closer to this for some newer search products than ever before, if it’s not already there. I’d argue that it is.

To be clear, Google may have “free” information listed in any of these areas because of data feeds it pulls in or some crawling it does of the web. But it was clear the intention for these products is really to be building a way to compare between services from companies that Google has a commercial relationship with. That’s a fairly big departure from Google’s traditional search products. Google News, for example, doesn’t only feature newspapers that purchase inclusion. Nor does Google Shopping only list merchants that pay to be considered.

Google Hotel Finder, launched last year, appears to be a hotel search engine similar to how Google has a search engine for finding images or videos or web pages. But unlike those other search engines, from talking with Google, it seems most if not all the content in Google Hotel Finder is for companies that it has a commercial relationship with or hopes to have one with — a commercial relationship meaning Google gets paid for leads.

Google Flight Search which also launched last year seems the same situation. Google was unclear about whether businesses were listed for free within the area or why some airlines had booking options or not, if that was only for those with commercial arrangements.

As for Google Advisor which rolled up various financial product searching tools last year, individual sections, such as the credit card area, currently say that Google isn’t paid for offers shown. Yet this area powers the comparison units in Google that are expressly noted as sponsored. Google told us the wording in Google Advisor is being updated, after we pointed out this mismatch.

Will More Paid Inclusion Come To Google?

Even though paid inclusion is fairly commonplace in the vertical space, it still feels somewhat surprising for Google to be doing it. Having a search tool for financial products using paid inclusion even goes directly against what Google’s founders said they disliked back in 2004, as part of the IPO filing’s “Don’t Be Evil” section:

Google users trust our systems to help them with important decisions: medical, financial and many others. Our search results are the best we know how to produce. They are unbiased and objective, and we do not accept payment for them or for inclusion or more frequent updating.

It makes me wonder if future Google vertical search products will go down this route. I’ll be following-up more with Google about this in the near future.

Pamela Parker contributed to this story. 

Related Articles

• Yahoo To Drop Paid Inclusion Program
• Open Letter To Google: Do The Right Thing, Divest Yourself Of Performics
• Google Experiments With Paid Inclusion & Does “Promoted” Meet FTC Guidelines?
• Google Blurs The Line Between Paid & Unpaid Results Again
• Bing’s Travel Search & Kayak Favoritism Angers No One, While Google’s Gets Headline Attention From WSJ
• Google Goes Big For Financial Comparison Shopping, Launches “Google Advisor”
• Search For Hotels With Google Hotel Finder
• Google Travel Search Takes Flight With First ITA Travel Product

The Mobile App extension — currently for smartphones only, not tablets — will help advertisers raise awareness of the fact that they have an app. When users are searching on their smartphones for a brand name or product category, they’ll see relevant ads displayed, and advertisers that have apps can append a link to their app below the ad description. If a user is searching for Wells Fargo or Walgreens, for example, the ad would allow users to see that they could get information via the brand’s app, rather than going to the mobile web site.

AdWords with the Mobile App extension will have two links — one from the headline to the regular landing page, and a second one on the app which will lead to Google Play or the Apple App Store.

To set up a Mobile App extension, advertisers need a mobile app that’s live within one of the two marketplaces. They’ll also need the Package Name, if it’s Android, or App ID, if it’s iOS, along with the URL of the page within the app store where users can install or learn more about the app. The Mobile App extensions are available within the Ad Extensions tab within the Campaigns tab in AdWords.

Google has also added additional information to click-to-download ads. When a user searches and sees the click-to-download ad, they will be able to see image previews, a description of the app and, when applicable, pricing and rating information. The additional information will be drawn automatically from Google Play and the Apple App Store.

If they’re marketing an Android app, advertisers will be able to track downloads from Google Play within their AdWords account — they will appear as AdWords conversions. Advertisers can set this up via the “Conversions” tab under the “Tools and Analysis” tab.

Finally, Google has introduced an ad type for app developer/publishers that want to incorporate Custom Search within their tablet apps. Custom Search Ads for Mobile Apps will appear along with organic search results when a user performs a search within the app. Google shares revenue generated with the publisher.

Three settings currently exist:

• Rotate evenly
• Optimize for clicks
• Optimize for conversions

The “rotate evenly” setting has been the preferred type for achieving apples-to-apples testing as ads were evenly disbursed at all times.   The change will now require users who are using ads to test landing pages or copy-based conversions to be limited to 30 days per trial.   Each time creative is enabled or edited the 30-day window will restart over again.

This change is being implemented to improve performance.  Google states:

“This change will enable us to provide users with the most relevant ad experience and should help advertisers improve the performance of their AdWords accounts. “

Any creative not changed within the past 30 days will be immediately converted to “optimize-for-clicks” next week.  For more information see the Inside AdWords blog.

With the Keyword Tool changes, advertisers will be able to see keyword ideas grouped by themes related to their search term. For example, if you search on “cashmere” you might see an idea for a “cashmere sweater” group that includes keywords like “cashmere sweaters,” and “women’s cashmere sweaters.” Once these themes are suggested, advertisers can choose to add ad groups directly to their accounts.

The Keyword Tool has also been integrated with the Traffic Estimator, so advertisers can see traffic estimates for the keywords selected.

The Traffic Estimator will now be available only to advertisers logged into their AdWords accounts, because it will begin using each account’s performance history to come up with its traffic estimates. The standalone Traffic Estimator is being retired.

Within the Traffic Estimator, Google has added the capability to graph performance estimates, so advertisers can determine the most effective bidding strategy. Users will also be able to create “draft” ad groups and view traffic and bid estimates for each draft ad group. When you’re satisfied with the draft groups, they can be added directly to your account.

The changes will appear in advertisers’ accounts over the next couple of weeks.