The New York Times Paywall Meters All Google Visits, Not Just Search Visits

The New York Times launched its paywall globally today, and I’ve been putting the barrier through its paces. Visits from search engines like Google, for convoluted reasons, are limited to five per day. But as it turns out, visits from any Google site — search or not — appear blocked.

To test how the new paywall works — and interacts with search engines — I triggered it for myself in Firefox. At first, I had no luck. After clicking on well past the allowed 20 views per month, I couldn’t get the wall to appear.

Restarting my browser (I was in Firefox) finally got the paywall to trigger. As was promised, I was still able to get past this when I came from links on social sites such as Twitter. In addition, I was able to get past the paywall when I came from an ordinary link, such as from River Of Times, which is a collection of New York Times links.

Blocked From Google, Yahoo & Bing

Search engines are also supposed to let you past the paywall, if you’ve hit it, but only for up to five visits per day. That’s a barrier that the New York Times has erected against whatever it considers to be a “major search engine.”

To discover which search engines got this special attention, I went to the Google.com, Yahoo.com, Bing.com, Ask.com and Blekko.com, found five stories from the New York Times in each of them, and clicked through.

When I tried to view more than five, I was blocked when coming from these:

  • Google
  • Yahoo
  • Bing

I wasn’t blocked when coming from Ask and Blekko. In addition, I wasn’t blocked when I came from any of these country-specific versions of Google that I tested:

  • Google Australia
  • Google France
  • Google UK

With Google UK (google.co.uk), I specifically tested to see if I could view more than five articles, in case the New York Time was perhaps applying a different limit for each version of Google. This doesn’t appear to be the case. I was able to view more than five pages, when coming from Google UK.

Blocking Google Reader

Next, I wondered if the New York Times was only metering Google search-related visits (which is what the New York Times said it would do) or instead metering visits from other Google services. As it turns out, any Google service may get blocked.

For example, I subscribe to the RSS feed from the New York Times in Google Reader. Once the paywall was triggered, I could no longer view articles from Google Reader without a paywall block coming up.

This shouldn’t happen. For one, the New York Times has said that links from social media sites wouldn’t be blocked, and I’d say Google Reader qualifies as a social media site. But more important, the New York Times has also previously said that ANY link from another site will let visitors through, unmetered, except in the case of search engines.

Google Reader isn’t a search engine, but the New York Times is treating it that way. More crazy, if I subscribed to the RSS feed in my browser, the New York Times consistently let me view whatever I wanted.

Blocking Google Buzz

Few would argue that Google Buzz isn’t a social media outlet. It might not be anywhere near as popular as Facebook or Twitter, but it is clearly a social media service. But in my testing, I wasn’t able to view an article I shared on Google Buzz, because I’d hit the paywall limit.

Again, this shouldn’t have been the case.

Blocking Gmail

I tried another test. I emailed myself a link to a New York Times article to my Gmail account, which is run by Google. Again, I got blocked. The same thing happened when I clicked from an emailed link sent to my Yahoo Mail account.

I’m pretty sure that if I’d clicked on these links from an email account not also run by a major search engine that I’d have gotten through — but that really shouldn’t be the case.

Postscript: HTTPS To Blame?

After I wrote my story above, a reader wrote suggesting that perhaps the secure connection (https) that Google provides for services like Google Reader, Buzz and Gmail might be to blame. The connection might not be passing “referrer” information that indicates someone is coming from these services, and so the paywall goes up.

I don’t think this is the case, but it’s worth noting. I did some further testing on the morning of March 29, the day after I wrote this. I found:

  • Clicks from Google Reader were now being allowed through, regardless of whether someone comes from a secured connection or not
  • Clicks from Gmail continue to be blocked, even when I turned the secure connection off temporarily
  • Clicks from Google Buzz continue to be blocked, even with the secure connection off

I also found clicks from Twitter, even from a secure connection, were allowed through the paywall. So, there’s more going on. Secure connections can pass referrer information, if they want, to my understanding. It’s also possible that the secure connection might redirect a click in a variety of ways to pass on different referrer information.

More Paywall Observations

The paywall is pretty leaky. It’s not going to stop anyone who wants to get past and knows even a little bit about the internet. Aside from admitting people via any link, you can also get through by:

  • Using the private or “incognito” mode of modern browsers like Firefox, Chrome and Internet Explorer
  • Clearing your nytimes.com cookies (and probably just one specific cookie, but I haven’t pinned that down)
  • Using another browser, if you have more than one on your compute

There are other tips, too, such as those Mashable has compiled.

You’re Blocking Search Engines Why?

The purpose of me writing about this isn’t to encourage people to avoid the paywall. Heck, the New York Times is deliberately going out of its way to keep it from triggering for many people who come to the site.

Instead, it’s to revisit the absurdity of how search engines ended up getting slapped with one of the few limits out there. It all started with the New York Times trying to keep limits that Google allows publishers as part of its “First Click Free” program, limits that make little sense when pretty much it’s “Any Click Free” now at the New York Times.

After questions were raised about why Google was getting singled out, the New York Times then had to concoct a story about how all major search engines would get blocked, with a confusing message suggesting that the New York Times was using tools that the search engines themselves provide for this.

They don’t. The blocking and metering is all done by the New York Times itself — and done so poorly, right now, that it is inadvertently blocking non-search services from these companies. Personally, I think the search engine limits should just get dropped.

Further Background

My article below covers more background on the New York Times blocking of search engines and the confusing statements it has made:

Over at my personal blog, I also did a send-up of the pricing plans for digital subscriptions (in short — buy a print subscription. You get digital access, should you really think you need it, for a much better price)

And below are some further background articles on search engines and newspaper sites.

Related Topics: Channel: Other | Top News

Sponsored


About The Author: is a Founding Editor of Search Engine Land. He’s a widely cited authority on search engines and search marketing issues who has covered the space since 1996. Danny also serves as Chief Content Officer for Third Door Media, which publishes Search Engine Land and produces the SMX: Search Marketing Expo conference series. He has a personal blog called Daggle (and keeps his disclosures page there). He can be found on Facebook, Google + and microblogs on Twitter as @dannysullivan.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • http://www.planetc1.com/ Michael Dorausch

    From watching you tweet throughout the day, it seems this story was perfect for you, made it fun to follow and wondered how long until a SEL post was up. That was fast!

    You’d think with all the money invested in this paywall they’d have dome more research and built a more sophisticated system. Perhaps though, they figure the average consumer is not going to take the steps necessary to avoid a block after 20 visits, and will pay up or leave.

    Will be interesting to watch the long tail suggested search terms as users seek out ways to continue getting articles for free.

  • http://snomanali.wordpress.com/ Kevin

    Google Block for NY times are you kidding me :o

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide