Study Shows No Clear Evidence That Google+ Drives Ranking

At SMX Advanced, I presented results from a study we had done that convinced me that links in shares from Google Plus and Facebook behave like traditional web-based links. My statements at SMX Advanced were later disputed by Matt Cutts during his keynote interview by Danny Sullivan, leading to a live discussion — Matt even invited me up on stage during the keynote:

Eric on Stage with Matt Cutts

Image Credit: BruceClay.com

As a result of my keynote visitation, Matt and I agreed that I would rerun the study, this time with some guidance from him on potential problems. While this input was limited in nature, it did lead to some ideas on how to improve the testing. As a result of that dialogue, I determined that there were flaws in the original study we did, and therefore we would need to run another one. This has proven to be a very difficult process, but the results are now in. So without further ado, let’s dig in!

Study Design & Methodology

We picked three different sites, all of which have been on the web for at least two years or more, and we wrote two different, relevant articles for each site. One of these was used as a “Test Page,” and the other was used as a “Baseline Page.” Both pages were implemented without any links to them from any source whatsoever. Both of them received an initial set of Google+ shares on July 19th, 2013. The Test Pages received 6 shares each, and the Baseline Pages received at least 30 shares on that date.

From there, the paths diverged. We provided the Baseline Page no further attention other than to monitor indexing and ranking behavior. For the Test Page, we sent additional shares in two waves, as follows:

  1. At least 25 shares on August 4th, 2013
  2. Four more shares from very authoritative profiles between August 28th and September 1st.

Throughout the process, we monitored ranking behavior for the pages on a number of different long tail search terms. The August 4th and late-August burst of shares are particularly important, because if Google+ shares are, in fact, a direct ranking factor, there should be noticeable changes in ranking after those shares occur. This is the basic premise of the entire study.

Goals & Objectives: What Makes Our Study Different

Plain and simple, this is not a correlation study. For the record, I believe that correlation studies are extremely valuable — but as we know, correlation is not causation, and the study results presented here were directly targeted at measuring causation.

To accomplish this, we restructured the study from the one we presented at SMX Advanced, and set the following goals:

  • Eliminate accesses to the Test Pages and Baseline Pages by humans or 3rd party tools prior to discovery of the pages by Google.
  • We took a great deal of care to minimize the risk of the Test Pages and Baseline Pages receiving links.
  • We then checked at every major step to see if there were any links implemented to the site (in spite of our efforts to prevent them).
  • We used a panel of hand-picked participants to work with us on the study. The reason we did this was so we could have better control over their behavior.

Click here to see the complete study methodology, additional data and analysis.

Possible Sources Of Error

There are three main possible sources of error:

1. Missing Links: It is possible that links were implemented to the pages that did not show up in our monitoring tools. This is not an insignificant potential problem, as by my estimate the cumulative links found by Open Site Explorer, Majestic SEO, Webmaster Tools, and Ahrefs is probably at best 50% of the total links to a site, and it may be as low as 30%.

I base this statement on my experiences with helping sites recover from link penalties. At Stone Temple Consulting, we have helped more than 50 sites recover from penalties this year, and it has happened over and over again that we would help these sites by cleaning out bad links, only to have Webmaster Tools report lots of new links the next time it was queried. The reported new links were not new, and I have no doubt that Google knew about them before but simply did not choose to include them in the Webmaster Tools report. However, once we cleaned out some of the bad ones, we got exposed to some more of the links residing in their database.

2. Ranking Churn: The study was vulnerable to general ranking movement and algo adjustments that our Baseline Pages did not enable us to perceive. This is also a pretty significant risk. In fact, our results suggest that the scope of changes we saw were less than the general churn and movement of overall rankings.

3. Other Environmental Factors: Let’s face it — what Google uses to rank search results involves hundreds of factors. There are lots of variables that could impact the results. In spite of our attempts to minimize people accessing the content, a small number of people chose to do so anyway. We nonetheless believe that the results have validity for reasons you will see explained below.

Detailed Results

As mentioned, we tested articles on 3 different websites. What follows is a representative sample of the results from two of the sites: the Stone Temple Consulting site and  New England travel website.

Stone Temple Consulting Results

The Test Page and Baseline Page tested on this site were articles written by two different users about notable search engine experiences they had. For the Test Page, we monitored the results for 11 different search phrases.

There was one search phrase that showed pretty dramatic movement. Here is the detail of the rankings of that phrase over time:

stonetemple-test-single-term

While there is significant movement, it does not occur until 24 days after the burst of shares on August 4th. This also happens to be just a few days after a lot of SEO community chatter occurred about a possible Google update (the chatter began on August 21st).

The behavior of the Baseline Pages is also pretty interesting to examine:

stonetemple-baseline-low-ranking-terms

stonetemple-baseline-high-ranking-terms

The movement of rankings on the Baseline Page occurs on the 29th of August, one day after the movement on the Test Page, and it moves in the opposite direction — the rankings get worse.

Travel New England Results

The Test Page and Baseline Page tested on this site were articles written by two different users about notable experiences they had traveling in New England. For the Test Page, we monitored the results for 10 different search phrases.

For this site, there was no material movement at all during the entire test:

tne-test-many-terms

For completeness, this is what we saw on the Baseline Pages:

tne-baseline-low-ranking-terms

tne-baseline-high-ranking-terms

No major movement was seen at any time during the monitoring of the results for the test on this site. The Baseline Pages actually show more movement around August 31st than the Test Page terms do at any time.

Analysis

Our study had 3 major goals: to see if Google Plus would drive discovery, indexing, and ranking. So, I will comment on that in three independent sections as follows.

Discovery

In my opinion, it is highly likely that Google Plus drove discovery of the content. Here is a sequence of accesses to one piece on the Test Pages extracted from the log file on one of the sites:

Log File Sequence

The line items that refer to “+https://developers.google.com/+/web/snippet/” are Google Plus sharing events taking place to the content in question. And, of course, +http://www.google.com/bot.html is GoogleBot. Notice how it takes less than 6 minutes for GoogleBot to come to the page after the first share of the page, and there is a visit by GoogleBot to the site for each share. There are no other accesses of any kind to the content in this time period.

In addition, note that the Google Developers page for implementing +1 buttons states the following:

By using a Google+ button, Publishers give Google permission to utilize an automated software program, often called a “web crawler,” to retrieve and analyze websites associated with a Google+ button.

Indexing

Initial shares of all six articles occurred on July 19th. All of the articles initially appeared in the index on July 29th — 10 days later.

After a detailed review of data from Google Webmaster Tools, Majestic SEO, Open Site Explorer and Ahrefs, we found no links to any of the three pages using the available tools as outlined above. This does not mean that the content received no links, only that the available tools did not have those links, and this is a possible source of error.

One other point of discussion is that the long delay between the shares and the initial indexing does raise some concerns. Clearly, this G+ share data was not leveraged in real-time. Why did it take 10 days?

But, in summary, with the information available to me, I don’t see any other signals that would have caused the posts to be indexed.

Ranking

Once we saw that a page was indexed, we were immediately able to find search queries for which the page ranked. However, this does not mean that the shares were driving ranking. As per the original Sergey Brin/Larry Page thesis, each page on the web has a small amount of innate PageRank. This PageRank by itself might cause an ability to rank for certain types of long tail queries, even in the absence of any other signals.

In addition, a page with no links may also gain some benefit from the overall authority of the domain on which it resides. How this works exactly is not known outside of Google. However, it is clear that we need more data to be able to conclude that we would see potential ranking benefits from G+ shares. This is the reason we sent two additional waves of shares in the direction of the pages being tested.

To me, the most remarkable thing about the data in this part of the test is how unremarkable it is. Based on this data, my interpretation is that this study did not show material evidence of Google Plus shares driving rankings movement for the Test Pages. Read more on my thoughts on this in the summary below.

Analysis Summary & Commentary

Here is how I would sum it up:

  1. Google+ shares do drive discovery.
  2. Google+ shares appear to drive indexing, but there is some possibility of error if the pages received links we are not aware of.
  3. We saw no material evidence of Google+ shares driving ranking.

Our test was designed to eliminate distracting signals, and hence we did something that, in its own way, was a bit unnatural: we attempted to minimize (and, in fact, eliminate) reshares. The text description used in the shares of the articles was not normal — it was a warning to others to not look at the content. We needed to do that to try and get as pure a measurement as possible. These constraints could possibly impact the validity of the findings.

(Note: Behaviors such as organic reshares, comments, and other social activity could help improve rankings, but ordinary Web links do not depend on comments and reshares to carry weight.)

My opinion is that if there is any impact from links in Google Plus shares, that they do not get treated the same way a regular link does, even though we can see that links in Google Plus pass PageRank. Google can filter link signals, including those in Google+, at many levels. Why would they allow shared links to pass PageRank if they did not want to use that PageRank in some manner? Perhaps they want to use it to help them identify more authoritative profiles. There are already many who believe that the posts of more authoritative Google Plus profiles get indexed by Google faster and are shown in their followers’ Google results more often.

I acknowledge that there are many ways to point at the holes in this study, and I have offered my interpretation of it. Let the debate begin. What do you think?

Click here if you want to see additional study data, the complete study methodology, and additional analysis.

If you want to see a panel of industry experts discuss the results in this study, and the other studies on this topic, I will be hosting a Google Plus Hangout on Air event (a live broadcast discussion) on September 19th at 4 PM ET (Boston time) with Mark Traphagen of Virante, Inc, Joshua Berg of RealSMO, Pete Myers from Moz, and Marcus Tober of Searchmetrics.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: All Things SEO Column | Channel: SEO | Google: +1 | Google: SEO | SEO: General

Sponsored


About The Author: is the president of Stone Temple Consulting, an SEO consultancy outside of Boston. Eric publishes a highly respected interview series and can be followed on Twitter at @stonetemple.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Joe Bergess

    Excellent study regarding Google+ shares – I would however, suggest that the title of the article be tweaked, as this isn’t a study about whether Google+ affects ranking; rather, it’s a study as to whether Google+ *shares* affects ranking. Another question – wouldn’t the structure of the articles written (keywords, meta data, etc) also have an impact in the ranking? Are the articles similar or identical in keywords and meta data? It’s so hard to complete these types of studies because of the variables. Very insightful article – thanks for sharing!

  • Joe Bergess

    Excellent study regarding Google+ shares – I would however, suggest that the title of the article be tweaked, as this isn’t a study about whether Google+ affects ranking; rather, it’s a study as to whether Google+ *shares* affects ranking. Another question – wouldn’t the structure of the articles written (keywords, meta data, etc) also have an impact in the ranking? Are the articles similar or identical in keywords and meta data? It’s so hard to complete these types of studies because of the variables. Very insightful article – thanks for sharing!

  • Matt O’Toole

    It’s always interesting to see theories put forward, dismissed, debunked, un-bunked, mythologised…

    I’d be interested to understand a bit more as to how you tracked the ranking data, though. I’m guessing you were working with “vanilla results”.

    The reason I ask is that Distilled presented data last year suggesting that the way +1s increase rankings is really very much in the event of a very personalised search – they gave the example of someone +1ing a brand’s Google+ page and that directly (positively) affecting the search results of friends of the original user when they subsequently used a relevant keyword.

  • Matt O’Toole

    It’s always interesting to see theories put forward, dismissed, debunked, un-bunked, mythologised…

    I’d be interested to understand a bit more as to how you tracked the ranking data, though. I’m guessing you were working with “vanilla results”.

    The reason I ask is that Distilled presented data last year suggesting that the way +1s increase rankings is really very much in the event of a very personalised search – they gave the example of someone +1ing a brand’s Google+ page and that directly (positively) affecting the search results of friends of the original user when they subsequently used a relevant keyword.

  • daveintheuk

    Shame really, because at least if it did have an effect on ranking it’d be worth using and have some kind of reason to exist…

  • http://www.koozai.com/author/ali-moghadam/ Ali Moghadam

    I think you’re onto something there Matt. Personalisation, +1s, Your Google+ network – it’s all got to be intertwined. I think Google’s looking for a more ‘human approved’ algorithm, weaving in Google+ as a form of user supported recommendation. We’ve already got things like the Disavow Tool, inviting webmasters to provide lists of human checked bad links. Human curators will shape future updates I reckon, there’s only so much they can do before they crack AI!

  • http://www.seo-theory.com/ Michael Martinez

    Creating visibility is more important than any individual ranking factor.

  • http://www.mazemedia.co.uk/ David Wilson

    There clearly are specific circumstances where Google+ clearly does impact rankings, particularly, as set out above, where people in your circles (or extended circles) have interacted with a piece of content.

    And if you’re a brand, you effectively get to pick which piece of fresh or deep content you want to rank alongside your homepage, because of the latest G+ post showing up in the sidebar.

    So I think to say it has *no* reason to exist is going a bit far, and it certainly is worth using for many sites, particularly those that are recognised brands.

  • http://www.mazemedia.co.uk/ David Wilson

    There clearly are specific circumstances where Google+ clearly does impact rankings, particularly, as set out above, where people in your circles (or extended circles) have interacted with a piece of content.

    And if you’re a brand, you effectively get to pick which piece of fresh or deep content you want to rank alongside your homepage, because of the latest G+ post showing up in the sidebar.

    So I think to say it has *no* reason to exist is going a bit far, and it certainly is worth using for many sites, particularly those that are recognised brands.

  • http://www.jaankanellis.com incrediblehelp

    Well we all know you can write the best content in the world on your site, but if it is not shared, linked to or discovered at all then it wont rank. That is correlation to me to cause it to rank.

  • http://www.rimmkaufman.com/ George Michie

    Great stuff, Eric. It is hard to generalize about any finding as it may have more weight in some contexts than others. I also have questions about whether rank checks mean anything given the degree of localization/personalization. All that said, this is a very cool study. Thanks for sharing.

  • Eric Enge

    Hi Joe – you are right in that it is more about Google+ Shares, and in addition, it is about Google+ shares in a vaccuum – without other signals.

    The articles were posted on 3 different sites, and were topically relevant to each site, so not related to each other.

  • Eric Enge

    Matt – totally agree that personalized search still applies. Google+ will definitely impact that. We were trying to measure the impact on “non-personalized” search in this study.

  • Eric Enge

    Michael M – completely agree!

  • Eric Enge

    David – totally agree that it has a reason to exist, and I do say that above, and in my extended article on the stonetemple.com web site. The conclusion is not that there is no impact, but that it is not the same impact as traditional web links, AND the benefits of personalized results still applies!

  • Eric Enge

    you are welcome, glad you liked it!

  • AdmiralLuke

    I think Google +1 has an indirect effect on ranking just not direct. Matt Cuttz chose his words carefully when he recently said it doesn’t have a ‘direct’ effect rather than saying it doesn’t have an effect at all. For him use those words, there’d have to be a logical reason why he didn’t simply say it has no effect period. It can easily be one of the multiple factors that play a role in contributing towards a natural link profile which has a direct effect on ranking, and Matt Cuttz alluded to this in the past when he said a site or page that gets bookmarked or shared is more likely to get natural backlinks. If a website has a lot of backlinks pointing towards it but no social media votes, it raises a red flag regarding the nature of the backlinks. Especially when it’s a lot easier to for random visitors to give a social media vote(Facebook like, Google +1, Tweet etc…) than it is for them to backlink to a site because they naturally like it. Not everyone owns a website to have the option of giving backlinks. But a lot more people exist who have social media accounts and can give votes.

  • http://bibianowenceslao.com/ Bibiano Wenceslao

    I agree: the title needs to be changed because it’s actually the number of G+ shares the study is focused on and not Google+ generally as a social media platform.

    Great article nevertheless, Eric. Thanks for writing this for the community.

  • Igal Stolpner

    First of all thanks for the study!
    Besides the fact that personalized search still applies with Google+, I honestly think that the results for non-personalized search might just change in the future.

    From my personal studies I got the same results but we still do care about Google+ both because of the personalized results (and they are extremely important these days) and as a long term investment.

  • L. van Eembergen

    Great article Eric, really great setup of your study. You are saying in your analysis summary; Google + drives no ranking.

    But I’m wondering, if the Baseline page were declining because the starting boost every page receives due to all factors was wearing off and the Testpages were gaining ‘authority’ due to receiving Google + shares instead of also declining.
    Looking forward to your discussion in the hangout.

  • http://www.addthree.com/ Miles Rossow

    Dave’s point is that there is no visibility on G+ because nobody uses it. I use the term “nobody” loosely.

  • Michael Cottam

    Great job on this, Eric! FYI I shared it on Google+, but….don’t expect that to help this article to rank :-p

  • Marc_Razia

    I will never understand this over used cliche about no one using G+. Its obviously not in Facebook’s league by any measure, but when I am on there I see plenty of communities with well over 100,000 members. There are also hundreds of celebrities each with well over a million people following them. I’m able to find new public posts daily on just about any topic. If fact anyone could verify these things if they bothered to actually look.

    That’s surely not everybody, not even close, but its definitely not “nobody”.

  • Eric Enge

    Marc – you are absolutely correct. G+ is a very active social network, across many different spaces.

  • Eric Enge

    AdmiralLuke – while we can’t confirm your speculation above, I think there is a good chance that you are correct!

  • http://www.dbsit.com.au/ Danny

    amazing analysis and study reports related to Google Plus and personalized search. :-)

  • Patrick Minard

    I agree with the whole concept that social metrics should influence where a page belongs in relation to another within search engine rankings. What frustrates me is that Google are trying to artificially create a social network by forcing people to share on via Google+. I would like to think that any shares via Facebook / Twitter would carry the same social weighting into the ranking value but I somewhat feel that more shares via Google’s own service will have a better effect.

  • eye9 Design

    I am with AdmiralLuke on this one. I think it is fairly naive to think that the king of search doesn’t use their own Social Networking environment to affect results. Google +1 has plenty of users and you can certainly see how Google could, and in my opinion does, use their own social media network to find out what sites are worthy of higher rankings. Furthermore, I suspect they will turn up the “value” of Google +1 in the future as they learn more about how it is used relative to search results.

  • http://www.carsmaniacs.com/ Silviu Fera

    yeah right and the Moon is not round..

  • Tom

    Interesting comment Admiral Luke and I think you’re right. I’ve had sites/posts naturally get to the top of search rankings without any backlinking and I suspect that it was mainly through Stumbleupon likes. If other social media is influencing SERPs I can’t believe that Google+ does nothing more than kickstart the indexing and I think it will only gain in importance.

  • Ray Parker

    I wish i could have bet on their findings. If google plus did influence rankings then it would be a clear cut anti trust violation

 

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide