Google: Unlinked URLs Are A Source For Indexing New Content

googlebotJohn Lincoln from Ignite Visibility spotted a Google Hangout where Google’s John Mueller confirmed that Google will use URLs or domain names within content that are unlinked, i.e. without an a href attribute, to discover and index new pages of content.

John went on to add that typically no PageRank or trust is flown from an unlinked URL or citation, but Google may indeed use it for content discovery.

You can watch John Mueller from Google confirm this 47 minutes and 32 seconds into this video:

John Lincoln transcribed part of it:

We use those kinds of links to try to discover new content. So for instance if we see that someone has been writing about a new domain name and we can recognize that as a domain name in the text even without a normal HTML link there, then that is something where we will try to pick that domain name up, try to crawl it and index it and see if that is something worth including in our search results.

Sometimes it happens that we pick up a whole URL like that. Sometimes someone will try to shorter a URL with just a ‘…’ in between and we try to crawl that URL so we get it wrong. But our goal here isn’t necessarily to pass any pagerank, which we don’t do with those kinds of links. But rather discover new URLs that we haven’t seen before. And if we see someone write about a URL that we haven’t seen before we will pick that up and try to index that for search.

I believe this may be the first time Google has gone on record about how they use URLs or domains within content that are unlinked, i.e. basic citations.

Related Topics: Channel: SEO | Google: SEO | Top News

Sponsored


About The Author: is Search Engine Land's News Editor and owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here. For more background information on Barry, see his full bio over here.

Connect with the author via: Email | Twitter | Google+ | LinkedIn



SearchCap:

Get all the top search stories emailed daily!  

Share

Other ways to share:
 

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.
  • Derrick Hicks

    I wonder how Google treats the sites that are “linking” to these urls? For instance, what would happen on a forum that has tons of user generated content. If they strip out all external links and just leave the text, Google wouldn’t “ding” the site publishing the “link” If users are promoting a “bad neighborhood” would they? It’s not a link so to say, but if Google is crawling that forum and then finding all of this text pointing to spammy sites, could that hurt the “linking” site? Thoughts?

  • Durant Imboden

    Now, if only we could get credit when a major media site mentions our site without linking to it!

  • Andrew SEO

    I believe Google is graphing all keywords just like links and using correlation between the keywords as a ranking factor. This is just the tip! Nice report Barry!

  • http://www.seo-theory.com/ Michael Martinez

    No, they’ve said they crawl unlinked URLs in the past.

  • http://www.irishwonder.com IrishWonder

    I believe they did mention before that unlinked mentions on authority sites do carry some sort of weight, not seen anything about actual crawling though

  • Peter Watson

    That would be nice. I think this is the direction they are heading in.

  • http://www.seo-theory.com/ Michael Martinez

    I don’t know about that but I’m surprised to see this come up as news. I thought everyone remembered that Google will crawl all sorts of things “for discovery”.

  • David Edwards

    Hey Bazza. I did a whitepaper on the different ways Google can discover URLs a while back that discovered this also. There were some interesting takeaways from it, if you want to check it out – http://www.clickconsult.com/whitepapers/Click_Consult_How_Does_Google_Discover_New_URLs_Whitepaper.pdf

  • SEO Expert – Rahul Trivedi

    I have to try this one

  • Scott Davis

    In the situation you’ve described, they’re not links.

  • Erik Dafforn

    John left a pretty similar comment here in 2011 in a thread about plain-text links:
    http://webmasters.stackexchange.com/questions/21359/how-does-google-treat-unlinked-urls
    “Given that we find links in all kinds of places and are primarily using them to discover unknown URLs, I wouldn’t worry about whether or not they pass PageRank.”

    We’ve seen plain-text (i.e., no “href”) links show up in GWT “Links to your site” reports for a while, so it’s disappointing (but not surprising, I suppose) that they’re not passing PageRank. For many sites, it’s just as valid a way to “endorse” a site as an href link, but the publisher (or CMS) may not be savvy enough to make it hot.

  • Morgan Akchehirlian

    I think like directories,forums are going to be 100% dead than. About 70% forums have tons of links and I am sure if this is the case the communities and blogs are going to hurt so badly.
    They hate spam but still index it?Strange but no solution.Surely the sites getting links from these forums are the first one to get affected by it.
    indirect citations and citations will than be really helpful.

  • Morgan Akchehirlian

    I think like directories,forums are going to be 100% dead than. About 70% forums have tons of links and I am sure if this is the case the communities and blogs are going to hurt so badly.
    They hate spam but still index it?Strange but no solution.Surely the sites getting links from these forums are the first one to get affected by it.
    indirect citations and citations will than be really helpful.

  • Derrick Hicks

    Hey Scott! Yep, I understand that they are not links. That’s why I called them “links”. :)

  • http://www.wigsalon.com/ Mike Aronesty

    Pretty sure they track urls via js widget codes as well. I’m curious how this gets… Google definitely picks up on the links though. Oh and they have been tracking non linked website mentions for a long time… At least a year or two.

    When penguin 1 happened, there was a domain with 10,000+ non linked mentions of one of my domains. It was the primary cause of the penalty, and once fixed they put me back. Again – These did not have href’s, the and was not a link … simply something like this http :// http://www.xyz. com

Get Our News, Everywhere!

Daily Email:

Follow Search Engine Land on Twitter @sengineland Like Search Engine Land on Facebook Follow Search Engine Land on Google+ Get the Search Engine Land Feed Connect with Search Engine Land on LinkedIn Check out our Tumblr! See us on Pinterest

 
 

Click to watch SMX conference video

Join us at one of our SMX or MarTech events:

United States

Europe

Australia & China

Learn more about: SMX | MarTech


Free Daily Search News Recap!

SearchCap is a once-per-day newsletter update - sign up below and get the news delivered to you!

 


 

Search Engine Land Periodic Table of SEO Success Factors

Get Your Copy
Read The Full SEO Guide