• Search Engine Land
  • Sections
    • SEO
    • SEM
    • Local
    • Retail
    • Google
    • Bing
    • Social
    • Resources
    • More
    • Home
  • Follow Us
    • Follow
  • Search Engine Land
  • SEO
  • SEM
  • Local
  • Retail
  • Google
  • Bing
  • Social
  • Resources
  • Live
  • More
  • Events
    • Follow
  • SUBSCRIBE

Search Engine Land

Search Engine Land
  • SEO
  • SEM
  • Local
  • Retail
  • Google
  • Bing
  • Social
  • Resources
  • More
  • Newsletters
  • Home
Social

Twitter’s New SEO Play: A Profile Directory Of All User Accounts

Twitter has quietly launched a user profile directory that it hopes will bring it more search engine traffic. It’s Twitter’s second SEO play in recent weeks, coming on the heels of a change in its robots.txt file. With that change, Twitter made its content-based search results — i.e., search results for tweets and hashtags, but […]

Matt McGee on October 8, 2012 at 8:45 am
  • More

twitter-new-logo-2012Twitter has quietly launched a user profile directory that it hopes will bring it more search engine traffic.

It’s Twitter’s second SEO play in recent weeks, coming on the heels of a change in its robots.txt file. With that change, Twitter made its content-based search results — i.e., search results for tweets and hashtags, but not search results for users — crawlable.

With the new directory, Twitter is finishing that process by making its user profiles more discoverable by Google, Bing and other search engines. When asked why Twitter launched the user directory, a company spokesperson said it’s “to help people find the accounts they’re looking for with various search engines.”

Is The Directory Helping Twitter’s Search Visibility?

Twitter’s user directory has been online for a couple weeks. It’s essentially a master site map of Twitter accounts listed alphabetically and, from an SEO perspective, follows best practices by showing no more than 100 profile links per page.

As I write this, a Google search for site:twitter.com/i/directory/profiles gives a superficial display of 718,000 matching results. If I click through to reach the end of the 1,000 results that Google will show, I can reach page 47 before getting the “we have omitted some entries very similar to the 469 already displayed” message. And if I click to include the omitted results, I hit the end at page 57.

twitter-google-directory

The 718,000 results isn’t right because Twitter’s directory doesn’t have that many pages. If you estimate that Twitter has about five million total accounts, the directory — with 100 accounts per page — would be about 50,000 pages, far less than 718,000.

UPDATE: I incorrectly typed and used five million accounts in the paragraph above, when I should’ve used 500 million accounts. My apologies. Based on 500 million accounts, the Twitter directory would have as many as five million pages, which means Google’s count of 718,000 could be an accurate reflection of how many pages it knows about.

But even if Google has only indexed about 57 pages (and those numbers aren’t perfectly reliable since I could be hitting a certain datacenter that has fewer results), it’s still a lot more than Bing. My searches show that Bing has so far only indexed the Twitter directory home page.

bing-twitter-directory

I have to assume that Bing will eventually crawl much deeper into the directory; Bing has gone on record admitting that its bot is slow, and that’s certainly the case here.

Of course, many Twitter profile pages have been crawled and indexed by Google and Bing for a long time now, so it’s difficult to say if the new directory is helping (or will help) the search engines find new, unindexed profiles.

The Checkered Twitter/Google Search History

What’s interesting is how Twitter’s recent search- and SEO-friendly moves fit into its checkered relationship with Google.

Google and Twitter used to be on good terms. Google had access to the full “firehose” of tweets and was primarily using that to power its Google Realtime Search product. But that ended in July 2011, forcing Realtime Search offline. Google promised that it would resurrect Realtime Search, but that never happened.

The end of that agreement meant Google no longer had access to the firehose — a special API feed of tweets — but Google was still able to do its regular crawl of Twitter profiles and content.

Except there was one temporary problem: Around the same time, in the summer of 2011, Twitter wasn’t very SEO-friendly. As we reported, Twitter was having technical infrastructure problems that hindered Google’s ability to crawl and index its site. Whoops.

In fact, Google says that it was cut entirely off from Twitter for a time. Here’s the head of Google’s search efforts Amit Singhal talking about the issue (at 1:35 in) earlier this year at our SMX London conference, as well as the challenge in crawling Twitter without a deal:

Twitter eventually fixed those issues, but its relationship with Google has been strained ever since the end of the companies’ agreement. In January, Twitter came out firing when Google launched its Search Plus Your World product, accusing Google of favoring its own social content ahead of Twitter (and other social content).

Google’s response then — and many times since — has been to mention how Twitter turned off the firehose and isn’t giving Google direct access to its content anymore. But Twitter engineers joined with peers at Facebook and MySpace to create a “Don’t Be Evil” tool that showed how easily Google could surface Twitter (and other social) content without firehose access. For more on that, see this recap of Danny Sullivan’s conversation with Google’s Eric Schmidt from January.

So today, while the war of words between Google and Twitter has died down, we now see Twitter taking a much more proactive approach to search engines — first with the robots.txt change and now with the SEO-friendly user directory.

For more on the directory itself and how it works, see my companion article on Marketing Land, Twitter Quietly Launches A User Directory.



About The Author

Matt McGee
Matt McGee joined Third Door Media as a writer/reporter/editor in September 2008. He served as Editor-In-Chief from January 2013 until his departure in July 2017. He can be found on Twitter at @MattMcGee.

Related Topics

Channel: SocialFeatures: AnalysisTwitterTwitter: Search

We're listening.

Have something to say about this article? Share it with us on Facebook, Twitter or our LinkedIn Group.

Get the daily newsletter search marketers rely on.
See terms.

ATTEND OUR EVENTS

Lorem ipsum doler this is promo text about SMX events.

February 23, 2021: SMX Report

April 13, 2021: SMX Create

May 18-19, 2021: SMX London

June 8-9, 2021: SMX Paris

June 15-16, 2021: SMX Advanced

August 17, 2021: SMX Convert

November 9-10, 2021: SMX Next

October 2021: SMX Advanced Europe

December 17, 2021: SMX Code

Available On-Demand: SMX

×


Learn More About Our SMX Events

Discover actionable tactics that can help you overcome crucial marketing challenges. Our next conference will be held:

MarTech 2021: March 16-17

MarTech 2021: Sept. 14-15

MarTech 2020: Watch On-Demand

×

Attend MarTech - Click Here


Learn More About Our MarTech Events

White Papers

  • Digital Marketing Report Q4 2020: Benchmarks and Insights for 2021
  • Data SEO – The Next Big Adventure
  • Getting Started with Email Marketing Automation
  • The State of Local Marketing Report 2020-2021
  • Quality CRM Data: The Key to Delivering Great Customer Experiences
See More Whitepapers

Webinars

  • How to Avoid the Digital Transformation Trap
  • How to Build a Marketing System of Record
  • Meet BIMI: The brand-boosting email security marketers must have for 2021
See More Webinars

Research Reports

  • Local Marketing Solutions for Multi-Location Businesses
  • Enterprise Digital Asset Management Platforms
  • Identity Resolution Platforms
  • Customer Data Platforms
  • B2B Marketing Automation Platforms
  • Call Analytics Platforms
See More Research

h
Receive daily search news and analysis.
Search Engine Land
Download the Search Engine Land App on iTunes Download the Search Engine Land App on Google Play

Channels

  • SEO
  • SEM
  • Local
  • Retail
  • Google
  • Bing
  • Social

Our Events

  • SMX
  • MarTech

Resources

  • White Papers
  • Research
  • Webinars
  • Search Marketing Expo
  • MarTech Conference

About

  • About Us
  • Contact
  • Privacy
  • Marketing Opportunities
  • Staff
  • Connect With Us

Follow Us

  • Facebook
  • Twitter
  • LinkedIn
  • Newsletters
  • Instagram
  • RSS
  • Youtube
  • iOS App
  • Google Play

© 2021 Third Door Media, Inc. All rights reserved.

Your privacy means the world to us. We share your personal information only when you give us explicit permission to do so, and confirm we have your permission each time. Learn more by viewing our privacy policy.Ok