Everything you need to know about SEO, delivered every Thursday.
Search 3.0: The Blended & Vertical Search Revolution
This has been a remarkable year. After years of no real dramatic evolution in search, the third generation finally arrived. Google calls it
Universal Search, and I’ve been tending to say “blended search” as a generic name for the change that’s now hit all the major search engines. But in doing the agenda for our upcoming SMX West conference, a better term for what’s going on finally clicked: Search 3.0.
In this article, I’ll cover the why and what of Search 3.0, taking in Search 1.0 and 2.0 along the way and touch on how Search 4.0 — personal and social refinement — is on the way.
Search 1.0: Location, Frequency, & On-The-Page Ranking Criteria
Let’s go back to the good old days, around 1996, when some of the big names
in web search were Lycos, Infoseek, Excite, WebCrawler, Open Text, Hotbot, and
AltaVista. Yahoo was there, too — but it was the exception, the search engine
that relied on humans to catalog the web, rather than the others that used
“crawlers” to automatically build listings.
Those crawlers were robots that automatically followed links across the web.
When they came to a new page, they would effectively make a copy of it. All the
copies were stored in an “index,” which was like a big book of all the pages they
had collected. The crawlers indexed millions of pages, which sounds like nothing
compared to the billions of pages that are harvested today. But back then, the
web was a lot smaller!
When someone did a search, the actual namesake “search engine” component
kicked in, using an “algorithm” to sort through all the pages in the index. The
algorithm was a system designed to return what a programmer thought the best
pages would be. Back then, the best pages were deemed to be those that used the
search words a lot in proportion to other words on the page, plus whether those
words appeared in key areas of a web page.
Consider a search for shoes. The algorithm might not like this page:
Walking Alone At Night
I hear behind me
Shoes softly scuffle
Friend or foe?
Keys, and I’m safe inside
Maybe the algorithm doesn’t like bad poetry written in a hurry. But see how
the word “shoes” appears only once? How relevant is this page really, then, if
shoes only gets a single mention. Now consider:
Buying Shoes Online
As the web grows, many are discovering
that shopping can indeed be done online
for all types of things, even shoes!
Yes, footwear fanatics. You can buy
shoes online. But what about trying
them on? Those selling shoes have
thought of that. In this review, we
look at online shoe sellers with great
In the second example, shoes is mentioned several times. Plus, it appears as
the headline of the story, and in the opening paragraph. Let’s say it also
appears in the HTML title tag. That means shoes not only appears with frequency,
but it also appears in key locations on the page. Algorithms back then liked
So first generation search? Search 1.0 was largely about looking at the
location and frequency of words on individual web pages to measure them against
Search 2.0: Looking At Links & Other Off-The-Page Criteria
It didn’t take long for savvy search marketers to figure out that if the
algorithm liked certain things, you should tailor content to suit. Thus, SEO was
born — search engine optimization, optimizing content for search engines. Some
SEOs focused on making relatively minor changes that still could have dramatic
impacts. Other figured if location/frequency was what the search engines wanted,
the more the better:
Shoes, Beautiful Shoes!
Shoes are the best shoes
anyone could hope to find
shoes online at our store
with shoes shoes shoes
blue shoes red shoes
walking shoes tennis shoes
Pretty ugly — but that’s OK — this text might be hidden from human view in
a variety of ways. The main point is that location and frequency worked better
in the trusted environments of libraries and controlled collections of documents
that web search grew out. Applied to the open web, where anyone could get
anything listed, it started to fall apart. Back in 1999, AltaVista founder Louis
Monier told me:
The good answer to the query ‘car’ has nothing to do with the text. I’d
rather use a medium with a crystal ball.
Instead of a crystal ball, two guys by the name of Larry Page and Sergey Brin
had fired up their
own search engine, Google, that relied on link analysis to rank pages. They
weren’t the first to look at links, and
PageRank was only
part of the overall algorithm. But the pair popularized a shift for all the
major search engines to make a second generation leap — depending on criteria
not actually on web pages to rank which pages are best. Call it Search 2.0.
It’s not just links that can be part of off-the-page criteria. Clickthrough
is a metric that has been and may still be used, where you determine if people
are clicking on a particular listing more often than one in a particular
position normally gets. If so, perhaps the listing should be ranking better.
Domain age and general traffic levels to a site are other things that can be
used. But it’s really the links that remain dominant.
Google in particular popularized the notion of links as votes, with the web
being “democratic” in how those votes were cast. And for a time, it worked. But
then we got active campaigns to win votes, Googlebombing that led to things like
the miserable failure
ranking for President George W. Bush. We also saw an economy of buying and
selling votes spring up, which to this day sees Google trying to
fight off paid links.
Search 2.0 still works, of course, despite these problems. But things could
be better — cue Search 3.0 to the stage, please.
Intermission: What The Hell Is Vertical Search?
I’ve long hated the term “vertical” search, which was popularized by
financial analysts more than search engines. I hate it because most people have
no idea what it means. I know. I ask audiences all the time if they understand
the term, and plenty of hands of those indicating “no” go up, followed by “ah
ha” looks after I explain it.
But vertical search as a term is here to stay, continuing to be used and even
having its own Wikipedia
entry, so let me explain it.
Go to Google, Yahoo, Microsoft Live, Ask.com and do a default search. What
you’ve done is a “horizontal” search. You’ve searched across the entire spectrum
of the web for a topic. To illustrate, consider just some of the topics that
pages are about:
… shopping video news sports weather research stocks lottery …
See, a nice long horizontal line. But if there’s some news event that
happens, you don’t want to search the entire web for information. Doing that
means you’re hunting through billions of pages to find the relatively few
updated with the fresh and relevant information. Far better if you focus only on
To help you do that, search engines build specialized search engines that
only go to news sites. Rather than search across the horizontal spectrum of the
web, they let you slice down “vertically” only into news — or into sports — or
whatever. Search for “fires” in a news search engine after a major fire in a
particular area, and pages about that fire will dominate the results, since you
can apply a time-based ranking criteria. Search for “windows” in a home
improvement search engine, and you shouldn’t get information about how to fix
Vertical search isn’t new. However, it faces a huge challenge. Searchers
simply don’t know vertical search engines exist. Put tabs, buttons, drop-downs,
you name it — it’s been tried (see
Why Search Sucks & You Won’t Fix
It The Way You Think for some illustrations). Users have ignored these
Here’s another problem. Consider this:
That’s my “Google 2005″ image that I made in 2001, for a keynote to a
librarians’ group on search trends. My slide was to illustrate that Google
couldn’t keep adding tabs for each vertical search engine it unleashed. More
important, even if it did, no one clicked on the tabs. Instead, I explained that
search engine would need to learn to push the right tabs for us, behind the
scenes, something I eventually
Search 3.0: Blending Vertical Search Results
Even in 2001, we had some of this mixing of vertical results happening,
usually at the top of pages in what Google called
OneBoxes and others gave different names. But finally this year, search’s
third generation really happened. The supremacy of “horizontal” web search
results being the default was hit by Google’s Universal Search rollout.
Our Google 2.0:
Google Universal Search article from May explains more about the inner
workings of Universal Search — and the headline might now seem odd — shouldn’t
it be Google 3.0 if we’re talking Search 3.0? Well, Google 1.0 was a Search 2.0
search engine already! That’s my story, and I’m stickin’ to it.
The short story is that Universal Search replaces some of the web search
results with listings that come from vertical sources, such as video, local, or
news. The results are mixed in, blended as appropriate, as our
Illustrated column, um,
illustrated in July:
It’s not just Google, however. In June, Ask
launched its Ask 3D
interface that used the “Morph” algorithm to automatically decide which vertical
search results to blend into the main page. Yes, web search listings still
remain front and center, with vertical search blended around web search — but
vertical results became far more prominent.
OK, I am stretching the last two a bit. In both cases, they’ve mainly
enlarged the “shortcut” style material that shows up at the top of a search for
some queries, such as illustrated below for Yahoo:
And here for Live:
Still, the vertical search listings are growing in how often they appear at
Microsoft and Yahoo, as well as with the amount of space they take up on the
search page. It’s enough for me to declare Search 3.0 now fully engaged by all
the major search engines.
For search marketers, Search 3.0 represents new opportunities. There’s less
content in various vertical search engines, so the odds of ranking well
naturally increase more. In addition, much of the content in vertical search
areas such as local and video seems poorly optimized. With just a little care,
people should be able to see improvements — and now improvements that may bring
them to the first page of the “regular” search results.
Want to learn more from a search marketing perspective? Aside from how we
cover verticals (and thus Search 3.0) through our
columns here and features,
as I said above, we’re going to have an entire track on Search 3.0 at our
SMX West show happening next
February, specifically these sessions:
Search 3.0: The Blended Search Revolution
Search 3.0: Video, Images & Blended Results
Search 3.0: Local Search & Blended Results
Search 3.0: Online Retail & Blended Results
Search 4.0 & Beyond
I’m certainly not the first to use the
phrase “Search 3.0,” and I know not everyone agree with how I suggest it be used
now. Time will tell! But I hope this article explains why I find it useful to
explain the latest generation that’s been unleashed, especially in a world where
everything seems more easily understood by some if it’s X.0 something.
I’d note that back in October 2005, Robert
for a “Search 3.0″ world where search engines learned from users, and Read/Write
in July about “Search 2.0″ companies that were using “third generation” social
features. That got me to
The third gen has commonly been considered the combination of personal data
— either refining results because of your own past history or that of others.
Lump it all into social search, and that’s your third gen. And since it’s
third gen, please call it Search 3.0 if you must and don’t force it into a Web
2.0 world just to try and mesh some Web 2.0 companies into a Search 2.0
But the human / social element does have a
role to play in search. It’s the other long expected evolution to hit, one that
I’ve long written
about being alongside vertical search as a third generational leap:
Vertical search is important because it’s one of the two major things I’ve
long talked about as being how search will advance. First generation search
analyzed words on a page to rank content. Second generation search tapped into
link analysis. Third generation search to me is looking at both user input
(what we visit; what we click on; personalized results) and making search go
So where’s the humanity in Search 3.0? It’s
not — I’m shoving it into Search 4.0, a fourth generation leap that Google’s
already making and the other major search engines have yet to push into.
Search 4.0 is another track at the show — and the topic for another article
next week (Now posted: see Search 4.0: Putting Humans Back In Search). Next time, more on personalized and social search. Until then, here’s
some key reading for those who want to look ahead:
- Google Search
History Expands, Becomes Web History
- Mahalo Launches
With Human-Crafted Search Results
- The Promise &
Reality Of Mixing The Social Graph With Search Engines