searchengineland.com » Search Features: Tagging

Yahoo! Announces Common Tag: Like The Meta Keywords Tag, But Even Better

Vanessa Fox — Mon, 15 Jun 2009 19:19:48 +0000

Yahoo! recently announced their role in creating and supporting Common Tag, a new semantic tagging format.

Yahoo! says that Common Tag makes “web content more discoverable” and enables the community to “create more useful applications for aggregating, searching, and browsing the web.” Their blog post mentions that they want to accelerate the structuring of the web, which aligns with their SearchMonkey launch last year, which they said was, in part, an attempt to encourage the use of structured data on the web. This brings to mind a few questions.

Why did the web need a new semantic standard? The Common Tag blog explains:

“Semantic web promises havens. The promise of machines understanding the data and acting on it semi-autonomously. But how do we get there? Not many are willing to put in a lot of work into publishing data in new formats. For the benefit that might or might not come years after enormous resources will be spent?

So why not start with something easy, say tagging? Marking up text with exactly defined concepts. “

OK, maybe “explains” isn’t the right word.

This question really goes to the heart of what’s curious about Common Tag. Yahoo called it a “new semantic tagging format” in its blog post, but when we asked them why the web needed something new, they clarified that it’s an RDFa vocabulary, not something made from whole cloth. The Common Tag About page also implies that this is really just part of the standards that all the major search engines have joined together to support. “In addition, search engines like Yahoo and Google have begun reading RDFa—the markup standard used by the Common Tag format—to acquire richer information about sites that use it… Google’s new Rich Snippets feature uses the information to apply similar enhancements to Google search results.”

In truth, none of the major search engines are using semantic markup in web search and Google is using existing standards (microformats and RDFa) to display enhanced listings. Both Google and Yahoo have told me that they could use metadata in web search in the future, if it proves to be useful and they can safeguard against spamming. So far, this hasn’t happened.

Yahoo did clarify to me that Common Tag is something they’re participating in as means to cultivate the structured data community, not something they’ve come up on their own and are trying to get the community to adopt. RDFa provides a structure from which you can create vocabularies and several companies who were using RDFa were interested in creating a tagging vocabulary. Since these companies used SearchMonkey as an application for their metadata, they asked Yahoo to help create and promote this new vocabulary.

So, how does it work? Common Tag is intended to be a common tagging format to standardize tagging of concepts. According to the commontag.org site, as “publishers, developers, and end users” join in support for this format, “more content related to a specific concept will be discoverable through a single tag.” Now, for instance, the concept New York City may be tagged with “nyc”, “new_york_city”, and “newyork”. You can adding this tagging markup to your pages manually, or you can use infrastructure such as that provided by founding company Zemanta. And you can eliminate the problem of multiple tags for the same concept by using data from a participating database, such as Freebase (also a founding company). You can then use this structured data in an application such as Yahoo! SearchMonkey.

For instance, the Common Tag documentation uses the following example of using the Freebase database to tag a page as being about U2:

You can also do more complicated tagging, such as of external resources, sections of your web pages, and concepts within your content. For instance, you can identify the paragraphs of text on the page as follows:

Everyone loves Buffy the Vampire Slayer.

Amber Benson was awesome in it.

And then create tags for those paragraphs:

Why is Yahoo! so hell-bent on covering the web with structure? If Yahoo! found structured data made the web easier to crawl and their search results more relevant, I could see the push. But Yahoo! doesn’t use any of the semantic formats they’re encouraging in web search. They already were encouraging hCard, hCalendar, hReview, hAtom, XFN, Dublin Core, Creative Commons, FOAF, GeoRSS, MediaRSS, RDFa, and OpenSearch. Why do they need web developers to start using yet another format when they haven’t yet figured out how to use all of those others in their core search engine? Sure, they are involved in Common Tag in order to support the structured data community they’ve been aiming to accelerate, but why is that so important to them?

Since Yahoo isn’t encouraging the use of semantic markup to help them get an edge in search, it seems they must be instead looking to increase adoption of SearchMonkey and BOSS, where these formats are used.They seemingly have diverted the energy they used to spend to help improve Yahoo’s search index via tools such as Site Explorer to working to raise adoption of BOSS. The last Site Explorer update was in August 2008, and that was simply a UI change. No new features were launched. For new features, you have to go all the way back to August 2007, for dynamic URL rewriting.

Even Yahoo CEO Carol Bartz isn’t talking about focusing their consumer search engine as a core offering, but rather something that’s convenient for Yahoo users who are already on the site for some other reason (emphasis mine):

Listen, Google has the search brand, there’s no doubt about it …. [Yahoo Search] is really designed for people that are on our sites and find something interesting, they want to look farther and they go to Yahoo Search.

It’s enough to make one wonder if Yahoo is quietly abandoning its consumer search engine in favor of accelerating new third-party search engines through BOSS. If you can’t beat ‘em, help their enemies attack them on all fronts, as the old saying goes.

Didn’t the search engines already try using meta tags? The idea of using meta data to tag web pages in order to describe them to search engines isn’t new, of course. The meta keywords tag has been around since at least 1995. And it’s easier to adopt than Common Tag. That U2 example? The meta keywords tag would only require this:

Indeed, Yahoo supported the meta keywords tag initially (and to some extent, still does), but when Google launched, they did not. It was too easy for site owners to stuff that tag with anything they wanted, rather than the true focus of the page. Search engines use smarter methods (starting with the content on the page and how external sites link to it) for determining relevance. Could Common Tag have the same downfall? After all, as the documentation explains “you can create as many Tags as necessary to describe the contents of a document.”

Not only does Common Tag seem to replicate the purpose of the meta keywords tag, it seems to also replicate Delicious-style tagging and external anchor text. From the site:

“Common Tags are not only useful for identifying the concepts covered in your content, but if you reference content elsewhere on the web, Common Tags can be used to indicate the concepts covered in that external content as well. This is useful for better describing and organizing the content of external resources from within your own content. For example, you could use Common Tags to publish bookmarks to identify the concepts described by a link, or you could use them categorize image collections stored elsewhere on the web.”

A microformat already exists for a similar purpose as well. rel=”tag” is intended to tag content, such as web pages or portions of them.

Anchor text is an established method for search engines to determine how others describe an external resource. As for tags, the study Can Social Bookmarking Improve Web Search, presented at the First ACM International Conference on Web Search and Data Mining (Stanford) analyzed 40 million Delicious tags and found that anchor text was a better signal for web search relevance. Part of the problem was scale of adoption. A lot of people have to adopt this new tagging method for it to be worthwhile to use across the web. And if Delicious tags don’t have the scale, how long will it take for Common Tag to?

When I asked Yahoo about it this, they acknowledge that it may not be something that’s adopted web-wide. Rather, it’s a format of interest to a particular group of developers who have needs beyond that which is available through means such as the meta keywords tag and rel=”tag”.

Why would anyone implement this? It seems like a lot of work. You can tag content now using methods like anchor text and well, tags, such as those available through most blogging platforms and bookmarking sites like Delicious. If content management systems and other content creation platforms such as blogging systems incorporate this structure (for instance, by automatically using the tags labeling a blog post), we might see some adoption, but this wouldn’t eliminate the issue of multiple tags for one concept. (Zemanta, one of the founding companies for Common Tag provides plugins for blogging platforms to insert Common Tagging.) And Wordpress strips out RDFa by default.

The answer is that web developers will use this structure, just as they’ll use any other structure, if it’s valuable for what they’re building. And what applications are ultimately made possible by this format remain to be seen.

Wikia Introduces “Evolution” Toolbar To Expand Reach To Other Search Engines

Greg Sterling — Thu, 07 Aug 2008 14:18:35 +0000

Like many before it, most recently Mahalo, Wikia Search has introduced a Firefox toolbar, called “Wikia Evolution,” to expand the site’s reach and engage more users in the process of adding and annotating search results.

In the notes, it suggests that Google is taking a page from its community playbook: “Already, we’re the cutting edge when it comes to incorporating user feedback into our search results, so much so that Google is experimenting with eerily similar features.”

The new Wikia toolbar allows users to add URLs/pages to the Wikia index (from wherever they are online) and edit or annotate the descriptions and titles of those pages. Users are asked to associate keywords and URLs:

While perhaps simplifying the process of adding to the Wikia index, the toolbar is primarily designed to attract participation and input from users who might be interested in or sympathetic to the Wikia project but use Google or Yahoo as their main search engines.

Microsoft Joins The Social Bookmarking Game

Barry Schwartz — Wed, 28 May 2008 12:51:38 +0000

Microsoft’s John Martin has shared a preview of the new Microsoft social bookmarking tool that will be demonstrated at the TechEd North America event in the upcoming week.

John provided a shortlist of features and two screen captures of the social bookmarking tool at his blog post. Based on the limited detail I can see, the tool is not all that innovative when compared to Delicious and other social bookmarking tools out there today. It seems that you can bookmark any URL across the web then apply tags to those URLs and also save the bookmarks in the “new My Bookmarks tool.”

The screen shots look very similar to other social bookmarking applications. In addition, Microsoft set up a support forum for this tool at Microsoft Forums.

Yahoo Travel Adds Personalization, New Maps

Greg Sterling — Wed, 09 May 2007 13:18:15 +0000

Yahoo Travel has redesigned and added a range of features, including personalization and new mapping tools to the site. It’s also seeking to more deeply integrate travel properties FareChase (its fares and rates “meta search” engine) and Trip Planner (its user-generated content site) into Travel. Flickr is also well represented throughout the redesign. I’m not going to be able to capture every nuance or aspect of the upgrade and all the new features, but I’ll highlight the main ones.

The new home page displays personalized recommendations, as well as more “social features” from Trip Planner. The personalized recommendations, prominently featured as still photographs in a horizontally scrolling menu, are drawn largely from explicit user activity and search history on the site. But there’s also reliance on user location and collaborative filtering (”those who liked Paris . . .”). Another feature of the new personalization is a pull-down menu (”my recommendations”) that changes recommendations depending on the category selected. Categories include Best Deals, Romantic, Beach, Nearby, Art/Architecture, Family, Hiking & Camping and Nightlife. The content of these recommendations come via a new structured tagging feature in Trip Planner.

One of very useful new features is the integration of a collaborative “flight planner” plug-in for Yahoo Messenger (only the desktop client right now). Built on FareChase, the idea here is that you can review, consider and book flights with friends and family in real time.

Another very interesting aspect of the redesign is what Yahoo is calling “remap.” I was told the new capability came out of Yahoo Hack Day and may be rolled out beyond Travel to other Yahoo Maps. Basically “remap” is the layering of additional maps and detail on top of existing Yahoo Maps.

Here are some examples:

You need to scroll to the bottom of the pages for the maps. You can play with the “opacity” of the overlaid image/map using a slider. After you take a look at one or both maps you’ll get the idea of how interesting this is as a way to highlight locations, routes or features of a destination. This also points toward “social maps” (user-generated content) that represent a somewhat different approach than what Google is doing with My Maps.

One of the problems with the travel category, arguably the most mature vertical, is that there’s so much information and so many tools online that it can be overwhelming. In addition, there’s also a lack of confidence that pricing information you’re seeing on any given site represents the best deal or lowest price.

By contrast, Yahoo’s range of tools, meta-search capabilities, user recommendations and personalization features is an impressive package that starts to look and feel pretty complete.

Happy Birthday, Flickr: Web 2.0 Pioneer Turns Three

Greg Sterling — Tue, 27 Feb 2007 20:43:55 +0000

On March 3rd, Flickr turns three. Flickr is in many ways the company that helped define “Web 2.0″ and was its poster child for quite some time. And Yahoo’s acquisition of Flickr, in March, 2005, after only a year in existence and, even then, a longish courtship was one of the best consumer moves Yahoo’s made in the past couple years.

Flickr and the community mindset it brought along, not to mention tagging, have been widely integrated across numerous Yahoo properties. Here’s the Business 2.0 article, “The Flickrization of Yahoo” that appeared in late 2005.

Indeed, Flickr has had a “cultural” impact on Yahoo and helped define its competitive strategy vs. Google, part of which is based on community and social media. In this realm, Yahoo was way ahead of Google, which only recently “got Web 2.0 religion.”

However perhaps the biggest story, in retrospect, is how inexpensive Flickr was for Yahoo given all the benefits the company has received. Reportedly the sale was in the $20 to $30 million range. Flickr was considering taking VC money at the time but instead felt the company would have a better home at Yahoo.

Here’s co-founder Caterina Fake’s blog posting announcing the acquisition in March, 2005. And here’s another, related post from her justifying the acquisition:

The best thing is we no longer have to worry about finance, HR, legal, or things at which we are completely incompetent and were taking our time away from building Flickr. Hot damn.

Now Fake is one of the movers and shakers behind Yahoo’s quasi-incubator and employee retention vehicle “Brickhouse.”

In a post responding to community fears about the impact of the Yahoo takeover on the day the acquisition was announced, Flickr co-founder Stewart Butterfield wrote:

And we prefer to think of it as Flickr taking over Yahoo rather than the other way around.

In a way he appears to have been right.

Happy Birthday, Flickr.

Search Patent Documents for 1-12-07 – Limited Access Documents in Search Results

Bill Slawski — Fri, 12 Jan 2007 17:34:10 +0000

A wide range of newly published patent applications and granted patents, covering such ground as subscribed content in search results from Google; tag searching, detecting similar audio files, and simpler support vector machines from Yahoo; geographic based searching from MetaCarta; and data center architecture and smarter results to queries from Microsoft, amongst others…

Patent Applications

Document access control Google (20070005595)

Focuses primarily on setting the ability to access documents for individuals. Where it starts to become interesting is when it discusses checking access rights before performing a search on an intranet or the internet. Might this lead to subscribed content showing up in search results along side web results? Hard to tell. added: the document is talking about subscription based sites or collections of images or calendar applications that aren’t normally accessible in search results, instead of the type of pages that show an abstract, and then charge you to read the rest of the document. It would be nice to be able to filter those out of search results.

User interface for navigating a keyword space Yahoo (20070011161)

Related to the patent filing I discussed in The Social Side of Trustrank, this patent application discusses searching through tags that you, or your friends may have used on web pages.

Building support vector machines with reduced classifier complexity Yahoo (200700411110)

Describes a way that the use of support vector machines (SVM) can be simplified, so that they may be used in sorting through search results to rerank them based upon some relevancy factors.

System and method for searching for a query TextDigger, Inc. (20070011154)

It’s possible that some synonyms for queries used in a search may be considered to be equivalent for that search (such as car and automobile), and could be used to provide relevant results for the query. This patent application explores the idea.

User Interface For Geographic Search MetaCarta, Inc. (20070011150)

Interesting to see an advanced geographic-based search engine from a company that isn’t one of the major search engines.

Granted Patents

Method and system for usage analyzer that determines user accessed sources, indexes data subsets, and associated metadata, processing implicit queries based on potential interest to users Microsoft (7,162,473)

I wrote a little about how some of the lessons learned from working on Microsoft’s desktop search and Vista were influencing search at live.com in Improved Information Retrieval – Looking at Context with Susan Dumais. This patent takes more ideas from that research, and discusses how it could be used to do things such as provide information from standing queries to users.

Architecture for distributed computing system and automated design, deployment, and management of distributed applications Microsoft (7,162,509)

There’s been a lot of recent stories in the news about new locations for data centers for search engines. If you’re interested in learning a little about how a data center works, the detailed description in this granted patent provides a fair amount of explanation and detail.

Methods and apparatus for indexing and searching of multi-media web pages Oracle (7,162,691)

Audio, Video, and multimedia content provide search engines with some indexing issues. This patent describes a way to automatically create text annotations, or meta data, to be used in indexing those pages and improve searches for the content they contain.

Information retrieval engine Yahoo, through MusicMatch (7,162,691)

If you search through YouTube, you might find the same video appearing multiple times, under different names and with different tags and information attached to it. The same is true with sites that search for audio. If there were a way to recognize that those were the same video or audio files, it might have implications for the way those are indexed. This patent focuses upon indexing audio files by comparing information derived from looking at the contents of the files themselves. The patent notes that the process is one that could be used for more than just audio.

Disclaimer: Patents are filed to protect ideas and methods developed as part of the intellectual property of a company, and may be used to exclude others from using the same, or similar processes, but the granting of a patent or publication of a patent application doesn’t necessarily mean that the processes involved have been fully developed, or will be in the future. Yet, the documents can provide some insight into the ideas that an organization is working upon, and may act as a starting point for more research.

Tagging Isn’t Indexing: A Study of Del.icio.us Tagging

Bill Slawski — Thu, 04 Jan 2007 05:40:05 +0000

Have you ever tagged an image in Flickr, a link in Del.icio.us, a video in YouTube? Have you used tags in these systems to find similar pictures or pages or videos? Is tagging is taking indexing away from professional indexers, and putting it into the hands of amateurs. Have you used a tag to indicate that you want to read something in the future? Is tagging a threat to indexing, or is it developing into a complementary way of collecting information about an object?

A study on tags, Patterns and Inconsistencies in Collaborative Tagging Systems: An Examination of Tagging Practices (pdf) – via Gary Price, explores the use of tags from a scientific perspective.

A co-word analysis of tagging practices, therefore, could provide insight into the patterns that are emerging through these practices, and the extent to which they are consistent with, and supportive of, conventional indexing and classification practices. Equally important, such an analysis might well show important differences between user tagging and conventional indexing: differences which indexers would do well to notice.

Some interesting observations within the paper. I’ll be thinking a little more about the tags that I use now that I’ve read it.

Jon Kleinberg On Communities At Yahoo

Danny Sullivan — Fri, 01 Dec 2006 23:12:20 +0000

Another one from the clean-out my email files. Earlier this year, I was visiting at Yahoo doing briefings. At another Yahoo campus, Cornell University professor Jon Kleinberg was visiting and giving a standing room only lecture. I’d have been standing with everyone as well, had I known. Kleinberg is legendary in the search world, and his work with links attracted attention at the same time Google’s did. I was pleased to meet him personally at the end of the day, when we both ended up at the same restaurant with Yahoos. You can see what the Yahoos got in his lecture here, which leads to a video of his talk on large social networks and how knowledge moves between communities.