In the not-too-distant past, I wrote about Google’s Data Highlighter for event data, a tool which allows webmasters to indicate structured data for events without having to actually mark up the site’s HTML code. It has the charming feature that the resultant extracted data is viewable by the webmaster only in Webmaster Tools in the Structured Data section; and, of course, the data is available to Google itself.

As there is no actual structured markup ever placed on the page (i.e., no schema.org, microdata or any other markup), the information is extracted via human-guided machine learning and possibly other techniques. This extracted/consumed information resides internally in Google and is not available to any other search or social engine for consumption. Thus, my question at the time was, “Is Google Hijacking Structured Markup?” As you read on, you will certainly realize the answer is affirmative.

Google Structured Data Markup Helper

While perusing the new types of structured data supported by the Data Highlighter, I came across a far more interesting tool — a means Google is giving webmasters to add structured markup to their sites.

The Google Structured Data Markup Helper is actually a pretty cool tool. It allows you to enter a URL and then highlight on-page elements for which you would like to generate structured data markup, automatically mapping them into the appropriate schema.org vocabulary with guided direction as to the relevancy of that element in the schema.org ontology. To test it out and illustrate how it works, I used this product page as an example.

To start, I selected “Products” from the options above, entered the product page URL, and clicked “Start Tagging.” This brought up the screen below: the schema for “products” and its associated data items appeared on the right-hand side of the screen, and the webpage itself appeared on the left.

In this environment, you can highlight any page element — when you do, a drop-down menu appears from which you can select an identifier (Product Name, Product Image, Price, Brand Name, etc.) from among available schema.org markup. Once selected, this information populates within the “My Data Items” pane on the right. On my example page, you can see in the screenshot below that I indicated the brand name (“Rolodex”) and the price (“$21.90″).

(Also of particular note for you data quality folks out there (and I am presuming this is anyone who is involved in Google Shopping at a minimum): note  that the date on which the price is verified above is recorded.)

After tagging all the page elements you’d like to annotate, click on the “Create HTML” button in the upper right-hand corner. This generates a new version of the source code for the page, with the added microdata markup (highlighted for your convenience). All you need to do is add the highlighted HTML markup to your page as shown. Very useful and elegant — and, unlike with the Data Highlighter tool, you actually do get the schema.org markup physically on your page (and thus viewable by other search engines, your Chrome plugins, etc.).

Another point worth observing is that Google gives you a choice of two formats. “Microdata” is selected by default, and “JSON-LD” is provided as an alternative option.

I was pleasantly surprised to see this, as I find it JSON-LD a far more elegant solution (see the JSON-LD code displayed below).

(For the record, Google does state that it prefers microdata for web content.)

As a final note, the Markup Helper supports a range of schema.org markup, but not all of schema.org’s data types. The types supported can be seen in the figure below, and more information can be found here.

Changes to the Google Data Highlighter

The Google Data Highlighter, found under the “Optimization” section of Google Webmaster Tools, was recently extended to support more than just events. As you can see, the new data types supported by the Data Highlighter are identical to those supported by the Structured Markup Helper.

There are clearly many other tools on the market that enable webmasters/users to generate structured markup for their webpages. However, the fact that Google has released two different tools to do this makes it clear that it intends to lead the charge (or at least be at the forefront) of the proliferation of structured data markup on the Web.

Google is definitely “hijacking structured markup” using the Data Highlighter, since this information is not consumable by the standard semantic Web community tools; so, its official support of JSON-LD within the Structured Data Markup Helper (an announcement of formal support to the semantic Web community) while simultaneously expanding Data Highlighter functionality is an interesting juxtaposition of events.

Key Takeaways

• Be sure to place semantic markup on your pages — search engines will continue to leverage this information to enhance SERPs, presumably in ways that searchers will find useful.
• Try to keep current with the latest supported microdata formats and schema.org markup (as well as other vocabularies supported by Google, such as GoodRelations).
• There are many tools on the market to generate static annotations using schema.org and microdata.
• Be on the lookout for commercially available tools that can dynamically interpret HTML pages in real time and apply relevant semantic markup.

The talk focused on what was then the current state of the Semantic Web, as well as upcoming formats/technologies in development and the research being done in the field of semantic search.

The idea that the Semantic Web would be central to search within just a few years was met with some skepticism at the time — back then, all most folks were tracking was the adoption of Semantic Web technologies and semantic search using primarily RDFa, embedded metadata, or semantic markup.

Search Becomes Semantic & Graph-Based

The prediction that search would become increasingly semantic and graph-based has certainly proven to be more than true. Not only have the search engines since adopted schema.org as a standard along with microdata as a syntax (Facebook RDFa and Open Graph are examples), but things are now elevated to the next level in this process of adoption.

Graph Search on consumed/verified/validated information, which is a core component of the Semantic Web, is now considered key for the future of search in both search and social engines.

Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch.

Google Knowledge Graph

Google originally started their rich snippets program in 2009 and finally announced their Knowledge Graph just a couple of days before Facebook’s IPO in 2012. For their “rich snippets,” Google initially utilized their own ontology/vocabulary, then switched to schema.org standards along with the three other major search engines (now four, as Yandex recently joined the mix).

Google’s Knowledge Graph is effectively using their version of graph search; however, it is largely focused on information and facts rather than searcher interests and social interactions. While there are some social signals provided via Google+, Knowledge Graph search is primarily based on Wikipedia-type information and other related/verified sources.

Excitingly, this is constantly being extended. (Those of you interested in further details and additional information may want to check out wikidata.)

Search Is Fast Moving

As an example of the incredible pace at which the industry moves, look at image recognition. Image recognition was a research problem in the 1990s. Several years ago, facial recognition made its way into everyday applications like i-photo, Facebook, Picasa and even your camera. Computer vision techniques are now being applied all over the Web — especially in retail to find “visually similar” items (e.g., Google Shopping and Amazon).

Google Play also now leverages facial recognition to enhance its Knowledge Graph capabilities. It’s pretty remarkable, but we are only seeing the beginning. 

The reality is that intelligent systems are driving e-commerce and sales. We now live in an era where forward deployed inventory calendars can leverage predictive stocking, making same-day delivery of online purchases possible. Technologies like graph search, graph analytics, predictive analytics and big data are par for the course, being leveraged under the hood every time a user makes a purchase online. Google itself is a master at these technologies.

Bing Tiles

Clearly, the other search engines have adopted similar data tracking techniques and are working to leverage them as well. Bing “tiles” are their version of rich snippets; their version of the knowledge graph, or graph search, is probably Bing snapshots. (Feel free to read more on Microsoft’s graph-based repository, Satori.)

From what I can garner looking at this in the context of Bing Tags, they are taking a more social approach to Semantic and graph-based search (presumably in line with their relationships with social media sites like Facebook, LinkedIn, etc), even going so far as to allow users to control their appearance in SERPs to some extent. Bing clearly needs this strongly differentiated angle to help increase its share of the search market.

Facebook Graph Search

Finally, enter Facebook Graph Search, which utilizes Facebook’s wealth of easily filterable user data to provide searchers with results that are tailored specifically for them. As my colleague, Gerald Burnand, writes in his piece about Facebook’s Graph Search, there are numerous possibilities for how this filtered data might be used.

Recruiters might look to Facebook to narrow down potential candidates by industry (although it has a long way to go if it wants to compete with LinkedIn). Similarly, Facebook marketers could use Graph Search to learn more about common Likes/interests among their fans, making it easy to create highly-targeted micro campaigns aimed at relevant demographics. 

Understanding & Using Search Data

Today, even the end consumer has access to these resources in the form of their mobile phones, tablets and computers. Remember, every time a consumer picks up a mobile device, uses an app, or performs a search, those devices are essentially acting as sensors, tracking search queries, geolocations, and many other consumer behaviors — all of which ultimately turn into information to be used to increase conversions.

Google’s status as a market leader in search grants them access to copious amounts of invaluable user data and product information. This information — obtaining it, understanding it and applying it — is the key to their success.

There is clearly a race among these companies to better aggregate and understand huge amounts of data. The Semantic Web has come far since 2010, but improvements and new technologies are continuously being developed. What new developments do you see on the horizon?

Graph Search was hardly a surprise to many of us, yet its implementation to date has been interesting. If well-understood, it can be leveraged by marketers and brands; but, if not well-understood, it could lead to some seriously controversial (not pretty or even possibly ugly) issues.

Given that I have a background in the semantic Web field, when Facebook launched the open graph protocol with support announced for RDFa, I was very excited. I avidly followed the “ontology” (or vocabulary) they used; so, when sitting down to write this article, the first thing I did was go to one of my “ontology reader” tools, usually Protégé, and point it to where I always do, namely, Open Graph Protocol.

However, upon visiting the above link that I’d used in the past, I realized that Facebook had pulled a lot of interesting information from that resource, which now provides just attributes. (Useful in their own right.) But do note that this means that plugins like the drupal one here that references the old URL, http://opengraphprotocol.org/schema/ (not redirected) will break!

Depicted in the figure below, is a user-centric version of the Facebook Open Graph schema. To dig deeper, I used a tool called Gruff, from Franz Inc, which provides beautiful visual displays of information in graph databases. I did take several “views” of the information as there were about 804 triple sets (subject, predicate, object ) which probably form the higher level data structure from which Facebook can derive its structured/graph search.

The How: Facebook Graph Search

From the graph above, you can see very clearly what Facebook perceives/defines as what the user is interested in and also what they want to know about the user:

• TV programs they like
• Games they like
• Books they like
• Albums they have and so on and so on

Of perhaps even more interest, is that all Likes are treated the same (See figure below).

However, as I mentioned, the version of OpenGraph I obtained was prior to the January 2013 data lockdown; so, please take my statements as an educated conjecture. This is an important caveat!

If you look at the diagram above, you can see this view is portrayed with respect to the “Open Graph Object.” What this actually tells you, specifically at a higher level, is what “Open Graph Object” is interested in or cares about. Things like:

• Pages
• Users
• Posts
• Accounts
• Groups
• Message Status
• Books
• Movies
• And so on

More about Likes? You can see them graphically depicted in a lot more detail below (using Gruff once again), as predicates in the (subject, predicate, object) structure and searching on it.

The What: Facebook Graph Search

At this point, you may be asking, what on earth has this to do with me, and in what way can it help me? For one thing, Facebook search clearly searches around its structured data types (defined above) and does this based on “Likes” or “interests,” which you can also see above. It is very clear what type of structured data they are searching on and how they are using the “Likes” and “interests”.

Understanding what the search and social engines do and what makes them happy is paramount to succeeding in any type of search or social campaign. Just as Facebook wants to make their users happy using Graph Search, understanding their intention of “making their users happy” or creating a great user experience is crucial to the success of an SEO- or SEM-oriented agency’s/individual’s goals.

Anything that furthers the goal of the search or social engines will be sure to work in an “SEO campaign” or “social campaign,” respectively. Taking this approach, you can reverse engineer whatever you want to understand/achieve online.

A Word Of Caution (The Ugly)

As you can very clearly see from the above diagrams depicting Open Graph Schema, it is very focused on “Likes” and “interests.” I have selected a couple of examples below of Actual Facebook Graph Searches taken from Tumblr. 

The example above illustrates how an organization could potentially suffer from a negative image by having their employees “Like” racism. I am sure there are those who would also run counter campaigns, for example, using Mechanical Turk with a major e-tail competitor and creating tons of Facebook accounts with employees that Like the opposite of “racism” or “equal opportunity” or whatever synonym you choose. (I could get into hot water here, so will opt out of volunteering any more synonyms).

There are several other blatant examples where Graph Search is inevitably using drill-down techniques (filters or faceted-type search) in their Graphical User Interface to leverage their structured data (as does Google) to provide relevant answers to queries.

However, there is one great difference here; there are potentially disastrous social consequences. (Even if the data is clean and scrubbed, there could be all kinds of social ramifications). I have included another example of actual Facebook Graph searches from Tumblr below, and you can find even more here.

It is clearly wonderful that both the search and social engines are striving to become answer engines and are actually succeeding. Serving up a valid answer for a search engine is clearly cool and useful. Do the social engines need an added complication? A conscience? Failures or errors in determining or creating “a conscience” clearly have long-term and long-reaching ramifications and social impact.

Last, but not least, a couple of oddities I detected in the last day or so.

Sincere thanks to Kinglsey Idehen of OpenLink Software for having both the foresight and ability to maintain backups for all this data; the lost information I retrieved is directly attributed to OpenLink Software.

Just an FYI: I seriously did look for older versions of Facebook’s OpenGraph schema online. Below you can see when all evidence was removed from the Wayback Machine! (The timeline appears to go back to August 2, 2010).

Very, very last, but not least: why on earth is Facebook putting Microdata using Google’s old ontology/vocabulary prior to schema.org on their public pages for folks who Like the pages and more? Example is illustrated below.

Summary & Takeaways!

Lots to make you go “hmmm…” In short:

• Be aware of what you personally like on Facebook.
• Try to retain the perspective (point of view) of a search or social engine if trying to optimize campaigns, pages, websites or whatever for them.
• If you make Facebook happy and you want to succeed with social engines, it will work!
• Also note the differences between Social Search and Generic Search for other types of facts.
• I believe semantic search is orthogonal to social search, and conscience is required in both, but applying the identical principals to social search as to standard search, looking for the perfect tangible object or fact is very different from a social perspective than looking for the perfect ….. ?

I would love to invite discussion on the ramifications of these issues from an SEO perspective or any other perspective!

Since Google has a history of applying successful strategies to solve different problems, the use of Paid Inclusion was not unexpected when it introduced Google Shopping and Product Listing Ads (PLA).

Retailers advertising in Google Shopping should know the definition of Google Product Rank is a combination of “relevance and bid price” (similar to Ad Rank, also a combination of “relevance and bid price”). What this means is merchants must use clean data in their data feeds.

What Does Clean Data Mean?

Both Product Rank in Google Shopping and Ad Rank in Google AdWords are specified as relevance multiplied by bid price. Relevance, in both cases, is defined as quality.

In the case of Google Shopping, it means all data you give Google in the data feeds must be accurate, reliable and not conflict with any other informational sources or “signals” being sent out. The more your data is verified from multiple sources, the stronger signal it will give the search engines.

Products are listed on Google Shopping based on a paid component and a quality or relevancy component. A typical example of paid ranking results for HD TVs is depicted below. You must pay to rank, but having better data quality and richer data will, in many cases, offset the bid price in terms of ranking in Google Shopping.

What End Users See In The SERPs

Google also makes the above explicit (sort of), to the end users, should they think of clicking on the unobtrusive blue button (tiny info boxes on the “Google Everything Page,” where the ads are prominent with enhanced displays such as the one below, or on the just-as-unobtrusive blue statement, “Why these products,” in Google Shopping itself).

However, the user experience is far more engaging than before, and many of the enhanced displays are stunning.

The Importance Of Data Quality

What does this mean for SEO folks wanting to appear high in SERPs while minimizing their bid price? You must adhere to the Quality Score guidelines as well as Product Quality. It is crucial to adhere to quality standards — the richer your data and the better your Quality Scores and Product Scores, the less you pay for your ad. Data quality is key!

From a search engine perspective, this makes complete sense. Search engines do not like to be spammed, and neither do users. No-one likes being deceived in Google shopping when they select an item with one price on Google Shopping and then find the price is different or out of stock when they go to the item on the page! No-one likes dirty, noisy data — from the search engines themselves right down to the end users. 

Google Shopping Clean Data Mandate

None of this is news, but if you put yourself in a search engines shoes, you can see they clearly must provide relevant results and create a great customer experience. If they can’t do this, they will lose market share. Clean, rich, well-attributed and relevant data is a must!

Paid inclusion is now a fact of life in Google Shopping, and Google made a simultaneous mandate requiring clean, relevant, rich data inputs to product feeds, data that is verifiable by cross referencing other sources. That means the following data must all be the same and in sync:

• Data on the webpage visible to humans
• On-page semantic markup
• Data in the product feed

Google’s mandate is strongly reminiscent of Semantic Web philosophy for dealing with data quality and provenance. As is the fact that Google uses some forms of Rich Snippets to expand its Knowledge Graph.

Paid Inclusion Good For Google & Merchants

Paid Inclusion has been a very effective strategy for Google, which had a strong fourth quarter, and was good for merchants as well. An example is listed below and derived from this report from Marin Software, published January 29, 2013. There are also many other reports indicating much better ROAS (Return on Ad Spend) with PLAs.

Marin Report Summary:

“… in the last year the click share of PLAs as a percent of total search clicks increased 210% as consumers increased their engagement with the ads, which appear as image results in Google Search and as product listings within Google Shopping. Google transitioned Google Shopping to a commercial model in October 2012, and the bet appears to be paying off with Marin finding that advertisers increased their share of search budgets directed towards PLAs by nearly 600% in the last quarter of 2012.”

What Does Rich Data Imply?

The easiest way to obtain a realistic understanding of data quality is to put yourself in your customers’ shoes. Customers want easily findable items and want to be able to quickly filter out SERPs they don’t need. They aren’t able to study the complexity of Data Quality, Provenance, Trust, Object Authority and the complexities of the Semantic Web .

However, adopting a basic KISS approach from a user’s perspective makes it very understandable. Think about your customers looking for black pumps.

If they go into Google Shopping, they can enter search queries that are filtered by the attribute, color property and also the size and silhouette. If merchants do not populate those fields in their data feed to Google, no matter how “clean” the data is, the items cannot possibly display in the SERPs.

Hence, it is essential to have rich, well-attributed information. Rich Data sends Rich Signals to the engines. Clearly illustrated in the image below are the filters on the left hand side. If merchants do not populate the color and other attributes, their items cannot show up in filtered search results. 

You can markup many of these attributes with properties from schema.org on your product page. It is key, however, to put them in the product feed submitted to the major CSE’s (Comparison Shopping Engines). Richly-attributed data, therefore, increases your visibility in Search Engines.

Google Merchant Data Quality Specifications

Here, we can take a look at what Google stipulates as Data Quality. The first item in the diagram below states the submitted price on the feed must match the price on the webpage. We discussed the necessity for that already.

You also must absolutely have a valid UPC or other identifier. Another important requirement is that the Product Category must be accurate, or else the product will be suspended. There are several such examples that may or may not appear obvious to a naïve user.

Looking at the Google Merchant Data Quality issues depicted below, several things stand out. Clearly, no user wants to be frustrated with a price mismatch, or items suddenly unavailable, or going to a non-existent or off-the-subject landing page when making a purchase decision on the buy URL (destination page). You can read more about the importance of this and other details on the Google webmaster site.

Key Information

To summarize the takeaways from the information above:

• Ensure your data is clean and scrubbed, with the same signals being sent from all sources
• Ensure all necessary fields are populated, like UPC, brand, product category correct
• Ensure your landing pages are valid and product availability accurately specified
• Ensure that your data is fresh and frequently updated with new changes
• Ensure that your images are properly enhanced
• Ensure that you mark up the appropriate information for rich snippets, including product videos and product images, to send stronger signals
• Ensure markup of specific items in schema such as reviews and any other items that drive traffic and increase “trust” or authority (providing you have that information visible to human readers, too)
• Ensure that your sitemap is optimized and easy for the search engines to crawl, making use of the last mod feature in your site map

Special Note: The three universities: UCSD, UCI, UCLA, are coming together to create a Semantic Computer Consortium that leverages both academic and industrial resources. They will be at the next San Diego Semantic Web Meetup (no cost to attend) I am hosting on Thursday, February 21, at 6:00 pm at the UCSD Supercomputer Center. Anyone interested in hearing Prof Shlomo Dubnov and Prof Phillip Sheu elaborate on these plans is welcome to attend.

In this article, I am going to focus on recent developments that are search engine and/or Google specific, then take a further look back in search engine history with the assumption (for you history and strategy lovers,) that a successful strategy used once, may well be used again in similar circumstances.

Image via presentermedia.com under license

Google & The Semantic Web

In the interim, since September of 2011, Google has taken increasingly more steps in becoming more semantic-web like in nature and in the migration of its SERPs, resembling those of an answer engine.

For example, Google has added explanations to the Knowledge Graph. Subsequent to that, on November 9^th, Google adopted GoodRelations as part of schema.org. Aaron Bradley wrote an excellent article about that, and you can read more details here if you missed it.

The Knowledge Graph was rolled out globally, on December 4, 2011, and various interesting changes were made to flight search and associated activities. All very interesting; however, not exactly earth shattering, but well worth noting.

Google Knowledge Graph

Google Data Highlighter

On December 12, 2012, Google rolled out a new tool, called the Google Data Highlighter for event data. Upon a cursory read, it seems to be a tagging tool, where a human trains the Data Highlighter using a few pages on their website, until Google can pick up enough of a pattern to do the remainder of the site itself.

Better yet, you can see all of these results in the structured data dashboard. It appears as if event data is marked up and is compatible with schema.org. However, there is a caveat here that some folks may not notice.

No actual markup is placed on the page, meaning that none of the semantic markup using this Data Highlighter tool is consumable by Bing, Yahoo or any other crawler on the Web; only Google can use it!

Google is essentially hi-jacking semantic markup so only Google can take advantage of it. Google has the global touch and the ability to execute well-thought-out and brilliantly strategic plans.

Image via presentermedia.com under license

You may think, how odd, why would Google go to all that effort to create standards for schema.org for all three search engines, (actually four at this point: Bing, Yahoo, Yandex and Google), and then create a tool useful only for Google? It appears such a charitable gesture on their part.

A Little Google History On Sitemaps & Schema.Org

Perhaps some history can help us understand this better. Schema.org was first announced on June 2, 2011 as the first time the three major search engines came together to produce a standard since sitemaps.org. Depicted below are the home pages of both schema.org and sitemaps.org. Note the striking similarity.

Sitemap.org & Schema.org Homepages

Even the terms and conditions appear similar, as can be seen in the figure below. Ok, so now you may be thinking, so what, they used the same template. It is much more interesting than that. As you know, Google is brilliant at applying successful strategies, and if it succeeds once, it is very likely to pursue that same tactic again. (All you historians out there know how history tends to repeat itself). Perhaps a look at where Google is going with the structured markup can be determined by what they did with sitemaps.org.

Robots.Txt File

On a similar note, www.example.com/robots.txt is a standard location for a sitemap to reside; however, Google provides the option to submit the sitemap directly. (Most search engines do, actually). Given that the search engines allow direct sitemap submission (and may even prefer it), it’s interesting in that there is no longer an actual need for a robots.txt file. Many sites do not have them.

In some sense, however, this is strongly reminiscent of the action being taken by Google with regard to the Data Highlighter for events. Net effect: it makes this information available to Google, but not necessarily to others.

Although up and down in certain situations, there seems to be a lot of controversy around this fact, you can see a recent example noted in Matt McGee’s article. (For the rest, I will leave it to historians to make the projections.)

Historically, it also seems, for both search engines and websites/webmasters alike, best practice is to adhere to standards, as that is what best practices ultimately boil down to and why standards are so important! Ultimately, they should be enforceable (one would assume).

How Small Merchants Can Remain Competitive In Google Shopping

Small Merchants can remain competitive in Google Shopping by adding rich snippets and more. Below are some mechanisms worth looking into, as it appears Google is facilitating the small merchant in Google Shopping to retain inventories that include more eclectic and interesting product offerings!

If you understand the big picture, it is easier to predict not only what to do, but how and why with regard to getting good SERP visibility. For example, I am embedding a portion of Google’s webmaster advice on rich snippets and products. You can find the link here. I checked the last update, which was December 3, 2012.

Considering how fast things are changing with schema.org microdata, there is a need for automation to remain both competitive and compliant.

On Google’s part, this would offer an ideal mechanism for small business owners with unique, boutique-style products or eclectic products to enter Google Shopping. This would enrich and augment the current inventory on Google Shopping. On the part of the small merchant, it would now be necessary to strongly consider this option.

On a similar note, I also found this link to be of interest on Google, posted the same day! It is a great guide on best practices. Note the sitemap and robots.txt stipulations as well as those on rich snippets. It also was last updated December 3, 2012.

It may be, that much of this falls on the heels of Microsoft’s Scroogled campaign, where in addition to much else, they comment on PageRank being replaced by Pay to Rank as an algorithm. It may have elements of truth, but PageRank is now reduced to only 1 of a couple of hundred signals Google uses to rank pages.

Essential & Professional Practices For 2013

Based on the information above, it seems to me that retailers can profit by following my suggestions below.

• Stick to industry standards and standards recommended by the search engines.
• Always provide unique and interesting content on your pages.
• Keep your information fresh and relevant.
• Provide clean sitemaps and make use of the <lastmod> feature.
• Mark up your pages for rich snippets, if you are not sure how to do that, engage experts to help you or look for automated software that can provide the same!

Lastly, in terms of 2013 predictions, count on the Data Highlighter being expanded to more than just events!

Included below is an illustrative guide highlighting currently available Chrome extensions, which you can leverage to both test on-site markup as well as expose any information regarding your competitors.

An example is illustrated below, and what follows is a guide to getting the information.

Installing The Chrome Extensions For Microdata & RDFa

The result set above, is merely to pique your interest, as an initial look at the current Chrome extensions/apps appear somewhat boring. Let’s take these apparently trivial apps for a ride and see what they can do! An additional FYI – they are all free.

To start off, I just checked out what I have installed in Chrome, and as a caveat here, I am only going to write about tools I have experience with and trust. As a first step, go to the Chrome store and type in the word “microdata.”

You can see my results below; I installed all four microdata extensions. This assumes you have Chrome installed as a browser.

I did the same for RDFa, which is illustrated below. Now, let’s take these tools out for a run to how you can leverage them.

If you have not previously installed the tools, simply select the blue “Add to Chrome” and you are ready to go. It is that easy.

Using The Microdata Tools For Verification

There are many ways you can leverage these tools that will enable you to understand the power of semantic markup, and ultimately semantic search; but, let’s take a sample case in terms of verifying your own markup.

Since you now have the tools installed on Chrome, just go to the page you are interested in. Merely for demonstration purposes, I selected an item page for a red dress from Macy’s.

Now, notice the little guys you have installed popping up in the top right corner. They are activated whenever the type of semantic markup they are defined for appears in the page. I enlarged them and added them as a display on the left hand side of the image for clarity.

Assuming you were working on such a webpage, you can then select any of the icons above and get the appropriate result set. You can verify right on the page that your microdata or Facebook Open Graph information is implemented as you would expect it to be by the results shown below.

Nothing too exciting so far, but at least you can test your markup on the page you are working on in an efficient manner.

Use The Tools To See How Competitors Are Using Semantic Markup

Assume now that you would like to see what someone else is doing in order to better enhance your own pages. Perhaps you are interested in television sets and want to sell to sell TVs in your store, such as the JVC 42″ Class LCD 1080p 60Hz HDTV.

The next step is to get the details of what you need, or examine someone else’s implementation.

Go to a webpage for that television set, and check the semantic markup on their page. The results are illustrated below.

You can clearly see the wealth and richness of the information displayed, which could possibly assist you in marking up your own pages. Or, merely use for casual research when doing shopping of your own. I only illustrated results for microdata and Facebook Open Graph information, but feel free to try the others on your own.

GoodRelations & RDFa

There is only one aspect we have not looked at in detail, and that is the ability to obtain information with pages marked up with GoodRelations vocabulary using RDFa.

There are several interesting aspects to explore here. On the page selected below, when looking at an item page, only the RDF icons are now active (none for microdata). For the benefit of readability, I enlarged the active icons on the left hand side of the page.

Also activated, and enlarged on the diagram is a GoodRelations extension called the “GoodRelations Amazon Checker.” This extension allows you to check whether a product or offering, as defined by the GoodRelations ontology, is available on Amazon. And yes, it works.

Selecting those blue links within the orange box will take you to the same product on Amazon itself. A novel use of semantic Web techniques, and I’m sure it is not going to be the last innovative mechanism leveraging semantic Web type technology.

The result of selecting the link is displayed on the right hand side of the figure below: an Amazon page for the product, or an Amazon page with similar results.

Last, but not least, I wanted to illustrate the resulting graph when selecting the green turtle. I could go into that in great detail, along with many interesting use cases, but that is a topic for another article.

Rich Snippets Send Rich Signals

In concluding, I will simply say there are many uses of these applications/extensions that leverage semantic markup by consuming it right on the webpage itself. I listed three or four use cases above (from debugging schema.org implementation or Facebook Open Graph implementation and obtaining competitive intelligence information to a novel usage of leveraging GoodRelations and Amazon referrals).

Since these tools are free, they are extremely useful if you want to either implement or leverage semantic markup or even understand its capabilities in a deeper fashion.

It is also clear that some of these verticals derive far higher revenues for Google than other verticals. Google Shopping, with its new pay-to-play model, looks like it is going to do just that.

According to a recent study, Google’s new PLA’s (Paid Listing Ads) have been found to significantly outperform text ads in click-through rates by 47% and conversion rates by 38%, resulting in a 25% ROAS (return-on-ad-spend). Take into account that the deadline for merchants to comply, October 17, just kicked in, and that we are only just hitting Q4, and this becomes even more significant.

Advantages Of Vertical Search

From a search engine perspective, one of the advantages of vertical search is the ability to far more easily determine user intent. When a user clicks on “Shopping” in Google, there is a limited number of actions they intend to take.

In Google shopping, that intent would either be to purchase a product or research the purchase of a product. Add something like location to the mix, and the context is even further refined. And oh yes, all your past search history, information from your Google plus account and anything else Google knows about you can be added to further refine the search and determine user intent.

This further refines the context, enabling and facilitating a search engine’s capability to provide the searcher with the information they are seeking. The other clear benefit from a search engine point of view is how it narrows down the search space for them,  as they can  ”activate” appropriate contexts within. This can indirectly improve performance as well as directly improve accuracy of results served.

Overall effect: this can provide the searcher as much as possible with what they are seeking,  facilitating the knowledge graph or “knowledge graph” type displays seen above.

The large online retailers, such as Wal-Mart or Amazon, have the funding and capability to provide internal semantic search.

From a search-engine perspective, another thing to note is that within a vertical search engine, the internal taxonomy with reference to relevant search may differ from generic search, e.g., chlorine tablets may pull up pool equipment; whereas, in a standard taxonomy or vocabulary, this would not be the case.

A typical example would be Walmart’s Polaris ,which incidentally caused a 10 -15 % lift in sales,  and began with the formation of @walmartlabs and the initial acquisition of Kosmix amongst other later acquisitions. Astronomically expensive and beyond the reach of smaller e-tailers with relevant products, wishing to contend and have their products findable in the major Comparison Shopping Engines such as Google Shopping itself.

The Future Of Vertical Search & E-Commerce

Getting back to vertical search and e-commerce, let’s take a look at some current reality. After Google’s earnings disaster last Thursday, one of the questions Larry Page was asked about was the future of vertical search.

In his response, he referred to the large investment Google made in the Knowledge Graph and the critical need for the search engines to get the correct information to the user at the appropriate moment in time. He also cited airline flights (presumably referencing their recent flight search capability) as an example  of a  recent vertical other than Google shopping (YouTube and others are clearly significant verticals, but Google’s flight search is more recent).

Given that Google’s Knowledge Graph, in essence, is a play at adopting semantic search and semantic-Web-like technology, Google has thus far put this to brilliant use with respect to anticipating the answer and leveraging the results − along with its extremely visually-pleasing Knowledge Graph displays.

A successful guess at the answer within that display provides the searcher with no further need to click through to another site. Google is even making a stab at providing explanations via associated relationships, doing a rather good job of it too, for a nascent technology, and perhaps better than the innovators of that technology, themselves!

However, Google is not the only player in the vertical search game. There are many other contenders, and it is possibly still a hot topic in the race for the end game (or even a piece of it).

Vertical Search Works is the name of a company based in New York, London, Vienna and Carlsbad that offers users an alternative to Google paid search ads. They also offer several niche vertical search engines of their own, such as wedding, food, woodwork and more.

An image from that site is selected below (in terms of its vertical searches) and does not include its semantically targeted advertising capability.

There are also many other vertical search, or semantic search engines, that have phenomenal underlying technology, and have either been overlooked for some reason, and which could, with a little capital and innovation, be players in the future.

Perhaps not included in this list, are other new and innovative startups in this arena. Feel free to add those not mentioned in the comments below. I believe the list is extensive. To name but a few off the top of my head, I would cite:

• Hakia.com
• Evri.com
• Blekko.com
• Dogpile.com
• Sindice.com – the Semantic Web index

I also received this, in a Google Alert from a pearltree user: Some great semantic search engines!

In addition, to state the evident, there are the other major CSE’s (comparison shopping engines) such as Amazon, E-bay, etc., or some great online retailers such as Overstock.com, which are pure e-commerce. As previously mentioned, feel free to add to the list in the comments below.

Key Takeaways & Action Items

In summary, online merchants should focus on the following during the holiday season:

• It is always worth submitting your feed to Google Shopping and any other major CSEs or niche search engines.
• Make sure you have excellent data quality; if not, you may be suspended.
• Make sure your site maps are readable and of good quality.
• Use semantic markup on your pages where possible, and ensure on-page content, on-page markup and the data you submit in your feed all match.
• Make sure you frequently update your product feed (at least once a day or more).
• Keep your eyes open for novel vertical search engines and recommendation systems (typically semantic in nature). They may just get you that unexpected lift.
• Last, but not least, here’s an unexpected takeaway from e-Bay: keep your images crystal clear, with an unfussy background!

We will look at an alternative vocabulary, namely GoodRelations, and take a deeper dive into the specifics of generating semantic markup for the e-commerce domain using GoodRelations.

To clarify, GoodRelations is a vocabulary for e-commerce. Microdata and RDFa are syntaxes. Schema.org for product is an alternate vocabulary for e-commerce.

The diagram below from Linked Open Commerce gives an excellent graphical depiction of how all entities are interlinked and related with the Web of commerce. It also gives an indication of what other novel applications can arise that can leverage proliferation of structured markup that adheres to standards. In this case, showing specifically those that pertain to e-commerce.

First off, we can take a look at the GoodRelations toolset. There are many. Even converters from RDFa to microdata. The easiest way may be to use a plugin if you use a tool like WordPress, but we will do a manual walk-through of the entire process, as it is more illustrative of precisely what is occurring.

What can it do for you, anyway? How does it make your website or business findable? An example is listed below.

How do you generate markup for this using GoodRelations? For our first step, take a look at the figure below. As you can see, there are many tools available, even crawlers and code that will consume GoodRelations on page markup. 

For the scope of this walk-through, we are going to select the GoodRelations Rich Snippet Generator. Part 1, depicted below, will generate a snippet for your organization. I went ahead and filled in the information for a store.

After entering the information above (it is not a real store), I got the result displayed below. You can see all of the generated RDFa once you fill in all the details for your store and select Generate Code Snippet.

It also tests the validity of your inputs, e.g., what opening times and closing times are valid, and will generate a specific error with suggestions for correct input formats. Pretty cool, right?

When it is happy with your input you get a green checkmark along with the validated semantic markup! (The markup is illustrated in the grayish box below”. It begins and ends with a <div>.

I now have the markup needed to add to my page for organization information. In our next step, we are going to get all the structured markup for the store, including opening hours!

Make sure to select “Copy address information from company” and then also select “Determine geo position from address,” in that order.

A lot of code gleaned from above will be populated, and you do not have to worry about deriving your longitude and latitude. It will be filled in for you!

After filling in the rest, we are now ready to select the Generate Code Snippet. Upon doing so, the necessary markup is generated as shown in the figure below.

Assuming you keep following the steps shown on the Rich Snippet Generator page, you can add product information and offers as well.

You now have the correct format to place on all your product pages, as well as the correct formats for local and business. You can continue to follow the next steps (two more) on the GoodRelations Rich Snippet Generator page until you have all the markup you need to edit your webpage.

Let us look at a store that is an actual user of GoodRelations. An example would be Peek and Cloppenberg. Below is a snapshot of one of its product pages:

I took the URL for the product, and then ran it through the Sindice Web Data Inspector tool. It extracted all the RDF or “triples” as you can see!

The graph view can make it a lot easier to see the inheritance hierachy. You can even use it for debugging to ensure your divs are correctly nested.

As you can see, if you want your site to be “Findable” in the Web of commerce, it is wise to add semantic markup.

GoodRelations is great for e-commerce. Schema.org is used by all three major engines and is less complex in some ways, but less rich in others. You may have to walk through it once. But there are many tools provided to get you on your way.

Just a caveat here. Some training may be required. But if you want to be findable in the Web of commerce, make sure you add semantic markup to your site and follow other best practice guidelines (like clean, verified and validated data feeds).

And one more thing! When making your decision, “rdfa vs microdata” the below chart is definitely something to keep in mind! For more background, also read this GoodRelations Primer.

Make sure your markup matches your data feeds and that you also do not “stuff” your pages with RDFa information or microdata that is visible to bots only and not to humans. (The latter is considered cloaking). But you do want to be findable in the Web of commerce. So, delve into it. But make sure you tread smartly.

A few final stats on the usage of markup in the current online landscape:

Below is more data on the above chart:

In closing, using GoodRelations and related markup is something all retailers should investigate using to ensure their online store is findable in the global graph of commerce!

This special series of forthcoming articles on semantic search will take a look at the history behind the development of semantic technology and why it has now become so commercially viable and topical. It will also take a look at how the technology enables “answer engines,” rather than simple search engines, to improve the user experience.

For example, look at the direct answer to a query for [Barack obama birthday] in Google.

According to announcements on Google’s Inside Search blog, this is only the beginning of building an “artificial intelligence” engine, or its “Star Trek” computer.”

Of note was the comment posted at the end of Google’s blog announcing the Knowledge Graph by Amit SInghal:

“We’re proud of our first baby step—the Knowledge Graph—which will enable us to make search more intelligent, moving us closer to the “Star Trek computer” that I’ve always dreamt of building.”

Semantic Search

Many Artificial intelligence, NLP (Natural Language Processing), or machine learning technologies may be cited as “semantic technologies.” Semantic means “meaning,” so semantic technologies in general do not merely include semantic search.

However, many of them can be leveraged to improve search, such as semantically targeted advertising, automated topic recognition and more. Many semantic technologies map to underlying ontologies, which are thought of simply as vocabularies or lexicons.

For the purpose of “Semantic SEO,” we will refer primarily to the concepts associated with the Semantic Web and the adoption thereof by Google and the other major search and social engines. I’m referring to the ontologies or vocabularies being used − which encompasses the semantics or concepts − and the syntax defined in metadata with on page structured markup.

Semantic Search, as it is used in current parlance is essentially the notion of using or exploiting metadata to improve search on documents. In the case of search engines, it more explicitly refers to embedding metadata in HTML5 (using semantic markup, the formats or HTML5 syntax currently supported by the search engines: RDFa Lite and microdata).

How Is Metadata Exploited By Search Engines?

One example is that of enhanced displays in the SERPs − Google’s Rich Snippets, Bing Tiles or Yahoo SearchMonkey. Enhanced displays also provide more visually engaging displays and interfaces with a corresponding increase in CTR.

Another aspect of exploiting this information for search engines is to search directly on this consumed metadata – examples would include Sindice.com or Google with the Knowledge Graph and the Knowledge Carousel.

This is a large part of the evolution of search engines from producing a series of probabilistic results or “blue links” to becoming “answer engines.” Users definitely find it tiresome running multiple queries to obtain (or not) an answer to a query. Relevancy in answer to a query is everything, and there are multiple ways semantic technologies can be leveraged to ultimately attain that goal.

Machine learning techniques can also be used to ensure/improve topic validation. A timeline of Semantic Web adoption is shown below:

• Yahoo! opens Search Monkey:  February 2008
• Bing acquires Powerset: July 2008
• Google introduces reviews and aggregate reviews using rich snippets: May 2009
• Google introduces specifying an image’s license using RDFa:  August 2009
• Google introduces RDFa support for videos:  September 2009
• Google encourages webmasters to “help us make the web better” by using rich snippets: October 2009
• Google announces use of structured data to describe an organization: March 2010
• Google announces rich snippets for recipes: April 2010
• Google announces rich snippets go international: April 2010
• Facebook announces open graph protocol based on RDFa: April 2010
• Google acquires MetaWeb: July 2010
• Google Refine is announced: November 2010
• Google announces rich snippets for shopping sites: November 2010
• Google, Yahoo, and Bing announce Schema.org: June 2011

Semantic Technology Helps Provide Relevant Answers

It is at this point that the three major search engines got together and announced support for schema.org, not only diverging from previous standards of markup (namely RDFa) and initially announcing support for microdata only (subsequently changed due to uproar in the Semantic Web community), as well as the logical acceptance on the part of the search engines to consume information that converts to high standards, data quality and standards body definitions.

It has long been the intent of any search engine to be able to provide not just a series of links, but actual relevant answers. These possible answers can be derived by leveraging the above mentioned mechanisms.

Determining user intent is yet another means of exploiting semantic technology. It can be done by:

• Correctly interpreting a portion of the query, or the query in its entirety
• Providing a “best guess” at an answer by reasoning directly with previously validated information from highly trusted sources
• Aggregating and growing this knowledge base or Web of data or graph database by adding consumed information and/or reasoning about it.

Producing or publishing this information in the form of embedded metadata in, say, HTML5 can be accomplished via adding microdata or RDFa Lite as defined in the Google blogs and other blogs. However, these are merely syntaxes that can be consumed  or understood by the search engines and are HTML5 compatible.

The other issue is that of vocabularies (or ontologies or taxonomies). Since standards are always an advantage in many arenas, the three search engines − Google, Bing and Yahoo − agreed to mandate a standard vocabulary or ontology, that of schema.org, announced June 2, 2011. Search engines have such a large user base that they actually have the power to mandate the ontologies or vocabularies to be used.

The Semantic Web community has many other defined ontologies/vocabularies and provides them as open source (GoodRelations for e-commerce, FOAF SIOC, Wordnet, DBpedia – derived from Wikipedia and more).

Schema.rdfs.org has a great set of resources for those of you wanting to get started as there are tutorials, software and tools to generate structured markup automatically, and more.

Let’s continue the timeline of Semantic Web adoption since the schema.org announcement.

More music formats: 08/2011

Microdata and sports stats from NFL:  8/22/11

Upcoming concerts:  2012/02

Search Engines Are Becoming Answer Engines

This timeline culminates at the time of this writing with introduction of the Knowledge Graph and the Knowledge Carousel in May 2012.

The Knowledge Graph is a direct extension of Freebase and is extended by other consumed information from “structured markup” as defined in schema.org or as deemed relevant by Google.

The Knowledge Graph itself is depicted on the right hand side of the query and is a further example of Google moving its queries to those of an “answer engine” from fact based or aggregated information.

In June 2012, Twitter announced its “Twitter Cards,” a way to “attach media experiences to Tweets that link to your content.” You can read more about this in a post on semanticweb.com.

In July 2012, Google Webmaster Central introduced the Structured Data Dashboard for Webmasters, allowing them to see consumed structured data. Further descriptive comments indicating Google’s commitment to this course of action could be interpreted along with the post.

“Structured data is becoming an increasingly important part of the web ecosystem. Google makes use of structured data in a number of ways including rich snippets which allow websites to highlight specific types of content in search results. Websites participate by marking up their content using industry-standard formats and schemas.”

Shown below is a site level view of about 2 million annotations for schema.org books:

Note the depiction of results for the band “Coldplay.” Rich snippets markup for schema.org (music, etc.) is clearly integrated into this display.

Below is a depiction of the Knowledge Graph combined with features of the Knowledge Carousel, namely the scroll bar on the top. The Knowledge Graph is extended from simply Freebase and other linked data sources via validated verified pages and trusted sources containing structured markup as per Semantic Web related techniques.

The query entered for the display below was “Tom Cruise Movies.” The Knowledge Carousel is globally available in English as of September 2012.

The example above is certainly indicative of how these enhanced displays consume SERP real estate.

CTR Increase With Semantic Markup

Another crucial aspect of Semantic SEO or schema is the increase in CTR for marked up items, and the incredible increase of screen real estate utilized by the Knowledge Graph/Carousel and Rich Snippets and other information aggregated by Google (like places and events on the RHS of Google, where the Knowledge Graph results typically display).

Example shown below:

 As an indication of where this is going, it is worth taking a look at Google insights for search and seeing the resultant graph. I typed in the term [Semantic Search] and the results can be seen below.

Semantic Technology Adoption Expands

It is interesting to see what terms emerge as the most searched. Also, the peak in the graph below illustrates the impact of WalMart’s Semantic Search Engine resulting in a 10 to 15 percent boost in business.

 Another point of relevance is the mapping of higher order terms in schema.org to the verticals in Google, Bing and other engines. When a user selects a specific vertical in the image below, it is clearly far easier to determine intent in a query.

Feel free to compare these to the full set on the official schema.org site. As an experiment, I actually loaded the schema (owl version) from the official schema.org site.

Using a tool called Protégé, and looking at the resulting hierarchical display, it actually gives a great graphical depiction of this. I focused on expanding “place,” but you can select any option you choose:

Semantic SEO Benefits

In summary, Semantic SEO and Semantic Technologies bring many strong benefits to the search engines.

• Enhancement of visual displays (Rich Snippets) in SERPs
• Search engines to search directly on that data for relevant answers to obtain more relevant results
• Classifiers and other machine learning mechanisms can be used by search engines to verify topic information on pages
• Assist in determining user intent (context improves recall/relevancy)

Future articles in this series will dive into the specific verticals in greater depth, clarifying in more detail how vertical search improves relevancy and defines user intent, taking a look at semantic technologies used in recommendation engines, semantic advertising and more!

So, how about putting yourself in a search engine’s shoes?

No one wants to be spammed or fed invalid data, so all information needs to be clean, verified and preferably from more than one source (all semantic Web or named entity extraction type philosophies).

From a search engine’s perspective, it wants (and needs) to serve up not just relevant information, but also information that is valid and verified from their end.

Search engines need to ensure that the user has a good experience on the site.

Details for products, and types of products, are essential to optimize the user experience and make your products appear in listings. After all, even if you have to “pay to play” in the game, you still have to know the rules of play, and have your items exist in the indexes and data sets that search engines maintain!

This may sound somewhat esoteric, but let me now jump to an example in Google Shopping.

Upon a search for “red pumps” in Google Shopping, I received the following display:

What you see is a very attractive display of “visually similar” items, and it helps create a great shopping experience. You may be asking yourself, “How does Google do this?”

Image recognition is still a hard problem to solve. Facial recognition was a research problem in the artificial intelligence arena in the 1990’s. Now, it is in everyday items like iphoto. Does this sound boring? Not if you can benefit from it by understanding it.

A search on Amazon provides a similar type of result, illustrated directly below.

Scrolling further down on the Amazon results page for “red pumps,” one then gets the option for “see visually similar items.”

Selecting a specific “visually similar” option leads to more of them, and a mechanism for honing in on the selection from a visual perspective.

Going back to Google shopping, I then did a search for “black dress,” which yielded the following result:

This time in the screenshot above, I included the search options on the left hand side. The very top item, BTW, is a check box under show only “in stock nearby.” This would be indicative of an edge over say, Amazon, or other e-tailers, were they going to directly compete with them. (I would actually believe that is a valid assumption.)

As a further incremental example in that line of reasoning, I had the luxury recently of obtaining a Nexus 7 tablet and playing with it. I did notice the absence of Amazon Prime for movies, etc., and could not even find it in the app store. (However, there was a Google version as part of the default OS and UI).

Back to going down our list on the left hand side, the other items listed are fields required for data feeds for apparel, ranging from color to brand and more. Ensuring accurate details are filled in and populated will make your items findable and appear in these eye-catching presentations.

I selected the first item on the top left hand side (the picture of the dress, not a search option) and received the following:

My assumption here is that Google is using results from the deprecated boutiques.com which was actually the result of the like.com acquisition by Google. It was an image recognition engine that turned to a focus on apparel such as handbags and other items with great success.

Under the “visually similar items,” I actually had a couple of pages of results. Scrolling down from the above screen capture, I have depicted the remainder of the results below:

Of note, is the line at the bottom of the page, which I will reiterate here in case the image is too small to read:

“Google is compensated by some of these merchants. Payment is one of several factors used to rank these results. Tax and shipping costs are estimates.”

Moving away from apparel and to electronics, I tried a search for a 60 inch LED TV.  This was a generic search, as I neglected to select the “Shopping” option.

The results on the right hand side yield a knowledge graph type display. Selecting the shopping option gives the typical Google shopping results, with many relevant searchable fields on the right hand side.

In this category, however, selecting an item does not give “visually similar” results as they are not in the apparel category.

However, the moral of the story is as follows: Supply as much accurate information as possible in any data feed sent to Google or other search or shopping engines. Ensure you add structured/semantic markup to your webpages and that it matches the data supplied in your feeds. Also, make sure you have good clean images and they are also marked up on your webpages.

For Google image search alone, it states they are using a combination of both computer vision techniques as well as text and semantic markup.

Google’s recent post, “On Web Semantics,” made it very clear that adding Semantic Markup is the professional thing to do. Add as much valid markup as possible, ensuring the information is displayed to both bots and users alike! It is leveraged by all the major search and social engines.