Yahoo Adds SearchScan Alerts To “Risky” Search Results

Yahoo Search has begun a partnership with McAfee, Inc. to provide SearchScan, which uses McAfee’s SiteAdvisor technology to flag URLs it deems "risky" in the search results. Results are flagged with the type of danger below the title. This new feature is primarily aimed at preventing spyware and other malicious software from being downloaded on searchers’ computers, as well as at preventing searchers from falling victim to sites that employ spammy email tactics.

The Yahoo Search Blog provides more information. Below, more details on what types of pages are flagged and how site owners can dispute incorrect flagging.

The types of behavior that causes a page to be deemed risky include:

• Download triggered upon page visit (these types of pages are removed from the search results entirely)
• User-initiated download includes spyware or other malicious sofware
• Site engages in spammy email tactics, such as flooding inboxes with mail

SearchScan is on to alert by default, but searchers can turn it off (or specify that flagged sites shouldn’t display at all) in their Yahoo preferences.

Why has Yahoo implemented this feature?
The press release being released tomorrow morning quotes a Decipher Inc Online Security & Web Search consumer survey from March of 2008 and says, "After children’s safety, 65 percent of Americans online are more worried about clicking unsecured search listings than the threat of neighborhood crime, getting ones [sic] wallet stolen or email scams." That’s an interesting claim, as I don’t know that 65% of online Americans know what an "unsecured search listing" is, but the point remains a valid one. Search engines present the web as a whole, and as the web include lots of malicious activity, search engines end up serving up malicious suggestions. This partnership is an attempt to serve up "safe" results without engaging in web censorship.

Google’s approach
Google has taken a similar tactic with a partnership with StopBadware.org. Any sites flagged by StopBadware.org include a message below the search result and Google directs searchers who click on these results to a page that provides more information and enables them to either continue to the page or go back to the search results.

If a site is flagged in Google’s search results, Google alerts the site owner via email and a Google Webmaster Tools message. Google also provides a dispute and resolution process in the cases where the site owner doesn’t agree with the label or makes changes to the site to abide by the StopBadware.org guidelines. In addition, the site owner can obtain more information from the page that Google directs searchers to for flagged pages in the search results.

The dispute and resolution process is actually forwarded to StopBadware.org and site owners can follow the process there.

Yahoo’s dispute process
Yahoo has a dispute process for site owners as well. When you hover over an alert in the Yahoo search results, an information box appears that includes a site owner link.

That links leads to the SearchScan form, which seems to be for both site owners and searchers. When I talked to Yahoo about this process a month ago, they said that, like with Google’s process, they forward the information to McAfee to resolve. I’ve asked them if they also provide proactive alerts to site owners and they said that if site owners are concerned that their pages may be missing from the search results due to SearchScan, they can turn off SearchScan and check the results:

If your site shows up in that experience, but not in the SearchScan On mode, (all other options such as SafeSearch remaining same) then you can believe that it is due to exploit rating on your site.

Note, to make this work, you should ensure that you keep all other elements of the search experience constant between the test with SearchScan off and on. That is

a) make sure you don’t change any other preferences

b) make sure you don’t change the computer you are searching from in case source IP or other changes affect the query routing

c) make sure you use the same Y! search destination – .com, co.uk etc. because sometimes there are regional rules which cause filtering (for example, france has stricter rules around nazi memorabilia sites etc.)

d) check multiple times across a couple of days. This is important because sometimes there is some localized maintenance going on which might temporarily affect what you see.

As part of the agreement, McAfee will distribute Yahoo Search to its user base. Distribution is arguably more important to gaining search market share than user interface improvements, and this distribution deal may provide clues to Yahoo’s strategy. (Two different hotels I’ve stayed at in the last month have featured a Yahoo search box on the wifi landing page — more signs that Yahoo is working hard at increasing distribution.)