As if SEOs don’t have enough things to worry about already, add malware to the list. Why does malware matter to SEOs? If the site you are working on gets infected, its search traffic will plummet. Search engines attempt to remove infected pages from their search results, or they label them with an ominous warning, such as This site may damage your computer.

Back in 2008 Google reported that malware infected pages had increased to more than 1% of all search results. Google posted a malware statistics update last week. Malware infections have more than doubled since April 2009. Search results containing a url labeled as harmful have remained level in the range of 0.5% to 0.9%, an improvement. While the web as a whole has become more dangerous, Google’s been doing an even better job clearing their search results.

I know one reason why there’s been a dramatic rise in malware on the Web since April. A nasty malware attack has been targeting web developers to steal their passwords. Stolen passwords are used by the bad guys to automatically deploy iframe injection attacks to innocent web page.

If you access web sites via File Transfer Protocol (FTP) or Secure File Transfer Protocol (SFTP), this attack is targeting you. All you need to do is browse an infected page using an insecure browser. Badware will be deployed to your machine, and it will find the files used by FileZilla, or possibly other FTP programs to store passwords, and silently send those files back to a server in China. Then an automated bot attack will use FTP to edit your web pages, infecting them with malware. Then your sites will drop out of the search results. Can you image the uncomfortable conversations when all your sites get hacked at once and you have to admit responsibility?

What can be done to reduce this risk of search Armageddon?

1. Use a more secure browser such as Chrome or Firefox with the NoScript add on for routine browsing.
2. Don’t use any FTP program that stores passwords locally in plaintext, such as FileZilla. To date, Dreamweaver has not been reported to have been compromised. Dreamweaver encrypts passwords and stores them in the Windows registry.
3. Consider using a Mac or Linux instead of Windows. As the most popular operating system, Windows is the most popular target for attacks.
4. Make sure your machine and server are fully updated and patched. Turn off unnecessary services and software to reduce the attack surface.
5. Register your site with Google Webmaster Tools and Bing Webmaster Center. Check regularly to see if there are any malware reports (or other issues) with your sites.
6. If you suspect a malware infection, check Unmask Parasites,
7. View your site’s reputation at McAfee SiteAdvisor.
8. Reduce the number of people and computers that have access to your web server.
9. Keep a backup copy of your web pages. In case of infection, it’s a race to see if you can fix the site before search engines (and users) discover the problem and dump you.
10. Choose the hosting provider that has the quickest response time, not the cheapest price. If your site gets hacked, you may need their help to change all the passwords.

As the web becomes more dangerous, customers become more suspicious, reducing opportunities for everyone. Please do your part to make the web safer, and to reduce your risks.

Ars Technica reports Google has informed the Federal Communications Commission (FCC) that they have been working on a new filtering system for videos, comments and communication on YouTube. It is believed that Google is working on these changes in order to make YouTube more kid or family friendly, by complying more with the FCC regulations for TV viewing.

Google wrote to the FCC:

Google Inc. (“Google”), by its attorneys, files these comments in response to the Notice of Inquiry (“NOI”) issued by the Federal Communications Commission (“FCC” or “Commission”) initiating a proceeding as required by the Child Safe Viewing Act of 2007 (“CSVA”) to examine the existence and availability of advanced blocking technologies compatible with various communication devices and platforms for programming parents deem indecent, violent or otherwise objectionable.2 As we explain, Google is committed to empowering and educating parents so that they can create a positive and safe online experience for their children.

A number of initiatives designed to give users and families greater control to moderate their YouTube experience, including the ability to filter video comments they find inappropriate.

Previously, YouTube had to create sophisticated software to protect copyright over music and videos. That software has been doing a pretty good job identifying videos or music on YouTube and quickly removing such video.

In other YouTube news, YouTube has been testing a super seekrit channel design.

Ask.com has partnered with Symantec to include a new “Safe Search” feature into the Norton 360 security suite, integrated into the Norton toolbar that is part of the suite. When a search is run from the toolbar, results are safety-rated by Symantec using a color-coded icon, to flag sites or sections of sites that may be malicious or harmful. Mousing over a rating will open a popup window offering more details about the Safe Search rating.

The toolbar works on both Internet Explorer and Firefox, and is set by default to not display “red” or potentially malicious web site in search results. There are options allowing you to tune the settings to meet your own comfort level when viewing search results.

The announcement is another part of Ask’s strategic focus of working with partners, in an effort to increase its market share among its target audience of families and kids. Danny wrote about this new focus a few weeks ago in Ask.com Partners With NASCAR, Says “Super Verticals” Will Put It Back In Search Race. The idea is that by leveraging the strength of its partnerships, Ask will be able to successfully compete with Google et al without necessarily doing battle directly with the major search engines.

Ask can use the help in acquiring new traffic. According to comScore, Ask’s share of the search market in the fourth quarter of 2008 declined to 4.0%, down from 4.4% in the third quarter of the year. Andrew Moers, president of the Ask partner network, declined to discuss specific traffic share the company hopes to gain, but said “It’s a big win for the Ask partner network. Symantec ships tens of millions of pieces of software per year, so it’s a very meaningful relationship.”

Google led the way in displaying malware warnings in search results when it began issuing warnings about potentially dangerous sites in February 2007. No toolbar is required to view these warnings—they show up automatically on a Google search result page when malicious sites are detected. Over the past weekend, Google’s system went haywire, briefly labeling all search results as malware.

Yahoo has had a partnership with McAfee since May of last year with SearchScan, which uses McAfee’s SiteAdvisor technology to flag URLs it deems “risky” in the search results. And Microsoft added malware warnings to Live Search results in December 2008.

The new Safe Search toolbar is part of the Norton 360 security suite, version 3.0, and will be available by download. The final version of the security suite will be released by the end of the quarter, according to Moers.

Recently, Microsoft Live Search added malware warnings to their search results. If a searcher clicks a result that Microsoft has detected contains malware, a popup warns then not to proceed to the site. As Matt noted yesterday, Google and Yahoo! also provide malware warnings to searchers.

How can you find out if Microsoft has flagged your site for malware and how can you let them know you’ve fixed the problem? As part of this update, Microsoft Live Search also launched an update to their Webmaster Center that added alerts about malware. You can generate a report of all pages on your site that have malware on them, see if you link to any external pages that contain malware, and submit a review request once you’ve fixed any issues.

Find out if your site has been flagged

To find out if your site has been flagged for malware, log in to the Webmaster Center (you have to verify site ownership) and access the Crawl Issues tab. Select the Malware Infected issue type and click Search. You’ll see a list of any flagged pages on your site that you can download for offline processing.

The Malware Infected report has been added to the existing crawl errors reports in the tool (File Not Found, Block by REP, Long Dynamic URLs, and Unsupported Content Types).

Request review

If you find that any pages are infected, you can request review once you fix the problem. Microsoft says the reinclusion  process should “take days, not weeks”.  Once they’ve determined that the pages are no longer infected, they’ll remove alert from the search results.

Find out if you’re linking to malware

The webmaster tools also include a new Outbound Links report that lists any external malware-infected pages that your site links to. This is important information to have, as you don’t want to send your customers to infected sites. To generate this report, simply access the Outbound Links tab, choose Show All Outbound Links, and select the Show only outbound links to malware option.

Learn more about malware and how to prevent it

Microsoft has added information to their help center that provides details on what malware is, how to protect your site from it, and steps to take if your site has been infected. This document notes the importance of fixing any security holes in your website that enabled the malware infection.

What about Yahoo! and Google?

Both Google and Yahoo also offer methods for requesting evaluation once malware issues have been fixed, but neither provide the robust reporting that Microsoft now offers.

Google alerts you to malware issues on the summary page of the site in webmaster tools and provides a sample list of URLs. Microsoft’s report may contain a more complete list of infected pages. However, once you fix the problem and request review, Google will let you know of any remaining pages with issues. Google has also published several blogs about what to do if your site’s been hacked and a how to prevent it.

Yahoo! doesn’t provide this type of alert or reporting in Site Explorer. When I asked about this earlier in the year when they launched “SearchScan Alerts”, they told me:

If your site shows up in that experience, but not in the SearchScan On mode, (all other options such as SafeSearch remaining same) then you can believe that it is due to exploit rating on your site.

Note, to make this work, you should ensure that you keep all other elements of the search experience constant between the test with SearchScan off and on. That is

a) make sure you don’t change any other preferences

b) make sure you don’t change the computer you are searching from in case source IP or other changes affect the query routing

c) make sure you use the same Y! search destination – .com, co.uk etc. because sometimes there are regional rules which cause filtering (for example, france has stricter rules around nazi memorabilia sites etc.)

d) check multiple times across a couple of days. This is important because sometimes there is some localized maintenance going on which might temporarily affect what you see.

Use Yahoo’s SearchScan form (linked to in the search results alert) to request review once you’ve fixed any issues.

Ultimately, search engines want to provide the best possible experience for searchers and don’t want to send searchers to sites that will infect their computers. By providing site owners with detailed information about pages on their sites that are infected, external links to malware, and how to prevent infection, Microsoft is going a step further beyond just alerting searchers to infected sites by helping keep the results free of malware in the first place.

MSN’s Live Search has announced the addition of malware warnings to their search results pages. In doing so, Live Search joins Google and Yahoo in taking a proactive stance against potentially dangerous sites; Google began adding malware warnings in early 2007. Yahoo added SearchScan alerts in May of this year.

The Live Search implementation is different from how the other two search engines show malware warnings. When a potentially harmful page shows up in the Live Search results, users see no warning until they actually try to click on the link. When they do click, a small “pop-up” box appears to the far right of the listing. Here’s what it says:

On the other hand, both Google and Yahoo show the warning immediately, without the need for a user click:

Google Malware Warning:

Yahoo SearchScan Warning:

It’s good that Live Search has joined Yahoo and Google in warning users about possibly harmful sites. But to be frank, the Live Search implementation of this warning suffers from poor usability. By placing the alert to the far right, it’s not where users are looking as they click. Because of that, and because the warning is in small text on a white background, searchers may miss the message altogether.

Webmasters whose sites are flagged by Live Search as being dangerous are encouraged to sign up for Live Search Webmaster Tools to get more information about fixing and securing their web site.

Google Is A Malware Site (Says Yahoo) from TechCrunch reports a funny bug at Yahoo that accidentally made it look like Google’s home page had Malware on it, based on the Yahoo SearchScan feature that was recently launched.

A search for Astalavista at Yahoo returned what appeared to be a Google.com result in the 10th position. But in reality, Yahoo somehow mixed up the astalavista.ms URL with a Google.com URL, and since astalavista.ms had signs of Malware on it, it appeared that Google.com had malware on it.

Here is a screen capture of what the search results originally looked like, when Yahoo labeled the astalavista.ms result and a google.com result:

But now, the same result is labeled correctly and looks like this:

Amit Bhawani commented at TechCrunch, explaining that if you clicked through to the detailed warning, it showed the warning was for astalavista.ms and not for google.com.

It now appears that Yahoo fixed the issue with the search result showing google.com instead of astalavista.ms. Yahoo has been known to mix up the display URL in the search results in some cases, specifically in the past. But I have been hearing some recent noise in the search forums that there are more signs of these Yahoo issues.

Yahoo Search has begun a partnership with McAfee, Inc. to provide SearchScan, which uses McAfee’s SiteAdvisor technology to flag URLs it deems "risky" in the search results. Results are flagged with the type of danger below the title. This new feature is primarily aimed at preventing spyware and other malicious software from being downloaded on searchers’ computers, as well as at preventing searchers from falling victim to sites that employ spammy email tactics.

The Yahoo Search Blog provides more information. Below, more details on what types of pages are flagged and how site owners can dispute incorrect flagging.

The types of behavior that causes a page to be deemed risky include:

• Download triggered upon page visit (these types of pages are removed from the search results entirely)
• User-initiated download includes spyware or other malicious sofware
• Site engages in spammy email tactics, such as flooding inboxes with mail

SearchScan is on to alert by default, but searchers can turn it off (or specify that flagged sites shouldn’t display at all) in their Yahoo preferences.

Why has Yahoo implemented this feature?
The press release being released tomorrow morning quotes a Decipher Inc Online Security & Web Search consumer survey from March of 2008 and says, "After children’s safety, 65 percent of Americans online are more worried about clicking unsecured search listings than the threat of neighborhood crime, getting ones [sic] wallet stolen or email scams." That’s an interesting claim, as I don’t know that 65% of online Americans know what an "unsecured search listing" is, but the point remains a valid one. Search engines present the web as a whole, and as the web include lots of malicious activity, search engines end up serving up malicious suggestions. This partnership is an attempt to serve up "safe" results without engaging in web censorship.

Google’s approach
Google has taken a similar tactic with a partnership with StopBadware.org. Any sites flagged by StopBadware.org include a message below the search result and Google directs searchers who click on these results to a page that provides more information and enables them to either continue to the page or go back to the search results.

If a site is flagged in Google’s search results, Google alerts the site owner via email and a Google Webmaster Tools message. Google also provides a dispute and resolution process in the cases where the site owner doesn’t agree with the label or makes changes to the site to abide by the StopBadware.org guidelines. In addition, the site owner can obtain more information from the page that Google directs searchers to for flagged pages in the search results.

The dispute and resolution process is actually forwarded to StopBadware.org and site owners can follow the process there.

Yahoo’s dispute process
Yahoo has a dispute process for site owners as well. When you hover over an alert in the Yahoo search results, an information box appears that includes a site owner link.

That links leads to the SearchScan form, which seems to be for both site owners and searchers. When I talked to Yahoo about this process a month ago, they said that, like with Google’s process, they forward the information to McAfee to resolve. I’ve asked them if they also provide proactive alerts to site owners and they said that if site owners are concerned that their pages may be missing from the search results due to SearchScan, they can turn off SearchScan and check the results:

If your site shows up in that experience, but not in the SearchScan On mode, (all other options such as SafeSearch remaining same) then you can believe that it is due to exploit rating on your site.

Note, to make this work, you should ensure that you keep all other elements of the search experience constant between the test with SearchScan off and on. That is

a) make sure you don’t change any other preferences

b) make sure you don’t change the computer you are searching from in case source IP or other changes affect the query routing

c) make sure you use the same Y! search destination – .com, co.uk etc. because sometimes there are regional rules which cause filtering (for example, france has stricter rules around nazi memorabilia sites etc.)

d) check multiple times across a couple of days. This is important because sometimes there is some localized maintenance going on which might temporarily affect what you see.

As part of the agreement, McAfee will distribute Yahoo Search to its user base. Distribution is arguably more important to gaining search market share than user interface improvements, and this distribution deal may provide clues to Yahoo’s strategy. (Two different hotels I’ve stayed at in the last month have featured a Yahoo search box on the wifi landing page — more signs that Yahoo is working hard at increasing distribution.)

In the aftermath of the major malware attack that hit search results, the Google Online Security Blog announced the launch of a new form to report malware software showing up in Google results.

Google Cleans Up Returns; Yahoo Not So Much from eWeek has a look at how Google has quickly cleaned up the malware from their search results, while Yahoo has taken a slower approach.

Subverted search sites lead to massive malware attack in progress from ComputerWorld reports that Google, along with Yahoo, Live Search have been targeted in a massive attack that puts links leading to malware sites into top search results.

Alex Eckelberry, Sunbelt Software’s CEO said “So far we’ve found 27 different domains, each with up to 1,499 [malicious] pages. That’s 40,000 possible pages.” The pages are specifically being returned by search engines in their organic results (not the paid results). Eckelberry explains, “They get themselves on to Google, then redirect people to their malware pages.”

The SunBelt Blog has screen captures of this issue happening in Google’s organic results.

But doesn’t Google have a malware filter tool? See our past articles:

• Google Steps Up Web Page Malware Notifications
• Google Search Results Now May Display Malware Warnings
• Google Webmaster Tools Adds Malware Review Form
• Matt Cutts On Appealing Google Malware Warnings

Google has now removed the result in SunBelt’s example. But those results looked very similar to the Chinese Look-Alike Google Spam we reported from early September. But this can easily be a different issue.

For more coverage of this topic, check Techmeme.

NOTE: If you saw a malware warning on Jan. 31, 2009, this was due to an error that briefly impacted all web sites. See Google Gets Fearful, Flags Entire Internet As Malware Briefly, for more.

Google Korea to censor search results from InfoWorld reports that Google Korea — like other search engines in the country — will require people to prove their age in order to get back unfiltered results for certain searches.

Those searching for any of about 700 terms judged to be adult in nature by the Korean government will be required to enter their name and national resident registration number. If they are under 19, results will be filtered.

Google Korea hopes to go live with this new feature by September 1, 2007.

Postscript: We sent the following questions to Google and received these responses:

1) What prompted this?

Google.co.kr complies with local Korean laws and regulations on juvenile protection from adult content. The government requires all search engines/portals to implement a process to protect minors from adult content.

2) Do other search engines indeed have to do the same?

All local portals have implemented the age verification process to comply with Korean law on juvenile protection from adult content. The words for adult content are recommended by the government to all portals/search engines.

3) Does this mean you’ll real time verify the information?

The database for age verification is provided by 3rd party agencies which are endorsed by the government. All Internet sites use the same agencies. The database information is regularly refreshed by the agencies.

4) Will it be stored somehow so people don’t have to reenter?

Once a user has been verified as an adult, the user does not have to go through the verification process again. The local portals use the same process.

5) Does this happen only on Google Korea?

Yes, it affects only google.co.kr.

Postscript Barry 3/14/08: Age Verification at Google Korea from Google Operating System shows a step by step screen capture of how the age verification process works at Google Korea.